Advanced search
Start date
Betweenand

Interpretability and efficiency in hypothesis tests

Grant number: 17/03363-8
Support Opportunities:Regular Research Grants
Start date: June 01, 2017
End date: May 31, 2019
Field of knowledge:Physical Sciences and Mathematics - Probability and Statistics - Statistics
Principal Investigator:Rafael Izbicki
Grantee:Rafael Izbicki
Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil
Associated researchers: Luis Ernesto Bueno Salasar ; Rafael Bassi Stern

Abstract

Hypothesis testing is a very common and widespread statistical tool. Unfortunately, such methodology still presents several challenges to statisticians. This project aims at developing hypothesis tests by filling several existing gaps.More precisely, the follows issues will be addressed: (1) Agnostic Tests. There is a disagreement about the interpretation of results from a hypothesis test: while some understand that a hypothesis test is able to either reject or accept the null hypothesis $H_0$, others believe its outcomes should be interpreted as either reject or not reject $H_0$. This often lead practitioners to have difficulties in understanding the conclusions from significance tests. In particular, the second (and most common) perspective is deeply linked to the development of non-inferiority tests used in clinical trials. Here, we propose an alternative formulation to hypothesis tests in which, besidesthe decisions “accept $H_0$“ and “reject $H_0$“, there is a third decision, namely the “no conclusion“ decision, which we call the agnostic decision. (2) Bayesian Nonparametric Tests. Because of the large volume of data available today in several applications, nonparametric methods have been gaining a lot of attention as they allow one to make less assumption about the data generating process. Unfortunately, there is almost no literature on Bayesian nonparametric tests, even though the Bayesian paradigm is widespread today. Here, we investigate new tests that try to overcome such gap. In particular, we investigate Bayesian nonparametric two-sample tests. (3) FBST in High Dimensions. Another challenge that exits in several applications is the issue of high dimensionality: in many problems, the number of covariates is very large; many times larger than the sample size. This makes sevaral standard methods fail. In particular, it has been observed that the Full Bayesian Significance Test has difficulties dealing with such situation. We will propose improvements in such method so that it is able to overcome the issue of high dimensionality, and we will investigate their theoretical properties. As a part of this project, we will also develop R packages that implementthe methods developed here. (AU)

Articles published in Agência FAPESP Newsletter about the research grant:
More itemsLess items
Articles published in other media outlets ( ):
More itemsLess items
VEICULO: TITULO (DATA)
VEICULO: TITULO (DATA)

Scientific publications (21)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
CEREGATTI, RAFAEL DE CARVALHO; IZBICKI, RAFAEL; BUENO SALASAR, LUIS ERNESTO. WIKS: a general Bayesian nonparametric index for quantifying differences between two populations. TEST, . (19/11321-9, 17/03363-8)
IZBICKI, RAFAEL; LEE, ANN B.; FREEMAN, PETER E.. PHOTO-z ESTIMATION: AN EXAMPLE OF NONPARAMETRIC CONDITIONAL DENSITY ESTIMATION UNDER SELECTION BIAS. Annals of Applied Statistics, v. 11, n. 2, p. 698-724, . (17/03363-8, 14/25302-2)
STERN, JULIO MICHAEL; IZBICKI, RAFAEL; ESTEVES, LUIS GUSTAVO; STERN, RAFAEL BASSI. Logically-consistent hypothesis testing and the hexagon of oppositions. LOGIC JOURNAL OF THE IGPL, v. 25, n. 5, p. 741-757, . (13/07375-0, 17/03363-8, 14/50279-4, 14/25302-2)
DINIZ, MARCIO ALVES; IZBICKI, RAFAEL; LOPES, DANILO; SALASAR, LUIS ERNESTO. Comparing probabilistic predictive models applied to football. Journal of the Operational Research Society, v. 70, n. 5, p. 770-782, . (14/25302-2, 17/03363-8)
VAZ, AFONSO FERNANDES; IZBICKI, RAFAEL; STERN, RAFAEL BASSI. Quantification Under Prior Probability Shift: the Ratio Estimator and its Extensions. JOURNAL OF MACHINE LEARNING RESEARCH, v. 20, . (17/03363-8)
DALMASSO, N.; POSPISIL, T.; LEE, A. B.; IZBICKI, R.; FREEMAN, P. E.; MALZ, A. I.. Conditional density estimation tools in python and R with applications to photometric redshifts and likelihood-free cosmological inference. ASTRONOMY AND COMPUTING, v. 30, . (19/11321-9, 17/03363-8)
M. MUSETTI; R. IZBICKI. Combinando Métodos de Aprendizado Supervisionado para a Melhoria da Previsão do Redshift de Galáxias. TEMA (São Carlos), v. 21, n. 1, p. 117-131, . (17/03363-8, 19/11321-9)
IZBICKI, RAFAEL; LEE, ANN B.. Converting high-dimensional regression to high-dimensional conditional density estimation. ELECTRONIC JOURNAL OF STATISTICS, v. 11, n. 2, p. 2800-2831, . (17/03363-8, 14/25302-2)
COSCRATO, VICTOR; DE ALMEIDA INACIO, MARCO HENRIQUE; IZBICKI, RAFAEL. The NN-Stacking: Feature weighted linear stacking through neural networks. Neurocomputing, v. 399, p. 141-152, . (19/11321-9, 17/03363-8)
INACIO, MARCO; IZBICKI, RAFAEL; GYIRES-TOTH, BALINT. Distance assessment and analysis of high-dimensional samples using variational autoencoders. INFORMATION SCIENCES, v. 557, p. 407-420, . (19/11321-9, 17/03363-8)
PULGROSSI, RAFAEL CATOIA; OLIVEIRA, NATALIA LOMBARDI; POLPO, ADRIANO; IZBICKI, RAFAEL; POLPO, A; STERN, J; LOUZADA, F; IZBICKI, R; TAKADA, H. A Comparison of Two Methods for Obtaining a Collective Posterior Distribution. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING, MAXENT 37, v. 239, p. 10-pg., . (17/03363-8)
CEREGATTI, RAFAEL DE C.; IZBICKI, RAFAEL; SALASAR, LUIS ERNESTO B.; POLPO, A; STERN, J; LOUZADA, F; IZBICKI, R; TAKADA, H. A Nonparametric Bayesian Approach for the Two-Sample Problem. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING, MAXENT 37, v. 239, p. 11-pg., . (17/03363-8)
DE ALMEIDA INACIO, MARCO HENRIQUE; IZBICKI, RAFAEL; SALASAR, LUIS ERNESTO. Comparing two populations using Bayesian Fourier series density estimation. COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION, v. 49, n. 1, p. 261-282, . (14/25302-2, 17/03363-8)
ESTEVES, LUIS GUSTAVO; IZBICKI, RAFAEL; STERN, JULIO MICHAEL; STERN, RAFAEL BASSI. Pragmatic Hypotheses in the Evolution of Science. Entropy, v. 21, n. 9, . (14/50279-4, 19/11321-9, 13/07375-0, 17/03363-8, 14/25302-2)
DALMASSO, NICCOLO; LEE, ANN B.; IZBICKI, RAFAEL; POSPISIL, TAYLOR; KIM, ILMUN; LIN, CHIEH-AN; CHIAPPA, S; CALANDRA, R. Validation of Approximate Likelihood and Emulator Models for Computationally Intensive Simulations. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, v. 108, p. 12-pg., . (19/11321-9, 17/03363-8)
VAZ, AFONSO; IZBICKI, RAFAEL; STERN, RAFAEL BASSI; POLPO, A; STERN, J; LOUZADA, F; IZBICKI, R; TAKADA, H. Prior Shift Using the Ratio Estimator. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING, MAXENT 37, v. 239, p. 11-pg., . (17/03363-8)
CEREGATTI, RAFAEL DE CARVALHO; IZBICKI, RAFAEL; BUENO SALASAR, LUIS ERNESTO. WIKS: a general Bayesian nonparametric index for quantifying differences between two populations. TEST, v. 30, n. 1, p. 18-pg., . (19/11321-9, 17/03363-8)
COSCRATO, VICTOR; INACIO, MARCO H. A.; BOTARI, TIAGO; IZBICKI, RAFAEL. NLS: An accurate and yet easy-to-interpret prediction method. NEURAL NETWORKS, v. 162, p. 14-pg., . (19/11321-9, 17/06161-7, 17/03363-8)
COSCRATO, VICTOR; IZBICKI, RAFAEL; STERN, RAFAEL BASSI. Agnostic tests can control the type I and type II errors simultaneously. BRAZILIAN JOURNAL OF PROBABILITY AND STATISTICS, v. 34, n. 2, p. 230-250, . (19/11321-9, 17/03363-8)
ESTEVES, LUIS GUSTAVO; IZBICKI, RAFAEL; STERN, RAFAEL BASSI. Teaching Decision Theory Proof Strategies Using a Crowdsourcing Problem. AMERICAN STATISTICIAN, v. 71, n. 4, p. 336-343, . (17/03363-8, 14/25302-2)
VILLAR COUTO, CYNTHIA M.; CUMMING, GRAEME S.; LACORTE, GUSTAVO A.; CONGRAINS, CARLOS; IZBICKI, RAFAEL; BRAGA, ERIKA MARTINS; ROCHA, CRISTIANO D.; MORALEZ-SILVA, EMMANUEL; HENRY, DOMINIC A. W.; MANU, SHIIWUA A.; et al. Avian haemosporidians in the cattle egret (Bubulcus ibis) from central-western and southern Africa: High diversity and prevalence. PLoS One, v. 14, n. 2, . (10/50406-5, 16/01673-7, 17/03363-8)