The rapid advancement in technology and data sharing has led to an explosion of detailed structural and functional brain maps. In modern neuroimaging research, identifying the relationships between these maps is increasingly crucial. However, conventional statistical methods often fail to consider the spatial characteristics inherent in brain map data. To address this, several new methods have been developed to create null distributions that preserve the spatial autocorrelation of brain maps, leading to more reliable statistical evaluations.
This article provides a thorough evaluation of ten established null frameworks used in statistical analyses of neuroimaging data. To assess their effectiveness under controlled conditions, we first applied these frameworks to a series of simulations, examining how data resolution and spatial autocorrelation affect their error rates. We then tested each framework using two empirical neuroimaging datasets to evaluate their performance in two scenarios: (1) determining the correspondence between different brain maps, such as correlating two activation maps, and (2) analyzing the spatial distribution of a specific feature within a defined area, for example, measuring the specificity of an activation map within a particular functional network. Furthermore, we explored how variations in the implementation of these null models might influence their performance.
Consistent with previous findings, our results indicate that basic null models that do not account for spatial autocorrelation consistently produce inflated false positive rates and overly permissive statistical estimates. While spatially-constrained null models offered more realistic and conservative estimates, these frameworks still exhibited elevated false positive rates and inconsistent performance across different types of analyses. Notably, changes in parcellation and resolution had minimal impact on the performance of the null models across our tests.
In conclusion, our findings underscore the continued need for the development of robust statistical methods for comparing brain maps. This report offers a standardized framework for evaluating and comparing future advancements in this critical area of neuroimaging research.
