Yes, it’s accurate that when data is processed without a left-out observation, the value of N, representing the number of observations, will be one less than when utilizing stan with doexactloo=FALSE.
There’s a common misconception that incorporating factors lacking predictive power introduces additional uncertainty, consequently diminishing the Expected Log Predictive Density (ELPD) of left-out observations in both exact and Pareto Smoothed Importance Sampling Leave-One-Out cross-validation (PSIS-LOO). This would suggest that the model is penalized for over-parameterization. If eliminating redundant factors doesn’t lead to an increase in ELPD, one might wonder whether the most effective predictive model is invariably the one encompassing all available factors. This is a point of understandable confusion.
Consider a scenario where the objective is to predict tree age without invasive core sampling, relying on easily observable characteristics. The relationship between the logarithm of tree age and predictive factors isn’t necessarily linear. Instead of predefining a transformation, we aim to empirically determine the optimal power transformations for each factor to achieve the best model fit. However, initial analysis reveals that only tree diameter exhibits a clear predictive relationship. Upon examining the 95% credibility intervals for the coefficients of other factors, they all encompass zero, indicating a lack of substantial evidence for their influence. Therefore, attempting to estimate exponents for these factors proves to be unproductive. A simpler approach might involve skipping exponent estimation and directly applying a log transformation to the diameter data. Nevertheless, it remains pertinent to investigate whether a power transformation, utilizing estimated exponents, outperforms a model with an a priori specified transformation in terms of predictive accuracy. This is a High Compare scenario we are interested in.
To visualize model performance, while the most appropriate Posterior Predictive Check (PPC) plot is debatable, violin plots offer valuable insights. Below are violin plots derived from a model that exclusively includes diameter as a predictor. Log-transformed age predictions are generated using predictive data for each tree, categorized by diameter classes: <30cm, 30-50 cm, and >50cm.
The plots illustrate that the distribution of both predictive distributions and actual data is broader in the narrowest diameter class (a). This observation aligns with biological intuition. Only older trees can attain large diameters, but trees can exhibit slow growth due to diverse factors, implying that narrow trees can be either young or old, leading to greater variability in age for smaller diameter trees. This high compare visualization aids in understanding model behavior across different data segments.
Your expert insights and feedback are highly valued in refining this analysis.
