ISMEM

Abstract

This paper introduces a novel intrinsic semiparametric mixed-effects model (ISMEM) to characterize the complex relationships between manifold-valued responses and Euclidean-valued covariates in the longitudinal setting. Manifold-valued data, characterized by their inherent nonlinearity, high dimensionality, and specific geometric structures, present significant challenges for longitudinal analysis. Most existing intrinsic models are developed for cross-sectional data and unsuited for longitudinal studies. To address this limitation, we propose ISMEM, an efficient framework designed for longitudinal manifold-valued datasets that capture such relationships at both group and individual levels.

Key features of ISMEM include (i) the integration of fixed and random effects on Riemannian manifolds, enabling analysis at multiple levels; (ii) a semiparametric structure combining parametric and nonparametric components, enhancing both flexibility and interpretability; (iii) the preservation of the geometric properties of manifold-valued observations, offering improved robustness and interpretability compared to Euclidean-based models.

In addition, we develop a two-stage iterative estimation procedure and validate our approach through simulations on the symmetric positive definite (SPD) manifold. Finally, we apply ISMEM to longitudinal data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), reconstructing and comparing continuous 3D shape trajectories of the lateral ventricle in Kendall shape space across distinct groups.

Real Data Analysis in Kendall Shape Space

Alzheimer's disease (AD) is a progressive neurodegenerative condition and a leading cause of dementia in the elderly. Structural MRI studies have shown that the lateral ventricle (LV), a subcortical structure, is a significant morphological marker of AD progression.

We utilized longitudinal T1-weighted MRI data from the ADNI-GO and ADNI2 phases of the Alzheimer's Disease Neuroimaging Initiative (ADNI), comprising 3685 scans from 1090 subjects. After preprocessing and segmentation of the left lateral ventricle (LLV), we performed surface reconstruction using area-preserving spherical mapping, then represented the surfaces via square-root normal fields (SRNF) and registered them to a global template.

Each registered surface was transformed into a point on the Kendall shape space by eliminating translation, scale, and rotation. This enables intrinsic statistical modeling on nonlinear manifolds. We applied the proposed ISMEM model to estimate group-level and subject-specific trajectories of LLV shape evolution.

Experiment 1: Reconstruct the LLV Shape Surface Evolutionary Trajectory in a Univariate Case

We then reconstruct the global mean evolutionary trajectories of the LLV using the trained RNLMM, LESA, and ISMEM models, as illustrated in Figure 1. In the figure, the horizontal axis denotes age (ranging from 60 to 90), and each shape is compared with the first one in the same row, which corresponds to the LLV shape at age 60. Local morphological changes are visualized through color differences at corresponding locations. The colorbar reflects the magnitude of 3D deformation relative to the baseline shape at age 60, with warmer (redder) tones indicating greater structural deviations.

From visual inspection, all three methods display warmer colors in similar anatomical regions, indicating that they capture the general aging trend in LLV morphology. As such, the global shape evolution trajectories derived from each method are broadly consistent. ISMEM is capable of detecting richer local deformations, particularly in anatomically critical regions such as the frontal horn, atrium, occipital horn, and temporal horn, yielding higher reconstruction accuracy and finer anatomical fidelity. In contrast, the shape evolution trajectory generated by RNLMM or LESA from age 60 to 90 appears overly smooth, potentially masking subtle but clinically relevant local deformations.

Owing to its enhanced sensitivity to such variations, ISMEM provides a more comprehensive characterization of LLV shape progression with aging. Simultaneously, we provide the 3D deformation field to visualize the corresponding evolutionary trajectory of the LLV, as shown in Figure 3. These subject-specific results further highlight the ability of ISMEM to capture localized deformations and subject-level variations.

Experiment 2: Reconstruct the LLV Shape Surface Evolutionary Trajectory in a Multivariate Case

In this section, we extend our analysis to the multivariate case, incorporating age, gender, and AD diagnosis (categorized as NC, MCI, or AD) as influential factors affecting the 3D shape surface of the LLV. Specifically, we define the covariates as x_ij = [x_ij¹, x_ij², x_ij³, x_ij⁴], where x_ij¹ denotes age, x_ij² indicates gender (male: 1, female: 0), and [x_ij³, x_ij⁴] signifies diagnosis status (NC: [0, 0], MCI: [1, 0], AD: [0, 1]). We also set z_ij = x_ij² to account for subject-specific effects.

We then apply the proposed ISMEM to analyze the ADNIGO2 dataset. Figures 6 and 7 display the reconstructed mean surface evolutionary trajectories and corresponding 3D deformation fields for females within the NC, MCI, and AD groups. Figures 8 and 9 present similar visualizations for males.

Conclusions

• Alzheimer's disease (AD) accelerates shape changes in the lateral ventricles, creating distinct shape differences between AD and NC groups.
• These differences between AD and NC are more evident in females aged 68–77, while in males, they emerge later, around ages 72–81.
• The degree of difference between the MCI and NC groups is smaller than that observed between the AD and NC groups. Distinct variations in the left lateral ventricle among the NC, MCI, and AD groups are primarily observed in the frontal and occipital horns.