The apolipoprotein E (APOE) e4 genotype is a powerful risk factor for late-onset Alzheimer’s disease (AD). In the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort, we previously reported significant baseline structural differences in APOE e4 carriers relative to non-carriers, involving the left hippocampus more than the right—a difference more pronounced in e4 homozygotes than heterozygotes. We now examine the longitudinal effects of APOE genotype on hippocampal morphometry at 6-, 12- and 24-months, in the ADNI cohort. We employed a new automated surface registration system based on conformal geometry and tensor-based morphometry. Among different hippocampal surfaces, we computed high-order correspondences, using a novel inverse-consistent surface-based fluid registration method and multivariate statistics consisting of multivariate tensor-based morphometry (mTBM) and radial distance. At each time point, using Hotelling’s T² test, we found significant morphological deformation in APOE e4 carriers relative to non-carriers in the full cohort as well as in the non-demented (pooled MCI and control) subjects at each follow-up interval. In the complete ADNI cohort, we found greater atrophy of the left hippocampus than the right, and this asymmetry was more pronounced in e4 homozygotes than heterozygotes. These findings, combined with our earlier investigations, demonstrate an e4 dose effect on accelerated hippocampal atrophy, and support the enrichment of prevention trial cohorts with e4 carriers.