Investigation of measurement invariance (MI) commonly assumes correct specification of dimensionality across multiple groups. Although research shows that violation of the dimensionality assumption can cause bias in model parameter estimation for single-group analyses, little research on this issue has been conducted for multiple-group analyses. This study explored the effects of mismatch in dimensionality between data and analysis models with multiple-group analyses at the population and sample levels. Datasets were generated using a bifactor model with different factor structures and were analyzed with bifactor and single-factor models to assess misspecification effects on assessments of MI and latent mean differences. As baseline models, the bifactor models fit data well and had minimal bias in latent mean estimation. However, the low convergence rates of fitting bifactor models to data with complex structures and small sample sizes caused concern. On the other hand, effects of fitting the misspecified single-factor models on the assessments of MI and latent means differed by the bifactor structures underlying data. For data following one general factor and one group factor affecting a small set of indicators, the effects of ignoring the group factor in analysis models on the tests of MI and latent mean differences were mild. In contrast, for data following one general factor and several group factors, oversimplifications of analysis models can lead to inaccurate conclusions regarding MI assessment and latent mean estimation.

Structural equation modeling is potentially useful for assessing mean differences between groups on latent variables (i.e., factors). However, to evaluate these differences accurately, the parameters of the indicators of these latent variables must be specified correctly. The focus of the current research is on the specification of between-group equality constraints on the loadings and intercepts of indicators. These equality constraints are referred to as invariance constraints. Previous simulation studies in this area focused on fitting a particular model to data that were generated to have various levels and patterns of non-invariance. Results from these studies were interpreted from a viewpoint of assumption violation rather than model misspecification. In contrast, the current study investigated analysis models with varying number of invariance constraints given data that were generated based on a model with indicators that were invariant, partially invariant, or non-invariant. More broadly, the current simulation study was conducted to examine the effect of correctly or incorrectly imposing invariance constraints as well as correctly or incorrectly not imposing invariance constraints on the assessment of factor mean differences. The results indicated that different types of analysis models yield different results in terms of Type I error rates, power, bias in estimation of factor mean difference, and model fit. Benefits and risks are associated with imposing or reducing invariance constraints on models. In addition, model fit or lack of fit can lead to wrong decisions concerning invariance constraints.