The last two decades have seen growing awareness of and emphasis on the replication of empirical findings. While this is a large literature, very little of it has focused on or considered the interaction of replication and psychometrics. This is unfortunate given that sound measurement is crucial when considering the complex constructs studied in psychological research. If the psychometric properties of a scale fail to replicate, then inferences made using scores from that scale are questionable at best. In this dissertation, I begin to address replication issues in factor analysis – a widely used psychometric method in psychology. After noticing inconsistencies across results for studies that factor analyzed the same scale, I sought to gain a better understanding of what replication means in factor analysis as well as address issues that affect the replicability of factor analytic models. With this work, I take steps toward integrating factor analysis into the broader replication discussion. Ultimately, the goal of this dissertation was to highlight the importance of psychometric replication and bring attention to its role in fostering a more replicable scientific literature.

This research explores tests for statistical suppression. Suppression is a statistical phenomenon whereby the magnitude of an effect becomes larger when another variable is added to the regression equation. From a causal perspective, suppression occurs when there is inconsistent mediation or negative confounding. Several different estimators for suppression are evaluated conceptually and in a statistical simulation study where we impose suppression and non-suppression conditions. For each estimator without an existing standard error formula, one was derived in order to conduct significance tests and build confidence intervals. Overall, two of the estimators were biased and had poor coverage, one worked well but had inflated type-I error rates when the population model was complete mediation. As a result of analyzing these three tests, a fourth was considered in the late stages of the project and showed promising results that address concerns of the other tests. When the tests were applied to real data, they gave similar results and were consistent.

Relative to their heterosexual peers, sexual minority youth experience significant mental health disparities. This, in part, is due to prejudicial encounters (e.g., discrimination, victimization) because of their sexual minority status, and potential compounding stressors from prejudicial experiences related to their ethnic minority status, which could lead to worse mental health outcomes due to intersecting minority stress processes. Surprisingly, even though religiosity has been identified as a protective factor in the general literature for adolescents and young adults, few studies have examined whether religiosity serves as a potential buffer of the relations between stress and mental health outcomes in sexual minority Latinx youth. Thus, the goals of this study were to examine: (1) whether ethnic discrimination and sexuality discrimination have additive or interactive effects on depressive symptoms, and (2) whether self or family religiosity moderate the relations between discrimination and depressive symptoms, in a sample of 377 sexual minority Latinx youth (Mage = 20.29, SD = 2.61, age range 14-24). Results showed that there was no interactive effect between ethnic discrimination and sexuality discrimination in predicting depressive symptoms. There was a significant interaction between own religiosity and sexuality discrimination in predicting depressive symptoms, in which own religiosity was negatively associated with depressive symptoms only at low level of sexuality discrimination. No significant interaction emerged between own religiosity and ethnic discrimination. Finally, there were significant interactive effects between family religiosity and discrimination (ethnic and sexuality), in which family religiosity was negatively associated with depressive symptoms only at average and low level of discrimination. These findings highlight the importance of examining the intersection of religion, sexuality, and Latinx minority status in relation to mental health outcome.

Past research suggests that both Alcohol Expectancies and Subjective Response are strong predictors of drinking. However, most studies do not account for the shared variance or relations between the two. Social cognitive and expectancy theories suggest that cognitions may distort reality, creating a discrepancy between expected and subjective effects. Only one study has tested the effects of such discrepancies (Morean et al., 2015), but that study was cross-sectional, making it impossible to determine the direction of effects. As such, the present study sought to test prospective associations between expectancy-subjective response interactions and future drinking behavior. Participants (N=448) were randomly assigned to receive alcohol (target blood alcohol alcohol =.08 g%) or placebo, with 270 in the alcohol condition. Alcohol expectancies and subjective response were assessed across the full range of affective space of valence by arousal. Hierarchical regression tested whether expectancies, subjective response, and their interaction predicted follow-up drinking in 258 participants who reached a blood alcohol curve of >.06 (to differentiate blood alcohol curve limbs). Covariates included gender, age, drinking context, and baseline drinking. High arousal subjective response was tested on the ascending limb and low arousal subjective response on the descending limb. High arousal positive expectancies and subjective response interacted to predict future drinking, such that mean and low levels of high arousal positive subjective response were associated with more drinking when expectancies were higher. High arousal negative expectancies and subjective response also interacted to predict future drinking, such that high levels of high arousal negative subjective response marginally predicted more drinking when expectancies were lower. There were no interactions between low arousal positive or low arousal negative expectancies and subjective response. Results suggest that those who expected high arousal positive subjective response but did not receive many of these effects drank more, and those who did not expect to feel high arousal negative subjective response but did in fact feel these effects also drank more. The results suggest that challenging inaccurate positive expectancies and increasing awareness of true negative subjective response may be efficacious ways to reduce drinking.

To make meaningful comparisons on a construct of interest across groups or over time, measurement invariance needs to exist for at least a subset of the observed variables that define the construct. Often, chi-square difference tests are used to test for measurement invariance. However, these statistics are affected by sample size such that larger sample sizes are associated with a greater prevalence of significant tests. Thus, using other measures of non-invariance to aid in the decision process would be beneficial. For this dissertation project, I proposed four new effect size measures of measurement non-invariance and analyzed a Monte Carlo simulation study to evaluate their properties and behavior in addition to the properties and behavior of an already existing effect size measure of non-invariance. The effect size measures were evaluated based on bias, variability, and consistency. Additionally, the factors that affected the value of the effect size measures were analyzed. All studied effect sizes were consistent, but three were biased under certain conditions. Further work is needed to establish benchmarks for the unbiased effect sizes.