We present an analysis of the stellar populations of 102 visually selected early-type galaxies (ETGs) with spectroscopic redshifts (0.35 ≲ z ≲ 1.5) from observations in the Early Release Science program with the Wide Field Camera 3 (WFC3) on the Hubble Space Telescope (HST). We fit one- and two-component synthetic stellar models to the ETGs UV-optical-near-IR spectral energy distributions and find that a large fraction (∼40%) are likely to have experienced a minor (fYC ≲ 10% of stellar mass) burst of recent (tYC ≲ 1 Gyr) star formation. The measured age and mass fraction of the young stellar populations do not strongly trend with measurements of galaxy morphology. We note that massive (M > 1010.5M☼) recent star-forming ETGs appear to have larger sizes. Furthermore, high-mass, quiescent ETGs identified with likely companions populate a distinct region in the size-mass parameter space, in comparison with the distribution of massive ETGs with evidence of recent star formation (RSF). We conclude that both mechanisms of quenching star formation in disk-like ETGs and (gas-rich, minor) merger activity contribute to the formation of young stars and the size-mass evolution of intermediate redshift ETGs. The number of ETGs for which we have both HST WFC3 panchromatic (especially UV) imaging and spectroscopically confirmed redshifts is relatively small, therefore, a conclusion about the relative roles of both of these mechanisms remains an open question.