Integration of dielectrics with graphene is essential to the fulfillment of graphene based electronic applications. While many dielectric deposition techniques exist, plasma enhanced atomic layer deposition (PEALD) is emerging as a technique to deposit ultrathin dielectric films with superior densities and interfaces. However, the degree to which PEALD on graphene can be achieved without plasma-induced graphene deterioration is not well understood. In this work, we investigate a range of plasma conditions across a single sample, characterizing both oxide growth and graphene deterioration using spectroscopic analysis and atomic force microscopy. Investigation of graphene and film quality produced by these conditions yields insight into plasma effects. Using a specially designed sample configuration, we achieve ultrathin (< 1 nm) aluminum oxide films atop graphene.