The Earth’s lowermost mantle large low velocity provinces are accompanied by small-scale ultralow velocity zones in localized regions on the core-mantle boundary. Large low velocity provinces are hypothesized to be caused by large-scale compositional heterogeneity (i.e., thermochemical piles). The origin of ultralow velocity zones, however, remains elusive. Here we perform three-dimensional geodynamical calculations to show that the current locations and shapes of ultralow velocity zones are related to their cause. We find that the hottest lowermost mantle regions are commonly located well within the interiors of thermochemical piles. In contrast, accumulations of ultradense compositionally distinct material occur as discontinuous patches along the margins of thermochemical piles and have asymmetrical cross-sectional shape. Furthermore, the lateral morphology of these patches provides insight into mantle flow directions and long-term stability. The global distribution and large variations of morphology of ultralow velocity zones validate a compositionally distinct origin for most ultralow velocity zones.