Voronoi Tessellation and Non-Linear Diffusion for Density Estimation

Density estimation is ubiquitous in statistical modeling and machine learning. It aims to reconstruct a probability distribution from a dataset. In this work, a non-parametric approach is developed using Voronoi tessellation and non-linear diffusion. The basic tessellation method introduces high variance in estimates. To address this, a consensus model is proposed, formulated as a system of ordinary linear differential equations, to continuously modify the dataset before applying Voronoi tessellation estimation. A regularization parameter (time) is fine-tuned by optimizing the mean integrated squared error (MISE) and least squares cross-validation (LSCV) criteria. While LSCV is less precise than MISE for selecting the optimal parameter, it has the advantage of not requiring the true distribution of the underlying data, making it more practical. One issue with regularization through consensus models is the buildup of density near the boundary. To mitigate this effect, weights are introduced into the consensus models to enforce a specific behavior of the regularizing sample at large times. Notably, using weights taken from a Gaussian distribution results in a superior fit with lower mean squared error. Finally, this approach is generalized to two dimensional space. Here, the natural order of a one-dimensional space can no longer be relied on. Instead, Delaunay triangulation is used to determine the neighboring graph of the dataset. This graph allows to generalize the consensus model in higher dimensions and to define a regularization method for tessellation estimation. Numerical examples are provided to illustrate the method further.