Using Machine Learning Algorithms for Privacy

Machine learning algorithms have a wide variety of applications and use cases. They are robust in the sense that they can continue to learn and improve long after they have been deployed without much programmer supervision. One key area that machine learning has been used for is in the detection and classification of objects in images and videos. This so-called computer vision has typically been used by companies to extract user information from the images and videos that they post. Meta (formerly known as Facebook) had been using such algorithms to automatically tag users in pictures that were uploaded to the Facebook website up until November 2021 [1]. Although these algorithms have been used to exploit user’s privacy, they can also be used to help ensure this privacy. For this creative project, I developed a machine learning model that could detect faces in a given picture and identify the area of the picture that these faces took up. Training a model from scratch can take millions of images of data and hundreds of hours on powerful GPUs. Since I didn’t have access to those resources, I began with a pre-trained model known as VGG16 by Karen Simonyan & Andrew Zisserman. From there, I took 90 pictures of myself and annotated where in the image my face was located. Since 90 pictures wouldn’t be enough data for this algorithm, I used an image augmentation algorithm to randomly crop, flip, change brightness, change gamma, and recolor the images to expand the dataset. In total, I used 5400 images to train the algorithm. The machine learning model had a loss value that hovered around 0.1 thanks to the VGG16 model. It was able to accurately detect my face and also adapt whenever I moved my face horizontally and vertically across a camera. However, the model struggled to draw a bounding box whenever I moved my face forward or backward in the camera shot.