The pervasive use of the Web has connected billions of people all around the globe and enabled them to obtain information at their fingertips. This results in tremendous amounts of user-generated data which makes users traceable and vulnerable to privacy leakage attacks. In general, there are two types of privacy leakage attacks for user-generated data, i.e., identity disclosure and private-attribute disclosure attacks. These attacks put users at potential risks ranging from persecution by governments to targeted frauds. Therefore, it is necessary for users to be able to safeguard their privacy without leaving their unnecessary traces of online activities. However, privacy protection comes at the cost of utility loss defined as the loss in quality of personalized services users receive. The reason is that this information of traces is crucial for online vendors to provide personalized services and a lack of it would result in deteriorating utility. This leads to a dilemma of privacy and utility.

Protecting users' privacy while preserving utility for user-generated data is a challenging task. The reason is that users generate different types of data such as Web browsing histories, user-item interactions, and textual information. This data is heterogeneous, unstructured, noisy, and inherently different from relational and tabular data and thus requires quantifying users' privacy and utility in each context separately. In this dissertation, I investigate four aspects of protecting user privacy for user-generated data. First, a novel adversarial technique is introduced to assay privacy risks in heterogeneous user-generated data. Second, a novel framework is proposed to boost users' privacy while retaining high utility for Web browsing histories. Third, a privacy-aware recommendation system is developed to protect privacy w.r.t. the rich user-item interaction data by recommending relevant and privacy-preserving items. Fourth, a privacy-preserving framework for text representation learning is presented to safeguard user-generated textual data as it can reveal private information.

The explosive Web growth in the last decade has drastically changed the way billions of people all around the globe conduct numerous activities including creating, sharing, and consuming information. The massive amount of user-generated information encourages companies and service providers to collect users' information and use it in order to better their own goals and then further provide personalized services to users as well. However, the users' information contains their private and sensitive information and can lead to breach of users' privacy. Anonymizing users' information before publishing and using such data is vital in securing their privacy. Due to the many forms of user information (e.g., structural, interactions, etc), different techniques are required for anonymization of users' data. In this thesis, first we discuss different anonymization techniques for various types of user-generated data, i.e., network graphs, web browsing history, and user-item interactions. Our experimental results show the effectiveness of such techniques for data anonymization. Then, we briefly touch on securely and privately sharing information through blockchains.