The purpose of this senior thesis is to explore the abstract ideas that give rise to the well-known Fourier series and transforms. More specifically, finite group representations are used to study the structure of Hilbert spaces to determine under what conditions an element of the space can be expanded as a sum. The Peter-Weyl theorem is the result that shows why integrable functions can be expressed in terms of trigonometric functions. Although some theorems will not be proved, the results that can be derived from them will be briefly discussed. For instance, the Pontryagin Duality theorem states that there is a canonical isomorphism between a group and the second dual of the group, and it can be used to prove $Plancherel$ theorem which essentially says that the Fourier transform is itself a unitary isomorphism.