Cognitive radio (CR) and device-to-device (D2D) systems are two promising dynamic spectrum access schemes in wireless communication systems to provide improved quality-of-service, and efficient spectrum utilization. This dissertation shows that both CR and D2D systems benefit from properly designed cooperation scheme.

In underlay CR systems, where secondary users (SUs) transmit simultaneously with primary users (PUs), reliable communication is by all means guaranteed for PUs, which likely deteriorates SUs’ performance. To overcome this issue, cooperation exclusively among SUs is achieved through multi-user diversity (MUD), where each SU is subject to an instantaneous interference constraint at the primary receiver. Therefore, the active number of SUs satisfying this constraint is random. Under different user distributions with the same mean number of SUs, the stochastic ordering of SU performance metrics including bit error rate (BER), outage probability, and ergodic capacity are made possible even without observing closed form expressions. Furthermore, a cooperation is assumed between primary and secondary networks, where those SUs exceeding the interference constraint facilitate PU’s transmission by relaying its signal. A fundamental performance trade-off between primary and secondary networks is observed, and it is illustrated that the proposed scheme outperforms non-cooperative underlay CR systems in the sense of system overall BER and sum achievable rate.

Similar to conventional cellular networks, CR systems suffer from an overloaded receiver having to manage signals from a large number of users. To address this issue, D2D communications has been proposed, where direct transmission links are established between users in close proximity to offload the system traffic. Several new cooperative spectrum access policies are proposed allowing coexistence of multiple D2D pairs in order to improve the spectral efficiency. Despite the additional interference, it is shown that both the cellular user’s (CU) and the individual D2D user's achievable rates can be improved simultaneously when the number of D2D pairs is below a certain threshold, resulting in a significant multiplexing gain in the sense of D2D sum rate. This threshold is quantified for different policies using second order approximations for the average achievable rates for both the CU and the individual D2D user.

This thesis aims to investigate the capacity and bit error rate (BER) performance of multi-user diversity systems with random number of users and considers its application to cognitive radio systems. Ergodic capacity, normalized capacity, outage capacity, and average bit error rate metrics are studied. It has been found that the randomization of the number of users will reduce the ergodic capacity. A stochastic ordering framework is adopted to order user distributions, for example, Laplace transform ordering. The ergodic capacity under different user distributions will follow their corresponding Laplace transform order. The scaling law of ergodic capacity with mean number of users under Poisson and negative binomial user distributions are studied for large mean number of users and these two random distributions are ordered in Laplace transform ordering sense. The ergodic capacity per user is defined and is shown to increase when the total number of users is randomized, which is the opposite to the case of unnormalized ergodic capacity metric. Outage probability under slow fading is also considered and shown to decrease when the total number of users is randomized. The bit error rate (BER) in a general multi-user diversity system has a completely monotonic derivative, which implies that, according to the Jensen's inequality, the randomization of the total number of users will decrease the average BER performance. The special case of Poisson number of users and Rayleigh fading is studied. Combining with the knowledge of regular variation, the average BER is shown to achieve tightness in the Jensen's inequality. This is followed by the extension to the negative binomial number of users, for which the BER is derived and shown to be decreasing in the number of users. A single primary user cognitive radio system with multi-user diversity at the secondary users is proposed. Comparing to the general multi-user diversity system, there exists an interference constraint between secondary and primary users, which is independent of the secondary users' transmission. The secondary user with high- est transmitted SNR which also satisfies the interference constraint is selected to communicate. The active number of secondary users is a binomial random variable. This is then followed by a derivation of the scaling law of the ergodic capacity with mean number of users and the closed form expression of average BER under this situation. The ergodic capacity under binomial user distribution is shown to outperform the Poisson case. Monte-Carlo simulations are used to supplement our analytical results and compare the performance of different user distributions.