Modern machine learning systems leverage data and features from multiple modalities to gain more predictive power. In most scenarios, the modalities are vastly different and the acquired data are heterogeneous in nature. Consequently, building highly effective fusion algorithms is at the core to achieve improved model robustness and inferencing performance. This dissertation focuses on the representation learning approaches as the fusion strategy. Specifically, the objective is to learn the shared latent representation which jointly exploit the structural information encoded in all modalities, such that a straightforward learning model can be adopted to obtain the prediction.

We first consider sensor fusion, a typical multimodal fusion problem critical to building a pervasive computing platform. A systematic fusion technique is described to support both multiple sensors and descriptors for activity recognition. Targeted to learn the optimal combination of kernels, Multiple Kernel Learning (MKL) algorithms have been successfully applied to numerous fusion problems in computer vision etc. Utilizing the MKL formulation, next we describe an auto-context algorithm for learning image context via the fusion with low-level descriptors. Furthermore, a principled fusion algorithm using deep learning to optimize kernel machines is developed. By bridging deep architectures with kernel optimization, this approach leverages the benefits of both paradigms and is applied to a wide variety of fusion problems.

In many real-world applications, the modalities exhibit highly specific data structures, such as time sequences and graphs, and consequently, special design of the learning architecture is needed. In order to improve the temporal modeling for multivariate sequences, we developed two architectures centered around attention models. A novel clinical time series analysis model is proposed for several critical problems in healthcare. Another model coupled with triplet ranking loss as metric learning framework is described to better solve speaker diarization. Compared to state-of-the-art recurrent networks, these attention-based multivariate analysis tools achieve improved performance while having a lower computational complexity. Finally, in order to perform community detection on multilayer graphs, a fusion algorithm is described to derive node embedding from word embedding techniques and also exploit the complementary relational information contained in each layer of the graph.

Software-defined radio provides users with a low-cost and flexible platform for implementing and studying advanced communications and remote sensing applications. Two such applications include unmanned aerial system-to-ground communications channel and joint sensing and communication systems. In this work, these applications are studied.

In the first part, unmanned aerial system-to-ground communications channel models are derived from empirical data collected from software-defined radio transceivers in residential and mountainous desert environments using a small (< 20 kg) unmanned aerial system during low-altitude flight (< 130 m). The Kullback-Leibler divergence measure was employed to characterize model mismatch from the empirical data. Using this measure the derived models accurately describe the underlying data.

In the second part, an experimental joint sensing and communications system is implemented using a network of software-defined radio transceivers. A novel co-design receiver architecture is presented and demonstrated within a three-node joint multiple access system topology consisting of an independent radar and communications transmitter along with a joint radar and communications receiver. The receiver tracks an emulated target moving along a predefined path and simultaneously decodes a communications message. Experimental system performance bounds are characterized jointly using the communications channel capacity and novel estimation information rate.

Cognitive radio (CR) and device-to-device (D2D) systems are two promising dynamic spectrum access schemes in wireless communication systems to provide improved quality-of-service, and efficient spectrum utilization. This dissertation shows that both CR and D2D systems benefit from properly designed cooperation scheme.

In underlay CR systems, where secondary users (SUs) transmit simultaneously with primary users (PUs), reliable communication is by all means guaranteed for PUs, which likely deteriorates SUs’ performance. To overcome this issue, cooperation exclusively among SUs is achieved through multi-user diversity (MUD), where each SU is subject to an instantaneous interference constraint at the primary receiver. Therefore, the active number of SUs satisfying this constraint is random. Under different user distributions with the same mean number of SUs, the stochastic ordering of SU performance metrics including bit error rate (BER), outage probability, and ergodic capacity are made possible even without observing closed form expressions. Furthermore, a cooperation is assumed between primary and secondary networks, where those SUs exceeding the interference constraint facilitate PU’s transmission by relaying its signal. A fundamental performance trade-off between primary and secondary networks is observed, and it is illustrated that the proposed scheme outperforms non-cooperative underlay CR systems in the sense of system overall BER and sum achievable rate.

Similar to conventional cellular networks, CR systems suffer from an overloaded receiver having to manage signals from a large number of users. To address this issue, D2D communications has been proposed, where direct transmission links are established between users in close proximity to offload the system traffic. Several new cooperative spectrum access policies are proposed allowing coexistence of multiple D2D pairs in order to improve the spectral efficiency. Despite the additional interference, it is shown that both the cellular user’s (CU) and the individual D2D user's achievable rates can be improved simultaneously when the number of D2D pairs is below a certain threshold, resulting in a significant multiplexing gain in the sense of D2D sum rate. This threshold is quantified for different policies using second order approximations for the average achievable rates for both the CU and the individual D2D user.

Fully distributed wireless sensor networks (WSNs) without fusion center have advantages such as scalability in network size and energy efficiency in communications. Each sensor shares its data only with neighbors and then achieves global consensus quantities by in-network processing. This dissertation considers robust distributed parameter estimation methods, seeking global consensus on parameters of adaptive learning algorithms and statistical quantities.

Diffusion adaptation strategy with nonlinear transmission is proposed. The nonlinearity was motivated by the necessity for bounded transmit power, as sensors need to iteratively communicate each other energy-efficiently. Despite the nonlinearity, it is shown that the algorithm performs close to the linear case with the added advantage of power savings. This dissertation also discusses convergence properties of the algorithm in the mean and the mean-square sense.

Often, average is used to measure central tendency of sensed data over a network. When there are outliers in the data, however, average can be highly biased. Alternative choices of robust metrics against outliers are median, mode, and trimmed mean. Quantiles generalize the median, and they also can be used for trimmed mean. Consensus-based distributed quantile estimation algorithm is proposed and applied for finding trimmed-mean, median, maximum or minimum values, and identification of outliers through simulation. It is shown that the estimated quantities are asymptotically unbiased and converges toward the sample quantile in the mean-square sense. Step-size sequences with proper decay rates are also discussed for convergence analysis.

Another measure of central tendency is a mode which represents the most probable value and also be robust to outliers and other contaminations in data. The proposed distributed mode estimation algorithm achieves a global mode by recursively shifting conditional mean of the measurement data until it converges to stationary points of estimated density function. It is also possible to estimate the mode by utilizing grid vector as well as kernel density estimator. The densities are estimated at each grid point, while the points are updated until they converge to a global mode.

Small wireless cells have the potential to overcome bottlenecks in wireless access through the sharing of spectrum resources. A novel access backhaul network architecture based on a Smart Gateway (Sm-GW) between the small cell base stations, e.g., LTE eNBs, and the conventional backhaul gateways, e.g., LTE Servicing/Packet Gateways (S/P-GWs) has been introduced to address the bottleneck. The Sm-GW flexibly schedules uplink transmissions for the eNBs. Based on software defined networking (SDN) a management mechanism that allows multiple operator to flexibly inter-operate via multiple Sm-GWs with a multitude of small cells has been proposed. This dissertation also comprehensively survey the studies that examine the SDN paradigm in optical networks. Along with the PHY functional split improvements, the performance of Distributed Converged Cable Access Platform (DCCAP) in the cable architectures especially for the Remote-PHY and Remote-MACPHY nodes has been evaluated. In the PHY functional split, in addition to the re-use of infrastructure with a common FFT module for multiple technologies, a novel cross functional split interaction to cache the repetitive QAM symbols across time at the remote node to reduce the transmission rate requirement of the fronthaul link has been proposed.

Dynamic spectrum access (DSA) has great potential to address worldwide spectrum shortage by enhancing spectrum efficiency. It allows unlicensed secondary users to access the under-utilized spectrum when the primary users are not transmitting. On the other hand, the open wireless medium subjects DSA systems to various security and privacy issues, which might hinder the practical deployment. This dissertation consists of two parts to discuss the potential challenges and solutions.

The first part consists of three chapters, with a focus on secondary-user authentication. Chapter One gives an overview of the challenges and existing solutions in spectrum-misuse detection. Chapter Two presents SpecGuard, the first crowdsourced spectrum-misuse detection framework for DSA systems. In SpecGuard, three novel schemes are proposed for embedding and detecting a spectrum permit at the physical layer. Chapter Three proposes SafeDSA, a novel PHY-based scheme utilizing temporal features for authenticating secondary users. In SafeDSA, the secondary user embeds his spectrum authorization into the cyclic prefix of each physical-layer symbol, which can be detected and authenticated by a verifier.

The second part also consists of three chapters, with a focus on crowdsourced spectrum sensing (CSS) with privacy consideration. CSS allows a spectrum sensing provider (SSP) to outsource the spectrum sensing to distributed mobile users. Without strong incentives and location-privacy protection in place, however, mobile users are reluctant to act as crowdsourcing workers for spectrum-sensing tasks. Chapter Four gives an overview of the challenges and existing solutions. Chapter Five presents PriCSS, where the SSP selects participants based on the exponential mechanism such that the participants' sensing cost, associated with their locations, are privacy-preserved. Chapter Six further proposes DPSense, a framework that allows the honest-but-curious SSP to select mobile users for executing spatiotemporal spectrum-sensing tasks without violating the location privacy of mobile users. By collecting perturbed location traces with differential privacy guarantee from participants, the SSP assigns spectrum-sensing tasks to participants with the consideration of both spatial and temporal factors.

Through theoretical analysis and simulations, the efficacy and effectiveness of the proposed schemes are validated.

Distributed wireless sensor networks (WSNs) have attracted researchers recently due to their advantages such as low power consumption, scalability and robustness to link failures. In sensor networks with no fusion center, consensus is a process where

all the sensors in the network achieve global agreement using only local transmissions. In this dissertation, several consensus and consensus-based algorithms in WSNs are studied.

Firstly, a distributed consensus algorithm for estimating the maximum and minimum value of the initial measurements in a sensor network in the presence of communication noise is proposed. In the proposed algorithm, a soft-max approximation together with a non-linear average consensus algorithm is used. A design parameter controls the trade-off between the soft-max error and convergence speed. An analysis of this trade-off gives guidelines towards how to choose the design parameter for the max estimate. It is also shown that if some prior knowledge of the initial measurements is available, the consensus process can be accelerated.

Secondly, a distributed system size estimation algorithm is proposed. The proposed algorithm is based on distributed average consensus and L2 norm estimation. Different sources of error are explicitly discussed, and the distribution of the final estimate is derived. The CRBs for system size estimator with average and max consensus strategies are also considered, and different consensus based system size estimation approaches are compared.

Then, a consensus-based network center and radius estimation algorithm is described. The center localization problem is formulated as a convex optimization problem with a summation form by using soft-max approximation with exponential functions. Distributed optimization methods such as stochastic gradient descent and diffusion adaptation are used to estimate the center. Then, max consensus is used to compute the radius of the network area.

Finally, two average consensus based distributed estimation algorithms are introduced: distributed degree distribution estimation algorithm and algorithm for tracking the dynamics of the desired parameter. Simulation results for all proposed algorithms are provided.

RF convergence of radar and communications users is rapidly becoming an issue for a multitude of stakeholders. To hedge against growing spectral congestion, research into cooperative radar and communications systems has been identified as a critical necessity for the United States and other countries. Further, the joint sensing-communicating paradigm appears imminent in several technological domains. In the pursuit of co-designing radar and communications systems that work cooperatively and benefit from each other's existence, joint radar-communications metrics are defined and bounded as a measure of performance. Estimation rate is introduced, a novel measure of radar estimation information as a function of time. Complementary to communications data rate, the two systems can now be compared on the same scale. An information-centric approach has a number of advantages, defining precisely what is gained through radar illumination and serves as a measure of spectral efficiency. Bounding radar estimation rate and communications data rate jointly, systems can be designed as a joint optimization problem.

Large-scale integration of wind generation introduces planning and operational difficulties due to the intermittent and highly variable nature of wind. In particular, the generation from non-hydro renewable resources is inherently variable and often times difficult to predict. Integrating significant amounts of renewable generation, thus, presents a challenge to the power systems operators, requiring additional flexibility, which may incur a decrease of conventional generation capacity.

This research investigates the algorithms employing emerging computational advances in system operation policies that can improve the flexibility of the electricity industry. The focus of this study is on flexible operation policies for renewable generation, particularly wind generation. Specifically, distributional forecasts of windfarm generation are used to dispatch a “discounted” amount of the wind generation, leaving a reserve margin that can be used for reserve if needed. This study presents systematic mathematic formulations that allow the operator incorporate this flexibility into the operation optimization model to increase the benefits in the energy and reserve scheduling procedure. Incorporating this formulation into the dispatch optimization problem provides the operator with the ability of using forecasted probability distributions as well as the off-line generated policies to choose proper approaches for operating the system in real-time. Methods to generate such policies are discussed and a forecast-based approach for developing wind margin policies is presented. The impacts of incorporating such policies in the electricity market models are also investigated.

In this dissertation, I propose potential techniques to improve the quality-of-service (QoS) of real-time applications in cognitive radio (CR) systems. Unlike best-effort applications, real-time applications, such as audio and video, have a QoS that need to be met. There are two different frameworks that are used to study the QoS in the literature, namely, the average-delay and the hard-deadline frameworks. In the former, the scheduling algorithm has to guarantee that the packet's average delay is below a prespecified threshold while the latter imposes a hard deadline on each packet in the system. In this dissertation, I present joint power allocation and scheduling algorithms for each framework and show their applications in CR systems which are known to have strict power limitations so as to protect the licensed users from interference.

A common aspect of the two frameworks is the packet service time. Thus, the effect of multiple channels on the service time is studied first. The problem is formulated as an optimal stopping rule problem where it is required to decide at which channel the SU should stop sensing and begin transmission. I provide a closed-form expression for this optimal stopping rule and the optimal transmission power of secondary user (SU).

The average-delay framework is then presented in a single CR channel system with a base station (BS) that schedules the SUs to minimize the average delay while protecting the primary users (PUs) from harmful interference. One of the contributions of the proposed algorithm is its suitability for heterogeneous-channels systems where users with statistically low channel quality suffer worse delay performances. The proposed algorithm guarantees the prespecified delay performance to each SU without violating the PU's interference constraint.

Finally, in the hard-deadline framework, I propose three algorithms that maximize the system's throughput while guaranteeing the required percentage of packets to be transmitted by their deadlines. The proposed algorithms work in heterogeneous systems where the BS is serving different types of users having real-time (RT) data and non-real-time (NRT) data. I show that two of the proposed algorithms have the low complexity where the power policies of both the RT and NRT users are in closed-form expressions and a low-complexity scheduler.