Virtual reality (VR) provides significant opportunities for students to experience immersive education. In VR, students can travel to the international space station, or go through a science experiment at home. However, the current tactile feedback provided by these systems do not feel real. Controllers do not provide the same tactile feedback experienced in the physical world. This dissertation aims to bridge the gap between the virtual and physical learning environments through the development of novel haptic devices capable of emulating tactile sensations found in physical science labs. My research explores haptic devices that can emulate the sensations of fluids in vessels within the virtual environment. Fluid handling is a cornerstone experience of science labs. I also explore how to emulate the handling of other science equipment. I describe and research on four novel devices. These are 1) SWISH: A shifting-weight interface of simulated hydrodynamics for haptic perception of virtual fluid vessels, 2) Geppetteau, 3) Vibr-eau, and 4) Pneutouch. SWISH simulates the sensation of virtual fluids in vessels using a rack and pinion mechanism, while Geppetteau employs a string-driven mechanism to provide haptic feedback for a variety of vessel shapes. Vibr-eau utilizes vibrotactile actuators in the vessel’s interior to emulate the behavior of virtual liquids. Finally, Pneutouch enables users to interact with virtual objects through pneumatic inflatables. Through systematic evaluations and comparisons with baseline comparisons, the usability and effectiveness of these haptic devices in enhancing virtual experiences is demonstrated. The development of these haptic mechanisms and interfaces represents a significant step towards creating transformative educational tools that provide customizable, hands-on learning environments in both Mixed (MR) and Virtual Reality (VR) - now called XR. This dissertation contributes to advancing the field of haptics for virtual education and lays the foundation for future research in immersive learning technologies.

This dissertation contributes to uncertainty-aware neural networks using multi-modality data, with a focus on industrial and aviation applications. Drawing from seminal works in recent years that have significantly advanced the field, this dissertation develops techniques for incorporating uncertainty estimation and leveraging multi-modality information into neural networks for tasks such as fault detection and environmental perception. The escalating complexity of data in engineering contexts demands models that predict accurately and quantify uncertainty in these predictions. The methods proposed in this document utilize various techniques, including Bayesian Deep Learning, multi-task regularization and feature fusion, and efficient use of unlabeled data. Popular methods of uncertainty quantification are analyzed empirically to derive important insights on their use in real world engineering problems. The primary objective is to develop and refine Bayesian neural network models for enhanced predictive accuracy and decision support in engineering. This involves exploring novel architectures, regularization methods, and data fusion techniques. Significant attention is given to data handling challenges in deep learning, particularly in the context of quality inspection systems. The research integrates deep learning with vision systems for engineering risk assessment and decision support tasks, and introduces two novel benchmark datasets designed for semantic segmentation and classification tasks. Additionally, the dissertation delves into RGB-Depth data fusion for pipeline defect detection and the use of semi-supervised learning algorithms for manufacturing inspection tasks with imaging data. The dissertation contributes to bridging the gap between advanced statistical methods and practical engineering applications.

In the rapidly evolving field of computer vision, propelled by advancements in deeplearning, the integration of hardware-software co-design has become crucial to overcome the limitations of traditional imaging systems. This dissertation explores the integration of hardware-software co-design in computational imaging, particularly in light transport acquisition and Non-Line-of-Sight (NLOS) imaging. By leveraging projector-camera systems and computational techniques, this thesis address critical challenges in imaging complex environments, such as adverse weather conditions, low-light scenarios, and the imaging of reflective or transparent objects. The first contribution in this thesis is the theory, design, and implementation of a slope disparity gating system, which is a vertically aligned configuration of a synchronized raster scanning projector and rolling-shutter camera, facilitating selective imaging through disparity-based triangulation. This system introduces a novel, hardware-oriented approach to selective imaging, circumventing the limitations of post-capture processing. The second contribution of this thesis is the realization of two innovative approaches for spotlight optimization to improve localization and tracking for NLOS imaging. The first approach utilizes radiosity-based optimization to improve 3D localization and object identification for small-scale laboratory settings. The second approach introduces a learningbased illumination network along with a differentiable renderer and NLOS estimation network to optimize human 2D localization and activity recognition. This approach is validated on a large, room-scale scene with complex line-of-sight geometries and occluders. The third contribution of this thesis is an attention-based neural network for passive NLOS settings where there is no controllable illumination. The thesis demonstrates realtime, dynamic NLOS human tracking where the camera is moving on a mobile robotic platform. In addition, this thesis contains an appendix featuring temporally consistent relighting for portrait videos with applications in computer graphics and vision.

This thesis explores the development and integration of a wrist-worn pneumatic haptic interface, Pneutouch, into multiplayer virtual reality (VR) environments. The study investigates the impact of haptics on multiplayer experiences, with a specific focus on presence, collaboration, and communication. Evaluation and investigation were performed using three mini-games, each targeting specific interactions and investigating presence, collaboration, and communication. It was found that haptics enhanced user presence and object realism, increased user seriousness towards tasks, and shifted the focus of interactions from user-user to user-object. In collaborative tasks, haptics increased realism but did not improve efficiency for simple tasks. In communication tasks, a unique interaction modality, termed "haptic mirroring," was introduced, which explored a new form of communication that could be implemented with haptic devices. It was found that with new communication modalities, users experience an associated learning curve. Together, these findings suggest a new set of multiplayer haptic design considerations, such as how haptics increase seriousness, shift focus from social to physical interactions, generally increase realism but decrease task efficiency, and have associated learning curves. These findings contribute to the growing body of research on haptics in VR, particularly in multiplayer settings, and provide insights that can be further investigated or utilized in the implementation of VR experiences.

This dissertation features three pieces of scholarship which showcase and demonstrate an ethic of queerness for engineering education research (EER). The concept of an ethic of queerness is introduced and constructed in Chapter 1 using tenets from the philosophy of pragmatism, systems thinking, critical theory, and the personal and collective experiences of queered communities immersed in normative spaces, such as engineering and engineering education. Chapter 2 is a scoping literature review on the state of research on the LGBTQIA+ engineering student experience compared to other relevant fields, revealing that EER is still nascent on the topic. Chapter 3 leverages arts-based qualitative inquiry to explore the opportunities and limitations of mixed-initiative creative interfaces (MICIs) when used as a tool for self care by queer(ed) subjects. Chapter 4 connects Patricia Hill Collins’ insider/outsider paradox framework to recent engineering education research through collaborative autoethnographies, illuminating the ways in which normative, oppressive social discourses are embedded within the EER system. Although Chapters 2-4 feature their own unique methodology and topic of inquiry, they are united through a motivation to deconstruct and re-imagine sociotechnical systems throughout engineering and EER through the lens of radical queerness. Chapter 5 summarizes how each of the prior chapters aligns with queerness as an ethic and explores avenues of future work from this dissertation. More specifically, each chapter represents a way of queering engineering education research methodology through the embrace of ambiguity and ephemerality, particularly with regard to the ways in which the author’s subjectivity and relationality to the roles of researcher, student, engineer, and engineering education researcher emerged throughout their doctoral education.

Speech analysis for clinical applications has emerged as a burgeoning field, providing valuable insights into an individual's physical and physiological state. Researchers have explored speech features for clinical applications, such as diagnosing, predicting, and monitoring various pathologies. Before presenting the new deep learning frameworks, this thesis introduces a study on conventional acoustic feature changes in subjects with post-traumatic headache (PTH) attributed to mild traumatic brain injury (mTBI). This work demonstrates the effectiveness of using speech signals to assess the pathological status of individuals. At the same time, it highlights some of the limitations of conventional acoustic and linguistic features, such as low repeatability and generalizability. Two critical characteristics of speech features are (1) good robustness, as speech features need to generalize across different corpora, and (2) high repeatability, as speech features need to be invariant to all confounding factors except the pathological state of targets. This thesis presents two research thrusts in the context of speech signals in clinical applications that focus on improving the robustness and repeatability of speech features, respectively. The first thrust introduces a deep learning framework to generate acoustic feature embeddings sensitive to vocal quality and robust across different corpora. A contrastive loss combined with a classification loss is used to train the model jointly, and data-warping techniques are employed to improve the robustness of embeddings. Empirical results demonstrate that the proposed method achieves high in-corpus and cross-corpus classification accuracy and generates good embeddings sensitive to voice quality and robust across different corpora. The second thrust introduces using the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. A novel regularizer, the ICC regularizer, is proposed to regularize deep neural networks to produce embeddings with higher repeatability. This ICC regularizer is implemented and applied to three speech applications: a clinical application, speaker verification, and voice style conversion. The experimental results reveal that the ICC regularizer improves the repeatability of learned embeddings compared to the contrastive loss, leading to enhanced performance in downstream tasks.

This thesis presents robust and novel solutions using knowledge distillation with geometric approaches and multimodal data that can address the current challenges in deep learning, providing a comprehensive understanding of the learning process involved in knowledge distillation. Deep learning has attained significant success in various applications, such as health and wellness promotion, smart homes, and intelligent surveillance. In general, stacking more layers or increasing the number of trainable parameters causes deep networks to exhibit improved performance. However, this causes the model to become large, resulting in an additional need for computing and power resources for training, storage, and deployment. These are the core challenges in incorporating such models into small devices with limited power and computational resources. In this thesis, robust solutions aimed at addressing the aforementioned challenges are presented. These proposed methodologies and algorithmic contributions enhance the performance and efficiency of deep learning models. The thesis encompasses a comprehensive exploration of knowledge distillation, an approach that holds promise for creating compact models from high-capacity ones, while preserving their performance. This exploration covers diverse datasets, including both time series and image data, shedding light on the pivotal role of augmentation methods in knowledge distillation. The effects of these methods are rigorously examined through empirical experiments. Furthermore, the study within this thesis delves into the efficient utilization of features derived from two different teacher models, each trained on dissimilar data representations, including time-series and image data. Through these investigations, I present novel approaches to knowledge distillation, leveraging geometric techniques for the analysis of multimodal data. These solutions not only address real-world challenges but also offer valuable insights and recommendations for modeling in new applications.

Huge advancements have been made over the years in terms of modern image-sensing hardware and visual computing algorithms (e.g. computer vision, image processing, computational photography). However, to this day, there still exists a current gap between the hardware and software design in an imaging system, which silos one research domain from another. Bridging this gap is the key to unlocking new visual computing capabilities for end applications in commercial photography, industrial inspection, and robotics. This thesis explores avenues where hardware-software co-design of image sensors can be leveraged to replace conventional hardware components in an imaging system with software for enhanced reconfigurability. As a result, the user can program the image sensor in a way best suited to the end application. This is referred to as software-defined imaging (SDI), where image sensor behavior can be altered by the system software depending on the user's needs. The scope of this thesis covers the development and deployment of SDI algorithms for low-power computer vision. Strategies for sparse spatial sampling have been developed in this thesis for power optimization of the vision sensor. This dissertation shows how a hardware-compatible state-of-the-art object tracker can be coupled with a Kalman filter for energy gains at the sensor level. Extensive experiments reveal how adaptive spatial sampling of image frames with this hardware-friendly framework offers attractive energy-accuracy tradeoffs. Another thrust of this thesis is to demonstrate the benefits of reinforcement learning in this research avenue. A major finding reported in this dissertation shows how neural-network-based reinforcement learning can be exploited for the adaptive subsampling framework to achieve improved sampling performance, thereby optimizing the energy efficiency of the image sensor. The last thrust of this thesis is to leverage emerging event-based SDI technology for building a low-power navigation system. A homography estimation pipeline has been proposed in this thesis which couples the right data representation with a differential scale-invariant feature transform (SIFT) module to extract rich visual cues from event streams. Positional encoding is leveraged with a multilayer perceptron (MLP) network to get robust homography estimation from event data.

Generative models are deep neural network-based models trained to learn the underlying distribution of a dataset. Once trained, these models can be used to sample novel data points from this distribution. Their impressive capabilities have been manifested in various generative tasks, encompassing areas like image-to-image translation, style transfer, image editing, and more. One notable application of generative models is data augmentation, aimed at expanding and diversifying the training dataset to augment the performance of deep learning models for a downstream task. Generative models can be used to create new samples similar to the original data but with different variations and properties that are difficult to capture with traditional data augmentation techniques. However, the quality, diversity, and controllability of the shape and structure of the generated samples from these models are often directly proportional to the size and diversity of the training dataset. A more extensive and diverse training dataset allows the generative model to capture overall structures present in the data and generate more diverse and realistic-looking samples. In this dissertation, I present innovative methods designed to enhance the robustness and controllability of generative models, drawing upon physics-based, probabilistic, and geometric techniques. These methods help improve the generalization and controllability of the generative model without necessarily relying on large training datasets. I enhance the robustness of generative models by integrating classical geometric moments for shape awareness and minimizing trainable parameters. Additionally, I employ non-parametric priors for the generative model's latent space through basic probability and optimization methods to improve the fidelity of interpolated images. I adopt a hybrid approach to address domain-specific challenges with limited data and controllability, combining physics-based rendering with generative models for more realistic results. These approaches are particularly relevant in industrial settings, where the training datasets are small and class imbalance is common. Through extensive experiments on various datasets, I demonstrate the effectiveness of the proposed methods over conventional approaches.

This thesis encompasses a comprehensive research effort dedicated to overcoming the critical bottlenecks that hinder the current generation of neural networks, thereby significantly advancing their reliability and performance. Deep neural networks, with their millions of parameters, suffer from over-parameterization and lack of constraints, leading to limited generalization capabilities. In other words, the complex architecture and millions of parameters present challenges in finding the right balance between capturing useful patterns and avoiding noise in the data. To address these issues, this thesis explores novel solutions based on knowledge distillation, enabling the learning of robust representations. Leveraging the capabilities of large-scale networks, effective learning strategies are developed. Moreover, the limitations of dependency on external networks in the distillation process, which often require large-scale models, are effectively overcome by proposing a self-distillation strategy. The proposed approach empowers the model to generate high-level knowledge within a single network, pushing the boundaries of knowledge distillation. The effectiveness of the proposed method is not only demonstrated across diverse applications, including image classification, object detection, and semantic segmentation but also explored in practical considerations such as handling data scarcity and assessing the transferability of the model to other learning tasks. Another major obstacle hindering the development of reliable and robust models lies in their black-box nature, impeding clear insights into the contributions toward the final predictions and yielding uninterpretable feature representations. To address this challenge, this thesis introduces techniques that incorporate simple yet powerful deep constraints rooted in Riemannian geometry. These constraints confer geometric qualities upon the latent representation, thereby fostering a more interpretable and insightful representation. In addition to its primary focus on general tasks like image classification and activity recognition, this strategy offers significant benefits in real-world applications where data scarcity is prevalent. Moreover, its robustness in feature removal showcases its potential for edge applications. By successfully tackling these challenges, this research contributes to advancing the field of machine learning and provides a foundation for building more reliable and robust systems across various application domains.