Head movement is a natural orienting behavior for sensing environmental events around us. Head movement is particularly important for identifying through the sense of hearing the location of an out-of-sight, rear-approaching target to avoid danger or threat. This research aims to design a portable device for detecting the head movement patterns of common marmoset monkeys in laboratory environments. Marmoset is a new-world primate species and has become increasingly popular for neuroscience research. Understanding the unique patterns of their head movements will improve its values as a new primate model for uncovering the neurobiology of natural orienting behavior. Due to their relatively small head size (5 cm in diameter) and body weight (300-500 g), the device has to meet several unique design requirements with respect to accuracy and workability. A head-mount wireless tracking system was implemented based on inertial sensors that are capable of detecting motion in the Yaw, Pitch and Roll axes. The sensors were connected to the encoding station, which transmits wirelessly the 3-axis movement data to the decoding station at the sampling rate of ~175 Hz. The decoding station relays this information to the computer for real-time display and analysis. Different tracking systems, based on the accelerometer and Inertial Measurement Unit is implemented to track the head movement pattern of the marmoset head. Using these systems, translational and rotational information of head movement are collected, and the data analysis focuses on the rotational head movement in body-constrained marmosets. Three stimulus conditions were tested: 1) Alert, 2) Idle 3) Sound only. The head movement patterns were examined when the house light was turned on and off for each stimulus. Angular velocity, angular displacement and angular acceleration were analyzed in all three axes.

Fast and large head turns were observed in the Yaw axis in response to the alert stimuli and not much in the idle and sound-only stimulus conditions. Contrasting changes in speed and range of head movement were found between light-on and light-off situations. The mean peak angular displacement was 95 degrees (light on) and 55 (light off) and the mean peak angular velocity was 650 degrees/ second (light on) and 400 degrees/second (light off), respectively, in response to the alert stimuli. These results suggest that the marmoset monkeys may engage in different modes of orienting behaviors with respect to the availability of visual cues and thus the necessity of head movement. This study provides a useful tool for future studies in understanding the interplay among visual, auditory and vestibular systems during nature behavior.

Sound localization can be difficult in a reverberant environment. Fortunately listeners can utilize various perceptual compensatory mechanisms to increase the reliability of sound localization when provided with ambiguous physical evidence. For example, the directional information of echoes can be perceptually suppressed by the direct sound to achieve a single, fused auditory event in a process called the precedence effect (Litovsky et al., 1999). Visual cues also influence sound localization through a phenomenon known as the ventriloquist effect. It is classically demonstrated by a puppeteer who speaks without visible lip movements while moving the mouth of a puppet synchronously with his/her speech (Gelder and Bertelson, 2003). If the ventriloquist is successful, sound will be “captured” by vision and be perceived to be originating at the location of the puppet. This thesis investigates the influence of vision on the spatial localization of audio-visual stimuli. Participants seated in a sound-attenuated room indicated their perceived locations of either ISI or level-difference stimuli in free field conditions. Two types of stereophonic phantom sound sources, created by modulating the inter-stimulus time interval (ISI) or level difference between two loudspeakers, were used as auditory stimuli. The results showed that the light cues influenced auditory spatial perception to a greater extent for the ISI stimuli than the level difference stimuli. A binaural signal analysis further revealed that the greater visual bias for the ISI phantom sound sources was correlated with the increasingly ambiguous binaural cues of the ISI signals. This finding suggests that when sound localization cues are unreliable, perceptual decisions become increasingly biased towards vision for finding a sound source. These results support the cue saliency theory underlying cross-modal bias and extend this theory to include stereophonic phantom sound sources.

This study consisted of several related projects on dynamic spatial hearing by both human and robot listeners. The first experiment investigated the maximum number of sound sources that human listeners could localize at the same time. Speech stimuli were presented simultaneously from different loudspeakers at multiple time intervals. The maximum of perceived sound sources was close to four. The second experiment asked whether the amplitude modulation of multiple static sound sources could lead to the perception of auditory motion. On the horizontal and vertical planes, four independent noise sound sources with 60° spacing were amplitude modulated with consecutively larger phase delay. At lower modulation rates, motion could be perceived by human listeners in both cases. The third experiment asked whether several sources at static positions could serve as "acoustic landmarks" to improve the localization of other sources. Four continuous speech sound sources were placed on the horizontal plane with 90° spacing and served as the landmarks. The task was to localize a noise that was played for only three seconds when the listener was passively rotated in a chair in the middle of the loudspeaker array. The human listeners were better able to localize the sound sources with landmarks than without. The other experiments were with the aid of an acoustic manikin in an attempt to fuse binaural recording and motion data to localize sounds sources. A dummy head with recording devices was mounted on top of a rotating chair and motion data was collected. The fourth experiment showed that an Extended Kalman Filter could be used to localize sound sources in a recursive manner. The fifth experiment demonstrated the use of a fitting method for separating multiple sounds sources.

Head movement is known to have the benefit of improving the accuracy of sound localization for humans and animals. Marmoset is a small bodied New World monkey species and it has become an emerging model for studying the auditory functions. This thesis aims to detect the horizontal and vertical rotation of head movement in marmoset monkeys.

Experiments were conducted in a sound-attenuated acoustic chamber. Head movement of marmoset monkey was studied under various auditory and visual stimulation conditions. With increasing complexity, these conditions are (1) idle, (2) sound-alone, (3) sound and visual signals, and (4) alert signal by opening and closing of the chamber door. All of these conditions were tested with either house light on or off. Infra-red camera with a frame rate of 90 Hz was used to capture of the head movement of monkeys. To assist the signal detection, two circular markers were attached to the top of monkey head. The data analysis used an image-based marker detection scheme. Images were processed using the Computation Vision Toolbox in Matlab. The markers and their positions were detected using blob detection techniques. Based on the frame-by-frame information of marker positions, the angular position, velocity and acceleration were extracted in horizontal and vertical planes. Adaptive Otsu Thresholding, Kalman filtering and bound setting for marker properties were used to overcome a number of challenges encountered during this analysis, such as finding image segmentation threshold, continuously tracking markers during large head movement, and false alarm detection.

The results show that the blob detection method together with Kalman filtering yielded better performances than other image based techniques like optical flow and SURF features .The median of the maximal head turn in the horizontal plane was in the range of 20 to 70 degrees and the median of the maximal velocity in horizontal plane was in the range of a few hundreds of degrees per second. In comparison, the natural alert signal - door opening and closing - evoked the faster head turns than other stimulus conditions. These results suggest that behaviorally relevant stimulus such as alert signals evoke faster head-turn responses in marmoset monkeys.