Selected Research Projects

Natioanal Research Council project (Dec. 2009-current)
Visual scan pattern analysis and misperception modeling in overland navigation

My main postdoctoral and current work cosponsored by ONR (Office of Naval Research) and NMSO (Naval Modeling Simulation Office) was to integrate psychophysiological measurement of trainees' cognitive states on helicopter overland navigation as cue for instructional intervention. My research team consists of engineers, psychologists, and aviators. Such a team composition allowed us to tackle the problem in an essentially multidisciplinary way. Helicopter overland navigation is a cognitively complex task that requires continuous monitoring of system and environmental parameters and years of training. This study investigates potential improvements to training simulation by analyzing the influences of flight expertise on visual scan patterns.  Nineteen male military personnel flew in a simulated terrain model of Twenty-Nine Palms, CA while their gaze data were tracking via two sets of faceLAB eye-tracking systems. Flight performance measures were not predicted by the expertise level of pilots. However, gaze parameters and scan management skills were predicted by the expertise level (For more detailed information, click here.) Flight and eye scan visualization tool (FEST, Fig. 1 or click here for an example video clip) was developed along with visual scan pattern characterization with respect to pilots’ expertise level.  Based on empirical data, we provided a framework to model visual misperception in overland navigation. Some pilots showed common visual misperception during the navigation task, which can be explained by the following errors 1) confusion between inference and evidence, 2) incorrect mutually exclusive assumptions on the data, and 3) biased sampling (inattentional blindness). More in-depth analysis and simulation results can be found here. We are currently expanding the project to develop a real-time instructional display and to estimate trainees' cognitive state by including target detection tasks, a wide field of view simulator, and supplementary physiological measures.


Fig. 1 Flight and Eye Scan visulization Tool (FEST)
Fig.2 Prior and posterior probabilities of A: Bayesian Agent, B1&B2: Misperception Type 1&2

Ph.D. project (Sep. 2004-Aug. 2007)
Drowsy driver detection and countermeasures

My Ph.D. research was funded by the Ford-MIT alliance and explored drowsy driver detection by utilizing driver-vehicle interaction data. Previously, a variety of driver drowsiness detection techniques had been developed by using physiological signals, physical changes, or secondary tasks. However, they had relied mainly on observing the symptoms of drowsiness. There has been little research on how and why the performance of drowsy drivers degrades and how to utilize those data in a systematic way. Thus, rather than developing another drowsiness monitoring technique, I focused on understanding in-depth characteristics of driver-vehicle systems with drivers at different sleep-deprivation levels. The camouflage nature of drowsy driving was revealed through human-in-the-loop experiments, which also led to the design of a detection system based on a Bayesian Network. I believe the novelty of this work comes from the initial idea as shown in Fig. 3 which originated from discussions with neuroscientists and the integrative research method was achieved by collaborating with automotive engineers, computer scientists, and sleep experts. A journal publication based on my Ph.D. research received IEEE SMC Andrew P. Sage Best Transactions Paper Award in 2009.


Fig. 3 A schematic diagram of driver-vehicle system: expected effectrs of drowsienss on the neural controllers are shown in different colors. If not all neural controllers are affected by drowsiness equally, which driving tasks will be influenced more and/or earlier?

Fig. 4 When encountering abrupt changes caused by environment, vehicle (SC: steering characteristics change), or challenging road geometry, drivers showed significantly different adatability to changes under different levels of sleep deprivation. However, if there was no excitation or disturbances introduces (e.g., LT: straight lane tracking), the drivers' performance showed little difference between different sleep deprivation levels.





Other selected research projects

Modeling peripheral vision on target detection and acquisition (Sep. 2010-current)

As part of an effort to develop methodologies to better represent individual combatants and improved search and target acquisition (STA), an improved model of unaided human vision is proposed based on empirical data. Twenty-three subjects participated in target detection and vehicle monitoring task scenarios while their gaze data and detection performances were collected with a 180-degree wide field-of-view simulator. Subjects were instructed to detect objects in their periphery, either UAVs in a rural scenario or pedestrians in an urban scenario. Peripheral object detection performance was predicted by peripheral angle and entrance speed. A mixed-effect model was applied to the data and an interactive 3-D visualization tool was developed.


Fig 5. Integrated gaze diagram in the urban envrionment

Assessing human-systems performance in a field environment (Oct. 2008- Sep. 2009)

To assess the performance of reconnaissance and surveillance objectively, we proposed an integrated method using the AWARE tool and demonstrated its potential application through a field experiment. There is no doubt that advances in technology have improved military performance. However, these advances require humans to adapt to new and complex systems, all too often resulting in mishaps. We provided an objective methodology to assess human system performance in a military context. We conducted two surveillance and reconnaissance field experiments at Camp Roberts, California, with 16 volunteers assigned to Blue and Red teams, respectively. The Red teams followed scripted scenarios that included terrorist-like activities, while their positions were tracked by Global Positioning Systems (GPSs). At the same time, the Blue team’s mission was to track the Red team’s vehicles and their activities, while their situated cognition and workload were recorded at 15-minute intervals using the portable AWARE tool. The field experiment demonstrated the potential use of the AWARE tool as a measure of objective performance. The data obtained from the AWARE tool, along with the GPS, provided quantitative measures of performance of surveillance and reconnaissance,  e.g., distances between the Blue team’s perceived and actual location of Red entities. The Blue’s perception varied over the course of the simulation, showing no improvement in situated cognition. We also applied the Dynamic Model of Situated Cognition to observe the information flow in human and technical systems.



Fig. 6 Snapshot of AWARE interface

Fig. 7 AWARE and GPS data exported to Google Earth


Complex alerting systems development and system-wide impact
(Sep. 2001-Aug. 2004)

Many threat assessment algorithms are based on a collection of threshold equations that predict when a collision is to occur. The fact that there are numerous algorithms suggests need to understand the underlying principles behind the equation design and thershold settings. We presented a methodology to develop appropriate alerting thresholds based on performance metrics. This also allows us to compare different alerting algorithms. The method is a performance-based approach in state-space, and can thus be utilized in conjunction with any chosen alerting algorithm or sensor system. Using carefully prescribed trajectory models (which may include uncertainties), the performance tradeoff with and without an alert can be predicted for different states along the course of an encouter situation. This information can then be used to set appropriate threshold values for the desired alerting logic. The development of the threshold criteria for a rear-end collision warning system is give as an example and the algorithm was implemented in Linconln LS concept vehicle. On-board GPS system was used to measure vehicle positio and velocity. Though the approach given is presented as athreshold design tool, the methodology is self-contained as a threat assessment logic. The possibility exists to compute the performance measures on-the-fly from wich alerting decisions can be made directly.


Fig. 8 Algorithm inplemented in Lincoln LS concept vehicle


Fig. 9 Hybrid control model of alering system