PUB - Publikationen an der Universität Bielefeld

While technical systems grow in complexity, human-machine interaction (HMI) still requires the user to adapt to the tools. By combining electroencephalography (EEG) and eye-tracking, we aim to create a multi-modal brain machine interface (BMI) to improve system adaptivity and resource effciency in HMI. We propose to analyze Fixation-Related Potentials (FRP) as indicators for cognitive demand and utilize this measure to automatically adapt a system to the user. To this end, we conducted two complementary studies to investigate variations between different tasks, i.e., an object counting task in cluttered scenes and a subjective choice task. Comparing the FRP grand averages (per task category over all trials and participants), we identified significant amplitude and frequency differences in early fronto-central N100 and P200 components. The dataset combining both studies was linearly classified with 66% (SD: 2.0) accuracy using temporal features, but 93% (SD: 1.8) using frequency features. This suggests that early FRP components vary with task complexity during natural scene exploration. Our results also suggest that cognitive demand may be identified on a single trial level using the proposed multi-modal approach. Thus, EEG data may contain discriminative information that eye-tracking data cannot provide alone, e.g., through Ffixation duration.