Motivation

BEDA supports use of multimodal data for longitudinal studies where common challenges are visualization, synchronization, and analysis of many sessions of data. There are many studies in Psychology and Special Education using mutlimodal and longitudinal data. Our tool was inspired by one of the studies we were conducting with special education researchers. An interdisciplinary team of researchers from Computer Science and Special Education conduct a study that explores efficacy of Sensory Integration (SI) therapy. SI therapy is often used to help individuals who have difficulties in integrating sensory inputs. However, we lack rigorous research showing relationships between SI therapy and changes in either behavior or arousal level. An occupational therapist who we collaborate with specifically recommended a pressure vest for one of the SI therapies. In this way, we explore whether wearing a pressure vest would produce changes in behavior and arousal levels.

Traditionally, a study like this would collect ...

In addition to these, we added extra sensor, which collects EDA, temperature, and acceleration data.

Organizing and analyzing the data using only traditional method is a huge amount of task. Adding extra sensor data makes the analysis more complicated.

Img


Main view for visualization, synchronization, and annotation

View video, data streams, and annotated behaviors (b and c).
One-click synchronization for video and multiple data streams (a).
Provide video (behavior) annotation by allowing users to define name, color, and hotkey for targeted behaviors (e).
Users can annotate behaviors on timeline by pressing a corresponding hot key (c).
Allow interval play mode for behavior annotation, which plays a video for x seconds in fast speed and y seconds in normal speed (d).
Users can annotate behaviors if the specified behavior occurred anywhere in the past y seconds.

Img

Analysis window for processing and analyzing data streams

Process and analyze data streams (e.g., physiological data, audio, or any type of time series data etc.) (a) by selecting a pre-programmed analysis script (b).
Allow for writing own scripts using MATLAB and R and import it to BEDA.

Img

Summary-graph view for visualizing analysis results

Visualize results of several weeks of longitudinal sessions ( b and c).
Clicking the names of sessions in the list box (a) opens them in the multi- sessions view.

Img

Multi-sessions view for comparing multiple sessions’ data

Compare and contrast multiple sessions’ actual data streams.
(e.g., If users drag one of the orange playheads (a-2), all the data related to this session (a-1 and a-2) move together while other data (b-1, b-2, c-1, c-2) stay.)

Img