External Application Data Recorder
External Application Data Recorder
The External Application Data Recorder allows you to record, save and synchronize sensor data while running external VR applications from any source (SteamVR game, Oculus, Web application, etc.). This uses the Nvidia Shadow Play recording feature to also have the option of recording and saving a video that can then be viewed back and matched up with the data recording.
List of some of the data that can be recorded (will be saved to the data folder, unless a different directory is chosen):
Facial Expression values (Meta Quest Pro, HTC Face Tracker)
Eye gaze rotation
Fixations and Saccades
Head orientation (needs to be manually aligned if not at same origin point)
Pupil Diameter (Omnicept, Vive Pro Eye, Pupil Labs)
Eye Openness Value (Omnicept, Vive Pro Eye)
Heart Rate (Omnicept)
Cognitive Load (Omnicept)
Interactions using the chosen even trigger
TIme Stamp
Gaze mat of eye rotation
Video Recording
All Biopac Acqknowledge physiological data if connecting to Biopac
Video recording can be imported to Acqknowledge to see how it matches up with physiological data
Gaze point overlay on video recording (currently may take some manual alignment depending on the field of view of the headset)
Some limitations:
Only will work with PC based applications (android applications running on stand alone not supported)
Gaze point position is not going to be accurate in the Z axis, as it's only hitting a collide box but not objects in the scene a user is actually looking at
Head orientation (Meta Quest Pro) would need to be manually aligned if not using the same origin starting point
Not able to tag regions or objects of interest
For headsets that go through SteamVR (Omnicept, Vive Focus 3, Vive Pro Eye) you will not get head orientation
To better synchronize head position if using a controller for navigation, will have to modify the SightLab vizconnect to match the external application's navigation controls
Note to record videos you need to install the pyautogui python library using the Package Manager, as well as install Tkinter from the Vizard installer
If getting error " A supported game is required" message, make sure you are using your dedicated graphics card and not the onboard one
This can also be modified to use other screen recording options
To Run:
Start Data_Recorder_External_Application script
Start External Application
Make Data Recorder screen active and press spacebar
After timer value that is set, the video recording and data recording will start. Alt-tab and go back to the external application within this time
Press spacebar to end recording
Quit External Application
Video recording will be saved in your UserName- Videos folder and data will be in 'data'
Can also leverage the Plotly visualization example to view a walk path and gaze path (requires the Plotly Python library)
Face Tracking Data can be viewed in utils/data and visualized using the facial_expressions_over_time script
If using Biopac Acqknowledge can open the video to see the synchronization with the physiological data. See here for how to do that
To view a replay run DataRecorderReplay.py that will show the gaze point and visualizations on a virtual screen that is tied to the user's head
List of configurable options using Data_Recorder_Config.py:
#Choose whether to communicate with Biopac AcqknowledgeBIOPAC_ON = False
#Key that can set when certain events happen, saves to data file and sends marker to AcqknowledgeNETWORK_SYNC_KEY = 't'
#Choose if wanting to send a network based eventUSE_NETWORK_EVENT = False
#Option to save all eye tracking gaze mat data to the data fileRECORD_GAZE_MAT = False
#Save facial expression data (Meta Quest Pro or Vive Face Tracker)RECORD_FACE_TRACKER_DATA = False
#Start and stop a video recording using Nvidia Shadow PlayRECORD_VIDEO_SHADOW_PLAY = True
#How long to wait for the video to start recording when the session startsTIMER_TO_START_VIDEO_RECORDING = 3
#Set path to video recording for replayPATH_TO_VIDEO = ''
