SightLab Class Documentation

• For help with coding in SightLab, can also use the custom SightLab GPT, details here 

• To see how to import SightLab into an exising project see this page

• For more information on the additional modules and code available with Vizard see the Vizard Documentation. Or  the Vizard Command Index

• This page does not include all code available with SightLab. Also see the rest of the documentation or the examples page

1. Importing and Initial Setup

You no longer need to import vizconnect, viz, vizfx, or viztask separately as these are automatically included in the settings file.

To initialize SightLab, use the following:

import sightlab_utils.sightlab as sl

from sightlab_utils.settings import *

# Create a SightLab instance

sightlab = sl.SightLab()

# Disable GUI

sightlab = sl.SightLab(gui=False)

Note: For multi-user use sightlab = sightlab = sl.SightLabServer() or sightlab = sl.SightLabClient()

2. Class Overview

The SightLab class initializes a visual experiment environment with attributes for controlling the experiment settings. 

Attributes:

• gui (bool): Enables the graphical user interface.

• pid (bool): Enable or disable the participant ID.

• biopac (bool): Enables integration with Biopac systems.

• timer (bool): Enables the timer.

• fadequad (bool): Enables the fade quad.

• headlight (bool): Enables the headlight.

• mouselock (bool): Locks the mouse cursor.

• gazepathvisible (bool): Makes the gaze path visible at the end of a trial.

• console (bool): Enables the console display.

• screenrecord (bool): Enables recording of the screen.

• gazepointvisible (bool): Makes gaze points visible on the mirrored view.

• trackingdatalogging (bool): Logs tracking data.

• replaydatalogging (bool): Logs replay data.

• experimentdatalogging (bool): Logs experiment data.

• timelinelogging (bool): Logs timeline data.

• continuekey (bool): Enables the continue key (default: spacebar).

• trialcount (int or None): Number of trials or None if unlimited.

• configPath (str or None): Path to the configuration file.

• repetitions (int): Number of repetitions for trial blocks.

• datafilesaving (str): PER_TRIAL or COMBINED for saving data files.

• hardwareconfig (str or None): Specify hardware configuration (e.g., "Desktop").

• vizconnectconfig (dict or None): Specify custom vizconnect configuration from a dictionary.

• vizconnectconfiglist (list or None): Specify what to show in the hardware dropdown from built-in options.

• Server/Client Only: avatarintersect (bool): Enables avatar intersection tracking (default: True). Only used when serverMode is enabled. 

3. Setting the Environment

To configure the environment for your experiment, such as loading 3D models or 360 media:

env = vizfx.addChild('sightlab_resources/environments/dojo2.osgb')

sightlab.setEnvironment(env)

Method Parameters:

• env: The environment object to be used.

• trial (str): The trial type, default is 'DEFAULT'.

• mode (str): The mode of the experiment, e.g., 'MODEL_3D' or 'MEDIA_360'.

• mediaType (str): Type of media, default 'video' or 'image'.

• mediaFormat (str): Format of the media, default 'stereo'.

• headlightIntensity (int): Intensity of the headlight, default 0.

4. Managing Trials

Setting Trial Count:

sightlab.setTrialCount(5)

Starting a Trial:

You can start a trial using keyboard/button events or programmatically within a script:

Using Keyboard Events:

vizact.onkeydown(' ', sightlab.startTrial)

Programmatic Start:

yield viztask.waitTime(0.1)

yield sightlab.startTrial(trialLength=60, trialLabel="A", startTrialText="Beginning Trial")

Parameters: 

startTrial(condition = NO_CONDITION, trialLength=-1, trialLabel="A", startTrialText=None, startExperimentText=None, textContinueEvent='triggerPress', textContinueKey=None, trackingDataLogging=True, replayDataLogging=True, experimentSummaryLogging = None, timeLineLogging = None, startPoint = None, startEuler = None)

• condition : Can choose what starts the trial (options are KEY_PRESS, TIMER, ON_MEDIA_END, ON_GAZE_TIME and CUSTOM_EVENT). Note this can also be left to None and use a yield viztask like "yield viztask.waitKeyDown" or this can be set up in the GUI. 

• trialLength (int): Duration of the trial in seconds. -1 means no time limit.

• trialLabel (str): Sets the trial label.

• startTrialText (str): Optional text to display at the start of the trial.

• startExperimentText (str): Optional text to display at the start of the experiment.

• startRatings: Set up a start rating scale

• textContinueEvent(str): event to start the trial (i.e. 'triggerPress')

• textContinueKey: key to start the trial (i.e. ' ')

• trackingDataLogging(bool): track data logging on or off

• replayDataLogging(bool): track logging for the replay

• experimentDataLogging(bool): track logging for the experiment summary

• timelineLogging(bool): track logging for the timeline

• startPoint: change the starting position

• startEuler: change the starting rotation

Ending a Trial:

vizact.onkeydown('e', sightlab.endTrial)

Parameters:

endTrial(condition = NO_CONDITION, endTrialText = None, endExperimentText = None, textContinueEvent = "triggerPress", textContinueKey = None, textContinueTimer = None, endRatings = None)

• condition = NO_CONDITION: Choose what ends the trial. (options are KEY_PRESS, TIMER, ON_MEDIA_END, ON_GAZE_TIME and CUSTOM_EVENT)

• endTrialText (str): Optional text to display at the end of the trial.

• endExperimentText (str): Optional text to display at the end of the experiment.

• endRatings : Set up an end rating scale

Direct Event Trigger:

To directly send an event to end a trial from within a function: 

viz.sendEvent(INITIATE_TRIAL_END)

Use this method when you need to trigger the end of a trial based on specific logic or external conditions. 

Scheduling SightLab

viztask.schedule(sightlab.runExperiment())

5. Setting Condition Based on Hardware Setup

You can set up specific conditions based on the hardware configuration being used. For instance, to check if the hardware is a "Meta Quest Pro" and adjust your experiment setup accordingly: 

if sightlab.getConfig() == "Meta Quest Pro":

# Add conditional logic for Meta Quest Pro hardware 

6. Adding and Managing Scene Objects

Adding Scene Objects

To add objects to the scene, use the addSceneObject method. This allows you to track gaze, grab interactions, and set visibility for the objects. Note: you only need to add the parameters you need. If you only add the name and sceneObject it will just be tracked for replay, but not gaze or grab data and interactions. 

sightlab.addSceneObject(name, sceneObject, gaze=True, grab=False, visible=True, avatar=False)

Parameters:

• name (str): Name identifier for the scene object.

• sceneObject: The instance of the scene object to add.

• gaze (bool): Enable gaze tracking for the object.

• grab (bool): Enable grab interactions for the object.

• visible (bool): Visibility of the object (default: True).

• avatar (bool): Treat the object as an avatar if True.

Multiple Objects:

You can add multiple objects in bulk:

sceneDict = {'Soccerball': [True, True], 'Baseball': [True, True]}

for item in sceneDict:

    sightlab.addSceneObject(item, env.getChild(item), gaze=sceneDict[item][0], grab=sceneDict[item][1])

7. Session Replay

The SightLabReplay class allows for replaying sessions, interacting with objects, and retrieving trial and gaze data. For more information and all the relevant code with the replay, see this page. 

Import and Setup:

from sightlab_utils import replay as Replay

from sightlab_utils.replay_settings import *

replay = Replay.SightLabReplay()

Session Replay Methods:

• Set Replay Position:

replay.transportNode.setPosition([x, y, z])

• Accessing Environment in Replay:

env = replay.getEnvironmentObject()

• Access Scene Objects:

screen = replay.sceneObjects[GAZE_OBJECTS]['screen']

• Get Trial Number:

trial_number = int(replay.currTrial)

• Get Gaze Point Object:

gazePointObject = replay.sceneObjects[GAZE_POINT_OBJECT]['1']

8. Ratings and Instructions

Ratings:

yield sightlab.showRatings('How are you feeling?', ratingScale=scaleList, pauseTimer=True)

• scaleList: Can be a custom list (e.g., "A", "B", "C" or "Yes", "No").

• sightlab.ratingChoice: Get the selected rating value.

You can also add a start or end rating scale within the trial using the startTrial or endTrial methods. See here for more information

Instructions:

yield sightlab.showInstructions('Press Space to Continue')

yield viztask.waitEvent('triggerPress')

yield sightlab.hideInstructions()

Can also add a start or end instructions or use images (see here for more information)

9. Gaze-Based Interactions 

You can trigger events based on gaze interactions within your scene. For example, detecting when a participant finds a target object: see here for more information

This is how you check which object is being viewed 

if e.object == sightlab.sceneObjects[GAZE_OBJECTS][TARGET_PAINTING]:

Example

def gazeActionEnd(e):

if e.object == sightlab.sceneObjects[GAZE_OBJECTS]['target_object']:

print('item found')

viz.sendEvent(INITIATE_TRIAL_END)

vizact.addCallback(sightlab.GAZE_TIME_EVENT, findItem)

10. Adjusting Fixations/Saccade/Dwell Parameters

Note: these need to be set after .startTrial()

sightlab.setDwellTimeThreshold(dwellTime = DWELL_THRESHOLD)      

sightlab.setFixationSaccadeThresholds(dispersionThreshold = 1, durationThreshold = 0.1)

11. Setting Position

Reset Position

viz.callback(viz.getEventID('ResetPosition'), sightlab.resetViewPoint)

This will reset your position back to the origin relative to your physical location. To change this point to a different value, will need to manually add the resetPosition code. 

Changing Starting Position

SightLab (can also add a Starting Point object in the GUI (see Creating a New Scene), or when calling sightlab.startTrial()

sightlab.transportNode.setPosition([0,0,0])

#or can use this

sightlab.setStartPoint((2,0,0))

Session Replay Starting Point

replay.transportNode.setPosition([0,0,0])

12. Miscellaneous  

• Change Default Folder:

sightlab.setEnvironmentDirectory('path_to_resources_folder')

• Set Starting Text:

sightlab.setStartText('Press Spacebar to start')

• Run a Loop for Each Trial:

for i in range(sightlab.getTrialCount()):

    # Custom logic for each trial

• Set Objects as Invisible:

sightlab.addSceneObject("wall2", wall2, gaze=True, visible=False)

• Getting Access to the Eye Tracker:

sightlab.getEyeTracker()

• Working with STIM Files

To work with STIM files, initialize the StimReader: see here for more information

from sightlab_utils import stim_reader

SR = stim_reader.StimReader(STIM_FILE_LOCATION)

SR.fillStimFileEntryList()

• Getting Child Object Position:

If you are using env.getChild to access your objects you need to use a .getTransform to access the object's position: 

targetTransform = env.getTransform('targetTransform')

Alternatively, use env.getChild on the object's GEODE.

• Get Current Time

sightlab.hud.getCurrentTime()

• Setting Maximum Distance for Gaze Intersection:

sightlab.setMaxDistance(1000)

• HeadLight On or Off

sightlab.setHeadLight(state, intensity = None, color = None)

• Trial End Event: The TRIAL_END_EVENT is now simply TRIAL_END.

• Vizconnect Config: VIZCONNECT_CONFIGS is now VIZCONNECT_CONFIG.

• Environment Constant: Use "ENVIRONMENT_MODEL" instead of "ENVIRONMENT".

13. Working with 360 Media

• Get Media Object:

sightlab.getMediaObject()

• Accessing Regions of Interest for 360 Media:

region1 = sightlab.getRegionOfInterest('tire')

14. Accessing the Environment Object

env = sightlab.getEnvironment()

Using the environment object in context

#Access after EXPERIMENT_START

def sightLabExperiment():

global env

yield viztask.waitEvent(EXPERIMENT_START)

env = sightlab.getEnvironment()

#If need access to it before use env = None, and put inside a function that can be called after it is set in the trial

def initialize_environment():

target = env.getChild(name)

def sightLabExperiment():

global env

yield viztask.waitEvent(EXPERIMENT_START)

env = sightlab.getEnvironment()

initialize_environment()

15. Changing Models to use for Avatar

head = vizfx.addChild(AVATAR_HEAD_RESOURCE_PATH+'/Male1.osgb'))

sightlab.setAvatarHead(head)

Other way

sightlab.setAvatarHead(head = vizfx.addChild(AVATAR_HEAD_RESOURCE_PATH+'/Male1.osgb'))

#Setting Hands

sightlab.setAvatarRightHand(rightHand= vizfx.addChild(AVATAR_HANDS_RESOURCE_PATH+'/empty.osgb'))

16. Toggling Visibility of Overlays

sightlab.toggleHUD()

17. Data Logging

For more information see this page

addCustomFlagColumn(self, columnName)

addTrackingTrialDataColumn(self, columnName, trackedObject, dataType='SIXDOF', mode=4)

sightlab.setExperimentSummaryData('condition',condition)

Adding a Custom Flag

sightlab.setCustomTrialData('name of flag')

Changing Data Directories

sightlab.setDataDirectory()

Setting Condition/Label

yield sightlab.startTrial(trialLabel = "A")

Adding additional file to the Data folder

fileName = '{d}_{p.id}_experiment_data._trial_{t}.pdf'.format(d=sightlab.dateTime, p=sightlab.participantData, t=sightlab.trialNumber)

fullPath = os.path.join(sightlab.getExperimentDataFolderPath(), fileName) drawChart(fullPath, sightlab.getPlotData())

18. Settings File

See here for information on the settings file located in the sightlab installation folder

19. Multi User

For code relevant to Multi User see this page

20. Hardware Events

Hardware button and keyboard mappings can be accessed to trigger functions. See the page on the Vizconnect Hardware Configurations page for a list. 

21. Method Reference

Environment and Setup:

• setEnvironment(env, trial='DEFAULT', mode='MODEL_3D', mediaType="video", mediaFormat="stereo", headlightIntensity=0): Sets the virtual environment for the experiment (e.g., 3D models, 360 videos).

• getEnvironment(): Retrieves the current environment object.

• setEnvironmentFolder(): Specifies the folder containing the environment assets.

• setEnvironmentDirectory(path): Sets the directory for environment files.

• getCurrentEnvironmentName(): Returns the name of the current environment. 

Trial Management

• setTrialCount(count): Sets the total number of trials for the experiment.

• getTrialCount(): Returns the total number of trials.

• setTrialLength(seconds): Defines the length of each trial.

• startTrial(trialLength=-1, ...): Begins a trial, optionally specifying trial length, conditions, and starting text.

• endTrial(endTrialText=None, endExperimentText=None): Ends a trial and optionally displays text at the end of the trial or experiment.

• resetTrial(): Resets the current trial.

• getTrialNumber(): Retrieves the current trial number.

• runTrial(): Runs a trial based on pre-configured settings.

• getCurrentTrialDirectory(): Returns the directory where the current trial data is stored.

• getCurrentRepetition(): Returns the current repetition number for repeated trials.

• setStartText(text): Sets the text displayed at the start of the trial.

Object Management

• addSceneObject(name, object, gaze=False, grab=False, visible=True, avatar=False): Adds an object to the scene for gaze, grab, or tracking interactions.

• removeSceneObject(objectName): Removes a previously added scene object.

• removeGazeObject(objectName): Removes an object from gaze tracking.

• removeGrabObject(objectName): Removes an object from grab tracking.

• removeInvisibleObject(objectName): Removes objects that were set as invisible.

• setGazePointObject(): Defines an object that represents the participant's gaze.

• getGazePointObject(): Retrieves the current gaze point object.

Avatar Management

• setAvatarHead(head): Sets the head model for an avatar.

• getAvatarHead(): Retrieves the current head model of the avatar.

• setAvatarLeftHand(): Sets the left-hand model for an avatar.

• setAvatarRightHand(): Sets the right-hand model for an avatar.

• getAvatarLHand(): Gets the current left-hand model of the avatar.

• getAvatarRHand(): Gets the current right-hand model of the avatar.

Gaze and Tracking

• setDwellTimeThreshold(seconds): Sets the amount of time a participant needs to gaze at an object for it to be logged as a "dwell."

• getCurrentGazeEventData(): Retrieves the current gaze data.

• getGazeTimeObject(): Returns the current object being gazed at.

• setFixationSaccadeThresholds(): Sets the thresholds for defining fixations and saccades in the eye-tracking data.

• showGazePath(): Displays the path of the participant's gaze over time.

• toggleGazePoint(): Toggles the visibility of the gaze point in the scene.

Data Logging

• setDataDirectory(path): Sets the directory where data logs are saved.

• setExperimentSummaryData(column, value): Adds custom data to the experiment summary log.

• addTrackingTrialDataColumn(name): Adds a new column to the tracking data logs.

• setCustomTrialData(value, column): Sets a custom value in a specified trial data column.

• setDataLoggingState(state): Enables or disables data logging.

• getPlotData(): Retrieves data for creating visual plots from the trial data.

• getExperimentDataFolderPath(): Gets the folder path where experiment data is saved.

Ratings and Instructions

• showRatings(text, options): Displays a rating scale for participants to provide feedback during a trial.

• hideRatings(): Hides the rating scale from the view.

• showInstructions(text): Displays instructional text during the experiment.

• hideInstructions(): Hides the instructions currently shown to the participant.

Recording and Replay

• compressRecording(): Compresses the recorded session for playback.

• screenCapture(): Captures a screenshot of the current scene.

• storeReplayExecCode(code): Stores executable code during replay mode.

• setCustomReplayFlag(flag): Sets a custom flag for replay purposes.

• getMediaObject(): Retrieves media objects for replay.

• addToTimeline(event): Adds an event to the trial timeline for later reference.

• getTimeLineData(): Returns timeline data captured during the trial.

Control and Transport

• resetViewPoint(): Resets the participant's viewpoint to a default position.

• setStartPoint(position): Sets the starting position for the participant in the scene.

• setTransportPosition(position): Sets the position for the transport object (if using one).

• setTransportState(state): Enables or disables transport mechanics (e.g., walking, teleporting).

HUD and Console

• toggleHUD(): Toggles the visibility of the heads-up display (HUD).

• setConsoleState(state): Enables or disables the console display.

• togglePoint(): Toggles the display of point markers in the scene.

• setHeadlight(intensity): Sets the intensity of the headlight (lighting attached to the participant's viewpoint).

Region of Interest (ROI)

• addRegionOfInterest(name, object): Adds a region of interest (ROI) to the scene for tracking focus.

• getRegionOfInterest(name): Retrieves a specific ROI object from the scene.

Experiment Control

• runExperiment(): Starts the experiment loop.

• resetExperiment(): Resets the experiment to its initial state.

• conditionEventManager(): Manages condition-based events for the experiment.

• experimentSetupCompleteStatus(): Checks if the experiment setup is completed.

Miscellaneous

• getConfig(): Retrieves the hardware configuration (vizconnect) of the experiment.

• setConfigDirectory(path): Sets the directory where configuration files are stored.

• setRecordingsDirectory(path): Sets the directory where recordings are saved.

• setMaxDistance(distance): Sets the maximum distance for collecting gaze data.

• setConditionName(name): Sets the name for the current condition of the experiment.

• getConditionName(): Retrieves the current condition name.

• setHighlighter(object): Enables a highlighter on the specified object to make it visually stand out.

22. Example Usage

import sightlab_utils.sightlab as sl

from sightlab_utils.settings import *

# Initialize SightLab without GUI

sightlab = sl.SightLab(gui=False)

# Set up environment

env = vizfx.addChild('sightlab_resources/environments/dojo.osgb')

sightlab.setEnvironment(env)

# Add scene objects

soccerball = env.getChild('soccerball')

sightlab.addSceneObject('soccerball', soccerball, gaze=True, grab=True)

# Set trial count and start the experiment

sightlab.setTrialCount(3)

def sightLabExperiment():

    yield viztask.waitKeyDown(' ')

    while True:

        yield sightlab.startTrial(trialLength=5, startTrialText="Look around!")

        yield viztask.waitEvent(TRIAL_END)

viztask.schedule(sightlab.runExperiment)

viztask.schedule(sightLabExperiment)