Face Tracking

To utilize face tracking sensors SightLab will support any devices that utilize the OpenXR standard for either Meta or HTC headsets.

Here are some of the built in examples/templates included in SightLab for Face Tracking

You can add collecting and saving of face tracking data to any script by adding the lines from sightlab_utils import face_tracker_data. This will save all the facial tracking data to the data folder
See the Mirror Demo to get a template on how to map the expressions to an animated avatar (can also just build from this code to create a script that maps the expressions to an avatar, remove the mirror code if not needed)
See the FaceTracker_Sliders to see sliders that move with your facial tracker data (this can also be pulled up in the Mirror demo)
facial_expressions_over_time will show a matplotlib chart of the facial expression data over time. This can load data from the mirror demo or other demos using face tracking

Here is some sample code on importing the face_tracker_data module and collecting and saving that data (which will save in the data folder:

import viz,vizfx, viztaskfrom sightlab_utils import face_tracker_dataimport sightlab_utils.sightlab as slfrom sightlab_utils.settings import *
sightlab = sl.SightLab()
# Set up with custom valuesface_tracker_data.setup()
def sightLabExperiment(): while True: yield viztask.waitKeyDown(' ') # Start the update process vizact.ontimer(0, face_tracker_data.UpdateAvatarFace) yield sightlab.startTrial()
yield viztask.waitKeyDown(' ') yield sightlab.endTrial() viztask.schedule(sightlab.experiment)viztask.schedule(sightLabExperiment)

Here are the parameters available for Face Tracking Expression:

For Facebook (Meta)

EXPRESSION_BROW_LOWERER_L = 0

EXPRESSION_BROW_LOWERER_R = 1

EXPRESSION_CHEEK_PUFF_L = 2

EXPRESSION_CHEEK_PUFF_R = 3

EXPRESSION_CHEEK_RAISER_L = 4

EXPRESSION_CHEEK_RAISER_R = 5

EXPRESSION_CHEEK_SUCK_L = 6

EXPRESSION_CHEEK_SUCK_R = 7

EXPRESSION_CHIN_RAISER_B = 8

EXPRESSION_CHIN_RAISER_T = 9

EXPRESSION_DIMPLER_L = 10

EXPRESSION_DIMPLER_R = 11

EXPRESSION_EYES_CLOSED_L = 12

EXPRESSION_EYES_CLOSED_R = 13

EXPRESSION_EYES_LOOK_DOWN_L = 14

EXPRESSION_EYES_LOOK_DOWN_R = 15

EXPRESSION_EYES_LOOK_LEFT_L = 16

EXPRESSION_EYES_LOOK_LEFT_R = 17

EXPRESSION_EYES_LOOK_RIGHT_L = 18

EXPRESSION_EYES_LOOK_RIGHT_R = 19

EXPRESSION_EYES_LOOK_UP_L = 20

EXPRESSION_EYES_LOOK_UP_R = 21

EXPRESSION_INNER_BROW_RAISER_L = 22

EXPRESSION_INNER_BROW_RAISER_R = 23

EXPRESSION_JAW_DROP = 24

EXPRESSION_JAW_SIDEWAYS_LEFT = 25

EXPRESSION_JAW_SIDEWAYS_RIGHT = 26

EXPRESSION_JAW_THRUST = 27

EXPRESSION_LID_TIGHTENER_L = 28

EXPRESSION_LID_TIGHTENER_R = 29

EXPRESSION_LIP_CORNER_DEPRESSOR_L = 30

EXPRESSION_LIP_CORNER_DEPRESSOR_R = 31

EXPRESSION_LIP_CORNER_PULLER_L = 32

EXPRESSION_LIP_CORNER_PULLER_R = 33

EXPRESSION_LIP_FUNNELER_LB = 34

EXPRESSION_LIP_FUNNELER_LT = 35

EXPRESSION_LIP_FUNNELER_RB = 36

EXPRESSION_LIP_FUNNELER_RT = 37

EXPRESSION_LIP_PRESSOR_L = 38

EXPRESSION_LIP_PRESSOR_R = 39

EXPRESSION_LIP_PUCKER_L = 40

EXPRESSION_LIP_PUCKER_R = 41

EXPRESSION_LIP_STRETCHER_L = 42

EXPRESSION_LIP_STRETCHER_R = 43

EXPRESSION_LIP_SUCK_LB = 44

EXPRESSION_LIP_SUCK_LT = 45

EXPRESSION_LIP_SUCK_RB = 46

EXPRESSION_LIP_SUCK_RT = 47

EXPRESSION_LIP_TIGHTENER_L = 48

EXPRESSION_LIP_TIGHTENER_R = 49

EXPRESSION_LIPS_TOWARD = 50

EXPRESSION_LOWER_LIP_DEPRESSOR_L = 51

EXPRESSION_LOWER_LIP_DEPRESSOR_R = 52

EXPRESSION_MOUTH_LEFT = 53

EXPRESSION_MOUTH_RIGHT = 54

EXPRESSION_NOSE_WRINKLER_L = 55

EXPRESSION_NOSE_WRINKLER_R = 56

EXPRESSION_OUTER_BROW_RAISER_L = 57

EXPRESSION_OUTER_BROW_RAISER_R = 58

EXPRESSION_UPPER_LID_RAISER_L = 59

EXPRESSION_UPPER_LID_RAISER_R = 60

EXPRESSION_UPPER_LIP_RAISER_L = 61

EXPRESSION_UPPER_LIP_RAISER_R = 62

CONFIDENCE_LOWER_FACE = 0

CONFIDENCE_UPPER_FACE = 1

For HTC:

EYE_LEFT_BLINK = 0

EYE_LEFT_WIDE = 1

EYE_RIGHT_BLINK = 2

EYE_RIGHT_WIDE = 3

EYE_LEFT_SQUEEZE = 4

EYE_RIGHT_SQUEEZE = 5

EYE_LEFT_DOWN = 6

EYE_RIGHT_DOWN = 7

EYE_LEFT_OUT = 8

EYE_RIGHT_IN = 9

EYE_LEFT_IN = 10

EYE_RIGHT_OUT = 11

EYE_LEFT_UP = 12

EYE_RIGHT_UP = 13

LIP_JAW_RIGHT = 0

LIP_JAW_LEFT = 1

LIP_JAW_FORWARD = 2

LIP_JAW_OPEN = 3

LIP_MOUTH_APE_SHAPE = 4

LIP_MOUTH_UPPER_RIGHT = 5

LIP_MOUTH_UPPER_LEFT = 6

LIP_MOUTH_LOWER_RIGHT = 7

LIP_MOUTH_LOWER_LEFT = 8

LIP_MOUTH_UPPER_OVERTURN = 9

LIP_MOUTH_LOWER_OVERTURN = 10

LIP_MOUTH_POUT = 11

LIP_MOUTH_SMILE_RIGHT = 12

LIP_MOUTH_SMILE_LEFT = 13

LIP_MOUTH_SAD_RIGHT = 14

LIP_MOUTH_SAD_LEFT = 15

LIP_CHEEK_PUFF_RIGHT = 16

LIP_CHEEK_PUFF_LEFT = 17

LIP_CHEEK_SUCK = 18

LIP_MOUTH_UPPER_UPRIGHT = 19

LIP_MOUTH_UPPER_UPLEFT = 20

LIP_MOUTH_LOWER_DOWNRIGHT = 21

LIP_MOUTH_LOWER_DOWNLEFT = 22

LIP_MOUTH_UPPER_INSIDE = 23

LIP_MOUTH_LOWER_INSIDE = 24

LIP_MOUTH_LOWER_OVERLAY = 25

LIP_TONGUE_LONGSTEP1 = 26

LIP_TONGUE_LEFT = 27

LIP_TONGUE_RIGHT = 28

LIP_TONGUE_UP = 29

LIP_TONGUE_DOWN = 30

LIP_TONGUE_ROLL = 31

LIP_TONGUE_LONGSTEP2 = 32

LIP_TONGUE_UPRIGHT_MORPH = 33

LIP_TONGUE_UPLEFT_MORPH = 34

LIP_TONGUE_DOWNRIGHT_MORPH = 35

LIP_TONGUE_DOWNLEFT_MORPH = 36