Data Classes and Enums

These are definitions of data objects and enumerations sent to and received from the robot.

class navel.Bbox2d(x1: int, y1: int, x2: int, y2: int)

Describes a bounding box with two 2D points.

Sets the limits of a bounding Box located in an image. The bounding box limits are described in pixels in XY plane.

Parameters:

• x1 (int) – Point 1 X coordinate in pixels.

• y1 (int) – Point 1 Y coordinate in pixels.

• x2 (int) – Point 2 X coordinate in pixels.

• y2 (int) – Point 2 Y coordinate in pixels.

x1: int

x2: int

y1: int

y2: int

class navel.Bryan(x: float, y: float, z: float)

Describes a Tait-Bryan angle rotation.

Parameters:

• x (float) – Rotation around X axis in radians.

• y (float) – Rotation around Y axis in radians.

• z (float) – Rotation around Z axis in radians.

x: float

y: float

z: float

class navel.CartSys3d(sys: CoordSystem, x: float, y: float, z: float)

Describes a 3D point in cartesian space.

Parameters:

• sys (CoordSystem) – Coordinate system used.

• x (float) – X coordinate in meters.

• y (float) – Y coordinate in meters.

• z (float) – Z coordinate in meters.

sys: CoordSystem

x: float

y: float

z: float

class navel.CartVec3d(x: float, y: float, z: float)

Describes a cartesian 3D vector.

Parameters:

• x (float) – X vector.

• y (float) – Y vector.

• z (float) – Z vector.

x: float

y: float

z: float

class navel.CoordSystem(value)

Enum class describing available coordinate frames. X generally points forward, Y to the left and Z upwards (from point of view of robot).

CAM_CHEST = 1

Origin in center of chest cam, not implemented yet

CAM_HEAD = 5

Origin in center of head cam.

COORD_LEN = 8

Max Value of enum, used only in calculations

EYE_LEFT = 6

Origin in center of left eye.

EYE_RIGHT = 7

Origin in center of right eye.

HEAD_REAL = 4

Fixed to head, x axis pointing in facing direction.

HEAD_REF = 2

Origin in head center, head facing slightly downwards.

HEAD_STRAIGHT = 3

Origin in head center, head facing straight.

class navel.DataType(value)

Enum class describing what data type is used.

F32 = 5

32 bit float

F64 = 6

64 bit double

I32 = 1

32 bit integer

I64 = 3

64 bit integer

U32 = 2

32 bit unsigned integer

U64 = 4

64 bit unsigned integer

class navel.PerceptionData(time: int = 0, persons: ~typing.List[~navel._data_structs.Person] = <factory>, sst_time_latest: int = 0, sst_tracks_latest: ~typing.List[~navel._data_structs.SstMeta] = <factory>)

Class containing infomation sent via perception socket.

The data contains the latest information about the perceived surroundings of the robot.

Parameters:

• time (int) – Timestap of image taken as basis for persons

• persons (list[Person]) – List of detected persons.

• sst_time_latest (int) – Timestamp of audio taken as basis for sst_latest

• sst_tracks_latest (list[SstMeta]) – Most current list of detected sound sources.

persons: List[Person]

sst_time_latest: int = 0

sst_tracks_latest: List[SstMeta]

time: int = 0

class navel.Person(id_score: float | None = None, uid: int | None = None, face: Bbox2d | None = None, landmarks: PersonLandmarks | None = None, head_position: Bryan | None = None, gaze: CartVec3d | None = None, facial_expression: PersonFacialExpression | None = None, dist_mm: float | None = None, gaze_overlap: float | None = None, g_eye_left: CartSys3d | None = None, g_eye_right: CartSys3d | None = None, g_nose: CartSys3d | None = None, g_head_position: CartSys3d | None = None, g_gaze: CartSys3d | None = None)

Defines attributes of a detected person.

Parameters:

• id_score (float) – Defaults to None.

• uid (int) – Unique user id. Defaults to None.

• face (Bbox2d) – Bounding box of face. Defaults to None.

• landmarks (PersonLandmarks) – Position of left end of mouth. Defaults to None.

• head_position (Bryan) – Position of head in Tait-Bryan angles (radians). Defaults to None.

• gaze (CartVec3d) – Vector describing persons direction of gaze. Defaults to None.

• facial_expression (PersonFacialExpression) – Person’s facial expression. Defaults to None.

• dist_mm (float) – Distance from robot to person (mm). Defaults to None.

• gaze_overlap (float) – How much person is staring at robot, 0 = not looking at robot, 1 = looking directly at robot. Defaults to None.

• g_eye_left (CartSys3d) – A list of the position of left eye in all coord frames. This might be out of frame, then sys is CoordSystem.UNDEFINED. Defaults to None.

• g_eye_right (CartSys3d) – A list of the position of right eye in all coord frames. This might be out of frame, then sys is CoordSystem.UNDEFINED. Defaults to None.

• g_nose (CartSys3d) – A list of the position of nose in all coord frames. This might be out of frame, then sys is CoordSystem.UNDEFINED. Defaults to None.

• g_head_position (CartSys3d) – A list of the position of head in all coord frames. Defaults to None.

• g_gaze (CartSys3d) – A list of the target of persons gaze in all coord frames. Defaults to None.

dist_mm: float | None = None

face: Bbox2d | None = None

facial_expression: PersonFacialExpression | None = None

g_eye_left: CartSys3d | None = None

g_eye_right: CartSys3d | None = None

g_gaze: CartSys3d | None = None

g_head_position: CartSys3d | None = None

g_nose: CartSys3d | None = None

gaze: CartVec3d | None = None

gaze_overlap: float | None = None

head_position: Bryan | None = None

id_score: float | None = None

landmarks: PersonLandmarks | None = None

uid: int | None = None

class navel.PersonFacialExpression(neutral: float, happy: float, sad: float, surprise: float, anger: float)

Class describing the facial expression of the detected person

Each emotion in this class is described by a magnitude where 1 describes maximum expression and 0 describes minimum expression.

Parameters:

• neutral (float) – magnitude of neutral.

• happy (float) – magnitude of happyness.

• sad (float) – magnitude of sadness.

• surprise (float) – magnitude of surprise.

• anger (float) – magnitude of anger.

anger: float

happy: float

neutral: float

sad: float

surprise: float

class navel.PersonLandmarks(eye_left: Point2d, eye_right: Point2d, nose: Point2d, mouth_left: Point2d, mouth_right: Point2d)

Class describing position of a person’s facial landmarks.

Each landmark is described by its 2D position within the host image.

Parameters:

• eye_left (Point2d) – Position of left eye.

• eye_right (Point2d) – Position of right eye.

• nose (Point2d) – Position of nose.

• mouth_left (Point2d) – Position of left end of mouth.

• mouth_right (Point2d) – Position of right end of mouth.

eye_left: Point2d

eye_right: Point2d

mouth_left: Point2d

mouth_right: Point2d

nose: Point2d

class navel.Point2d(x: int, y: int)

Describes a 2D point using pixel coordinates.

Parameters:

• x (int) – X coordinate in pixels.

• y (int) – Y coordinate in pixels.

x: int

y: int

class navel.SstMeta(id: int, activity: float, loc: CartSys3d, is_dynamic: bool)

Available information about tracked sounds.

Parameters:

• id (int) – ID per separable sound source, reused.

• activity (float) – Activity of signal (0 to 1).

• loc (CartSys3d) – Spacial source of sound.

• is_dynamic (bool) – If track is dynamic or statically defined.

activity: float

id: int

is_dynamic: bool

loc: CartSys3d

navel.run(app)

Run a coroutine with a robot connection as the only parameter.

This is used to start navel “apps”. This function is only provided as a shorthand for the most common use-case, calling the run method on any of the connection classes accomplishes the same goal.