Canvas

HOST.onstartup-README

Kinect host - HOST.onstartup.toe

Reads the three Azure Kinect sensors, puts them in room coordinates, merges them into one skeleton per person and sends everything to the lab network.
Runs on the Kinect NUC. Students never open it; this README is for whoever maintains it.

In: 3 × Azure Kinect · Out: all joints of all people (OSC), camera images (NDI)

How to use

  1. Start the NUC. The project opens by itself at startup.
  2. Check that the three Kinect Azure CHOPs show no error and that each shows data when someone stands in its view.
  3. Inside Sensor_Azure_Multiple_HOST, check the Script CHOP fusion: no flag, persons counts the people in the room, and heartbeat keeps changing.
  4. Walk through the room. You should stay p1 the whole time, also when you pass from one sensor's view into another's.
  5. Calibration check: set DEBUG = True at the top of fusion_callbacks and read the spread lines in the Textport (see Normal, not bugs).

Settings

On each Kinect Azure CHOP:

  • Max Players: how many bodies one sensor tracks. More bodies cost more GPU.
  • World Space Positions: on (positions in metres, before calibration)
  • Bone Confidence: on. The fusion uses it to prefer the sensor with the best view of each joint.

In the Transform XYZ CHOPs (transformxyz_Y2, _Y19, _Y13):

  • Translate and Rotate: the measured pose of that sensor. Only change these after moving a sensor.

At the top of fusion_callbacks (the fusion script):

  • MATCH_DIST (0.75 m): how far a person may move between two frames and still be the same person
  • MERGE_DIST (1.20 m): bodies from different sensors this close together are one person. Lower it once the calibration is tighter.
  • CONFIRM_FRAMES (5): frames a new person must be seen before getting a number
  • DROP_FRAMES (15): frames a person may be unseen before their number is released
  • MAX_SLOTS (8): highest person number, p1 to p8
  • DEBUG: prints the decisions once per second in the Textport

Outputs

  • p{N}/{joint}:tx ty tz: every joint of every tracked person, metres, room frame (X width, Y height, Z depth)
  • p1/present … p8/present: 1 while that person is tracked, always sent, also as 0
  • persons: how many people are seen right now
  • heartbeat: changes every frame while the host runs
  • OSC UDP 1030 (skeletons), 1031 (frame rate); NDI per sensor (AK1-3)

Normal, not bugs

  • A person gets a different number after leaving and coming back: numbers belong to tracks, not to people.
  • Numbers don't follow the order people entered once someone leaves: a free number is reused, the lowest one first.
  • A new person appears about 0.2 s after entering: they must be seen for 5 frames first, so a one-frame glitch never takes a number.
  • spread above 0.5 m in the debug lines: the calibration of that sensor pair is off. Re-measure the pose of the sensor that differs.
  • Hands are less steady than torso and head: the Kinect itself rates hands as uncertain. The fusion then averages all sensors.
  • Left and right hand swap when someone turns their back or pulls up their shoulders: the Kinect assumes everyone faces it. Not solved yet.
  • Dark hair, long hair, dark skin or dark clothing track worse: less infrared light comes back to the depth camera.
  • Two people hugging become one person for a moment: they are within merge distance.