Azure Kinect sensors

OSC In (all_oscin1) ─► null1 ─┐                                 ┌─► joint_sel_0 ─► out_0
   or, in a copy:             ├─► joints_resolved (Script CHOP) ─┼─► joint_sel_1 ─► out_1
   shared_stream (Select) ────┘        ▲ input 1                 └─► …
Mouse_base (tx ty tz) ─────────────────┘
joint_utils (shared code) · joint_watch (Execute DAT, every frame) · fps_oscin ─► merge_view

• joints_resolved (Script CHOP, code in joints_resolved_callbacks) makes every per-frame decision about values: live, mouse or fallback, and smoothing.
• joint_watch (Execute DAT) maintains the structure: it builds the outlets from the joint list and decides which copy is the receiver.
• joint_utils holds the code both share: reading the joint list and the parameters, and finding the receiver.

The joint list on the component (Add joints) is the single source of truth. It is saved with the project, and nothing that arrives from the host can change it. That is what keeps the list editable without a connection, and what keeps the outlets from disappearing when the connection drops.

The host sends every joint of every tracked person over OSC (UDP 1030), named p{N}/{joint}:tx/ty/tz, plus p{N}/present and a heartbeat (see Azure Kinects calibration). Joints are matched by name, never by position in the stream, so the order of joints on the host and in the component never has to agree.

Output values are in metres in the lab's room frame: X = width, Y = height, Z = depth. This is the same frame used by Virtual objects and the sound system, so you can compare joint and object positions directly without converting.

joints_resolved cooks every frame (it reads the clock on purpose, so it also notices when the stream stops). For each joint in the list it decides, in this order:

Mouse on? ──yes──► mouse position (same value for every joint)
   │ no
Usable live data for this person? ──yes──► tracked value ─► One Euro smoothing
   │ no
Fallback origin on? ──yes──► Position
   │ no
0 0 0

OSC has one property that drives most of this design: a channel that stops arriving keeps showing its last value forever. A person who left, a frozen host and a dropped network all look like someone standing perfectly still. The component therefore treats live data as unusable when any of these is true:

1. No stream is connected.
2. The person is absent: the host's p{N}/present for this component's Person number is 0.
3. This person's values have stopped changing for 0.5 s (STALE_SEC). Real tracking always jitters by a fraction of a millimetre, so values that are exactly constant mean the data has stopped. This check works even with an older host that sends no presence flags or heartbeat.
4. The host's heartbeat hasn't changed for 0.5 s: the host has stopped, frozen or lost the connection.
5. None of the chosen joints exist in the stream for this person.

If live data is unusable, every joint of this component goes to the fallback at once. The outlets themselves never change; only their values do.

Live values pass through a One Euro filter (Casiez et al., CHI 2012), per channel. It is an adaptive low-pass filter whose cut-off frequency rises with the speed of the joint:

• When a joint is (nearly) still, the cut-off is low (MIN_CUTOFF), so sensor jitter disappears.
• When it moves fast, the cut-off rises by BETA × speed, so the filter hardly smooths and adds almost no delay.

A Lag CHOP smooths everything equally, so it always trails behind fast movement. In a simulation with 1 cm sensor noise and a fast 1 m hand swing, both reduced jitter at rest from 8.6 mm to about 2 mm, but the lag trailed by 133 ms and the One Euro filter by 50 ms. There are no Lag CHOPs left in the chain, on the host or in this component.

Two details matter in practice:

• The filter only updates when a new sample arrives. The sensors deliver 15–30 samples per second while TouchDesigner runs at 60 fps. Repeated values between samples would otherwise read as "standing still" and distort the speed estimate.
• The filter resets whenever live data stops. A person who comes back snaps to their real position instead of gliding in from the fallback origin.

The constants are at the top of joints_resolved_callbacks: MIN_CUTOFF (Hz), BETA and D_CUTOFF. Tune in this order: stand still and lower MIN_CUTOFF until the jitter is gone, then move fast and raise BETA until the lag is gone. SMOOTHING = False switches the filter off for comparison. Mouse mode and the fallback are never smoothed.

joint_watch compares the joint list and the Person number with the previous frame. Only when one of them changes does it rebuild:

• one Select CHOP (joint_sel_i) per slot, picking p{Person}/{joint}:tx/ty/tz and renaming it to tx ty tz
• one Out CHOP (out_i) per slot, labelled with the joint name

Existing Out CHOPs are reused, never recreated, and only outlets beyond the end of the list are removed. Wires from a student's patch therefore stay connected when the list changes, and nothing that arrives from the host (people entering or leaving, a dropped connection) can ever change the outlets. A joint chosen twice gets one channel in the viewer, but each slot keeps its own outlet. Configure has nothing to do with this: it only re-initialises the sensors on the host.

Each copy of the component follows one person, set with Person. The copies share one network connection, because two OSC In CHOPs can't listen on the same port. The decision is made automatically by joint_watch, about once per second:

1. Find all copies of the component in the project (every component with a joints_resolved and an OSC In).
2. The oldest copy is the receiver: the one with the lowest operator ID, normally the original.
3. The receiver keeps its OSC In, OSC Out and Configure active, and its fps_oscin plugged into the view.
4. Every other copy:

• switches its OSC In and OSC Out off;
• greys out Configure, because only one component should be able to re-initialise the host's sensors;
• unplugs fps_oscin from merge_view;
• creates a blue Select CHOP, shared_stream, that reads the receiver's stream, and wires it into joints_resolved. Its comment says which component it reads from.

1. If the receiver is deleted, the next oldest copy becomes the receiver within a second: it removes its shared_stream, reconnects its own stream and switches everything back on.

Nothing is deleted in a copy, so every copy stays a complete component. Settings are found by their label in the panel, not by their internal name (Mouse is internally Sensor, Joint is Skeleton, Fallback origin is Index, Position is Posu/Posv/Posw).

With Mouse on, every joint follows the mouse in a chosen plane (Mouse orientation, for example XZ = across the floor), already mapped to room metres by Mouse_base. This lets you test downstream logic without the lab. The mouse is never used as a fallback when Mouse is off.