Institution: LUCA School of Arts (LUCA)
Partner: FilmEU, as part of WIRE
Coordinator: Jan Duerinck (jan.duerinck@luca-arts.be)
Website: <https://www.researchcatalogue.net/>

The Spatial Computing Lab (LUCAscl*) at LUCA School of Arts explores real-time spatial interaction, augmented and mixed reality, and immersive sound. Integrating panoramic video, depth sensors, and tactile interfaces, it fosters artistic research into the dynamic relations between digital and physical space. The lab aims to develop critical, experimental, and participatory approaches to spatial computing and media art. As part of WIRE, it contributes to interdisciplinary research across XR, installation practices, and interactive design.

• Augmented and mixed reality
• Spatial computing
• Interactive media

• Advance interdisciplinary research in XR and spatial computing.
• Strengthen doctoral and practice-based research within LUCA and FilmEU.
• Strengthen collaborations.
• Expand the lab's technological capabilities.
• Build a sustainable framework that supports this research.

• Lab Tools (technique: the lab's components)
• Inventory (software and hardware in the lab)
• Lab model (the room in 3D)
• Partner FilmEU, Support STAGEtech (projects and partners)

• FilmEU-Spatial_Computing_Lab-Luca_School_of_Arts-Long_read (summary)

Layer: spatial logic
In: 2 POPs (point or shape)
Out: 1 CHOP

CALC measures how two spatial entities relate to each other: two points, a point and a shape, or two shapes. It reduces that relation to one control signal. Four measurements are available: centre-to-centre distance, signed distance to a surface, an inside/outside test, and a smooth fade between two concentric spheres. Together these are enough for most body-to-object interactions in the lab, such as proximity, entering a zone, and gradual transitions. They come from a single component with one predictable output.

CALC turns geometry into control data. A typical setup has a tracked joint from Azure Kinect sensors on input 1 and a Virtual objects on input 2. The resulting CHOP drives any parameter: the opacity of a projected layer, the gain of a sound, the state of a scene.

Let A be input 1 and B be input 2.

• Distance between points. The Euclidean distance between the centres of A and B:
  center_dist = ‖c_A − c_B‖ (metres).
• Distance between shapes. The distance from A to the surface of B, or between the surfaces of A and B. The result is signed: positive outside, negative inside. This works for every closed shape. For boxes and cylinders, the distance is measured outward from the shape's centre, so values near edges and corners are slightly larger than the true Euclidean surface distance.
• In or outside of a shape. A binary containment test: inside = 1 if A is inside B, otherwise 0.
• Fade to sphere. A continuous value between the inner sphere (radius r_in) and the outer shell (radius r_out) of a sphere-type virtual object. It is 1 inside r_in, 0 beyond r_out, and changes smoothly in between. This makes a soft, non-binary interaction zone. It requires B to be a sphere with Outer shell turned on.
• One calculation at a time. Only one calculation can be on. The others are greyed out. This keeps the output unambiguous and keeps the GPU/CPU cost per instance predictable. For several measurements, use several CALC instances in parallel.
• POP inputs. Both inputs are POPs, so the calculations work directly on the GPU-side geometry from Virtual objects. A single tracked joint counts as a one-point POP.

• Fade to sphere greyed out: input 2 is not a sphere with Outer shell on.
• Wrong distances: the shape is not closed (it has an open side).
• Box or cylinder distances slightly too large near edges and corners: this comes from measuring outward from the centre (see above).
• Yellow warning inside the component when a point is on input 1: harmless.
• Raw tracking noise passes straight through. For stable triggers, smooth the joint input, or add hysteresis to the inside output.

Since the October 2026 revision, CALC replaces four earlier components: DIST_between_points, DIST_to_object, IN_or_OUT_object and FADE_to_sphere. Distance to a shape now works for any closed shape and between two shapes.

• ← Azure Kinect sensors (input 1)
• ← Virtual objects (input 2, or both inputs)
• → any parameter, e.g. in Ambisonics sound control, Wall projection or Floor projection content

• User README: CALC-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

Layer: sensing
In: none
Out: one TOP per sensor

This component gives the live image streams of the lab's three Azure Kinect sensors as TouchDesigner TOPs, one per sensor. It is for work that uses the camera image itself, such as live video in a projection, image analysis, or material for compositing. Skeleton data is not included: for body positions, use Azure Kinect sensors.

• Source. The video comes from the Azure Kinects calibration as NDI streams, one per sensor (<host> (AK1), (AK2), (AK3)), and not from the workstation itself. This spreads the processing load: the high-bandwidth camera streams do not compete with rendering on the workstation. The component lists the NDI sources it finds on the network when the project starts.
• Viewpoint. The sensors are mounted on the ceiling grid, so all images are high-angle views looking down into the room.
• Stream selection. Image sets which stream is delivered (for example colour). The choice is sent to the host as an OSC broadcast (UDP 27), and the host switches what it publishes. Other stream types the sensor provides, such as depth or infrared, can be selected here if the host publishes them.
• Outlets. There is one outlet per sensor in a fixed order (1, 2, 3), which matches the physical sensors. To find out which outlet is which corner, wave at a camera.
• Resolution and frame rate are set by the sensor and the stream configuration, not by the component. Fit or resample downstream to match the target resolution of a projection component.

• Outlets 1–3: one TOP per sensor, in fixed order.
• Recommended convention: attach a named null to each outlet (null_cam_left, null_cam_right, …).

• Black or empty outputs: you are not in the lab, or the sensors are off.
• All views look alike: they show the same empty room from different corners.
• No change after switching Image: press Configure.
• Unexpected resolution or frame rate: this comes from the sensor or stream. Use a Fit TOP or resample downstream.
• Privacy: these are identifiable camera images. When recording or passing on material from public sessions, you need visitors' consent.

• → Wall projection, Floor projection (via your own image processing)
• ← Azure Kinects calibration (NDI images)
• ↔ Azure Kinect sensors (same sensors, position data)
• Hardware: Inventory (the installed sensors, and other cameras available in the lab)

• User README: Azure_Kinect_camera-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

The Spatial Computing Lab (LUCAscl*) gives artists, students and researchers a shared set of custom TouchDesigner components ("toxes"). These components hide the lab's specific hardware behind a small number of stable interfaces. There are eight components, grouped into three layers:

• Sensing: body tracking and camera images from Azure Kinect sensors.
• Spatial logic: virtual objects, and calculations between bodies and objects.
• Output: wall projection, floor projection and third-order ambisonic sound.

Together they let a user go from a tracked body in the room to an image or sound that responds to it, without rebuilding the lab's signal chain. The components are meant to be used both at home and in the lab. Projects can be prepared and simulated remotely, and then run at full resolution and with live sensor data on site. On the sound side, the components are complemented by a rendering patch in Max/MSP. This note describes the system the components belong to, the conventions they share and how the other notes in this collection fit together.

The components are tied to one physical space and its hardware:

All components follow the same conventions. Once you know them for one component, you know them for the rest.

• Units and axes. Positions are in metres. X = width, Y = height, Z = depth. The sound system uses the same frame: Z = 0 is the north wall, Z increases towards the entrance, and the reference listening point is at X = 0, Y = 1.5 m (ear height), Z = 2.5 m (room centre).
• Data types. Each component uses the native TouchDesigner family that fits the data:
• images are TOPs
• positions, distances and control values are CHOPs
• geometry is POPs, the GPU-based point operators that replaced SOPs in October 2026
• Configure pattern. Components that build their own outlets or windows (camera, projections) rebuild only when you press Configure. This keeps a running performance from being rebuilt by accident. The skeleton sensor is the exception: its outlets follow the joint list by themselves, and its Configure re-initialises the sensors on the Kinect host.
• Named nulls. Users attach a named null to each outlet (null_head, null_cam_left, …) and work from those. If a component is reconfigured or updated, references to the nulls still work.
• Home/lab parity. Every component that depends on the lab can run without it:
• the skeleton sensor can follow the mouse instead of a body
• the projection components have a License switch that changes between home resolution, which stays within the 1280 × 1280 limit of TouchDesigner's free Non-Commercial licence, and full lab resolution, which the lab's Educational licence allows.
• all components behave the same either way, so a patch built at home runs unchanged in the lab.
• "Normal, not bugs". Each component documents behaviour that looks like a fault but is expected, such as empty outputs outside the lab or a source that takes a moment to appear. This saves time when troubleshooting on site.

          SENSING                    SPATIAL LOGIC                    OUTPUT
 ┌──────────────────────┐   ┌──────────────────────────┐   ┌──────────────────────────┐
 │ Azure Kinect sensor  │──►│ Calculations (CALC)      │──►│ Ambisonics sound control │
 │  (joints, CHOP)      │   │  distance / inside / fade│   │  (Max/MSP, 16 + 1 ch)    │
 │                      │──►│ Virtual object viewer    │   │                          │
 │ Azure Kinect camera  │   │  (verification only)     │   │ Wall projection          │
 │  (images, TOP)       │   │                          │   │  (L / C / R, panoramic)  │
 └──────────────────────┘   │ Virtual object           │──►│                          │
                            │  (shape POP, pos CHOP)   │   │ Floor projection         │
                            └──────────────────────────┘   └──────────────────────────┘

A typical interaction chain:

1. A tracked joint (for example the right hand) comes from the sensor.
2. A virtual object (for example a sphere with a soft outer shell) is placed in the room.
3. CALC turns their spatial relation into one control value.
4. That value drives the content, which reaches the audience through the wall and floor projections and through ambisonics sound control. The sound component can also take the virtual object's position directly, so the object can be heard in the room.

Each note follows the same structure: an opening summary, role in the setup, technical specification, parameters, outputs, limitations and relations to the other components. The shorter user READMEs are stored next to each .tox and inside the example .toe. Other equipment in the lab (alternative sensors, controllers, software) and learning resources are listed in Inventory.

The components are distributed as one folder: an example .toe project file and a toxes folder that holds each .tox and its README.

• Keep the .toe and the toxes folder together in one project folder with a name of your choice. This makes it easy to move the project between your own computer and the lab workstation.
• You can rename the .toe freely. Do not rename the .tox files or their READMEs, because updates replace them by name.
• To update a component, download the new .tox and replace the old one in the toxes folder. Components are built so that the parameter values stored in your project survive the update.
• TouchDesigner does not autosave. Save often with Cmd/Ctrl + S. TouchDesigner keeps the numbered backups itself.

Revision, 2 October 2026 (SOP → POP).

• virtual_object and virtual_object_viewer now output POPs, which run on the GPU, so scenes with many objects are noticeably faster.
• Each virtual object has its own colour.
• Four separate calculation components (DIST_between_points, DIST_to_object, IN_or_OUT_object, FADE_to_sphere) were replaced by one component, CALC. Its output channels are center_dist, shell_dist, inside and fade.
• Patches made before this revision need to:
• swap the old calculators for CALC
• update their channel references
• give a POP to anything that expected a SOP

The components follow one lesson from running the lab: visiting researchers have limited time on site, and that time should go to the parts of the work that need the physical room. Interfaces stay stable and the lab can be simulated, so most of the setup can be done remotely. Users can work with the lab at different depths:

• treat each component as a black box
• open it up and adapt it
• skip it entirely and work with the raw data streams

• Component repository: <https://github.com/SpatialComputingL/Td-updates>
• The components are released under the Apache License 2.0. You may use, modify and redistribute them, including commercially, as long as you keep the licence and copyright notice and state any changes.
• Context: Abstract Spatial Computing Lab

Layer: spatial logic
In: position (CHOP, optional)
Out: shape (POP, blue), position + size (CHOP, green)

A virtual object is an invisible geometric volume (a sphere, box or cylinder) that you place in the physical room for other components to interact with. It does not draw anything itself. It is a spatial definition that other components can measure against, listen to or display. It is the basic building block of the lab's interaction logic: zones, triggers, attractors and sound sources are all made from virtual objects.

Virtual objects connect the tracked body to the content. A tracked hand has no meaning until it can be related to something. The virtual object provides that "something" in the shared room frame. Its shape output feeds Calculations and the Virtual object viewer. Its position output can place a sound source in Ambisonics sound control.

• Primitives. You can choose a sphere (radius), a box (width X, height Y, depth Z) or a cylinder. Only the size fields that belong to the current shape are active.
• Transform. Position XYZ and Rotate XYZ place the object in the room frame (metres; X = width, Y = height, Z = depth). If you wire a CHOP into the optional input, it drives the position, so an object can follow a tracked joint or an animated path.
• Soft boundary. For spheres, Outer shell adds a second, larger radius around the core. The space between the two radii is a transition zone that CALC uses for its Fade to sphere calculation.
• GPU geometry. Since the October 2026 revision, the shape is a POP (point operator), so geometry is processed on the GPU. Scenes with many objects stay responsive. Each object gets its own colour, which the viewer picks up automatically.
• Closed volumes. Distances and inside/outside tests only make sense for closed shapes. All built-in primitives are closed.
• Position capture. Get pos (shift + M) captures a position interactively instead of typing it.

• Blue (POP): the shape geometry, for CALC and the viewer.
• Green (CHOP): centre position tx ty tz and a size value taken from the radius. In Ambisonics sound control, this size value sets the source's Rangegain.

• Size fields greyed out: they belong to another shape.
• Outer shell greyed out: the shape is not a sphere.
• Preview looks squashed or off-centre: this is the preview viewport. Switch viewports.
• Nothing visible in the room: this is expected. The component only defines the shape. To see it, use the viewer or render it into a projection.

• → Calculations (input 2, typically)
• → Virtual object viewer
• → Ambisonics sound control (position and size of a sound source)
• ← Azure Kinect sensors (optional: the object follows a joint)

• User README: virtual_object-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

Framework: FilmEU, the European University for Film and Media Arts (FilmEU+, Erasmus+ European Universities)
Related research project: WIRE

FilmEU is an alliance of European higher-education institutions in film and media arts. Its Imagination Centre connects the partners' local labs, each with its own expertise:

• virtual production and motion capture
• immersive sound and Dolby Atmos
• digital fabrication
• generative AI

The Spatial Computing Lab is LUCA's contribution, and one of the first labs in the network that is fully operating. It brings a rare combination to the network: immersive projection on walls and floor, a 16-speaker third-order ambisonic sound system, and real-time body tracking in one room.

Within FilmEU the lab has two roles:

• education: students from across the alliance work in it through coursework, electives and joint programmes, such as the planned BA Music and Sound in Immersive Multimedia Environments
• artistic research: through the WIRE project, visiting researchers develop and present work in the lab

In June 2026, at the FilmEU Summit in Brussels, the lab hosted two WIRE pilot showcases:

• Agnes Meng (Lusófona University): Por Este Rio Acima, an immersive documentary installation on Northern Portuguese folklore
• Diego Barajas (Tallinn University): Eclosion, an interactive immersive experience

These visits shaped one of the lab's central principles: preparation should happen remotely. The documented components and simulation tools let visitors arrive ready to spend their time on site on the work that needs the room. FilmEU is also developing shared access and governance models, such as a token-based system for booking labs across partners.

Related: Abstract Spatial Computing Lab · Lab Tools

Partners: LUCA School of Arts, Corda Campus, imec
Format: artistic research trajectory through an open call
Role of LUCA / SCL: artistic and technological expertise, knowledge and feedback

Technology doesn't make art. Artists do. StageTech Flanders is a programme led by imec, supported by VLAIO and the Flemish government, in which makers, companies and higher-education institutions experiment with new technologies for the stage and for live experiences. LUCA School of Arts is one of its education and knowledge partners. The pilot phase (2026–2027) runs at Club Wintercircus, Corda Arena and deSingel, with five pilot projects on immersive audio, content creation for immersive experiences and stage robotics.

Within StageTech, LUCA, Corda Campus and imec run an artistic open call. Artistic questions lead, and production comes after experimentation. The first call (deadline 15 September 2026) supports a trajectory from October 2026. It has a project budget of €30,000 and ends in a public presentation at the Corda Arena, planned for April 2027.

Layer: output (image)
In: 1 TOP
Out: projection window + resolution CHOP (resX, resY)

This component is the output stage for the lab's floor projector. It works the same way as Wall projection, but for a roughly square surface with no natural "up". The floor matters in spatial computing because it is where tracked bodies, virtual objects and projected images line up most directly. A position in the room's XZ plane corresponds to a point on the floor image.

The floor projection completes the lab's image output. It is especially suited to interaction based on location: zones, paths and traces that respond to where visitors stand. These are typically made by combining Azure Kinect sensors data with Virtual objects zones and Calculations.

• Resolution contract. As with the wall projection, the component publishes resX, resY as a CHOP that you wire into your source TOP.
• Recommended chain: moviefilein → fit → floor_projection. A Fit TOP before the component makes sure that content made for other aspect ratios (for example wide wall content) is fitted on purpose and not distorted.
• Home/lab switch. License off = reduced home resolution (within the 1280 × 1280 limit of the Non-Commercial licence). On = full lab resolution (Educational licence on the lab workstation).
• Orientation. The floor has no inherent up. By convention, orient the content towards where the audience enters (the south side, where the entrance is) and write the orientation down in the project documentation.
• Mapping to room coordinates. Because the room frame uses X = width and Z = depth, positions from the sensor and from virtual objects can be mapped to floor pixel coordinates with a linear scale to resX, resY. This mapping is the user's responsibility, as the component does not apply it.

• Resolution CHOP: resX, resY for the user's source TOP.
• The projection window on the floor projector.

• Wall content looks wrong: the aspect ratio is different. Add a Fit.
• Small resolution: License is off. Even at full lab resolution, the floor projector has a lower resolution than the wall projection, so avoid fine detail and small text on the floor.
• Image "upside down": the floor has no up. Orient the content towards the entrance and write it down.
• Nothing visible after a change: press Configure, then Open Window.
• Occlusion: visitors cast shadows on a projected floor. Take this into account in the content design, for example by keeping important information away from where people usually stand.

• ← any content TOP, e.g. driven by Azure Kinect sensors + Calculations
• ↔ Wall projection (same model)
• Hardware: Inventory (projection surfaces and projectors)

• User README: floor_projection-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

Host: Windows NUC, Max 9.1.5
Layer: output (sound, back end)
In: OSC positions (UDP 10000) + 16 audio channels over Dante
Out: 16 loudspeaker feeds + 1 subwoofer feed over Dante

This patch is the rendering engine behind Ambisonics sound control. It receives the positions of up to sixteen sound sources and their audio, encodes them into a third-order ambisonic sound field, and decodes that field to the lab's measured, irregular layout of sixteen loudspeakers plus a subwoofer, all in real time. Most users never touch it: the TouchDesigner component is the interface, and the renderer runs on a dedicated machine. Splitting the work this way keeps audio rendering apart from the graphics and tracking load on the main workstation. It also keeps the decoder calibration in one place, independent of any project.

The renderer is the second half of the lab's spatial sound chain. Ambisonics sound control decides where each sound is and what it plays. The renderer turns that into speaker signals for this particular room. Because the calibration lives here, any TouchDesigner project that sends positions and audio in the expected format sounds correct in the lab without project-specific tuning.

1. Position intake (OSC). udpreceive 10000 receives the positions. The patch routes the addresses /tx1 /ty1 /tz1 … /tx16 /ty16 /tz16 (one triplet per source), a tracked listener /lx1 /ly1 /lz1, a global /gain_range, per-source ranges /range_gain1 … /range_gain16, and /_samplerate (information only).
2. Cartesian → spherical. Each source position (X = width, Y = height, Z = depth, metres) is converted to azimuth, elevation and distance relative to the reference listening point (X = 0, Y = 1.5, Z = 2.5). A sign-preserving minimum-magnitude floor on the depth term removes the singularity when a source is at exactly the reference depth.
3. Gain. Each source's level falls with its distance to the listening reference. The gain range is the distance in metres at which the level reaches zero. It is set globally (/gain_range) or per source (/range_gain1 … /range_gain16); per source, it is fed by Rangegain in TouchDesigner. The reference is either fixed or tracked. An inversion toggle mirrors the tracked listener through the room centre to correct for the orientation of the tracking. Direction never comes from the listener, because listener orientation cannot be tracked reliably. A minimum floor on the range parameter prevents a division by zero while the patch loads. A global "gain active" switch sets all sources to full gain for direction-only tests.
4. Encoding. ambiencode~ 3 16 (ICST) encodes the 16 audio inputs (adc~ 1–16, from Dante) into a third-order AmbiX field: 16 channels, ACN order, SN3D normalisation.
5. File input. Two looped players (buffer~ + groove~, 16 channels each) add pre-encoded third-order B-format WAV files to the field before decoding. Each player has its own toggle and is off by default. (In development.)
6. Decoding. ambidecode~ 3 16 (ICST) decodes the field to the 16 measured loudspeaker positions. Each speaker is defined by azimuth, elevation and distance in a decoder grid that can be edited live.
7. Subwoofer. The omnidirectional W component (ACN 0) is taken from the encoder, low-pass filtered (lores~, 100 Hz) and sent to channel 17.
8. Output. dac~ 1–17 sends the speaker feeds to Dante.

Speaker positions were measured in the room and converted to decoder coordinates with a purpose-built interactive calibration tool, which has a top-down view and an elevation view. The tool exports azimuth, elevation and distance per speaker. These values are typed into the decoder grid, take effect immediately, and need no new patch version. The encoder's speaker-matched default positions (used in calibration mode) are derived from the same data, so the two stay consistent.

Convention: azimuth 0° points from the listening point towards the north wall. Positive azimuth turns towards +X. Z increases towards the entrance. The table also appears in the Lab model.

A debug panel shows each source's tracked position, its distance to the listener and the gain value sent to the encoder. This makes reactivity problems visible without console output.

• Re-evaluation in expr. In practice, Max's expr only reliably fires on its leftmost inlet. The distance and listener-blend expressions therefore route all their inputs through pak/unpack, which forces re-evaluation whenever any input changes.
• Sample rate. All content has to arrive at 48 kHz. Dropouts with 44.1 kHz material turned out to be a resampling problem on the sending side, not in the renderer.
• State. An encoder instance that has been corrupted by invalid coefficients does not recover when the patch file is reopened. Max has to be fully restarted. The same applies to stale test-audio buffers.

• Directional rendering. Checking the panning across this irregular layout is ongoing work. A controlled comparison with a regular reference layout is planned.
• Listener model. There is one listening reference. Multiple listeners and listener orientation are outside the current scope.
• Built-in distance model. ICST's own distance attenuation is disabled in favour of the custom gain model above. It remains an option.
• Speaker aiming. The decoder stores position only. It does not store per-speaker aim (yaw/pitch).

• A Dante input for B-format audio rendered in TouchDesigner, mixed in at the same point as the file players.
• A control interface in TouchDesigner, so live operation no longer requires editing the Max patch.
• A head-tracked headphone mode ("Augmented Audio Reality") alongside the loudspeaker array.

The renderer is not part of the public Td-updates repository. It runs on the lab's NUC and is maintained by the lab.

• ← Ambisonics sound control (OSC positions, Dante audio)
• ↔ Lab model (speaker positions)
• Overview: Lab Tools
• Hardware: Inventory (loudspeakers, renderer computer, Dante network)

• ICST Ambisonics externals for Max: Institute for Computer Music and Sound Technology, Zurich University of the Arts.
• AmbiX format: ACN channel ordering, SN3D normalisation.

Layer: output (sound)
In: one position per source (CHOP) + audio (CHOP)
Out: third-order ambisonic mix on the lab's 16 + 1 loudspeaker array

This component places sound sources in the three-dimensional space of the lab. Each source has its own position, size and audio input, and the component renders them through the lab's third-order ambisonic loudspeaker system. The user works only in TouchDesigner: wire in a position (for example a Virtual objects or a tracked joint), point to an audio signal, and set a range. Encoding, decoding to the irregular speaker layout and transport over the Dante network are handled by a dedicated rendering chain in Max/MSP. This gives everyday users of the lab position-driven spatial sound, which would otherwise require specialist knowledge of ambisonics.

This is the sound counterpart of Wall projection and Floor projection. It is where the spatial logic of a project becomes audible. A virtual object wired into a source makes that object heard at its location. A tracked joint makes a sound follow a body.

• The ambisonic mix of all sources on the speaker array (16 + 1 channels).
• A 3D view inside the component: one sphere per source, sized by Rangegain.

• Posin shows the middle of the room: nothing is wired into that source's inlet.
• Rangegain is a plain number instead of a bind: nothing is wired in, or the input is not a virtual object.
• A new or removed source takes a moment to appear: the script rebuilds about once per second.
• A manual rewire inside the component reverts: the script restores the internal chain. Never rewire it by hand.
• Sample-rate mismatch: 44.1 kHz material has to be resampled to 48 kHz before it reaches Dante. The component does this for its own inputs.
• Single listening reference: the decoder is optimised for one central listening area. Listeners far from the centre will hear the spatial image less precisely, which is normal for loudspeaker-based ambisonics on an irregular layout. Checking directional rendering on this layout is ongoing work.

Planned work includes:

• a TouchDesigner control interface for the renderer
• a direct Dante input for B-format audio rendered in TouchDesigner
• per-speaker aiming data
• a headphone-based, head-tracked "Augmented Audio Reality" mode alongside the loudspeaker system

• → Max-MSP ambisonic renderer (OSC positions + Dante audio)
• ← Virtual objects (position + size, via the green output)
• ← Azure Kinect sensors (position)
• ← Calculations (control values, e.g. to drive audio parameters)
• Hardware: Inventory (loudspeakers, Dante network)

• User README: Ambisonics_sound_ctrl-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• ICST Ambisonics externals for Max (Zurich University of the Arts)
• Overview: Lab Tools

This note lists the software, hardware and applications available in the Spatial Computing Lab, together with the learning resources that support them. It starts with the fixed installation, which is the hardware the Lab Tools components are built for. After that come the equipment around it (alternative sensors, interfaces and capture devices) and the software used for content, mapping and sound.

Software that runs in the lab. TouchDesigner runs the lab's own components (Lab Tools), and Max/MSP runs the Max-MSP ambisonic renderer.

Small software, plug-ins and apps.

External links for inspiration

• OLIB: TouchDesigner components
• Derivative: TouchDesigner download, tutorials and forum
• AllTouchDesigner: learning community
• YouTube channels: The Interactive & Immersive HQ, Bileam Tschepe, Matthew Ragan, Supermarket Sallad, Torin Blankensmith

Learning material (on the lab's SharePoint, for LUCA users)

• TD Fundamentals 100 and 200
• TouchDesigner POP examples and Learning About POPs
• The Interactive & Immersive HQ templates
• a TouchDesigner book and a beginner course

Snippet library (on the lab's SharePoint, for LUCA users)
A collection of small TouchDesigner examples and community components. It includes MediaPipe, multi-Kinect calibration and merging, OSC tools, UI layouts and utility patches such as counters, value mapping and camera fly-throughs.

• Lab Tools (core components, which run on this hardware)
• Azure Kinect sensors, Azure Kinect cameras, Kinect host (tracking hardware)
• Wall projection, Floor projection (projection hardware)
• Ambisonics sound control, Max-MSP ambisonic renderer (sound hardware)
• Lab model (where everything is in the room; 3D scanning apps can capture it)

Layer: spatial logic (verification)
In: positions (CHOP), shapes (POP)
Out: 3D preview

The viewer shows tracked points and virtual objects together in one 3D scene of the lab. Its only job is to check that everything is where you expect it to be. It does not change any data. The physical room is otherwise blind to what the system "thinks", and the viewer makes that state visible. Most spatial interaction design depends on this view for debugging and for explaining a setup.

Interaction logic in the lab is invisible: virtual objects have no physical form, and tracking errors cannot be seen in the room. The viewer is where the sensor layer and the spatial-logic layer can be checked together before content is connected to the outputs.

• Scene. A wireframe box shows the physical room. Inside it, every tracked point wired into the green inputs is drawn as a sphere, and every Virtual objects in the project is drawn in its own colour.
• Automatic discovery. Virtual objects in the project are found automatically. They do not need to be wired one by one.
• Viewports. The perspective view gives a quick overall check. The top view works as a live floor plan, which is useful for placing zones relative to the entrance and the projection surfaces.
• Read-only. The component has no side effects. You can add or remove it at any time without changing a patch's behaviour.

• Preview: a 3D render of the scene. The node preview is low resolution; open a larger viewer when you need detail.

• Everything in one spot: nothing is wired, or the sensor isn't running.
• Spheres frozen: the source is frozen. Check the sensor.
• A sphere outside the box: the position is out of range, or outside the sensor's field of view.
• Low-resolution preview: this is a node preview. Open a bigger viewer.

• ← Azure Kinect sensors (green inputs)
• ← Virtual objects (automatic)
• ↔ Calculations (the viewer shows the geometry that CALC measures)

• User README: virtual_object_viewer-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

Layer: output (image)
In: 1–3 TOPs
Out: projection window + resolution CHOP (resX, resY)

This component is the output stage for the lab's wall projectors. It opens the projection window, sends the images to the wall surfaces, and tells the user the exact resolution the source images must have, both at home and in the lab. The resolution comes from the component as a CHOP that the user wires into their own source. This keeps content and output in sync by design, so there is no stretching, cropping or black bars.

Together with Floor projection and Ambisonics sound control, this component forms the lab's output layer. Any image from a project, whether generated, filmed (Azure Kinect cameras) or controlled by interaction logic, reaches the walls through this component.

• Projection modes.
• Three screens (default, both switches off): three separate images on the left, centre and right walls (inputs 1, 2 and 3).
• Panoramic projection: one wide image across the whole wall surface from input 1. Inputs 2 and 3 are ignored.
• Single projection: output to one projector only, for tests and small setups.
• Wall layout. The three channels correspond to the three fixed walls: left, centre and right as seen from the entrance (west, north and east). The fourth (south) wall is mobile and is not installed by default. When a project uses it, its output is set up for that project and is not part of this component's standard configuration.
• Resolution contract. The resolution CHOP (resX, resY) always gives the resolution for the current mode and licence. Users wire it into their source TOP instead of typing numbers. If the mode or licence changes, the content follows automatically.
• Home/lab switch. License off gives a reduced home resolution that stays within the 1280 × 1280 limit of TouchDesigner's free Non-Commercial licence. On gives the full lab resolution, which the lab workstation's Educational licence allows. The patch stays the same either way.
• Fit Input sets what happens to an image of the wrong size, as a safety net. Following the resolution CHOP is still the preferred way to work.
• Window management. Open Window and Close Window switch the projection on and off. Keep the window closed while editing, so that it does not take focus from the TouchDesigner editor.

• Resolution CHOP: resX, resY for the user's source TOP.
• The projection window on the wall projectors.

• Small resolution: License is off (home setting).
• Inputs 2 and 3 do nothing: Panoramic is on.
• Stretched, cropped or black bars: the source isn't following the resolution CHOP.
• Content on the wrong wall: the input order is left, centre, right.
• Nothing visible after a change: press Configure, then Open Window.

• ← any content TOP, e.g. from Azure Kinect cameras or controlled by Calculations
• ↔ Floor projection (same model, different aspect ratio)
• Hardware: Inventory (projection surfaces and projectors)

• User README: wall_projection-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

Host: Intel NUC, TouchDesigner
Layer: sensing (back end)
In: 3 × Azure Kinect
Out: fused skeletons (OSC), camera images (NDI)

The Kinect host is the computer that the three Azure Kinect sensors are connected to. It runs a TouchDesigner project that reads each sensor, calibrates its data into the room's coordinate frame, decides which bodies seen by which sensors belong to the same person, fuses those views into one skeleton per person, and publishes the result to the lab network. Users never open it. The Azure Kinect sensors and Azure Kinect cameras components on the workstation are its clients. Keeping sensing on its own machine means the processing for depth, body tracking and video encoding does not compete with rendering on the main workstation. It also means every project in the lab receives the same calibrated data.

The host turns three independent sensors into one room-wide tracking system. It does three things:

• Calibration: each sensor's data is moved from the sensor's own viewpoint into the shared room frame (metres; X = width, Y = height, Z = depth; Z = 0 at the centre wall).
• Decision making and fusion: every frame, it decides which detections are the same person, which person they are, and which sensor to trust for each joint. Each person appears once, with a stable number (p1 … p8).
• Distribution: every joint of every person goes out over OSC, together with presence flags and a heartbeat. Camera images go out as NDI video streams.

The host deliberately makes no choice on behalf of the user. It always sends everything. Which person and which joints a project uses is decided on the client side (see Azure Kinect sensors), so several projects or users can use the same stream at the same time.

The Azure Kinect DK went out of production in 2023, and an Orbbec sensor is on the lab's purchase planning for 2026. The options discussed at the workshop meeting of 24 September 2026:

Working recommendation: three Femto Bolts, a sync hub, three tested active USB-C cables and a PCIe USB card with a separate controller per port, on a Windows host. Because the fusion works on room coordinates, swapping the sensors changes nothing for the clients or for projects built on them.

• Calibration is static. The poses are fixed values in the Transform CHOPs. If a sensor is bumped or moved, its pose has to be re-measured, or the fused skeletons drift apart. The left sensor currently differs most from the other two (see Calibration).
• Left/right swaps. Body tracking assumes the person faces the sensor. Seen from behind, or when someone pulls up their shoulders so the neck "disappears", a sensor can return a mirrored skeleton with left and right exchanged. A correction based on the shoulder axis and face-joint confidence was tried and reverted in September 2026 because it didn't improve real results.
• Skin, hair and clothing. The depth camera measures reflected infrared light. Dark hair, long hair (which also hides neck and shoulders), darker skin and dark clothing return less of it, so skeletons are less reliable for those people. This is a limitation of the sensor type and needs testing with sensor placement and settings.
• Close contact. People closer together than the merge distance can be merged into one person. When they separate, one of them gets a new number.
• Numbers are tracks, not individuals. Someone who leaves and comes back may get a different number. Content should not assume that a particular visitor keeps a particular number for a whole session.
• Hands. Hand joints are rated uncertain by the Kinect most of the time, so they are noisier than torso and head. Wrists are a steadier alternative.

• → Azure Kinect sensors (OSC skeletons)
• → Azure Kinect cameras (NDI images)
• ↔ Lab model (sensor poses)
• README: HOST.onstartup-README
• Overview: Lab Tools
• Hardware: Inventory (sensors, host computer and network)

Layer: sensing
In: none
Out: one CHOP per selected joint (tx ty tz, metres)

This component gives live three-dimensional positions of human body joints, measured by the lab's Azure Kinect sensors. You choose which joints you need, and each one gets its own outlet as a CHOP with the position in room coordinates. Several people can be tracked at the same time (active since the first week of October 2026): each copy of the component follows one person. When you are away from the lab, a mouse mode stands in for the tracked body. This lets you develop and test interaction logic without sensors, and the patch keeps the same structure in the lab.

The component is the main way bodies enter the system. Its outputs feed:

• Calculations, to relate a body to a virtual object
• Virtual object viewer, to check the tracking
• Ambisonics sound control, to place sound sources on a moving body

It is the client of the Kinect host. The host decides who is in the room and sends every joint of every person. This component decides what this project uses: which person, which joints, and what to output when there is no usable data. It covers position only. For images from the same sensors, use Azure Kinect cameras.

• One outlet per slot, in slot order, each with channels tx ty tz in metres.
• The component's viewer shows the chosen joints with their full names (p1/head:tx, …).
• Recommended convention: attach a null named after the joint (null_head, null_handl, …) to each outlet and reference that null downstream.

• Values at 0 or at Position: you are outside the lab, nobody with that Person number is in the room, or no new data has arrived for 0.5 s. Turn on Mouse to keep working.
• Numbers are tracks, not individuals: someone who leaves and comes back may get a different number. When person 1 leaves, person 2 keeps their number and p1 stays empty until someone new arrives. How numbers are assigned is explained in Azure Kinects calibration.
• Hands are less steady than head and pelvis: the sensors rate hands as uncertain most of the time. Wrists are steadier and move almost the same.
• Left/right swaps when someone turns their back to the sensors, and less reliable tracking for dark or long hair, darker skin and dark clothing: limitations of the sensors, see Azure Kinects calibration.
• External .tox: the component is loaded from Azure_Kinect_sensor.tox, and copies are loaded from that file. After changing the component, save it back to the .tox from the original, never from a copy, and make new copies.
• Hardware lifecycle: Microsoft stopped producing the Azure Kinect DK in 2023. In the long run the lab will need a successor sensor. Because the component's output is a sensor-independent CHOP interface, patches built on it should keep working when the hardware changes.

• → Calculations (input 1, as a point)
• → Virtual object viewer (green inputs)
• → Ambisonics sound control (source position)
• ← Azure Kinects calibration (OSC skeletons)
• ↔ Azure Kinect cameras (same sensors, image stream)
• Hardware: Inventory (the installed sensors, and other sensors available in the lab)

• User README: Azure_Kinect_sensor-README
• Repository: <https://github.com/SpatialComputingL/Td-updates>
• Overview: Lab Tools

This 3D model is an interactive spatial reference for the Spatial Computing Lab. It shows the three fixed projection walls, the floor projection, the mobile fourth wall, the ceiling grid, the sixteen loudspeakers, the Azure Kinect sensors and the reference listening point, all in the room coordinate frame used by every lab component. It is meant to answer where is what? before a visit: to plan where zones and virtual objects go relative to the walls and the entrance, to understand the speaker layout behind Max-MSP ambisonic renderer, and to compare positions in TouchDesigner with the physical room.

Approximate elements are drawn faded and labelled approx. in the viewer. The subwoofer is not drawn: it radiates omnidirectionally and its position has no effect on decoding.

Metres. X = width (west −, east +), Y = height, Z = depth. The origin is on the floor at the middle of the north (centre) wall. Z increases towards the entrance (south). This is the same frame used by Azure Kinect sensors, Virtual objects and the sound system, so a position read in the viewer can be typed straight into a component.

• Views:
• 3D overview
• Floor plan: top-down, with north at the top
• From entrance: what a visitor sees on entering
• Listener: from the reference listening point, looking north
• Layers: walls and floor, mobile wall, loudspeakers, sensors, ceiling grid, helpers and labels can each be switched on or off.
• Details: click any object to see its coordinates. For speakers this includes azimuth, elevation and distance to the listening point.
• Download .glb: exports the geometry as a standard glTF binary, for use in Blender, game engines or other 3D tools.
• Add ?view=top (or entrance, listener) to the URL to open the viewer on a specific view.

• One small file. The model is not a stored mesh. It is built in the browser from a short data block at the top of the HTML file, with three.js loaded from a CDN. The file is about 15 KB, so it hardly uses any of the website's storage. To update a position, edit the numbers in that data block. Nothing else needs to change.
• Not a scan. The model is schematic. A photogrammetric or LiDAR scan of the room can be added later as a .glb and loaded into the same viewer.

The interactive viewer is next to this note on the map.

• Lab Tools (system overview)
• Max-MSP ambisonic renderer (speaker table)
• Azure Kinect sensors, Wall projection, Floor projection