Residency
31 août 2026

Research Residency

In August 2026, we were invited for a two-weeks residency at Sporobole (Sherbrooke, CA). We started to experiment with It Moves Like You, showing a playable public demo at the end of the residency there. The idea here is to dig into possibilities for the HORIZON project we’re part of, REALM. REALM explores issues relating to distributed embodiment, identity instability, and new types of interactions in the world of extended reality (XR), screens, and physical space.
We would like to thank Sporobole for hosting us and Moverse.ai for supporting us. We had a great time with the nicest people! Québécois hospitality is very real.

Research
Our main research axes were:
-> Asymmetry: screen-based and VR participants with different access and control. The immersant is the performer in that case.
-> First-person characters in VR, using markerless motion capture (with our partner moverse.ai) and hand tracking.
-> Distributed embodiment: one character controlled by multiple people, or vice versa.
Visuals were inspired by our working environment at the moment, using recolorored 3D scans of elements around Sporobole and Sherbrooke.

Storytelling

Imagine you get ten million dollars and immortality, provided that a super-intelligent immortal snail will keep approaching you, and touching it means death. We have a very concrete and persistent goal: to stay away from it. In XR, movement guidance is a notoriously hard task: instructions are not followed, not understood or actively resisted. However, under pressure, the body finds its way before cognition does.
We use this “snail problem” as our minimal narrative startup, in order to research the impact of intention, fear and anticipation on movement in an immersive environment.

A demo in three acts

At the end of the residency, we demonstrated a short showcase in three acts.
First, a race: three players use gamepads to represent the snails that run along the power lines towards the Sporobole building.
Then, a negotiation: the two winners are brought into a terrarium. The loser is brought to the middle of the room, in a VR headset and finds himself or herself in the same world with the full body using motion capture. Using text chat, the two captive snails try to persuade the immersant to release them and get rid of their rival.
Finally, a guided walk: the chosen snail leads the immersant to the giant shell along the spiralling path. Once they arrive there, the shell expands to encompass the whole player, while the snail grows even larger. Now it is in a position to eat the immersant or let them live.

Technicalities
The combination of Moverse.AI’s markerless full-body motion capture with the Pico headset’s hand tracking provided the user with a body that could be seen: hands, torso, legs, mirroring the immersant’s actions in real-time. This also revealed certain current limitations of using two tracking sources at the same time: latency issues when prioritizing quality over speed, or a jittery body if speed is the priority. Another issue was that our storytelling and gameplay were not yet intertwined enough.
There are still some pending interactive possibilities. Could other people in the motion capture space interact with the person in VR? Could the immersant be confined to the motion capture space only, or can he travel freely anywhere? How can gamepad- and text-controlled characters or interactive cameras contribute more to this multimodal system?

Sporobole & CSJB, Sherbrooke, CA
6 août 2026–18 août 2026