2023
In collaboration with Paul Wang

[Detours & Deviations is an ongoing project. This post only shows its early development. All updates can be found on the project website.]

Detours & Deviations is a dynamic world perpetually reshaped by memories. Players traverse a landscape, where their actions give birth to “memories” that, in turn, influence the evolving environment. Different experiences converge and intertwine, creating a collective journey that never ends. The project’s ambition is to manifest the evolution of memories through AI-driven, procedural spatial reconstructions, and to allow players the freedom to explore an ever-evolving narrative woven with their own and others’ memories.

The perpetual transformation of the space is closely tied to the presence of agents, referred to as “actors,” who serve as custodians of various scenes, encapsulating and gathering memories imbued with subconscious behaviors. These actors possess distinctive personalities and methods for shaping their surroundings. Each begins their journey within a designated region but may venture into other territories, either to spark events or influence the inclinations of fellow actors.

THE OPENING SCENE

The opening scene, like many other scenes and assets in this project, was created by digitizing physical objects/models. For this particular scene, we scanned a physical model with photogrammetry, before combining the scanned model with video recordings. Since the scene is “projected,” the player can easily sense the distortions and mismatches once they move away from the center of the projection (where we put the camera). This approach intends to blur the distinct sensory/experiential differences between physical and digital environments.

THE ACTORS

Ahmed

In Ahmed’s domain, every action by players becomes a catalyst for the perpetual metamorphosis of the environment. Ahmed captures and compiles memories infused with players’ subconscious behaviors—ranging from movement speeds to footsteps and pauses—and translates them into spatial tendencies to create a spatial memory. When Ahmed sparks events in other realms or influences fellow actors, players may encounter imperfect copies of elements they observed before, as Ahmed prefers to record things in a rather casual manner.

m23

As players immerse themselves in m23’s “ocean,” they observe a dynamic interplay where the seabed continually shifts in response to their movements. Overhead, the diver actor accompanies them, reshaping the topography using displacement maps derived from shadows. When m23 interacts with other actors or explores different territories, expect the topographies to undergo transformative shifts, potentially metamorphosing the surroundings into an underwater-like environment.

The man with three names

As players venture into the ruins of the man with three names, their movements initiate the mirroring of paths, each one rescaled and transformed uniquely. This mirroring effect is orchestrated by three distinct yet interconnected entities, each displaying a unique movement behavior and transformation method. Together, these entities render the surroundings simultaneously recognizable and unfamiliar. 

Drawings & Diagrams

In Detours & Deviations, the scenes and their “acts” are developed with “portraits” (artifacts like drawings and models) of specific memories and places. The process of portraying involves exploring materialities and performance. The events of representation were digitized and compiled into the virtual environment by intentionally including the physical qualities of the hand-built artifacts and the psychological aspects of the performance.

Ahmed is an actor that traces a pattern but they do that in a rather casual manner which results in misalignments and transformations. The portrait of Ahmed is made by tracing a rectangle over and over again. Each time the tracing referred to the previous one but shifted a little. The more tracings there are, the easier it gets to lose track of the lines, and that influences the consistency of the tracings. The coverage of the tracings ultimately becomes a floor plan of a gallery. Ahmed collects data such as player traveling direction, velocity, and attention, and passes it to the space itself, which contains a series of boolean operations.

m23 comes from an experience diving in Aqaba. The actor perceives the space through a diving mask and their visibility is affected by the sand in the water. They learn about topography and depth by reading the various shades of blue. m23 uses displacement maps to generate topography. A movement model helps them predict the player’s location through the best-fit curve and then projects their displacement in relation to the player on the UV tile of the ground.

The man with three names always lives in memories. When they recall their past they are like looking through a rearview mirror. They only see one side of things and they try to fill in the side not seen with their assumptions and imaginations. We built a model and scanned the model with mirrors. Photogrammetry has a tendency to read mirrored images as an actual space and they construct the space using what they can “see” in the mirror and what they believe may look like on the other side. 

In terms of Actor behavior, we implement a primitive form of swarm intelligence derived from collective movement found in nature, such as a school of fish. movements such as swarm, polarize(directional) movement, and milling. All three agents are aware of the other agent’s location and pass relevant information about the player to each other. The spaces become increasingly fragmented through their manipulation of location, rotation, and scale.