If you can own one magic object from the world of Harry Potter, what would be?


My answer would be the PENSIEVE.


It preserves our precious memory clips, good or bad in four dimensions.


Makes them shareable, modifiable. With it, we can relive.






"I use the Pensieve. One simply siphons the excess thoughts from one's mind, pours them into the basin, and examines them at one's leisure."

—Albus Dumbledore



- PENSIEVR -


ADVISOR
Panagiotis Michalatos | Allen Sayegh
The need for experience capturing foster the development of relevant technology, and vice versa. At the same time, technology renders the memory recording process more and more impersonal. But even with the most recent technology, we never stop to be selective in the genre and angle of our capturing.

This project aims at creating a new art form with emerging technologies by exploring the hidden law behind the development of the expressive medium and producing a series of interactive experience.
Learning from the history, no matter how technology evolves, the fundamental way we en- dow technology with the meaning of life never changed. Even though it has already empowered us to capture and mimic the actual reality, we sometimes still tend to use blurred, abstract form as expressive medium.



From this perspective, this project explores the possibilities in simplest geometry -- dot as the paintbrush of a new visual expressive medium. The Legos of this new representation of experience will be point clouds, captured from one's subjective vision, and affected by various environmental parameters and user input.



WHEN AR GOES WILD

Inspired by Keiichi Matsuda's video of "HYPER-REALITY", which depicted a future an imaginary future when augmented reality has fully bridged the information gap between tangible and virtual world. With the Apple ARKit and Google ARcore,it seems there is a great chance that this vision is coming to the realm. If we are inevitably going to live in a world where augmented reality has been carried to an extreme, the cameras on these devices are going to record every single frame of our daily experience. And with these consecutive frames, we are able to reconstruct these experiences.

CAPTURE AND RECONSTRUCT

With current technologies (e.g. structure from motion), by analyzing the frames captured by the cameras, computers are able to process the pixel data and reconstruct the 3-dimensional environment and the tracks of the camera motions.


WHY PARTICLES?

Even though technology has already empowered us to capture the actual reality, we sometimes still tend to use blurred, abstract, emotional medium to store our experience. From this perspective, this project will explore the possibilities in simplest geometry -- dot as the paintbrush of a new visual expressive medium. The Legos of this new representation of experience will be point clouds, captured from the subjective vision, and affected by various environmental parameters and user input.
The criteria of the interaction design are to trigger the spontaneous reaction as much as possible. Comparing to all other display methods, Virtual Reality(VR) could bring a stronger sense of existing, strengthen the measurement of the environment, hence allow more retrospective elements to merge and intersect. The spatial tracking technology frees the users from traditional input devices like mouse and keyboards and enables natural body interaction.
Same with the application in the capturing process, eye tracking as control are more intuitive than convention gaze input in the user experience design in VR. The customized eye tracking module, plugged into the VR headset, potentially empowers the environment to read our focal point and attention level.
In the book of Listening and Its not, the author, Patrick Famer, invited a group of people to visit 10 miles due north of their homes and describe the audio cues in words. Objectively, it transforms the experience into an intriguing form of representation. Some subjects to polish their words as a whole, however, some tend to write down discrete keywords or phrases to extract a series of sections from memory and still kept them readable. These lines show the potential of using words and voice as user input to design a various nat- ural interactions.
Moreover, I prototyped a glove controller for in-VR interaction. The glove could bind with the functions on a regular VR controller and allows more instinctive input from the user to the environment virtual space. It also has potential to trigger intimate interaction among users when more than two users are sharing or combining their experiences.

- MORE -
As an expressive medium, it will morph with the constantly evolving technology like all precedents. By far, capturing and recreating the experience in this project are mainly from the visual perspective. The development of technology makes capturing all fives senses and even emotion possible.

Meanwhile, this medium potentially benefits from the emerging augmented reality(AR) technology. Imagine the world where video cameras walk with humans eyes everywhere, capturing visual, emotional, environmental data for us, recreating our experience in real-time, that device is indeed another version of us. The ultimate vision of this project lays in the same piece of technology and same pool of information.