Please access the final design document HERE.
Exhibition spatial setup
There are three main stages to experiencing this project: Spectating —> Viewing —> Examining. The first stage is observation from afar. At this stage, the audience would notice the device and its display, and can see how others are interacting with it. They can see that there is a main display, and may notice that there is a secondary reflective surface parallel to ground. As they begin to get interested in the installation, the second stage is observation in close distance, yet without a specific task. The audience is now directly interacting with the device, they are touching the device and feeling the physical controls (such as handles) on it. They are viewing the simulation, and trying to rotate the device with the handles. The final stage is observation specifically with regards to the dual imagery effect. The audience would closely examine the intersection of the mirror and display, and begin to notice the difference between the two worlds.
Prototyping
Thesis Pop-Up Show
Pop-Up Setup diagrams
Interactive Scenes
User Scenarios
For the complete user scenario design document, please visit this link.
Group1
This group of people is your average museum and gallery goers. They are interested in art and culture on a casual level, and would appreciate seeing art of all forms as they do not invest their time into researching and reading about the art world. They would be intrigued by new media art, but would not necessarily recognize the technology it employs until that medium has received considerable recognition (like VR). They sometimes see the value of new media art in the way it’s presentable in the context of social media.
Group2
This group of people are new media artists and art enthusiasts. They have either read about or worked extensively on new media art. They are familiar with the industry standards and would have no problem relating what they see to other creative technology projects. They have the ability to speculate, if not instantly recognize, the technology that is involved in the making of this installation. They are familiar with the use of Kinect, Leapmotion, Arduino and other software and hardwares, and will recognize demo creative coding patterns. They will not be solely impressed by the technicality, and they seek to evaluate and challenge the concept and how it fits the delivery of the project.
Group3
This group of people are of tangential interests to the installation. They are namely museum curators and grant makers, who are interested in showing or funding new media artwork similar to this project. They are greatly familiar with different forms of interactive installation artwork and new media. They will be my potential collaborators/employers.
I have gathered a list of the events and conferences where I could potentially apply to showcase my work. They all share a focus in showcasing digital, new media artwork. Some events including the NYC Media Lab are oriented around student work from university and institutions, others such as SIGGRAPH and Ars Electronica are created around practising artists. I have also included overseas opportunities such as ARTECH and Reasons to:.
Domain & Precedent research
Access the Domain & Precedent research paper HERE.
Prototype 1b
Design question:
How do people translate immediate sensory inputs and use the result to mentally construct an environment around them?
Prototype 1b
In this iteration, I will start engaging with my subjects in a common area, blindfold them and bring them into a room, have them wear earplugs and conduct the following experiment. After the experiment, I will first interview my subjects. I will then ask them to take on my role of performing the same experiment, with the next subject.
Setup:
- The subject will be brought into a room and stand on a slightly elevated platform, small so they can feel the edges easily and should not leave the platform.
- I will instruct them to use their sense of touch, then have them wear earplugs for the duration of the experiment.
- They will be entrapped on 3 or more sides by cardboard, which is cladded with various materials.
- As they touch each wall, I will use various instruments to hit the surfaces around them to create senses of vibration through the connected surfaces.
Props:
- A structure that encapsulates the subject on at least three sides (cardboard w/ dowels)
- Textures: felt, reflective paper, cardboard, mirror;
- Props: bouncy ball, wool roll, duct tape,
Findings:
- The subjects find it harder to visualize the objects from their sound, than the other way around--looking at an object and trying to imagine the sound it makes.
- I put too much emphasis on the guided blind-tour before the actual exercise. However it did provide a good segway for people to transition mentally into an unfamiliar space.
- The subjects did not establish any kind of correlation between the textures they touch and the sound they hear. They sometimes would take the sound as instruction or encouragement for them to continue to move.
- The various sounds do inspire people to visualize objects in their head, which are often inaccurate.
- People tend to understand enclosure better as they sense a familiar form (for example a cubicle).
Next steps
In the next prototype, I would like to create a environment that has disorienting visual clues that are constantly changing and/or responsive. This could be in a VR space or a physical space where the subject's vision and movement are limited.
Prototype 1
Concept statement (Sep 19th)
This thesis project is an exploration on the terms “unreal realities” and “real unrealities”. I plan to create an experience in which the audience encounter a series of choreographed sensory inputs, in an unusual combination which causes cognitive dissonance. In the form of an interactive installation, this project aims to examine the idea that we are subconsciously constructing the reality we are a part of. As a result, I would like my audience to leave with a mindset of skepticism for their old practice of perception.
Stemming from this updated concept statement, I created my first prototype which focuses on the following design question:
How do people translate immediate sensory inputs and use the result to mentally construct an environment around them?
In this iteration, I invited my test subjects to join my on a journey, blindfolded. They were led from their various original location on D12 into a quiet room, instructed to sit still in a spot but free to move otherwise. The only sense they have to refrain from using is sight.
The series of Inputs I provided were:
- Clinking sound from a glass bottle, far away
- Push vertical boards close to the person to create a cubical
- Fan right next to the person to create a breeze, from the direction of where the person touched the cubical
- Light the lighter
- Sprinkling water from a wet napkin
I ran four tests total, on 2 DT students, 1 non-DT student, and another DT student. After the experience, I led them back to their original spot, still blindfolded. I then asked them to visualize the environment they just experiences both in words and in sketches. See below for the form I used for their response:
I conducted my interview right after each test-run, so I was able to adjust the next accordingly. Some changes I made in the process were:
- I started taking detours and span the subjects around, so the ones that are familiar to the floor plan of D12 would lose their sense of orientation.
- I added more events so that there are inputs from 360 degrees around the subject.
- I moved the close by chairs after they sat down to "trap" them in their seats.
A summary of interesting findings:
- Our sense of touch can sometimtes be incredible inaccurate in the way it helps us construct a mental image of objects.
- The most immediate input may overwhelm one's other senses and cause one to become unaware of other inputs, especially if the most immediate one is an input that you do not recognize.
- We are sensitive to the voice of others talking, 2 out of 4 subject described the room as a "get-away, meditation space" far away from the crowd on D12.
- The sense of time got elongated during this experience.
- One felt the room was a radically different size, and another thought there were multiple people in the room.
Prototype 2 proposal
In the next iteration of my prototype, I would like to further challenge the audience's control of their senses, and give them an opportunity to digest the disconnect between their perception and the physical environment. The way I'm planning to do so it:
I would instruct the subjects to wear a blindfold and earplugs, bring them into a room and do a choreographed experienment similarly. From then on, I would bring the subject back, interview them, and ask them to join me and perform the same experience to the next subject. Afterwards, I would interview both the performer and the subject.
With this format, the subjects will be forced to try to remember what they experienced, and think of the reality that could have generated their perception. As they look around the room to look for that piece of reality, they may or may not find it, and will need to re-access their memory and perception, while at the same time trying to create that perception for the next person.
Impetus
I consider myself a design maker who works through the language of space. I have always invested my time and energy into creating something that is tangible and beautiful at the same time. It is important that my work bridges different disciplines, I strive to use technology not only as a tool but also incorporate it into my ideation process. I strive to create art pieces in which technology would be constructive yet not intrusive, and thus give the audiences an opportunity to appreciate the content of the pieces.
In architecture school I was trained to think in terms of space and how people move in relationship to space. I try to see not only the structure, but also the volume that is created in between. I found it interesting to take note of not only the positive shapes, but also the negatives — they are equally important. I believe that our understanding of the environment is constructed of both what we can perceive and the lack there of.
The million dollar questions is: “How is reality constructed?” I’ve chosen to look into some specific aspects to this question in attempt to focus my research. Firstly, who am I researching for? I’m discussing reality in the context of what it means to us as humans, which are capable of reasoning and logical thinking. From then on, how do we—as humans— approach and understand what reality is? I begin to imagine that this process can be generalized into two parts: we are first passively absorbing the sensory inputs we get from our surroundings, and when we actively interact with the surroundings, we are able to collect feedback. Finally, where do we go from there? We build on top of our existing knowledge of the reality that we are in, we pass that knowledge onwards and form a societal opinion on the world where we exist.
Let’s take a closer look at the second questions from above, and further break down this process of “perceiving, reacting, and . How can the physical space help or mislead us on understanding reality, and how does our senses and perception play a role in this process? What are the limitations to our senses? As technology advances, we seem to see the world in a more precise manner than we did before, and I wonder if that means we are getting closer to reality.
Through this project I aim to create an alarming disconnect between the physical environment and the perception of my audiences. I hope to provide an opportunity for the audiences to become aware of the possible disconnect between their sensory inputs and the physical reality. More specifically, I would like to create the following situations:
unreal realities
Bring attention to how our perception can be deceiving.
real unrealities
Imagine and realize what could exists beyond our human range of senses.
The terms “unreal realities” and “real unrealities” are taken from the short story “School of Constructed Realities” by Dunne & Raby. I’ve used them here in the context of my thesis, which does not necessarily align with the authors’ originally definition. In order to properly continue my exploration, there are several more terms I will need to clearly define: environment vs. reality, and sense vs. perception.
In one sentence, I hope this project would inspire my audience to re-assess what they believe to be or not be the reality.
Concept statement WIP
I plan to create an environment that challenges the constituents of our perceived reality. I hope to provide an opportunity for the audiences to become aware of the possible disconnect between their sensory inputs and the physical reality. Facilitated by space design, kinetic sculptures and AR technology, the project space will bring surprises, confusion and excitement of revelation. I would like my audience to walk away noticing things they've taken for granted and overlooked; with a mindset of skepticism for their old practice of perception.
Research directions
To begin the research--the overarching theme I would like to explore is:
How is reality constructed?
To answer this questions, I've divided it into smaller parts and I'm trying to identity the different entry points from which I can begin to tackle this great question:
Who am I researching?
Discussing reality in the context of what it means to us -- humans.
How do we approach and understand reality?
Our sensory inputs help us form an understanding of the environment.
We get feedback from when we’re interacting with the environment.
Where do we go from there?
We build on top of our existing knowledge of the reality we’re in.
With these directional questions as a starting point, I begin to narrow my research.
The following list is WIP.
Research questions:
To what extent can we be fooled by our senses?
What is the relationship between our senses and our perception?
What role does society play in forming our understanding of reality?
What is the experience when one's perception is fragmented and conflicting?
How are magicians manipulating their audience's perception? What are the psychological and physical tricks?
Research resources:
One Place After Another, site specific art and locational identity. Miwon Kwon
School of Constructed Realities. Dunne & Raby
Interactive art and embodiment. Nathaniel Stern.
Physics and Philosophy, The Revolution in Modern Science. Werner Heisenberg. “We have to remember that what we observe is not nature itself, but nature exposed to our method of questioning.”
James Turrell.
DT Thesis precedents
As precedents, I referenced the following six thesis projects from past MFADT students:
Melaine Bossert
Shuangshuang Huo
Eric Jiang
Christina Lafontaine
Zeqing Hong
Heloise Sampaio
I'm paying a closer look to the three following projects:
Path
In this project, Eric Jiang explores the concept of an "auralized landscape". It discusses the relationship between our environment and our senses and how technology facilitates our perception. The audience's raking motion would trigger audio of the chanting of monks. It's also interesting that the technological interaction is embedded into a ritual that is commonly perceived as purely analog. Through interacting with a virtual sand garden, the audience can curate and use sound to visualize their own version of a zen garden - their own reality.
Ethereal Ecologies
Christina Lafontaine's project explores the concept of parallel worlds and the illusory liminal spaces that bridge the boundary between our accepted reality and the unknown. She states that "my goal is to inspire curiosity and discovery about what exists beyond our human range of senses", and I consider that to be successfully achieved. I --and I imagine many else-- have always been intrigued by the concept of parallel universes and hidden portals which take us to another world. Her multi-media installations, although tiny in scale, illustrates a landscape that could expand into an entire world. I enjoy her photography skills and her documentation gives a clear explanation that these landscapes are to be stumbled upon and discovered within a seemingly ordinary urban landscape.
Blind Spot
How do we perceive the reality? Zeqing Hong's project builds on the common assumption that our eyes are the cameras that constantly monitor reality, and begins to ask what are the limitations to our eyes and thus our perception. In this interactive installation, the audience encounters the self from an outside perspective through seeing footages of themselves. This project asks existential questions and points the focus of our perception back to ourselves.
Concept Statement
Context: what are you researching, experimenting with or studying; and why?
Intention: what do you plan to make, and why? How is this making a response to your inquiry?
Impact: what is the goal of your project, why is it important?
I plan to create an environment that challenges the constituents of our perceived reality. I hope to provide an opportunity for the audiences to become aware of the possible disconnect between their sensory inputs and the physical reality. Facilitated by space design, kinetic sculptures and AR technology, the project space will bring surprises, confusion and excitement of revelation. I would like my audience to walk away noticing things they've taken for granted and overlooked; with a mindset of skepticism for their old practice of perception.
Domain exercise
This is the documentation of an in-class Domain Mapping exercise.
Sep 7th, with Professor Jess Irish.
Top level domains:
- Interactivity
- Public Art
- Space Design
- Psychology
Some design questions:
- How can I use an installation to document and reflect the traffic that passes through a space in an artistic way? I want to gain a perspective that is meaningful by synthesizing the perceived information.
- How can I use interactive artwork to influence the way people perceive a space, and generate emotional response accordingly?
- How can I test the factor that determines how likely a person is to interact with an object, what can I learn from it and how do I translate that into an artwork?
- How does something man-made --a sculpture or a machine-- live? How can I make it appear to live, breath, sleep and how should or should it not reflect human behaviors?
I then re-organized the questions into three possible design concepts, and for each one of them listed three precedents.
Reflection and revised design question:
- How do we define reality? What are the aspects of it?
- How does the physical space help or mislead us on understanding reality, and how does our senses and perception play a role in this process?
Grid of 9_Part 2
On Aug 29th I met with Aaron Lee to review the exercise, I received the following feedback:
- Think of different media and different responses
- excitement/ fear/ others...
- Polar opposites?
Why the medium(tool), and why the space?
Public Art - what are the other senses: sound, smell, etc.
Why ARKit?
- Amazing experience on top of ordinary space, or both curated?
INTERACTION DESIGN - how to make technology less intrusive?
In addition to my research concepts:
1. AR + home entertainment
I would like to research the current projects/demos made with ARKit to understand the application of it, and make lists of physical activities that people can do within a small private space, and cross link these two lists for inspiration.
Goals/responses:
- gameplay/entertainment
- home improvement
- emergency alarm
- critical design?
- exercise
2. Interactive Art
I would like to research the difference between high art and public art, and understand what makes artwork more approachable. I will create a list of the physical mechanisms as well as conceptual cues that makes it so and compare them with my current skillset.
The Times article by Ken Johnson, Well-Behaved Street-Corner Sculpture: “...outdoor art serves rather to divert, amuse and comfort. ...contemporary outdoor art tends to offer unobjectionable, mildly decorative or entertaining and relatively empty experiences."
- Public art can express community values, enhance our environment, transform a landscape, heighten our awareness, or question our assumptions. Placed in public sites, this art is there for everyone, a form of collective community expression. Public art is a reflection of how we see the world – the artist’s response to our time and place combined with our own sense of who we are…. As our society and its modes of expression evolve, so will our definitions of public art. Materials and methods change to reflect our contemporary culture. The process, guided by professional expertise and public involvement, should seek out the most imaginative and productive affinity between artist and community. Association for Public Art
3. Social media + Advertising
I would like to research the process of creating an advertising campaign, and try to understand the different entry points where technology can be injected, to serve the goal of making a campaign more effective and gain more exposure.
- Define target audience
- Conduct market research + look at precedents/competitors
- Choose marketing platform/medium*
- Create appropriate branding
- Send consistent and repetitive message*
I have then revised my Grid of 9:
Grid of 9_Part 1
The Grid of 9 is the first exercise in the ideation process for my thesis.
Rule set summary: write down 3 different areas of questioning and a related initial concept to pursue each. Then, consider 3 completely different ways (tools, methods, forms) for exploring each concept. Tldr: 3 ideas, each made in 3 different ways.
Develop three completely different sets of domains, issues or questions you are interested in investigating.
I am interested in how AR technology can be applied to entertainment for private spaces such as home and offices.
I am interested in the creation of approachable, interactive, generative public art.
I am interested in the study of social media/content sharing in its relation to the retail and advertising industry.
Next, develop that area of investigation into potential thesis concepts:
AR + home entertainment
I would like to research the current projects/demos made with ARKit to understand the application of it, and make lists of physical activities that people can do within a small private space, and cross link these two lists for inspiration.
Interactive Art
I would like to research the difference between high art and public art, and understand what makes artwork more approachable. I will create a list of the physical mechanisms as well as conceptual cues that makes it so and compare them with my current skillset.
Social media + Advertising
I would like to research the process of creating an advertising campaign, and try to understand the different entry points where technology can be injected, to serve the goal of making a campaign more effective and gain more exposure.
For each concept, think of three completely different project based responses, using different forms of media / methods to realize that concept. Consider using entirely different ways of working, that can address the subject or theme from different perspectives.
For each of the 9 cells, list pros and cons, and rate your relative level of excitement for each. Prepare a 7 minute presentation to share your Grid with your class.
Aug 29 Discussion with Aaron:
Think of different media and different responses
- excitement/ fear/ others...- Polar opposites?
- Why the medium(tool), and why the space?
- Public Art - what are the other senses: sound, smell, etc.
- Why ARKit?
- Amazing experience on top of ordinary space, or both curated?
- INTERACTION DESIGN - how to make technology less intrusive?