“Connected Colors” or real time face-mapped. © WOW
Japanese media artist Nobumichi Asai from WOW design studio awes and teases our senses with his face-mapped video projections and shape-shifting holograms. As his latest work is exhibited in Tokyo, we caught up with the artist for a techno-mystical interview.
Tokyo, from our correspondent
Ever since the international success of his spectacular real-time face tracking and projection mapping video Omote in 2014, Nobumichi Asai has gone into overdrive with new projects that continue to explore this technology of juxtaposition, especially in the performing arts. His holographic work Light of Birth, currently exhibited at Media Ambition Tokyo in Japan, spotlights even more acutely our blurred perception between virtual and physical. Reflecting on his process, this master of illusion speaks from science and touches upon theology. But what exactly is reality?
What is the relationship between the face-mapping of “Omote” and the “Light of Birth” hologram?
These are two similar works using the same code: Open CV. Both virtual CG space in the computer and physical space overlap, so that images on the PC can appear in physical space. These images track and react to the movement of the subject.
You used open source programs to create these works?
OpenCV was used to create this work. But this work could be created even without it, because OpenCV itself is a compilation of mathematical and physical laws put into code. The code for this spatial calibration uses the deterministic concept of homography in mathematics. OpenCV started in 1999, and is now offered as an open source project by Intel.
The universal laws of mathematics and physics are open to everyone and available to be known. In general, industries develop coding, and the rights to them are protected by special patents, but having OpenCV open source for anyone to use by Intel is of very deep significance to me. Because the things that are essential to us are shared by everyone, just like water and air.
“Connected Colors” by WOW (Nobumichi Asai), 2016:
According to you, the entire natural world can be broken down into algorithmic laws. What do you mean by “In the beginning there was the Code”?
When you look carefully at the natural world, you find that all sorts of things have the same structure as code. For instance, living creatures are described by DNA. The four bases A, T, G and C combine to form 21 types of amino acid, and the combination of these mere 21 types of amino acid makes the huge variety of high molecular proteins and enzymes which form a cell. In turn, these form the systems and organs which make up the human body. These cells number in excess of 37 trillion.
How this works is extremely similar to programming. In a computer, the combination of zeros and ones creates basic instructions, and the combination of these form modules, which contain the basic functions called classes. These, in turn, are combined to create systems with complex functionality.
In quantum mechanical terms, it can apparently be argued that nature (space) itself is made of mathematical operations. For instance, a chemical reaction such as H2 + O2 -> 2H2O has fixed rules, and can be interpreted as a mathematical operation. In addition, arrangements of atoms and molecules are themselves code, and can be viewed as variables. Taken in this manner, it can be said that the world is born of a vast number of parallel processes which proceed simultaneously. This overlaps with the idea of an omnidirectional virtual reality which is created by parallel processing in a GPU.
It is my feeling that space and the natural world are generated by code (= laws). From this perspective, in this work using holograms, I have shown the generation of physical existence from code.
“Light of Birth” by WOW (Nobumichi Asai), 2016:
There is a phrase in Buddhist scripture: “Unlimited duties arise from a single principle.” Within this, it preaches that all things in creation come from a single original principle that reigns over the universe. Did the sages deeply capture the essentials of this universe?
We have talked about the views of digital materialism up to this point, but I am not a materialist, because even though humans are made up of a conglomeration of atoms, conglomerating a bunch of atoms together will not make a human.
Within these boundaries, a mystical, miraculous power that is irreversible is operating. The complex, sophisticated code that created lifeforms was not born from coincidence all of a sudden. So who wrote the code that divides living beings from ones with no life? I feel like the “existence” of “things” that are not things is being carved out of the ether through the evolution of technology.
What future applications do you envision for such precise real-time projection mapping, either on living things or as holograms?
Today, the technological vector that virtual images will be actualized into reality has become inevitable. I feel the vector of human creativity crossing over the boundaries of humanity itself. I think holograms and projection mapping are a sign of that undercurrent. It has also affected the robotics movement (attempting to artificially create humans), which has become a hot topic in recent years. With robots such as BigDog and Petman, Boston Dynamics are always coming up with new innovations.
I see technology as a part of human evolution. I recognize myself as a member of the flow of that evolution, and am excited by it. Even art and entertainment are bringing forward a breakthrough with the expression of the evolution of technology, and will surely continue to surprise and excite people.
Japanese photographer Nobuyoshi Araki once said, “The human face is actually a nude.” A person’s face expresses everything about that person: emotions, age, physical condition, personality, individuality, ideals… to control the image of that face holds huge expressive potential.
For example, in a recent project, the music video for a song called Ending Theme by Amazarashi, we took on the challenge of transforming the artist in his 30s from his 20s to 100 years old. Experiencing representation of old age and death in just moments, reminds you that a human life is not eternal, but rather has limits.
“Ending Theme”, Amazarashi, music video by WOW, 2016:
The appeal of this representation is not that of fiction like a movie, but the point that the depiction is “something that is happening right here and now”. In other words, it is presented as a “real experience”, rather than as a simulation.
I think that from here on, improvements in the speed of PCs, sensors, and projects will result in the possibility of depicting even more lifelike realities, rather than in increased precision. From here on, these representations that utilize computer vision technologies will rapidly blur the line between the real and the virtual. However, the truth is that already, to humans, the difference in meaning between real and virtual doesn’t exist. To the brain, real experiences and virtual experiences are both encoded. This is because they are equivalent; nothing more than abstract electronic signals.
“The important thing is not ‘real or virtual’ but simply ‘truth or untruth’.”
For instance, if we take notes and coins to be real, then the money in your bank account is virtual (just numbers, a code). If it’s truth, we can use it; if it’s untruth, we can’t. In other words, the only thing that is important is: Is the meaning it holds reality, or not?
What are your upcoming projects?
I’m making a device that projects your face as a “3D object” in a remote location in real-time. In these times when Skype is so commonly used, the theme is to teleport humans as an “object”, not as an image. Something similar to the teleportation device in Star Trek. As a matter of fact, we can already do things that were done in the movie Avatar. In virtual reality terminology, this is called “telexistence”. It’s one of the major topics in relevant technology.
One can gain an interesting insight in light of traditional religions and world views. The concept of “Heaven” in Christianity and the “Pure Land in the West” in Buddhism, brings to mind the idea that perhaps human consciousness is somewhere faraway and is only transferred to this earth through life and death. Who knows, the world we believe as reality might just be a virtual space like the Matrix?
«Light of Birth» is currently exhibited at “Media Ambition Tokyo” 2016, through March 21 in Tokyo