Immersive Realities: Artform of the 21st Century?
ACT 1 (Article in 3 acts)

Breaking Boundaries: Public Spaces, Digital Immersion, and the Evolution of Art

In a century of exponential technological evolution, we are seeing a rebirth, a radical shift in the ways artists and performers captivate and interact with their audience. Current technology trends have unlocked the ability to merge the physical and digital space. And, this convergence of art and emergent technologies, including virtual reality (VR), augmented reality (AR), and sprawling metaverses, is challenging traditional artistic boundaries.

But what are these “boundaries” really? Are they defined by artists or technology? New advances or human authenticity? We believe that in this modern moment of art, artists utilizing technology are engaging poetic, almost magical realist expression; beauty, expanse, and grand feelings of immersion are showing up in the work of artists around the world. It is as though the great murals, ceilings, and three-dimensional sculptures have come to life. And, it is the use of augmented imagery that is the current tool of choice.

For those in the industry of creating this technology, enigmatic and powerful people have doubled down on AR. Executives like John Hanke, the founder of Google Earth and now CEO of Niantic Labs (makers of Pokemon Go) recently told the BBC, “2024 is going to be a breakthrough year for AR, especially as it converges with artificial intelligence (AI).” And, while it’s games like Pokemon Go that have made the news with millions of kids on the hunt for imaginary AR projections on their apps, artists have been pushing these immersive boundaries for years.

Art and Technology: An Historical Interplay

When art meets technology people are often skeptical. Some think technology has wormed its way into
the heart of the creative process, extending the artist’s canvas into hype. And, those same people worry
if art will remain the same introspective, deeply human endeavor in this brave new world, or will it
morph into something unrecognizable?

Consider artists like Sarah Sze, whose large-scale sculptural masterpieces (grounded in the physical world as compilations of thousands of tiny objects) evoke intimacy and connection in some of the most interesting public spaces in the world. Often compared to the spatial sculpture of Alexander Calder, Sze took the world by storm early in her career, and over the past twenty-five years, she has only grown as an international innovator. Utilizing her own specific way of compiling tiny objects to develop structures, architectural in nature, though oddly organic in form, her work sometimes looks like enormous new-found DNA stands, yet quirky and human. Sze famously stated about her objects:

“I don’t think of them as truly inhabiting any space. I think an artwork is a moment in time.”

In the face of such a thoughtful, philosophical artist, whose tactile immersions make people stop in their
tracks, can a VR/AR experience, however vast and intricate, truly compete?

Sarah Sze, Triple Point

We believe that there is no competition. That the boundaries Sze is pushing touch those working in the digital realm. Because, throughout history, artists have been at the forefront of technological advancements, pushing their own boundaries of what is possible. From the Paleolithic era to the Renaissance to the 20th century’s murals, later, video installations, and Sze’s hypnotic exploration of form, artists have made it up as they go along using whatever tools they have at the ready – or, those they must invent. “Art and Technology” are not opposing forces; they are companions to each other, and the artists’ vision and voice.

The Creative Economy’s Evolution

In the context of the Creative Economy, valued at an astounding 2.2 trillion dollars globally, the specialized genre of immersive experience is a powerful and innovative force. Often overlooked as spectacle, these experiences offer a visceral opportunity for culture, funders, and the art world to redefine how they engage with this artistic expression. We urge this because the creative economy is wildly underestimated, and often willfully misunderstood. Audiences now see deep value in creativity, and its impact is being felt throughout entertainment, gaming, and technology itself. Artists, always at the forefront of trends and cultural shifts, once again are proving that a creative economy is a real, needed, and vibrant one.

We interviewed Neil Redding, technologist and near futurist in the VR/AR field to explore this genre. Redding has worked at the convergence of digital and physical for decades and is an expert in spatial computing, AR, AI, and convergent brand ecosystems. When we asked him what he sees as the most exciting possibilities and challenges for artists in the new immersive landscape, he told us:

“The job of the artist is to cause us to see and experience things in new ways — whether visually,
conceptually, audibly, structurally, or any of countless other modes. When profoundly new types
of technology-enabled experiences show up in the culture, it’s essential that artists and the art
world take them seriously — as new materials, new prompts and provocations, and new
phenomena to critique and explore. VR/AR and Metaverse modalities absolutely meet these
criteria, and we believe they represent as valid a domain for artistic expression as any other medium.”

Some of the most exciting experiments in the creative economy are taking place in public spaces, blurring the lines between art and innovation. These public and large-scale digital immersive experiences and installations have impressive impacts:

  • Cultural Enrichment: Immersive digital experiences introduce new forms of artistic expression, enriching the cultural landscape.
    Immersive experiences are a forced perspective on place and connection to culture.
  • Preservation of Cultural Heritage: Immersive experiences can be used to preserve and present cultural heritage. For instance, historic sites can use VR or AR to recreate past eras, allowing visitors to better understand and connect with the culture of that time.
    Or, said another way, by augmenting heritage we look once again at what makes it radical, or remarkable, now.
  • Fusion of Art and Technology: These installations blend art and technology, fostering innovation in both fields. Artists and technologists collaborate to push the boundaries of creative expression, resulting in new art forms.
    This in turn influences, or creates new audiences, new energy and a fresh eye towards what is always under artistic “progress” new meaning.
  • Meeting New Audiences Where They Live: Younger audiences in particular are drawn to emotionally vibrant and physically stimulating experiences. This may be in response to being on small screens 24/7, but public (and large) immersions have a near cult following.
    Why? Because feelings are even more important to a culture reimagining community, and exploring reality is a natural search for a world finding itself without borders.

This being said, despite the immersive and transformative nature of virtual and augmented reality experiences in the art world, it is crucial to acknowledge and address certain counterarguments. While immersive experiences in art present a captivating frontier, there are valid concerns that merit consideration. Critics express apprehension about the potential commercialization of digital art, fearing market-driven demands might compromise artistic authenticity. Additionally, the accessibility of these experiences raises ethical questions, with the cost of necessary technology potentially contributing to a socioeconomic divide in art consumption. The issue of user privacy is another ethical dilemma, as augmented reality often involves data collection, prompting concerns about informed consent and data usage. Furthermore, there is a need to address the potential psychological effects of immersive art on participants, balancing the innovative push with responsible ethical practices.

Addressing these ethical concerns head-on is crucial to ensuring that the integration of technology into art is not only groundbreaking but also mindful of the well-being and rights of the individuals engaging with these immersive experiences. Though, culturally speaking, immersive experiences seem to resonate deeply with the modern audience. They flock to public events to experience large-scale awe in groups up to the tens of thousands. Digital mapping on architecture is always experienced as exciting and new. And, the inner/outer space of AR/VR has been adopted as a legitimate place of collaboration and community. People, plainly speaking, love them, and forward-thinking funders are supporting them.

Kiira Benzing is a Virtual Reality director crossing the mediums of theater, hybrid, and virtual reality. An award-winning artist and founder of Double Eye Studio, she represents the new guard working in immersive VR experiences. She told us that it is the film festivals (specifically Sundance and Venice Film Festival) which have embraced her work, and those of the immersive reality artists. Benzing shared with us:

“The film festival world gets it. It’s a funny thing since we are really making theater with game mechanics, and a game engine. So, we are somewhere between the theater world and the game world and yet we keep getting programmed in film festivals. The film festival landscape has made this category for XR, Immersive entertainment, and Installation work in a way that I haven’t seen the art world make a space quote yet. They (the art world) do sometimes in a curated way when they do work with Laurie (Anderson) and Marina (Abromovic) so I feel like we (the VR artists) are in an ‘out of the box’ place.”

As this AR/VR work relates to the Creative Economy, the film and technology worlds are the more adventurous funders – and this is no surprise, they can because they (Film and Tech) have such powerful economic engines themselves to support innovation. Benzing has been funded by High Fidelity, HP, and her own funds to develop trilogies of immersive work playing with improvisation, as well as with scripted scripts. Benzing pushes what she wants to create, and how, and this, in turn, pushes the technologists (and funders) to follow her into the unknown. This is just one example from Benzing of what is considered by many as a healthy economic ecosystem for innovation that is driving the Creative Economy.

Virtual Reality and the New Frontier of Immersive Art

With the advent of virtual reality (VR) and augmented reality (AR), artists have entered a new frontier of immersive experiences that challenge and expand the limits of perception and imagination. VR allows viewers to step inside digital artworks and explore entire worlds, while AR augments the physical world with digital elements, creating new layers of meaning.

Redding agrees, stating:

“AR brings all the dynamic interactive algorithmic capability of software to physical spaces — making it possible to visualize and realize just about anything that can be delivered by current AR hardware. In other words, software/AI can be invited into participatory artistic expression in physical space via AR — and we’ve really just seen the earliest glimmers of the possibilities this participation will create.”

Artists like Laurie Anderson and Marina Abramović have embraced these technologies early on to create immersive experiences that transcend traditional art forms. Anderson’s VR project “To the Moon” presented by The Public Theater in New York offered viewers a journey to the moon, immersing them in a poetic, lunar dreamscape. Abramović, known for her groundbreaking performance art, has delved into the possibilities of AR to create works that merge the real and virtual. Abramović’s piece “Rising” was a full VR experience at the 58th Venice Biennale, exploring the effects of climate change and rising sea levels. (Many of the immersive experiences that have caught the world’s attention deal with climate change, and we believe that they fall into “preservation of cultural heritage” because they take on the vastness of the topic of earth as home.)

From performance artists to presence in virtual worlds, we turned again to Kiira Benzing. She believes artists creating virtual worlds are called worldbuilders, and that she and her colleagues are creating a new art form. When asked why virtual reality is perceived as technology and not art, her response was about language:

“In the realm of virtual reality, there is an art form that is emerging…those worlds sometimes have a narrative, but more often than not, they are just incredible experiential places with social interaction. And, I like to think that when you go to a virtual world you are going to a new destination. It’s like we are going to a place to have an experience. But then the question is how it is curated and how (those building the world) are thinking about the people at the center. I like to think of them (at the center) as an audience.”

She continued, thoughtfully about how we view audiences, and how memory itself is evolving:

“If we are building an interactive experience I flip the word to participants. This is the foundational language for how we build our art. When I work with developers (while in the tech world) we say the word: users. This is probably why people label the work as tech. But it’s far deeper because when you move through a virtual world, and especially if you participate interactively, you experience the world in the first person and you create a memory of that experience. Worldbuilding (as much as I love theater) is even deeper, it is the next level of what theatrical performance can be because you live the memory.”

Benzing, who is known for her work with dancers, feels VR has given her a way to take the human body and push the boundaries of film. Her work with cultural phenom (musician, comic, spoken word artist), Reggie Watts, was part of the official selection by the Sundance Film Festival. The project, Runnin’, utilized volumetric capture technology to enhance the 3D experience and provided the viewer to be included in the dancing, as well as find their own superpower.

(Volumetric capture or volumetric video is a technique that captures a three-dimensional space, such as a location or performance.[1] This type of volumography acquires data that can be viewed on flat screens as well as using 3D displays and VR goggles.)

Reggie Watts, Runnin’ – Made With Unity: Runnin’ – YouTube

Benzing and Watts were the first artists to work with volumetric technology on the Intel studio stage, the largest volumetric stage in the world. When asked what volumetric technology has given her that she did not have at her disposal before, she said:

”Oh, it’s wonderful. I think this entire medium that allows us to work in the three-dimensional and has the presence of the human body is magical. What Reggie and I could do (using the Intel stage) was pull all my dancers at once, we could then replicate what started as ten dancers and then place them all over the world, so they added up to fifty in that virtual world. You get their authentic human presence at the moment, but then, with the tools of virtual reality, the possibilities of enhancing that human form is absolutely… magical.”

Though Benzing offers that she has ethical positions in her work she does not cross. She does not allow VR human bodies (those she captures) to be entered by audience members. She has done testing to support her position and found that women audience members particularly found the experience of taking over a body deeply uncomfortable, so she’s found ways to interact, without the ethical dilemma of crossing boundaries. Perhaps some of this sensitivity has been supported even more given that the VR creative world is driven largely by women. Benzing and her colleagues are focused on how audiences truly experience an authentic, emotional experience, while also driving the storytelling in a way that’s interesting and exciting.

And, in the end, the question, “What is real?” is intrinsically the deeper question of immersive, art and the time we are living in. Historically, theater and dance try to take the most complex subject and break it down, putting a laser focus to it to find that which is simple. That simplicity is what hits people between the eyes, forces an epiphany; and stays with them as a memory. The Double Eye Studio, and those projects Benzing leads are working even harder now to develop experiences that imbue the audience with both a central theme (like joy or hope) as well as the experience of the community.

Though, now, we have extra devices of perception found in the technology of augmentation and virtual reality. These devices are being tested to explore, as art does, what is real. The innovation may appear to be in the technology, though really they are with the artists – and sometimes the audience as participants – to challenge what we perceive; how we are feeling, and why. This particular time in art is no more different than any other search for meaning, yet now, we move at such a speed that our longing for depth is even greater. Therefore, the deeper, the more immersed, is how we choose to seek feeling and this desire is not breaking walls, it is crashing them down – for good.

(Watch for Act 2 of this Article Series. Next up “Deus Ex Machina or An Evolution of Theater”)

Immersive Realities: Artform of the 21st Century?