Tell us about what you do Brenda!
So, my job is really weird! I work at a company called Wave where we build virtual concerts. I help design, create, and perform all our shows and have taken on roles ranging from director and technical artist to VJ and mocap test performer. At Wave we use Unity, which is a realtime game engine to build what is almost like a video game virtual world. After that we layer in interactions and effects to make our shows more engaging and to add a sense of magic. Then on the day of the concert, we have the performer come into the office and put on a motion capture suit. That motion capture suit records the performer’s movements and is used to control a virtual avatar. Then, we stream all of that to the internet. So an audience member could be able to see, say, John Legend controlling his virtual avatar, and interact with the environment. It’s like a collaborative, intimate concert experience between fans all over the world and with the performer. The thing that makes Wave concerts so cool is that they’re performed live. Not only are the artists playing their music and dancing in realtime, but we also have a full team behind the scenes controlling different aspects of the show. For example, as a VJ, I trigger effects like lasers, lights, and color palette changes. We can also do things like pilot a drone camera through the virtual world, switch interactions, and spotlight real-world audience members on our virtual screens via Zoom.
That sounds awesome. Can you take us through the journey of a project?
Sure! First we find out who the artist is and pick a show director. That person will then do a lot of research on the artist and come up with the creative direction for the show. Afterwards we create a deck which shows the world we want to transport them to and its accompanying interactions which we then run by the artist and their creative team. When they’re happy with it, the next step is to build all the assets. This includes 3D modeling, texturing, compiling all the scenes in Unity, lighting, VFX, and adding interactions.
When we’ve finished building the show, and it looks great, we start testing it. One of our team members will jump into a mocap suit and pretend to perform and we’ll do a bunch of rehearsals where we practice cutting cameras, triggering different interactions, and general show flow. We like to experiment with different ways of performing and edit along the way until we’re happy. We’ll test, experiment, and learn, and then when the artist comes in we’ll do it all for real!
Who are your closest collaborators on a project like this?
Since I’m more on the creative art side, the people that I work with the most are the engineering team. They make my job so much easier! They’ll build special tools in Unity that help make the production process quicker and more efficient. For example, they’ll build things like a tool that will allow us to replace a set of objects within a scene so that we don’t have to go through the tedious process of duplicating and laying out objects from scratch. Or they’ll help us find and fix weird bugs that arise.
Let’s take a step back. How did you get to be doing this job?
It’s been a long journey! I always loved drawing when I was younger and thought I was going to become a fine artist. But then in high school a friend introduced me to an animation summer camp where I learned about and fell in love with experimental animation which inspired me to study animation in college. I thought I was going to become an indie filmmaker but then in my freshman year at USC, I accidentally wandered into a VR lab by accident. There I tried VR for the first time and was like, “Holy shit. This is the coolest thing I’ve ever seen in my life.”
Since that moment, I started exploring VR and immersive media. I began learning more about game development and creating my own projects with my friends in school. Together we built our own VR experiences and submitted them to tech conventions and events around the country. We ended up getting into a few festivals and we ditched a lot of college classes to go to those!
As I started to learn more about VR and developed more complex project ideas, I realized that I needed to expand my skill set in order to be able to build whatever I envisioned in my mind. So, I started learning how to 3D model, rig characters, and make textures and materials.
I also took programming classes so that I wouldn’t have to always rely on an engineer for every little thing that I wanted to build. I think if you’re an artist who wants to create immersive media experiences, it caps you at a certain point if you don’t know how to code. Over time, all these random skills that I had picked up along the way just fused together and I went from becoming a fine artist to being a technical artist now.
In my final year of school I went to a tech convention called TechCrunch where I saw Adam Arrigo, the CEO of Wave, give a talk. The next day, my friends and I decided to go to Beeple’s party in LA and to my dismay, they all ditched me to go networking. So, I was all alone and freaking out, but by chance, I noticed that Adam was also standing alone nearby so I went to go talk to him. We had a conversation about VR and he invited me to tour the Wave office.
When I emailed him to arrange a tour, I signed my name, but also put my website underneath. I found out later that he had clicked on the link, liked my art, and sent it to others on the team. When I went to tour the office, I thought I was just going for a tour, not a job, but we ended up talking more and eventually he hired me to be a technical artist.
Most people don’t meet their future boss at a party. It seems super random, but in hindsight, a lot of things led to this very opportune moment.
That’s a really good way of thinking about it. What do you think those moments were that kind of led you to that?
In college I was always a self-starter. I had loads of random ideas and just built them. For example, I designed a fish god character named Santiago and was like, “Oh, my God, it would be so cool if I made a giant sculpture of him.” So, I did and then my friends and I used the sculpture to build a multi-sensory VR installation. In the experience, you enter Santiago’s magical realm in VR and when you reach out to touch him, you actually feel him because you’re touching the sculpture. And then when you tap his eyeballs, you can trigger funky sounds and play him like a musical instrument. It was dumb shit like that.
But through projects like that I met another VR developer named Priyam. He told me about the Oculus launchpad program. I applied and somehow got in. They flew me and 100 other people to San Francisco for a two day bootcamp. After the bootcamp we were given three months to come up with a proposal for a project with the potential to receive funding.
I went home, and came up with another really stupid fish-themed project. Somehow I was able to scrape together a team, build a prototype, and submit it. Long story short, we got a grant for 50k, which at that point was the most amount of money I had ever seen! It was the hardest project I ever worked on. I was managing 21 people across five time zones while also building a significant chunk of the project myself and juggling school full time. But it taught me a lot about product management, time management, budgeting for a project, and running a team. While it was super hard at the time, all of these skills eventually came in handy for my job. And also, it’s a badass portfolio piece!
I think I’ve been very lucky and very fortunate to have been able to come across those opportunities. But, also, if I didn’t put in the work to learn new skills, I wouldn’t have been able to get those opportunities when they showed up.
What would you say to someone who wants to do the kind of things you do?
I would tell them to make a Twitter account. There are so many cool people in the VR, AR, XR industry on Twitter. It’s still such a small community, but everyone is so enthusiastic and very willing to learn and share. I would also tell them to just start building a personal project for fun and post about it on Twitter so people can see it. I’ve gotten so many weird job opportunities and met so many cool people through Twitter.
What’s next for you?
Recently, I’ve been working on an alien-themed virtual tarot deck. Right now it’s just a series of virtual cards but I’m going to be turning it into a physical installation soon. I also want to make my own fashion line for fun because I enjoy sewing in my spare time. But, my ultimate goal is to buy my own building and create a community tech art centre that’s an art class and communal workspace by day and a music venue by night. I just want to create a cool artsy space that brings people in the community together to learn and play. It’s going to be awesome.
Follow Brenda Chen on Twitter at @elevatedtrash
See more of her work at brendachenart.com