My guest today is Nonny de la Pena, co-founder of the Emblematic Group. This episode was recorded on Aug 12th, 2016.
Nonny and I talk about immersive journalism in VR, covering both technical, ethical and future aspects of it.
Yuval Boger (VRguy): Hello Nonny and welcome to the program.
Nonny de la Pena: Thank you so much for having me.
VRguy: My pleasure. Who are you and what do you do?
Nonny: My name is Nonny de la Peña, and I’m known for having pioneered the use of virtual reality for news, and impact on non-gaming content.
VRguy: I saw a movie that you did or a feature that you did about Syria children, as well as several others. Could you talk a little bit about the type of effort? How long does it take? How many people are involved in doing such a film?
Nonny: Virtual reality is really like any other medium. The more people you need to interview or have a new piece, the more places you need to go, the kind of intensity of the graphics, all that’s going to shift how long something takes. That said, with Syria, we had very little money. We had about $35,000. We had a 6-week deadline to get this in time for the World Economic Forum that was happening. This was back in 2013. It really was the first piece to use virtual reality, for this type of very strong and important content.
We all had a lot of late nights, and a lot of people volunteered their time, and there were many individuals who came in and worked even on short pieces, to help me construct the whole thing. The way that I dealt with that short time frame, with to spread the paint a lot to people, so it wasn’t too much for any one person to overcome. On the way to the World Economic Forum, I think I had about 2 hours to pack my bag and get to the flight. It was pretty crazy to get it finished. We had that piece help show the world that virtual reality journalism had potential.
I’d already made Hunger in Los Angeles previous to that, which had put me in the Sundance Film Festival in January of 2012. Hunger in Los Angeles, we used real audio of a day, a man waiting in line at a long food bank, didn’t get food in time. He had diabetes and the blood sugar dropped too low, so he collapsed into a diabetic coma. That piece took me almost 2 years to make. That I really had no one really committed. I had no funding. I had to become a better creator myself. The end of that was pretty amazing. The reactions of people at Sundance were incredible.
We showed up with 2 duct tape pair of goggles, and one of the guy who constructed those goggles was crashed in my hotel room, and helped drive the truck back almost like an intern, was Palmer Luckey, who then 9 months later did the kick-starter for Oculus Rift that he sold to Facebook for $2 billion. It’s amazing that you could take something like a story about hunger, and show the world the power of virtual reality.
VRguy: 6 weeks, 10 weeks, something like that. It’s more of a fit to a 60 Minutes type, if we take news analogy as opposed to the evening news, right?
Nonny: It depend, because again, we did a piece on Trayvon Martin, and we had that done about a week and a half, in which really just 1 person fulltime. With a couple of people working on that, we would have been able to turn it up very quickly. Where we could get things like an exact replica of George Zimmerman’s car, off of a modeling site like Turbo Squid for 100 bucks and have it within 5 minutes. There are stories that can be made very quickly for the evening news, and there are some stories that just require a little bit more time.
VRguy: I get it. What do you call this medium? Do you call it news, journalism, reenactment, documentary? What do you think is the best term of art for this?
Nonny: Initially, the first paper that was published about this subject in the MIT Journal presence, along with my co-authors Peggy Weil, and one of the greatest VR with pictures around Mel Slater, we termed it immersive journalism. I had actually put up the website before the paper, and had come to the idea that this virtual reality that was immersive, that put people on scene, was certainly valuable in the way that it provided news content, and therefore deserved a title in journalism. That said, when you do documentary, do you call it journalism or not? Sometimes documentary is journalism, sometimes it does have breaking news in it.
It’s hard to encompass all of these nonfiction experiences, which is often why I call it non-gaming, just because people think of games as being frivolous. Although there is this wonderful genre called Serious Games, which are games that tackle important subject matter. I do call it journalism, and certainly I think I’ve had some of the strongest skeptics spend a little time with us, and see the intensive research we do. The meticulous way that we approach subject matter, to make sure as accurate as any journalism that you would see depicted in current traditionally accepted forms of media, such as newspaper, or radio, or television.
The Knight Foundation jointly awarded Frontline and my company, Emblematic Group half a million dollars to do 3 very journalistically-oriented pieces. The fact that Frontline, which is one of America’s most traditional, fantastically renowned for its diligence and careful approach to document a film, have become a partner. Gives you an idea that we both believe that this medium has a way to be accurate, ethical, and an important way for particularly younger audiences to understand the world.
VRguy: Typically, you use real life recorded live audio, and then a virtual reality scene that’s reconstructed, or reenacted with great care for detail and accuracy. Would that be correct?
Nonny: Yeah, absolutely. The deal is that when you put things in headsets that let you walk around, which means you can turn your head any way and you could jump up and down, you can bend over. Then the screen in front of your eyes has to refresh an exact parallel with your movement, that’s called real time processing. The only place that you could get real time processing is with game engines. Would you do some 360 video? 360 video still does not let you walk around the scene.
How then can we have the photographic quality, and still let people walk around the scene? New technologies are really coming together now. We, along with Frontline, they did an amazing work on a solitary confinement cell in a main prison. We were allowed to go into the main prison, and actually take photographs of that exact solitary confinement cell, where one of the characters spent time. Now you can walk around the cell, and it’s photographically exactly accurate. That’s a brand new procedure that we’re using to also add the “realism” to the digital experience.
Similarly, there’s something now called videogrammetry, which allows us to film characters, and you can walk around the characters. Now, that’s brand new as well, and that’s something that’s going to be happening more and more. You’ll be able to just scan your friends, and yourself, and be in an environment that lets you move around. I hope that makes sense. It’s a really new technology I know.
VRguy: No, it does make sense. Basically, because of the walking around aspect, you’re saying that just traditional filming, even if it’s 360°, doesn’t give you the right sense of immersion, especially when you have people that are in the scene very close to you, and like the diabetes video that you mentioned earlier?
Nonny: Yes. With Hunger and pieces like that, when you feel the person is actually at your feet. When you could bend down, and the feeling is that there’s a person there, versus in a 360 video, if you bend down, you’ll never get any closer to the person. That intensity of feeling, I think is what made the work that we do so successful. We recognize how to create what I call special narrative, that is narrative that happens all around you. Actually, it really mirrors more closely how we experience the real world, more on scene.
The stories happening all around us, is not happening in a box. That’s what I think is so exciting about this medium.
VRguy: What kind of equipment does one need to experience one of your creations?
Nonny: We do offer many cross-platform ways to experience the content. The lowest tiny preps is our domestic violence, which is available on The New York Times VR app, NYTVR. It’s called Kiya, Kiya. The walk-around experiences we’ve been putting up on Steam, which is run by Valve. You need a HTC VIVE headset to see that currently. We are also putting step-up on Oculus Share. I think also very interesting to know is that this fall, the Sony PlayStation, we’ll be putting up their headset, the Project Morpheus headset. It’s a beautiful headset. That is associated with the fact that there’s 60 million PlayStation users out there.
This is going to be a $300 add-on. At that point, some of the bottlenecks of distribution of being able to put on headsets, and see all this content, are going to go away.
VRguy: At the moment, you have to have one of the higher-end headsets. You don’t offer something for say a Google Cardboard I guess?
Nonny: With the Google Cardboard, you can certainly see like I said our piece in The New York Times, Kiya. To be able to walk around the scene, you need a positional tracking headset, or a positionally-tracked headset. The binoculars are just starting to be made, and that what that means is that it can track your position and space. The headsets that let you walk around, are by far the ones that deliver the most engaging experiences.
VRguy: Right now, what you do is fairly unique but very compelling. As a result, do you expect that there are going to be many other studios, or groups that start putting out similar type of media?
Nonny: Luckily, there’s just so many people who are jumping into this field. I think it’s really a growth area. I think a lot of journalism organizations too, are now both adopting the 360 model, and beginning to look at the position with tracked work as well. That just means that there have been a lot of resources to us solving the technical problems, and a lot of amazing thinkers creating really interesting stories. We’re so excited that other people are now joining our journey.
VRguy: Understood. What project would you like to do that you can’t at the moment?
Nonny: Do you mean what story could I not do?
VRguy: If you could get a genius grant that says, “Do whatever you want.” What would you do next?
Nonny: It’s a very funny question and I’ll tell you why. You have this dream in life. You have this dream of something called the immersive journalism. You dream of the use of virtual reality to give people an opportunity to be on scene, and really understand more deeply stories. You dream about this thing being real, like it’s not me by myself trying to tell the world, “This is important.” The world embracing it, and major news organizations getting involved, and all these incredible stories coming to the forefront, and all these artists engaging with the medium. For me, that dream has been coming true. My company, we’re growing.
We already have 11, 12 people now working for us. We’re hiring more. We’re making the most groundbreaking content, and we continue to push the envelope. Or build this really interesting platform that lets people have positionally-tracked trailers, which is very different right now. You can’t do that, and we’ve built it because we needed it. We should give people trailers of our content, they can mainly pick what they want. These things are really … I get to do every morning what I want to do. I bought an electric bike and I zoomed to my little studio in Santa Monica.
It’s not so little, we just rented more space. I’m having the absolute time of my life. It’s my dream are coming true.
VRguy: That’s excellent. So you’re living your dream?
Nonny: I really feel that way. If there was any project that I wish I could work on, I’ll tell you. I’ll put it out there. I wish I could do the VR piece to go along with the novel, Kindred. I know HBO has got the rights and Darren Aronofsky is scheduled to direct it. If you know this sci-fi book by Octavia Butler, that is a story that I really wish I could do the VR component. That’s a dream. I have to tell you really, it’s like pinch me every day, because I can’t believe that this is just … It’s not easy. There’s no doubt I have to work really hard. Given the fact that I knew it went bankrupt when I started this to now, it’s pretty amazing.
VRguy: That’s great. Just a couple of questions before we wrap up, it looks like people have a very strong reaction once they experience your content. If there was one thing that you say, “This is the cause of that strong reaction.” Is it the directional audio? Is it the ability to kneel down and be very close to the object being filmed? What do you think makes them so compelling, as opposed to just to pick a hockey example. A few years ago, there were these Chinese reenactments of the Tiger Woods scandal, and they were a half-joke. You do these high quality journalism that people react so strongly to. What is it that makes the reactions so strong?
Nonny: This is one of those extremely difficult to describe unless you’ve actually done it. I cannot tell you again how many colleagues thought what I was doing was absolutely crazy, until they put on the goggles and actually walked around one of the pieces. It’s a very before and after moment. That said, we have an ability to connect to other people’s stories. We can hear their stories and we can feel them in a certain way. That’s how we share our world, and our community, and build communities, and our cultures. This is a natural thing. On the flip and very funny side, or not funny side I guess.
There was a study of folks watching their guys, watching their favorite football team. They would get so engaged in the game that when their team would lose, they were literally having heart attacks and dying. They were finding a connection between literally being so deeply engaged with their football team, that they would have a heart attack in their chair. We connect to our stories. VR is a different thing.
Nonny: We connect so deeply to story, we form community and culture. We can hear what people say, and we can read these stories, we can listen to them, we can watch movies, and we feel very intense. You watch a film and something scary happens, you might jump in your chair. Virtual reality, it takes it 1 step further, because it’s not just your eyes that are engaged, your whole body is engaged. The duality of presence is what I call it. You know you’re here but you feel like you’re there too. That feeling like you’re there, that’s what makes this medium so powerful.
VRguy: Excellent. You know, Nonny, my company, Sensics drives a lot of the OSVR open-source development, and obviously you have friends at Oculus, and VIVE, and Sony and so on. If you could drive our collective technical work plan for the next 18 months, what would you have us headset and software manufacturers do to assist your efforts?
Nonny: We certainly as a company, we’ve been very bootstrapped. It would be great to have some support on helping build our platform, helping me get a little bit more of our production teams out there. We had one team working in the Nuba Mountains in the longest running conflict in the world. I’ve got another team going out to Iraq shortly. These are amazing things to be doing as an independent studio. Certainly, I could use some more support equipment, any technical support can also go a long way for some of these teams that I’m sending out to in the world.
Also, if I already have a dream, I wish I had just a little fund, where I could go out and find people who clearly have interests, but are from communities who don’t easily have access to technological environments like mine. I would like to be able to bring people in with no skills, and give them an opportunity to work here. That’s a dream. Those are my two things. One is just like, “Yes, we can always use funding.” I can use a lot of more equipment and support, and then ultimately, an ability just to diversify our community now. I want to add one thing, that Valve has been incredibly generous with us, and they’ve been incredibly supportive.
It’s been amazing the way that they have helped me and my studio grow, and be able to tackle these important stories.
Nonny: That was no cash. They were just purely providing equipment and encouragement.
VRguy: Excellent. What is the best way to connect with you on the web, and learn for listeners about the work that you’re doing?
Nonny: I think the best thing to do is come to our website. I think we’re updating it. It’s emblematicgroup.com, and we are putting lots of stuff up now. Emblematicgroup.com is the best way to find some of our pieces, and we’re in the middle of updating it. That would give you a good idea of the stuff we have been doing.
VRguy: Nonny, thank you so much again for coming to my show.
Nonny: Thank you so much. Bye bye.