To drive its giant virtual world, Ready Player One needed a custom A.I. engine

Ahead of the 91st Academy Awards on Sunday, our Oscar Effects series puts the spotlight on each of the five movies nominated for “Visual Effects,” looking at the amazing tricks filmmakers and their effects teams used to make each of these films stand out as visual spectacles.

Ernest Cline’s 2011 novel Ready Player One was once thought to be un-adaptable with its legions of licensed characters from television, movies, video games, and comic books assembling for a sprawling adventure within a virtual universe known as OASIS. And then along came Steven Spielberg to prove the skeptics wrong.

Spielberg’s adaptation of Ready Player One not only managed to translate the grand scope of its source material, but it also managed to deliver a film jam-packed with the iconic characters and pop-culture references that made the book so popular among a certain generation of readers. It did so with the help of a talented visual effects team led by four-time Academy Award nominee Roger Guyett, who was tasked with not only building a virtual universe populated by a host of familiar and not-so-familiar characters, but also making sure that the digital avatars of the film’s lead characters were capable of conveying just as much emotion as their human counterparts.

Digital Trends spoke to Guyett about the experience of bringing Ready Player One to the screen, building virtual universes, and finding genuine emotional depth among more than half a million CG creations.

Digital Trends: With Ready Player One, you were essentially tasked with creating an entire virtual universe without any real limits. What was your initial reaction to the scope of the project?

Roger Guyett: It was breathtaking, really. To be honest, we underestimated just how much work was involved. There was the design challenge obviously, but we’ve done huge shows before. This was definitely a couple of notches harder and intensive, just on the sheer scale of it, but we did have a great team and great collaboration with production designer Adam Stockhausen.

Steven [Spielberg] was incredibly collaborative and wanted to make sure that there was a consistency in the ideas all the way from the writing through the work that we did. Ernest Cline and Zak Penn, the writers, were also involved — so that helped enormously. But without a doubt, we underestimated the complexity of it. It was designing on a literally world-building scale.

The sheer number of characters represented in Ready Player One is massive. How did you approach the process of deciding which characters to include and when?

You just have to start by laying out some of the scenes and start getting a feel for the spread of the characters and the kinds of characters that you’re using. What we did at one point was to have a card for every single character that we considered to be a “hero” character within the movie, and we put them up on a board and Steven and everyone else spent hours assigning them to scenes. We’d say, “This is where we want whoever it would be … Batman, Chun-Li, or whoever.” And then we went through the whole movie doing that. As the scenes developed, we got an understanding of how many characters we needed.

As far as the characters themselves, we always needed more — but we wanted to get a good spread of all types of characters, whether they were movie characters or cartoon characters or superheroes or whatever. It was a very complicated process and we wrote a number of different tools to try and help us keep track of it all. Those are the kinds of processes you can often really underestimate until you get into it.

What was involved in assembling that massive roster of characters from so many different properties and other sources?

Obviously, we had characters that we completely designed and built ourselves. Then you have characters that, to some extent, were known and we could appeal to some of the companies that were involved in licensing those characters. We’d say, “Hey, do you have a three-dimensional version of this character?” and quite often they did, or they gave us a starting point for them. The other aspect of this that’s really complicated is the licensing, which is really tricky. You have to be very sensitive about how you’re playing some of those characters.

So we had built characters that we did, we had characters that we tried to source versions of, and then there was a competition where people submitted character designs — and I think there were a dozen or so of those. That was another sort of fun thing. We also developed this thing we called “clans,” with characters of a certain type — maybe a [Ray] Harryhausen skeletal army character or something like that — and variations on that theme. If it was a U.S. Army soldier or something like that, for example, the clan would have one guy with a jacket, one guy with a shirt, one guy with a bazooka, another guy with a rifle, one guy who’s short, and one guy who’s taller, and so on. You can build up a large group of characters with that kind of approach.

Once you knew the characters that were going in particular scenes, how did you find the balance between managing these crowds of characters and still retaining their individual, recognizable elements?

That was a big question we had to answer: How do you generate massive numbers of characters and have them all behaving sensibly and abiding by certain rules and doing something vaguely intelligent? The way we achieved that was by writing our own crowd system. That was key to the success of the project. The guys did an absolutely incredible job of creating a completely new and bespoke crowd system.

We had a pretty good crowd system at the beginning of the project, but not a system that would allow us to do what we ended up doing. That was really the “jumping off a cliff and hoping that the parachute was going to open” moment for us. Everyone was very confident we could pull it off, but it was time-sensitive. There are around half a million characters or more in some of those shots at the end of the movie. It’s crazy.

… [The crowd system] had this incredible level of artificial intelligence, [and] you could kind of direct it, too. You could give groups of characters certain goals, and alter their behavior along the way. That was the thing that really made this so impressive.

One thing that really struck me about Ready Player One is how much genuine emotion the avatars have in the OASIS. How did you maintain that level of emotion when transitioning from the human characters to their avatars?

I am a very strong believer that you can have all of this technology and everything, but if you’re not following that emotional thread of the character, it’s all for naught. … Dave Shirk, the animation supervisor, and everyone else spent an enormous amount of time worrying about making sure each character mirrored the performance of the actors.

For example, if [Art3mis actress Olivia Cooke] looked flirtatious, or she looked sad, or she had a certain type of smile, you had to put that on the character and it needs to translate and get the same kind of reaction from the character. You try to automate it as best you can, but what you really need to make sure of is that when that translation happens, it’s 1:1 mapping … .

You have to adjust your mapping so that you feel the same way about whatever the character’s face is doing that you would feel about the actor … . This is really where the art meets the technology. The technology allows you to track the facial structure, and the fidelity of the motion-capture system that we were using for Ready Player One was considerably more impressive than anything else that we’ve used before, but it’s not just about that.

You need to make sure that Steven can do a side-by-side comparison of the two performances, playing back the performance of the actor and the performance of the character, and say, “Yeah, you’ve captured that performance.” … if you’re not getting that performance out of that character then you need to go back and try and understand why. That process can be very painstaking, and very specific to each actor.

Some early reports indicated that VR came into play in designing the movie’s version of OASIS and that Spielberg actually used VR goggles to explore some of the worlds you created. What did that involve?

One of the challenges with a motion-capture project is that the actors and the director don’t really understand the world they’re in, or they don’t get enough visual feedback from it. If you’re directing a scene in the live-action world, you simply walk into a location and all of your senses react to that space. We wanted to give Steven the same feeling, so we built more or less sophisticated versions of each of the locations we were trying to use for the movie in virtual environments. If it was Planet Doom and you’re doing a scene on the battlements, we would bring that into the system and both the actors and Steven could put on virtual reality goggles and move around that space and be inspired by it.

That feedback really helps you. You can understand what it feels like to stand on the battlements 100 or 200 feet high, and we could make adjustments, too. Steven would say, “It would be great if I could see a view of … whatever,” or he would ask, “Can we move that tower and get a better view of the field?” That’s the great thing about that kind of virtual world: You can make those adjustments. … Steven totally understands the technology and he really uses all the advantages of it, but the main thing for us was making the actors feel familiar with the space and feel inspired by it.

Ready Player One strikes me as a film that someone who works with visual effects can really identify with, as OASIS is a fantastic environment that works best when you don’t realize how much work it takes to create and maintain. Visual effects are all about creating things that don’t feel manufactured or artificial. Did that resonate with you?

I always felt that there was this kind of weird thing where we were constructing the OASIS. It was weird to be mirroring the story and many of the things that they were doing in the movie. If you were going to do something like traveling into The Shining, for example, then you probably would have done it very much like we did it. It was fascinating.

Was there a particular character or scene from Ready Player One that really encapsulates the experience of working on this film for you?

It’s hard to separate out just one or two things because I think one of the most important aspects in a project like this is to try to be consistent and try to fall in love with every moment, so that you bring each of those moments to the best it can be.

More Oscar Effects Interviews

• How ‘invisible’ effects brought Winnie the Pooh to life in ‘Christopher Robin’
• How big screens and small explosions shaped the VFX of Solo: A Star Wars Story
• Why First Man’s Oscar-nominated visual effects are a giant leap for filmmaking
• How Avengers: Infinity War’s Oscar-nominated VFX team made Thanos a movie star

But I just loved working on the New York race for all sorts of different reasons, because it was so much fun. But I also loved working on The Shining, because I loved hearing Steven’s stories about [Stanley] Kubrick and making The Shining movie and building that world. I am such a huge fan of the movie, and I have such a passion about it. So being able to just rebuild it like that was incredible.

Ready Player One premiered March 29, 2018. The 91st Academy Awards ceremony kicks off February 24 at 8:00 PM ET on ABC.