INTERVIEW – ‘Avatar: The Way of Water’ WETA VFX Team on 3,500 Water Shots, the Challenges of Facial Rendering and More!

Like the first Avatar film, Avatar: The Way of Water is another mind-blowing technological marvel. Recently, I visited the Walt Disney Studios lot to have a few sit downs with the key creative minds behind the stunning visual effects of the film at Wētā FX:

• Joe Letteri (Sr. VFX Supervisor)
• Richie Baneham (Lightstorm Visual Effects Supervisor)
• Dan Barrett (Sr. Animation Supervisor)
• Wayne Stables (VFX Supervisor)
• Pavani Boddapati (VFX Supervisor,  Metkayina Village and reefs)
• Jonathan Nixon (FX Supervisor, water, fire FX)

Editor’s Note: Some of these interviews did not take place concurrently, but the interview subjects were asked the same questions for consistency. These interviews have been edited and combined for clarity.

Avatar: The Way of Water is in theaters now. Watch it on the biggest and best screen possible.

ELIJAH ISAIAH JOHNSON: First I wanna congratulate you on Avatar: The Way of the Water. I know this movie is a massive undertaking. What do you feel like makes Avatar the premier 3D experience and sets it apart from other films?

PAVANI BODDAPATI: So personally, I’m actually not a huge fan of stereo other than Avatar. I find other stereo movies I’ve seen give me migraines. You know, you’re just not…you’re uncomfortable. And I watched the first Avatar in stereo and I watched this one in stereo. And I think it’s all credit to Jim [Cameron]. He has designed every frame of this movie to be extremely pleasing in stereo. It’s not intended as a gag, it’s not intended like, you know, something going into your eye. It is meant to be immersive….If you’re seeing an underwater shot or you feel like you’re in there, it was smooth. Like you didn’t actually feel like you had glasses on. And for me, I think it’s all credit to Jim and when he does a turn over, when he’s turning over shots, he will talk through that process. He will say, “Just make sure this particulate is this close and this big, because in stereo it’s gonna feel like you’re flowing left and this thing is going right.” So huge amount of detail. And we try as visual effects artists to keep that vision that he has until that final image.

JONATHAN NIXON: And look, I think a lot of other films, stereo is the afterthought. Or it was before, right? It’s like, okay, we made a movie, now let’s make it stereo. And I think you have to think of using stereo as you’re filming it in order for it to work. And I think that’s why Jim is so great at filming stereo films, because a lot of other times it’s just, “Let’s just make it post-conversion because we gotta have a stereo version of this,” versus, “This is a stereo film. This is what we are creating and this is the best way to see this.” So for me…it’s always the planning. If you don’t start with thinking of it that way and you’re just gonna do it after the fact, it’s never gonna come out.

ELIJAH ISAIAH JOHNSON: So that actually leads into my next question, because I know you mentioned that you were going to see some of the proofs like one year into the making of the film. How early on is your department brought into this particular kind of film? Because you say that it’s not so much post-production, it’s a little more integrated into the process.

JONATHAN NIXON: Yeah, so I would say specifically for the effects department, we started back in 2017, I believe. So yeah, about, you know, five to six years. And a lot of that is working with Lightstorm to start with what they’re doing on stage. So…our simulation R&D team actually built a GPU (graphics processing unit) water system called Water Waves, that we actually used on stage for them to be able to time the waves in 3D that then we could take into our world.

And then to be able to take that information, the data – How fast are the waves moving? What’s their amplitude? When will the wave crash? When won’t it crash? And then to be able to put that into the simulation world, that takes a lot of time to build that relationship with LightStorm and with our artist.

So that was kind of the first step. And then we evolved based on that, right? So we evolved. Based on scale, that’s starting kind of at a small scale with live action and characters. And then we scale that up to, you know, big sea creatures breaching the water. That obviously is something that you can’t film.

So the first three years it was the water task force. That’s what we were all part of. It was just a group of people from various disciplines working on really simple things like a tennis ball underwater. Right, you know, like start from ground up. Yeah. Like he said, this is all we did for three years. We just set up the system cuz we knew there were 3,500 shots to go in three years.

ELIJAH ISAIAH JOHNSON: Wow. I feel like the computers and some of the rendering is almost one of the actors in the film. What is the ideal rig to kind of run something like this? Because I feel like 64 gigabytes of ram probably isn’t enough.

JONATHAN NIXON: Yeah, it’s funny because…

PAVANI BODDAPATI: Five hundred! (Laughing)

JONATHAN NIXON: Yeah. I was gonna say for the rendering, we had these really…I mean we got all new machines maybe about a year ago for the render wall, which are like AMD machines that are 512 gigs of RAM that we absolutely needed to render and simulate this movie. So for our artists, I think we started on 128 gigs of RAM. And then at some point all of our FX teams had 256, because working on multiple scenes, just seeing the sizes of data…like we were creating sims that were terabytes per frame and we’re simulating a 48 FPS movie, so terabytes per frame. All of a sudden you start getting 40, 50, 60 terabytes worth of data for one shot on a very basic simulation that then needs to be pushed to the next, you know, resolution and make higher fidelity.

So yeah, the computing power is a lot, but also, you know, you can sometimes rely too much on that when sometimes you need to make better creative decisions. A lot of it is, you know, can we find some of these better techniques to render things faster?

PAVANI BODDAPATI: And also there’s another component, even in traditional compositing: you can render things in layers, right? You can say, “This is a table, I’m gonna do these four things as layers.” And if you are doing layer rendering, you can reduce your memory footprint You can have cheaper machines and then you can put it together in comp.

Now if you’re looking at that poster (pointing to Avatar: The Way of Water poster), a shot like that, with all of those swimming and the riders in the water, that’s all rendered together, right? Because it is very hard to decouple a creature or a character in water with all of that fine interaction. And that’s where some of the high memory comes from, is you are rendering less layers, which means you’re using more memory.

JOE LETTERI: So we’ve got pretty nice workstations for the artists to work in, but it’s the back-end power. In fact, we maxed out what we had built for ourselves. So we used Amazon. We did a lot of this on the cloud rendering for this one. I don’t have the exact specs, but like you’re talking, you know, petabytes of data and a lot of, a lot of computer power.

ELIJAH ISAIAH JOHNSON: Doing some of the troubleshooting from the first movie to the second movie, what were some of the brand new challenges that you encountered? What was easier now and what was surprisingly difficult?

PAVANI BODDAPATI: Water is easier. If I work in a water shot at the first film, it’s the shot where all these Ikrans are flying past this cliff. It took only about like two years to do that one shot. Um, we’ve obviously pumped out 3,500 shots. (Laughing)

ELIJAH ISAIAH JOHNSON: You say, “It only took two years.” (Laughing)

JONATHAN NIXON: I had the luxury of not working on the first film, so I was maybe a little naive to the fact of…yeah, I’ve worked on big water shows before, but not knowing the scale of, as Pavani said, you look at this one shot that we kept looking at going, “Okay, we at least have to make sure we’re better than this one shot. Right? That took two years and it’s been out there since 2009.” And then being able to do that repetitively over and over and over again…you can’t have one person working on all of the movie to make it look as good as it does. You have to build a bit of a small army, right? To be able to do this creatively.

PAVANI BODDAPATI: And this last year of production since the beginning of this year, this system was so finessed that you could get a shot in a day because we had already delivered example shots last year. We had done water shots. We had done a fire shot. We kind of knew that, okay, Jim likes these shots, let’s make this the template. So this year we delivered most of our shots really fast!

JONATHAN NIXON: We have some sims, it was one take. You set up the scene. Yeah, you run it, it works. Showed Jim. He goes, “It looks awesome” and we delivered it. So by the time we got to the end, there’re a lot of shots to get through, but it wasn’t a lot of iteration time. It wasn’t a bunch of, “Well how do we do this?” It was a lot of, “Okay, if we need to make this a bit of a process to get the shots done.” And so being able to have an artist that’s fairly new to the company, show them a workflow, they run it, it gets approved by Pavani and the show and the shot gets out the next day, is pretty remarkable.

PAVANI BODDAPATI: Yeah. We are glad we got there in the last year. (Laughs)

JONATHAN NIXON: (Laughs) It would’ve been tougher…it was tough, but it would’ve been even tougher if we weren’t at that point.

ELIJAH ISAIAH JOHNSON: Water is pretty much a character in the film. When you approach it, do you approach it just as “The key is just to make the water look as realistic as possible?” Or do you embellish the way the water looks in certain areas to kind of tell the story?

PAVANI BODDAPATI: I think the performance is usually the starting point because there is some performance that’s already in the water, right? We shot reference, a high speed reference, of this girl coming out of the water in a tank at Lightstorm. And we’ve looked at the reference like hundreds of times and all we are looking at is just beads of water on her face, and how quickly the beads of water come out. Different types of hair starting with, you know, curly hair to straight hair, how quickly does a drop of water come off the hair?

RICHIE BANEHAM: The visuals are, in a way, to carry the emotional, right? And truth is, you know, if you don’t feel the movie, we can look spectacular, we can be a spectacle movie, but…if we don’t hit you here, we don’t hit you in the heart, you don’t carry the emotion, we don’t have anything.

ELIJAH ISAIAH JOHNSON: So you guys have been working with James Cameron for a long time. What is the process like for that? Has it become a little bit more streamlined since the first go around?

RICHIE BANEHAM: Well, I think tool set allow us to be a little cleaner with the execution. But the interaction with Jim’s the same. Jim knows what he wants and he’s not shy about telling you. And when he—when we need to explore, he is a fantastic collaborator because he’s open to ideas and he really, I think, invites the right voices in the room. I think that that makes a big difference. Myself and Joe [Letteri] have worked together for a very long time, as a large crowd of the crew has. I think that that continuity really helps.

JOE LETTERI: I also think one big difference is, you know, we were all learning on the first film what was gonna work, what wasn’t going to work, on this film. We had that as a guide and something to fall back to. So Richie, you actually picked up a lot of the stuff on the stage, you know, like prepping stuff for Jim because you kind of knew what Jim might want to see here…

RICHIE BANEHAM: It was the language!

JOE LETTERI: Exactly. Or even throwing new ideas at him. And a lot of times Jim just said, “Yeah, that looks like what I would’ve done, so we’ll just run with it.” You know?

ELIJAH ISAIAH JOHNSON: As far as the highs and lows of making a film like this, what was one of the biggest challenges you guys found and what turned out a little bit easier than you expected?

DAN BARRETT: I think what remains difficult…facial is quite difficult. So it’s one thing that we did a lot of, especially in terms of interaction. You know, Spider’s arm band for instance. We might just replace the arm from the arm band down if there’s interaction. So we replace, maybe just replace his legs or…there are times when we just replace from the neck down….We sort of cut ’em up.

RICHIE BANEHAM: Well, I think the facial is probably the biggest challenge. Having to rebuild the facial form from ground up in order to encompass an ensemble cast. If you look at the evolution of the facial system from Gollum [from Lord of the Rings] all the way through Alita [from Alita: Battle Angel], and myself and Joe worked on the original rig together…and that has an end of life…but I think that the new facial system to me is probably the biggest breakthrough and maybe the hardest thing today.

And the other part of the question was, “What was the thing you found the easiest or easier than we expected?”

ELIJAH ISAIAH JOHNSON: Yes.

RICHIE BANEHAM: Nothing!

JOE LETTERI: Jim was saying, “Oh, I’m not worried about the jungle, that’s no problem.” I thought, “Oh, that just jinxed it!” (Laughs)

ELIJAH ISAIAH JOHNSON: What were some things that in real life that you kind of tried to implement into the animations of the characters? Wat were some things that you implemented and you felt like this will be the base?

RICHIE BANEHAM: Sam [Worthington] is an avid surfer, so he has great balance. So—and we do have a stunt crew as well—but we actually built a skim wing replicator machine for all intents and purposes called a jetevator, and got up and running in the water. So when he came down into the water, he came down into the water. That is so the best way to make it feel real is to make it real. (Laughs)

ELIJAH ISAIAH JOHNSON: Is there anything that you feel like you nailed it this time around; is there anything that you feel like, “Okay, next time around, there’s something we need to get really, really, really good at for the next one?”

JOE LETTERI: I think the facial system for me is a good example of that. I think we finally cracked understanding what goes on in the face in this new way of looking at it, in this new tool that we built. But we can make it so much better now as a tool for artists to use now that we have this new baseline established.