PCW: How big was the team that worked on the digital effects?
Ivins: Our company had about three 3D artists and about a dozen compositors. Disney also sent a lot of work to a facility they own for a lot of the simpler rod removal. We did about 350 shots in the movie--we were considered the lead effects facility. The effects supervisor just felt more comfortable being able to sit down with us and go over some of the more creative effects that had to be developed.
PCW: Other than removing the visual evidence of puppeteers, what were some of the more memorable sequences that your team worked on?
Ivins: I recall a sequence where Kermit walks through his mansion, and he has pictures of the band, Fozzie, Gonzo, and Chef on the wall. He starts singing a song, and as he's going through it, the various pictures come to life. We made a "picture" version of them that they blew up and printed large for the set, and then we took that and transitioned into the puppet responding to Kermit.
That was just a real finesse job. It turns from the painting into something a little more realistic, going from the flat surface to the three-dimensional without using a straight dissolve or anything that would take you out of the movie. It was just a lot of subtle manipulation through lots of versions of it. For those things, it was much easier if we were able to come and see the director, and even see his hand gestures. Those kinds of things were the most challenging in the movie--not really a technical challenge, just the finessing, the keying, and the right balance of what the interior and exterior levels look like, and all that stuff.
There's also a dream sequence in what the old television style looked like. They have scan lines on the puppets when they come flying out of the TV. All the little nuances don't become apparent until you start applying all the different looks to it, and you figure out where you're going to end up creatively.
In one helicopter shot we built a crowd up and down Hollywood Boulevard--that was one of the more fun shots we did. We were on our own particle system to drive about 20 different people we built, to make the crowd energetic and happy. At first, [the result] was like, "They don't really move around--they look almost like a still with a bit of noise on it." So we took a percentage of them and made them walk through the crowd and avoid each other and things. And then we added a little more motion, and then a little more motion ... and then whoa! It looked like fights were breaking out everywhere. We found out that the difference between a happy crowd and a riot was about 10 percent of motion.
PCW: What kind of hardware and software setups did your team use for the effects? It's probably not the kind of thing you can pull off with an app on a smartphone.
Ivins: That's coming around the corner--I'm surprised that now you can do almost all of it on a laptop. Just plug in the mouse and go to town.
Software-wise, Nuke is our workhorse compositor, but we also used AfterEffects. We use Maya for our 3D work. The more computing power, the better, especially for things such as the crowd scenes. We used Nvidia Quadro graphics cards. Watching these computationally intensive simulations, you need to be able to see about a million polygons at the same time. With some of the 3D stuff, you get render times that are way up there. When you see the whole crowd, I think we got up to an hour per frame.
We have several racks of render machines, as any CG company does now--machines with 24 threads that we split into about 8 threads apiece, because it turns out to be more efficient overall. We could render [the scene] in about half the time if we used all 24 threads, but you lose efficiency as far as how much rendering you get done with the amount of power you have available.
PCW: After all was said and done, how did the puppeteers react to the CG work?
Ivins: I think the puppeteers liked the way we used technology. It wasn't like, "Hey, we're going to take the puppets and make CG versions, and you guys are out of a job." It let the puppeteers cover a bigger repertoire of things that they could do with [the Muppets]. I think James Bobin, the director, wasn't thinking, "I have puppets, so I have to shoot a certain way." Instead, he could think, "I'm shooting a movie here, and my actors happen to be a frog, a pig, and whatever that thing is." I think that freedom is what modern visual effects added to the puppeteering and the movie.
They seemed pretty ecstatic to be able to do it. When they put on the blue suit, they thought they were invisible [laughs]. "No, no, no, hold on. We can't see through you."
They were like, "The handcuffs are off, man!" It was pretty obvious that they were excited about being able to be in the shot, and we'd do things like quickly throw together what a shot would look like after it was composed. Sometimes, it's hard to visualize exactly what [the scene is] going to end up looking like. So a couple of times, we just threw together something on a laptop on the set, so they would get an idea of what we were doing with it. Especially early on--I think the first day I was on the set, we were doing a big crowd scene where Kermit talks to everybody in the foyer of a theater. We were positioning them in groups, the way we do a traditional crowd-duplication shot: Shoot a group here, have them change their costume a bit, and then put them in the next chunk. We were doing that with the Muppets, and [the puppeteers] were fascinated by what we were going to do with their performance, and it was really great to see that.
After living with it for almost a year and being on the set, you sort of realize that puppeteering is, in a big way, the original cinema. I think shadow puppets on the cave wall were the first form of projection [laughs]. Shadow puppets in China go back thousands of years. This is the modern form of it, but it's so close to that original art form that's been around for thousands of years. And these are almost like the original visual-effects guys. So there was something really cool about working on [the movie] and being able to do it the way we did it. I'm proud of the way they handled it on the production side, and I feel lucky to have been a major component of the whole thing. This is one of those things I'm going to tell my kids about in five years, and I'll be wishing that I had taken that photograph with Kermit.