Before they could transport a trio of curious, wide-eyed porgs into your living room, the designers at ILMxLAB were having some very in-depth conversations about finding the perfect “toe spread” on a migrating space puffin.
The stars of the new mixed-reality experience Star Wars: Project Porg have the weight of a real bird-like creature, and the perfect doe eyes for pleading for food that also looked entirely realistic. “Getting their eyes right is just so difficult,” says Michael Koperwas, ILMxLAB’s mixed reality supervisor. “Getting enough specular highlight that they feel like they’re there, getting everything to feel like they’re really in the world with you, that the shadows are as good as they can possibly get, they feel grounded, they feel there.”
The porgs’ movements, from flying and landing to cocking their head to one side in mimicry, had to be precise. Designers even created an entire porg language, a catalog of squeaks and sqawks that, for the discerning user or practiced “porg whisperer,” can audibly cue their specific needs in the moment, says Alex Elsayad, the senior technical designer for Magic Leap, which partnered on the project. “There’s an entire language there. They can actually request specific toys if you pay enough attention and spend enough time with them.”
The final version, essentially the next step in virtual pets, is ILMxLAB’s latest experience and first public experiment in mixed-reality storytelling, a new avenue that aims to ground virtual creations in real surroundings. Available now exclusively on the Magic Leap One, an untethered headset, the project differs from previous ventures that transported fans to a droid repair bay through virtual reality or touched down on Mustafar with all the sights and smells in Secrets of the Empire. In traditional virtual reality, “You’re shutting out the rest of the world,” Koperwas says. “Once you aren’t doing that, once you start putting on this device that you can wear it and go about your regular tasks, it occupies a very different space of entertainment. We’re very interested and keen to find out where some of those things can work….The ability to interact with other people together through this experience, these are all super exciting parts that we want to start to play at.”
Direct from Ahch-To
Slip on the Magic Leap One headset and spatial scanning software creates a grid-like map of the space around you before the porgs touch down in a specially-designed device complete with a holographic C-3PO providing very polite on-screen instructions. With a hand controller, you can feed the porgs nutrient-rich cubes, play with a twine ball crafted by Chewbacca himself, or even maneuver a laser-pointer to send the porgs hustling and chirping around the room. Pick up a porg and place it on a table or even your friend’s head. And, if you happen to drop one of the bird-like creatures, the tiny porg will flap gently and gracefully back down to the nearest surface.
The longer you play, the more in tune you and the creatures become, with each other and the surroundings. Plant a blade of grass on your living room carpet and when you return days later you will see it sprout into an entire patch of grass on the same spot. Interact with the porgs enough and they’ll begin to mimic the way you tilt your head, imprinting upon their new caregiver over time. “A lot of work has gone into making sure the porgs were aware of each other at all times and then through various APIs (application programming interface), they can always be aware of not only where a user is but where they’re looking,” says Elsayad. “It’s a very powerful tool to be able to understand where people’s attention is because you can read a lot about their intention. On the technical side, we had a small touch of content persistence. There is actual grass that you can plant. For that grass to be at the same location the next day, as long as it recognizes the space, a single blade turns into a full patch that the porgs can then enjoy.”
‘Really cute and mischievous’
The porg storyline is nearly four years in the making. That’s when the collaboration between Magic Leap and ILMxLAB began, and the creators started running experiments to explore “how to best express this new medium,” Elsayad says. “We’ve been trying to figure out what would be the best fit for a long time. And for the past year we’ve been focused on this vignette.”
“We’ve tried a surprisingly large variety of different things, characters, cinematic tie-ins and scale,” adds Koperwas. “Some worked for various reasons and really didn’t work for others.
If you want to do something huge, you can do it, but you need a really big space and not everyone has that. If you want to do something human-sized, it doesn’t come across as well to everybody. Really small things almost instantly give you this intimate feeling.” That made the porgs the perfect creature to focus on. “They’re things you want to take care of, things you’re not afraid to approach. (Porgs) can have a mind of their own, they can have their own opinions and desires. You can project a lot of emotion onto them and they’re just, they’re just really cute and mischievous.”
“For Magic Leap, one of the important things was to explore, ‘What does it mean to have meaningful character interaction in mixed reality?’” adds Elsayad. “There is something very emotionally and viscerally satisfying about having a character that pays attention to you in your own space in a way that I don’t think any other medium really comes close to.”
To begin building the experience, the designers had to anticipate how people would interact with the tiny Ahch-To natives.
“You literally have to put it in front of as many eyes as possible,” says Elsayad. “There’s always going to be somebody who surprises you. Any one perspective is never enough. I remember someone at Magic Leap’s L.E.A.P. Conference playing porg golf,” he adds with a laugh. “That was not a thing that we really expected anybody to do. But because you can pick up the porg, what happens if I then use the porg to push stuff around? We quickly realized there’s interactions that you want to encourage and there’s patterns you want to find ways of discouraging. We eventually had to come up with a way of preventing people from shaking porgs.”
To date, the team has had over 150 testers explore the project as they fine tuned the details. “In the early days, we made this very simple first prototype and it’s surprising how close we ended up to that,” says Koperwas. “We had a couple of toys and the porgs would walk around and jump from location to location. The simplicity of the prototype was that it was made for a very specific space; everything was pre-built, but it was delightful.”
They also combed through video reference of real-world animals and footage of the porgs from The Last Jedi, created using a mix of physical effect puppets and CGI magic, to complete the effect. Although the experience is the first time fans will encounter an adolescent of the species, allowing artists working on the project to design the feathery, Mohawk-like tufts of a fuzzy porg offspring while also dabbling in recreating the adults and hatchlings seen on film.
“There are so many tricks that you can do in movies that you just can’t when they’re in the room with you,” says Elsayad. “The mechanics have to be right because you can look at it from every perspective, you can’t force the perspective so it has to be true to life.”
“Toe spread was very, very important,” Koperwas says, an essential step in giving the creature a natural-looking step off and landing. “They’re firmly planted. It gives them a greater sense of being actually on the ground.” But perfecting a porg hop proved to be a sticking point for the designers as they creatures meandered from surface to floor. “It’s something that you kind of think, ‘this should be as simple as can be,’” says Koperwas. But the effect relies on a perfect mix of animation, lightening, end environmental awareness triggered seamlessly by artificial intelligence technology.
Pet birds, like budgies, and the original porg inspiration, the regal puffin, helped designers with movement. But they also looked to more common household pets. The porgs are essentially a cross between “the cat personality, being much more on their own and just kind of solitary and wanting to find a warm space and just hang out, and dogs that are much more anxious and eager to play and interact with you,” says Koperwas. “Being able to hold your hand out and have them follow your hand. Sometimes they run away, sometimes they preen a little bit.”
Elsayad, who has two cats at home, incorporated a laser pointer after seeing his own beloved pets go wild for the toy. It also serves as a solid example of how realistic the experience feels when you’re immersed in it. “On top of having porgs run around, which is obviously good fun, the laser pointer will stop at a wall, or on the ground or at a table, which further anchors the entire experience into your space.”
And Unreal Engine 4 helped engineers build the porgs’ artificial intelligence. “One of my favorite moments in the entire experience is when porgs start mimicking your head movement,” says Elsayad. “It’s one of the things that we kept coming back to. ‘How can we make the porgs pay more attention to you?’ And sometimes it’s just as simple as slowing things down and allowing the porgs to look at you in the eyes for a second before they do something. The difference in emotional response between a porg that’s trying to do something to impress you and a porg that just stops, turns at you, looks at you, smiles metaphorically and then does the thing is really powerful.”
The future is now
In the future, the team hopes mixed reality can be an added layer to the human experience of everyday life, used for entertainment in off hours or even a distraction on a long commute to work. Video calls connecting colleagues separated by time zones and vast oceans could transition into mixed reality. “I’m looking forward to being able to have a meeting where the people who are remote are actually sitting in that chair and aware of where I’m looking,” says Elsayad. “I can look them in the eye.”
“What I’m most interested in is this eventually emerging medium of telling stories over time,” adds Koperwas. “We’re getting to this point where you can start to gain a relationship to a character and that story can be told through your direct relationship with that character. It’s no longer this sort of, ‘Oh they’re talking to the camera’ or breaking the fourth wall. No, it’s you. You’re part of the story. You are integral to it and we’re going on this story together.”
Koperwas expects that mixed reality will one day alter the way we lay out our homes, with rooms often configured around a glowing television screen. “And what that means for design, what that means for communication, eventually architecture as mixed reality gets more integral to our lives and becomes much more accessible, I think everything about the world is going to be very different from what we have now.”
But today, the team is just excited to see the first fans step into this larger world of porg caregiving. “One of the most profound joys is watching somebody else put on a headset and almost instantly forget that other people are there and just focus their attention on this new little creature that they’ve maybe wanted to meet for a long time,” Koperwas says. “That’s just the most heartwarming.”
Associate Editor Kristin Baver is a writer and all-around sci-fi nerd who always has just one more question in an inexhaustible list of curiosities. Sometimes she blurts out “It’s a trap!” even when it’s not. Do you know a fan who’s most impressive? Hop on Twitter and tell @KristinBaver all about them.
Site tags: #StarWarsBlog, #ILMxLAB