FEATURE: How Salford has embraced motion capture technology and how our students are revelling in it
“Once the capabilities of the technology became clear, I could see that the writing was on the wall.”
It is becoming increasingly clear that motion capture technology is here to stay. From its earliest iterations in Jar Jar Binks in Star Wars: The Phantom Menace (1999), to the box office-conquering James Cameron’s Avatar (2009) to helping unleash the raw emotional performance of its actors in Naughty Dog’s The Last of Us (2013), motion capture technology (or mocap) is establishing a new performance medium in how productions are made, whether its film, video games or television. As extended realities are becoming more and more part of our everyday viewing experience, an increase in the availability of the technology to universities is also helping students tap into the unbounded potential that it can provide for their studies.
The University of Salford’s School of Arts, Media and Creative Technology acquired mocap technology in 2017 in the form of the OptiTrack Motive system and initally developed an asset for students where they could get into a mocap suit and see themselves in a virtual space, albeit in a skeleton form. Fast forward to the onset of the Covid-19 pandemic in 2020 and opportunity struck for BA Media and Performance creative technical demonstrators Connoll Pavey and Jake Louden to really get stuck in.
“The opportunity came for us to look into the technology more and start to figure out more of its capabilities and what we could do with it,” Connoll said. “This was also around the time that The Mandalorian came out and everyone in the industry was suddenly talking about how virtual production was going to be a game changer and that everyone was going to be using it. Unreal 5 then came out and suddenly the opportunities to us were endless.”
Unreal Engine 5 is a state-of-the-art computer graphics game engine created by Epic Games that came out in May 2020. The engine is compatible with OpiTrack and allows a more realistic and detailed environment with characters displaying greater lifelike movement.
The OptiTrack Motive system captures the movement of the person in a mocap suit through cameras with high intensity infrared light and that real-time tracking data is then processed into the Unreal Engine which creates the world and displays the character’s movement within that world.
Connell said: “It just blew my mind. Suddenly, I had a PhD student who was looking at doing a dance performance in motion capture and I was thinking that we could make it work for them based on the leap in technology.
“I’m a visual effects artist by trade and once the capabilities of the technology became clear, I could see that the writing was on the wall and this was the future of production. I was reminded by [legendary visual effects artist] Phil Tippett realising his work was obsolete when Jurassic Park (1993) was being filmed.”
The Phil Tippett story is well-known in the industry as an example of how technology can completely alter a deep-rooted industry approach to filmmaking. Whilst prepping Jurassic Park, Tippett began designing the dinosaur’s movements through his tried-and-tested go motion technique. However, his work was suddenly obsolete when director Steven Spielberg realised he could create the dinosaurs in computer-generated imagery (CGI) for the film, which lead to the historic line from Tippett, ‘I’ve just become extinct.’
The more that Connoll and Jake experimented with the technology, the more they were able to start to build more of a character and an immersive environment on screen.
They developed a quicker and refined way to show participants what the technology could do to the point that during the filming process, the setting up and integration process was so quick that people could see their virtual character on screen right away.
Connoll adds: “We kept going deeper and deeper. We were thinking about what we could develop for students. At first, it felt like we were developing a module for acting students but then we realised that was too small scale. More and more programmes and students were talking to us about it and we started thinking bigger. What if we could help a student actually walk within a building they just designed?”
Word of mouth spread and the interest in the technology spiked across school programmes, from screen acting to architecture, from dance to games design and production, students were lining up to use the technology that could become a game changer in their studies and a develop a skillset with unlimited opportunities.
Below is a sizzle reel of some of the work that has been created at the University of Salford using motion capture technology.
“Mocap allows you to unleash your childlike imagination”
The career opportunities that could be provided by learning the skillset is what drove Third Year BA Theatre and Performance Practice student Sophie Stewart to get in touch with Connoll and Jake to use the technology.
"I was very interested as I wanted to create a portfolio of my work that made me more employable. I believe that they are going to need more trained actors to use this technology going forward and I thought I'd use this chance to get that skillset and put me in the best place to meet that demand when it comes,” she said.
Sophie put mocap at the centre of her dissertation, creating a show reel of her embracing the various movements within the technology to showcase all the key acting techniques that it can offer. She showed both herself in the outfit and her on-screen character through the Unreal Engine to demonstrate how even the most minimal movement in her body could be replicated in the virtual environment.
She described using the technology as a tool to unlock the childlike imagination of screen actors.
"When you are a child and you are playing, your imagination is constantly creating this world around you. This felt very similar. You're using your body and voice to unleash your imagination, but this time the technology is then bringing that to life in front of your eyes.
"From a creative perspective, it is immense fun as you are moving all the time and the endorphins you get provides such a rush. You're moving far more than any other performance-based type of acting. But as you can imagine, it's really demanding as it places physicality at the heart of it."
Image below shows a student dancing in a mocap suit and her virtual character replicating the movement as part of a project for BSc Games Design and Production students.
“There are so many opportunities in this line of work”
In the world of screen acting, the skills learned from using the software are expected to increase in stature as the industry continue to incorporate more virtual production into the filming process.
For MA Screen Acting graduate Katherine Jarema, the ability to use the equipment put Salford on her radar when she was looking to continue her studies following her undergraduate degree and it is a decision she feels has been well rewarded.
"A lot of acting programmes outside of London are just not looking at it and that could be because they are either too traditional or see it as too expensive. But I thought that it sounded really fun to try and it was a skillset that I never would have been exposed to through conventional teaching methods.
"Now that I've used it and had that experience, I really feel that to do mocap in a screen acting role, you have to have worked with the equipment before and most importantly, worked with people who really know what they are doing. I am very grateful for the opportunity that Salford provided."
Katherine came to Connoll and Jake to use the technology for her final project, a production in which her characters are 'sucked into' the virtual world from the real world.
"It was honestly one of the most rewarding things I've ever done. In stage acting, there are a lot of movements that you learn but motion capture takes every type of acting and uses it all at one time. It requires a lot of technique as you have to be able to convey everything with your body."
She said that the experience has provided her with a skillset that will prove to be increasingly useful as she enters the acting industry.
"I feel confident that if I had to do this [mocap performance] in future, I now have that foundational knowledge that would mean not feeling lost or uncomfortable in the suit. There are so many opportunities in this line of work that people either don't know about or aren't talking about but it's exciting."
Image below shows Katherine in a mocap suit peforming the fight choreography for her project
“We are helping to provide students with the skills that they cannot get easily elsewhere.”
Jake and Connoll say they that their work with mocap is helping to strengthen our students and graduates’ positioning in the industry so they can hopefully be at the head of the queue for roles with the skillset that the industry is looking for without having to do any further training.
Connoll added: “We have been speaking to industry about what they need so we can start training people in those skills now. I remember being at a visual arts event in London many years ago and was told that there were 2,500 jobs for visual artists at the time and universities were pumping out double that number of graduates. But the issue was that none of them were trained in the right skillset to do that job and that really got to me.
“We are trying to become that rapid responder to university by identifying key areas of skills that will be necessary for the key jobs of the future. I see mocap technology as titanic in its breadth and reach. People cite Gollum [From Peter Jackson’s Lord of the Rings (2001-2003) trilogy, Caesar [from the latest Planet of the Apes Trilogy (2011-2017) and Thanos (from Avengers: Infinity War (2018) and Avengers: Endgame (2019) as notable characters that mocap made possible. Now that its becoming more and more well-utilised, the whole realm of digital performance is about to get very big and its not just movies and video games.
“We are teaching a skillset that our students can carve out their own niche in their career. There are no non-viable options with this kind of technology and we are helping to provide them with the skills that they cannot get easily elsewhere.”
Connoll and Jake are looking to make Salford the place to study digital and vitual performance by opening the technology up to as many different students as possible and training them how to use it quickly and efficiently.
Jake added: “We know that we are on par with how the industry is exploring the technology right now and ultimately, we believe we have the passion and desire to make this work.”
Image below shows a performer using facial recognition technology in a mocap suit to provide the voice and facial movements for a character in a virtual environment
For all press office enquiries please email communications@salford.ac.uk.