Brad Scoggin
Very, cool resume. lots of work in Hollywood on some major, motion films. Also now doing some work in the automotive industry. today we get to sit down with Marty Romance’s. Marty is in the Bay area now, originally from Barcelona. He is the creative director and the co-founder of, territories Studios. thanks for joining us today, Marty.

Marti Romances
Thanks, Brad. It’s great to be here. Big fan of what you guys are doing.

Brad Scoggin
Well, I love to hear kind of how you got your start, into mixed reality.

Marti Romances
I think, as you mentioned, like, we come from a slightly different industry. We started 14 years ago, and our main focus was how we tell stories through graphics and moving graphics in the film industry. And we learned a lot, with that, because we’re talking in the film industry you’re talking to a broader audience. Everyone needs to understand what’s happening in these graphics. And directors use us to kind of create visual shortcuts to tell a story. Like if these guys are going from A to B and everyone sees a line that goes from point A to B, everyone in an instant gets it. And it’s a universal language, design. So we started doing more and more on that. All the way to working with Marvel, the visual graphics gave us the opportunity to start inventing, technologies of the future, how these superheroes what technologies are using. Now we’re looking at things that come from another galaxy sometimes, and that, of course, ended up with great iconic designs like the Iron Man heads up display or the Black Panther technology.

All of those started to infuse even before – with Prometheus and other projects like The Martian – started to infuse like the idea of what could be possible if we had technologies that are capable of doing these things. And very quickly we realized that, on the sidelines, we had the whole virtual reality industry starting to grow and evolve and mature to the point where, we started to see headsets with augmented reality from Oculus to Magic Leap’s first iterations. And this is really what got us into the industry. People that were looking at the visualizations and these prototypes that the world of Hollywood was allowing us to do without restrictions. That’s a beauty. Right? We don’t have to develop it. I work with Nintendo, and I know how hard it is to develop something that works. In the films, you just have to show how it could be. It’s just the idea is visual conceptualization and that to have no constraints allowed us be free to dream. In certain scenarios, people that were then looking to develop things for real, real products started to get in touch.

And by that, by the time that that happened, we already started to have a digital vertical in our company that we were focusing mainly on automotive designs for the dashboards. The automotive industry was asking for the same things that Hollywood was asking, like how we can tell very complex stories to a broader audience when they sit in driver’s seat.

And all of a sudden we need to tell them that the car is going to park by itself, or the car range of battery now needs to show that. So this electrification of vehicles over seven years ago opened the doors for us to do the same that we do with films, tell stories to a broader audience, through the user experience.

I think that combination is where we started to see the most successful stories with mixed reality, which was including mixed reality into these cars of the future.

Brad Scoggin
So you had the creative liberty with Iron Man and other films and then putting that into the heads up displays and other displays in cars. I mean, that’s a that’s a pretty cool transition.

Marti Romances
It takes time, but it’s a cool transition to dream of.

Brad Scoggin
Yeah, yeah, yeah, yeah, that’s very cool. Well, just before we move on to some enterprise of all the stuff you did in the film industry, is there one that was the most fun?

Marti Romances
I mean, if we had to link it to VR and mixed Reality, I think Ready Player One probably is the, the one that speaks the most to that type of industry of gaming in a virtual world while you still have a real world behind it. I think that that was very iconic, working with Steven Spielberg as well is always iconic, but it’s hinting at lots of opportunities on the physicality of certain user interfaces. as we move from the flat screen or the flat visor to something that is augmented in front of us and gives us a bit of a more 360 scenario in terms of what the opportunities are in terms of interaction between the human and the machine.

And this is certainly what we are seeing working with Magic Leap very closely and doing actual prototypes for automotive industry like Audi or Mercedes. You starting to unlock the future of these cars that now you don’t need screens because everything is overlaid on the cabin. But yeah. Ready Player One, I would say. Very iconic. And there was one that’s a film for Apple TV called Swan Song. And that was a very interesting case because we were asked to not design for the science fiction futuristic superhero, but instead design for a more plausible future. What if, these augmented reality that these people are using in their everyday lives was designed by Google, Apple, Meta? Let’s just bring it down to that more corporate scenario.

And we started designing all of these interfaces. And that was a couple of years before Vision Pro was presented. And the funny thing happened when we saw the Vision Pro UI designs and we compared it to what, two years prior to that release, we did with that film, the similarities were – it was a funny coincidence, maybe – but we basically ended up in the same in the same spot, like, surface scattering solutions, very simplistic, the use of very neutral colors and the use of light and shadows. So, yeah, that was another one that was pretty interesting in the sense.

Will Stackable
Something I’m fascinated about – You’re kind of hinting at it here, but the fact that your job was to visualize and imagine what XR could be in the future with no limitations, you’re not having to think about what current headsets can do. You’re thinking about what Iron Man’s billion-dollar suit can do. With not having those constraints and be able to sort of dream – What are some of the things that you explored or learned? You mentioned Apple Vision Pro. Maybe the some of the interaction, the UI, UX – what are some of the things that you’re now seeing, or maybe that you expect to see, in the future with headsets as capabilities on the hardware side grow?

Marti Romances
I see it there’s two points that keep getting closer and keep evolving the hardware, as you said, if we were looking back, very clunky hardware. I think we’re still there, but it’s not an excuse anymore. I think we can start prototyping now. But it’s also this software and some of these solutions that we were seeing. I see two things happening there. Happening exponentially. And I think they’re going to meet at the right point, probably in the next couple or three years. One is always getting smaller: the technology, the chips, everything – even the batteries, the solutions. But on the other end you see – on the software side – you see things like radiance fields, Gaussian splatting solutions creating very quick, efficient 3D that is running in real time. You see Nvidia doing things like NeuralAngelo. They are now meshing these radiance fields and giving us a very, very quick solution, meshing it into polygons so we can do relighting scenarios which is very important for AR. I see these two things. And of course you have Gaussian splatting that’s fully infused by AI and the algorithmic –

Will Stackable
Can you give us like ten seconds on what is Gaussian splitting basically?

Marti Romances
So a nerve – it’s a 3D imagery base of radiance fields, which means I can take a couple of pictures of a shoe from a few different angles. It’s not the same as photogrammetry. You know, photogrammetry. You need to take as many pictures so it’s using the images but it needs images from many, many different angles. With radiance fields, what you have is a machine learning process. An algorithm behind, basically calculating every single point of light – how it will bounce onto a specific surface, identifying what type of material that surface might be and how these might reflect light to another area in order to triangulate all of these points, to create an angle that you haven’t take a picture of.

So basically recreate how the light will be reflecting onto a site that you’re not seeing which, for transparencies – which, are very important in my opinion in terms of user experience and in this case of UI’s that exist and float next to you – for transparencies [avoiding] high polygon dense or very, very heavy data 3D elements.

This is what we need for these handsets that don’t have, like, two Nvidia graphics card running. So that have what Gaussian Splatting is basically the same as what a Gaussian blur will do visually. It really takes two points of data, a one and a zero, and it creates a medium, a medium point. It creates like what the gradient will be. If you do that with black and white, the solution will be a grayscale. But if you do it with millions of points of refraction and reflectance, information, normals and lots of different passes that 3D data have, then Gaussian splatting smooths out to almost a perfect image solution.

What the nerve technology is trying to is almost a supercharged nerve with AI algorithmic solutions. When you have that, it’s a radiance field. So there are points of light floating. You don’t have geometry which was the solution of. It’s so light because you don’t have geometry.

But if you want to start doing visual effects in it, or if you want to start doing interactions where your hand goes behind it, you need to then cast shadows your hands. If you want to do VFX and do, I don’t know, fire around it, there’s going to be light that will need to bounce into that geometry, but that geometry doesn’t exist.

So what we see now happening is Nvidia and all the other great innovators creating papers and solutions like Neural Angelo, which is basically you meshing nerve radiance field solutions and 3D solutions. These are the things are very, very quickly being developed at the same time that we finally see hardware being capable of what we have dreamed of for years.

And I think that’s the conversion of these. And this is a very long-winded answer to your question: that the conversion of the two is what we are all waiting for and what it’s going to unlock will be the use of this technology for the everyday user. Not only that, of course – content will be king. Without content this is not going to go anywhere, but at least we are seeing hardware and software and getting it to very, very exciting points.

Brad Scoggin
One of the to go back in time a little bit. You had a really cool project with a company called Medivis. Can you tell us about that?

00:14:12:23 – 00:14:39:21
Marti Romances
Yeah. It was trying to pioneer, basically, how do we use augmented reality for an enterprise level, in a way, but more for health and, in this case, for surgeons. How can we just show surgeons a more accurate representation of what they’re going to find inside after the incision or things like these? [It was] mainly for neuro-type surgery. This was way early in the days, and all we did was helping them putting our fictional hat on, help them imagine how this could look and then take that fictional hat off, get a reality check of how impossible what we were imagining was back in that time, and then trying to put more of a software developer hat on and try to create something as close as we could from the UI and user experience side of things. They went and developed these. But if I fast forward to now with what we’ve been doing with Magic Leap, we see now a super accurate solution tracking with Magic Leap too. And we actually, most recently, have been helping them create, a series of solutions – visual solutions – for enterprise, some for supply chain, some for learning and training in factories.

And another one was for medical solutions. And it’s interesting they ended up in the same spot with what the best use case is: have a very accurate representation of where the issue [is in the brain] and where the incision [is needed]. So, yeah, the medical field. I think it can benefit a lot.

We work with Intel in a solution for showing augmented prosthetics for patients. So for them to understand “this is what your new knee is going to look like, this prosthetic is going to be here”, and you can see that mandate in your knee. So that’s what we’re going to do. We’ve seen things like this happening for a great cause that’s not just for entertainment in medical. But also for elders. That means driving experiences that put them back in the driving seat without putting them in a car. So we’re very interested with [more than] entertainment. [we’re also interested in] the real use cases that will transform and augment the human experiences, and transform them if we can, with our graphics and prototypes.

Brad Scoggin
Well, we’ll put it in the show notes, but checking out the video with Medivis is pretty cool. I mean, it’s a really accurate 3D rendering of the body of the brain, and I thought that was that was really cool. Well, you just mentioned the thing with helping elderly people get back in the car. So is that giving them the experience of getting in back in a car or teaching them to be more comfortable driving again?

Marti Romances
No, I think it’s more like the transporting them somewhere else – the places that they miss. The reality here is [that] there’s a big difference to us – I think for everyone – between pass-through VR and true augmented reality. They do different things. And I think the most important part from the beginning is choosing the right solution. Because, for things that are immersive or transport you somewhere else, VR and pass-through VR are great for things that really need to be working right now as you’re driving your car. For example, there’s no way that regulations will ever accept a machine that can turn off the cameras outside and you’re up in the dark with the latency that these [machines have]. So, for transportation, we have been doing a lot of work on VR headsets. And to me, I’m always most excited about AR [compared to] VR and Astro VR, but that’s maybe a more of a personal preference.

Will Stackable
Going back to the Iron Man heads-up display, which is an incredible heads-up display that you can kind of imagine. Eventually we might have something similar with pair of glasses like the ones I’m wearing. What do you see coming down the pipeline soon? And maybe what lessons did you learn and have you learned about creating user interaction and use a good user experience within an AR framework?

Marti Romances
[I’d like to answer] this last one first. One of the things we realize is that we started to see the data points and we could trace a certain path of speculation being very close to latest technologies. We were prototyping for a mini reality – contact lenses that were hitting your retina cones.

Crazy things are happening here in Silicon Valley. Like we always say, we come from Hollywood and we’re bridging to Silicon Valley. But some of these things really exciting, and in terms of where things are going – the path is clear and is going into more portable, more efficient, in terms of battery and computation solutions.

And it’s going to be very, very fine between what is AR, what is the real world as your environment versus what is something that needs a lot of more power because you’re going to be fully immersed somewhere else. There’s going to be a division of product there.

I think they’re going to get lighter, they’re going to get better and more sufficient. That’s one thing. Just to answer like the second question – what was the second question?

Will Stackable
So yeah, first question was where’s this going? And so question is what did you learn?

Oh yeah.

Will Stackable
What have you learned about designing a great AR interface.

So what we have learned? That’s the best part. We always say to our clients and our brands “okay, we all know that maybe this is not ready for prime time, but you’re going to be late if you’re not developing now.” And that’s why we’re developing with people like Mercedes or Audi or Nike, Prada, Adobe.

We’re developing things in VR just to be prepared and just to have the real foundation that these enterprise companies need to be able to quickly react when these things are ready for prime time in two years. I think the people that are not doing these now are going to be late in two years.

Why I’m saying that is because that’s exactly the answer to your question. We’re learning things by doing. We’re not imagining things. I remember when I first landed in San Francisco eight years ago, I was in the meeting with Tom Chee, who was at the board of the Google Glass and the amount of meetings we had about deciding which color the Google Glass UI will be like. Theory basically told us that red was the best one because it uses less energy. And so the theory was telling it, and there was so much time wasted. Because the reality is that once you start doing and not talking about it, you start realizing that red is the worst-case scenario for visibility outdoors and all that stuff to end up being like blue and white.

But my point is, we learned that by doing. I always say rapid prototyping has been what brought us to San Francisco was through different type of Y Combinator activations and acceleration and events that we participated in where we realized we can do this, we can rapid prototype, and we’re going to learn very quickly about the technology.

And that’s exactly what’s happening. I remember when we were doing the Audi active sphere, which if I had to highlight a project from us in terms of where we are going, the Audi active sphere is the best representation of where we are going and what we’re doing. And I think what we’re doing very well, by doing these things. You’ll learn things like, for example, we will learn that when we were having certain knob controls floating in front of the driver or the passenger, we’re realizing that putting those fully frontal to our headsets wasn’t allowing the headset to read where the fingers were because it was just seeing the back of our hands. It wasn’t seeing a knob. It’s just one of many examples, but if we were putting the knob at a slight angle, the headset cameras we’re able to have at least a little bit of visibility of our fingers. So when we were rotating fingers, it was tracking very, very accurately. It was a temperature gauge that we designed. So things like these, you will only know by doing it, failing and doing it again. [Consider the position] of things like elbows, [they] need to always be next to your ribcage. Forget about anything that makes your arms go up. That’s going to fail completely. It’s like the most uncomfortable thing. But these are small things. [Such as] where is the area of operations for humans to kind of a reach, at least with our hands, because right now we have lots of other solutions that Apple came up with [cleverly] around gazing. But if you don’t do those, you’re not you’re not learning.

What we kind of tell our clients is “look, you’re not coming here just for the good design, good software developers and our creative technologies. What you’re going to get is 3 or 4 years of knowledge that we acquired by failing” And you look at Medivis and that was done eight years ago. You know, and you look at like even concepts that we did for Forza, I don’t know if you play the [game] Forza, but we designed augmented graphics on the cars to tell things about the tire, [for example]. If you go to your gaming section in the website, you will find Forza. That was done 11 years ago. 11 years VR wasn’t quite there yet. But back then again, we were prototyping.

How cool it will be that when you open the hood of your car, you can see all of the graphics augmented where they need to be placed. And that’s why I think this power of computation is helping so much human machine interface and user experience in cars because [what we’re used to in] cars is a light [that] turns on with [a red] icon. This is as much as you were getting before. [The car] is telling you something is wrong with your oil right now. We can have the render, you know. You know, a render of your car. Not a generic car, but your specific car. And we can show you exactly where the issue is through 3D real time rendering that is showing you that the sensor picked up something in the suspension frontal right. And that’s the beauty of having access to all of this information that we can then create a better user experience. We can expand that human experience.

Brad Scoggin
I love what you said. the learn by doing. I mean, that’s such simple but such powerful advice. I’d be curious. – You’ve worked with several large companies, from Siemens, to some of these automotive companies. Any advice you would have for someone else who is either a content creator or maybe an XR champion inside one of these companies, and they’ve got some idea? and a lot of our listeners are in training, but there are definitely some doing similar things to what you’re doing. Any advice you would have in just getting a project off the ground and selling that internally?

Marti Romances
Well, I think it’s kind of what we’ve been doing for years. Like you need to develop anything, visualize it. If you can visualize your idea, fake it until you make it. That’s what we’ve been doing for years. We’ve been faking things because they were not possible. Now we’re making things because they are possible. But even before we start anything, even before we start before we start putting our engineers to work, we are going to visually prototype and try things like making sure that at least we have a very good understanding on where are we going to go, what we’re going to build. We’re designing, we’re conceptualizing things. So our experience with 3D designs –we can even take videos from POV and we have many things like “will it work if this thing appears here, what if we are doing that but also mixing like our expertise with mixed reality?” We don’t develop anything until we have a very good idea and our client has a very good idea on where the time will be spent because we know that as soon as we you start developing these things, then all of a sudden, they take a different pace.

You know, within 2 or 3 weeks, we can create a video for your product that conceptually explains everything and how it will look, but it won’t work. It’s going to be like a fake version of it, but at least it’s going to allow us to kind of take some boxes before we spend time. And our plan to spend money on really developing that, so would be what I would suggest. Yeah, doing this concept phase first is super important. And then when you start developing. I think there’s so many new tools that are democratizing the access of many, many different things. I mean, we started to use this AI solution called “move” just by creating quick videos of ourselves, and then you get like a full rep model doing that animation within seconds. Like before you were seeing [some] are spending millions on a motion capture stage in Vancouver. I said “I feel bad for these guys now because you take a video on your phone and you pass it through this machine learning and you get a rate model. These things are evolving so fast that I will say they can really help and supercharge small teams, like ours.

When we jump into doing these small teams to prototype certain things and to get things done. S, don’t lose touch with these new, solutions that are appearing in the market because they’re going to allow you to again, supercharge your development, to supercharge the opportunities of a small team really doing amazing things.

Brad Scoggin
Yeah, that’s good. That’s really cool. And I think it’s, it’s always better to hear advice from someone who’s had your level of success, I think has a little bit more authority. But I love that. Marty, this has been really good. Where can people find you? if they want to follow you and keep up with your work?

Marti Romances
I mean, you can always go to Territory studio.com. This is our website. As you will imagine, because we’re diversified through different verticals, we had to make sure that people can be guided through the website. So if you’re interested in, like our Hollywood work, you will go to the VFX or screen graphics if you want to see some of the stuff that we’re doing with digital product, you will see lots of the stuff that we discuss about automotive and whatnot, so you can find us there and feel free to reach us to in through any social media, X, Instagram, territory studio. I’m myself as well. If you, are listening to these and like, “I want to just get in touch with Marti, jump on a call.” My email is marti@territorystudio.com, and if not, you can find me as well in all of social media with Marti Romances with an I, not a y, I guess. And, we are always welcoming people, maybe people that want to work with us or one is starting in this industry and wants some advice, and what we would suggest. We’re very open. We love hearing from the industry at all levels.

So yeah, that will be the best way.

Brad Scoggin
Very nice. Marty, I appreciate that. Marty with an eye. and thanks again. I know you’re busy, man, so I appreciate you taking the time to sit down with us and share with our listeners. and we will catch up with you soon.

Marti Romances
Love a good chat.