A few weeks ago, I tried on a Vision Pro for the first time. The one aspect of the technology that really caught my breath in my throat wasn’t the immersive movie Submerged, it wasn’t browsing the internet, nor was it trying out the digital workspaces. No, it was looking at my spatial photos.
This discussion is available as both a written story and as a special audio-only episode of the PetaPixel Podcast which can be listened to on Apple Podcasts, Spotify, through the embedded player below, or wherever you get your podcasts.
Apple has a monumentally challenging task: it has to convince people that its Vision Pro and visionOS are worth it with the huge barrier of “you have to try it to understand” in the way. Yes, you can make an appointment to try a headset out at any Apple Store, but even that is asking a lot of folks who look at the price of the Vision Pro and immediately dismiss it. I don’t blame these people: it is expensive.
Some might believe that Apple isn’t invested in the future of the platform either given the niche appeal or the high price, but after speaking with Della Huff (a member of the Product Marketing team at Apple, who oversees all things Camera app and Photos app) and Billy Sorrentino (a member of the the Apple Design Team who works across the company’s entire product line), I left feeling that Apple has every intention of pushing forward in this space.
The two explain that Apple is very much invested in Vision Pro and visionOS because it views the experience they provide as integral to the future of photography. Coming from the company that makes the most popular camera on the planet, that opinion carries significant weight.
The Evocative Power of Spatial Photos
“We kind of have thought about our photography suite as helping people capture and relive their most cherished memories in powerful ways, and I think our colleague John McCormick said this recently, but we see these as personal reflections of something that truly happened, but we’ve been doing that for a lot longer than spatial photography,” Sorrentino says.
“We’ve been doing it with Live Photos, panoramas, Memory videos and photos, and even in maybe more personal expression — places like widgets on our lock screen wallpapers, but we want our camera and photography ecosystem to celebrate the best moments of our lives. And again, while folks love using the term memories in many aspects of photography, hopefully, you saw earlier today, nothing comes close to reliving a moment or a memory quite like spatial, right?”
Sorrentino is referring to my first experience using Vision Pro to turn photos I shot on film, spatial. The latest version of visionOS added this feature and it works shockingly well. While the most recent iPhones are capable of capturing rich amounts of data for spatial viewing, Apple recognized that it was also important for older archived photos — memories — to be enjoyed in spatial, too. The tool works in seconds and with stunning accuracy.
Seeing a photo I took turned spatial triggered something in me that seeing the photo in the original flat space did not. It felt like being back there, at the moment the photo was taken. It tickled my brain in a way that is difficult to explain.
“I think what’s truly incredible about spatial photography is that it is literally adding another dimension to photography in a way that has never happened before,” Huff says.
“And to your point about it just triggering memories in ways that a still photo just cannot, I was actually a psych major in college and I studied memory and what blew me away when I first experienced these spatial photos is that it triggers memory differently because your brain encodes memory in three dimensions. If you go back and think about a memory from your childhood, you don’t experience that as a 2D photo or video replaying in your head,” she continues.
‘If you go back and think about a memory from your childhood, you don’t experience that as a 2D photo or video replaying in your head.’
“It’s a spatial memory. That is what our brains are doing. We record 3D space and time and people, and so that’s why it is so evocative. I think when you see these spatial memories in Apple Vision Pro, it triggers your brain in just a different way than looking at a still two-dimensional photo or video, which I think is just so incredibly powerful. So it makes you feel like you’re back in the moment in a way that really nothing else can. It’s just like how you remembered it because it’s literally how your brain encodes it.”
“We’re capturing a moment disparity between two lenses, playing it back into your two eyes, and then we process the image or the video just like we process the real world. So that’s why it feels like a memory,” Sorrentino explains.
“Your brain is really the thing that is making it so much more than if it were a CG scene or a perfect 3D-created moment. It really is the disparity of the two coming into your brain, pulling that together and it’s sort of that conviction in making these memories that led us to pull on it further than just capturing on Vision Pro, certainly bringing it to iPhone — which we’re so excited about — but then also designing this powerful spatial ecosystem and end workflow that we’re excited about bringing this to more folks and to more content creators.”
An Authentic Memory
There has been plenty of conversation recently about photography, memory, and how AI can play a part in that. After introducing AI features into its new smartphone, a Samsung executive claimed that there is no such thing as a photo. After Google launched its new phones that were packed with AI, it said we as users should be more concerned with how a memory made us feel than the reality of it.
Apple has taken a different stance, stating that it wants to keep photos true to life — to how something actually happened.
“There’s nothing more authentic than recording a memory spatially. I think it’s almost the fullest expression of authentically recording a memory is when you record something spatially,” Huff says.
Even using visionOS to turn an existing photo spatial, Apple does so with integrity to the original capture.
“We’re still using the original pixels of that image. We’re not creating anything that wasn’t already there in the image. We are creating a depth map, but it’s still all of the original pixels. So again, it’s very authentic to the original image. It is the original image, just projected into a 3D plane,” Huff continues. “We think that a moment or a photograph or a video is a celebration of a moment.”
Apple’s Vision of Photography’s Future
The feeling of seeing a photo spatially — one that Sorrentino admits is difficult to imagine until a person tries a Vision Pro — is one that Apple firmly believes is the future of photography. The company believes in it so much that it is shifting its entire company’s outlook to accommodate it.
“We’ve invested a ton into building the incredible technology, hardware, and user experience that Vision Pro is, but to create this spatial video and photography ecosystem, almost every team at Apple is touching it at this point,” Huff says.
‘To create this spatial video and photography ecosystem, almost every team at Apple is touching it at this point.’
“Everything from the Final Cut team to the WebKit team, to all of the teams that you would expect on Vision Pro, to the photos team, the camera engineering team, the camera hardware engineering team who did the impossible of moving the camera modules around to accommodate for this. There’s tons of teams that are invested and I think that just speaks to we’re literally putting our money down on this future because we think it is so important,” she continues.
Huff explains that Safari is becoming more spatial on Vision Pro and starting later this year, web developers will be able to easily add spatial photos and videos to their websites, which greatly enhances the experience of browsing the web on Vision Pro.
“And the cool thing is that they’re all fungible into 2D assets,” Huff adds. “So the people, your 2D readers on a MacBook Pro, will still be able to see the photo and video. It will just be 2D.”
Both Sorrentino and Huff heavily stressed that Apple firmly believes in this future of enjoying photos, which is why the company believes that if you’re using an iPhone that can capture spatial photos, you should be doing so even if you don’t have any way to view them in spatial yet. They strongly believe that users of iPhones today will be grateful for that spatial information in the future.
iPhone 16, Apple’s latest series, of course, can shoot spatial photos and videos, but one thing that may have gone unnoticed is that the iPhone 15 Pro and Pro Max gained the ability to shoot spatial video with iOS 18.1.
“We’re trying to make it as available as many iPhones as possible,” Sorrentino explains.
‘I think our job, certainly in the design and engineering end, is to just make sure that we are delivering delightful experiences’
“The one other thing that I would say is the idea that we recognize the reality that most people are still enjoying their content in 2D. That’s why spatial videos are very carefully made so that you can share them with other people who are on flat platforms still. Same thing with facial photos,” Huff says.
“If you capture a spatial photo, you’re also getting a 24-megapixel high-res still. You’re also getting a Live Photo. And we’re also, if you’re capturing a photo of a person or a pet, capturing that portrait depth map so that you can then make it a portrait photo and post. So you get all of it all at once. And so you don’t have to make these trade-offs.
“We’re invested in ideas that emotionally resonate with folks, and we know some of these things take a little bit of time and we want to invest in things that feel exciting and feel creative and feel emotionally connected,” Sorrentino says.
“So to us, that is very much in the spirit of how Apple has done things for a very long time. So I’d be curious to look back at this at a later date and see if people are emotionally connected to it like we certainly hope.”
‘I think it’s almost the fullest expression of authentically recording a memory is when you record something spatially.’
Sorrentino says that spatial media is, obviously, something he feels very strongly about. But he explains that is just one thread that expands out from a web where at the center is just the idea of making something that brings joy to people.
“I think our job, certainly in the design and engineering end, is to just make sure that we are delivering delightful experiences that get people excited about it, want to try it, and want to tell friends and family. As long as we’re making delightful connective experiences, we feel really great about pushing this forward. And again, it being day one, we’ve got a long run ahead of us, but it’s one that we’re really excited to be on.”
Image and video credits: Apple
