Projects
Experience the Future of Shopping Now The Behind-the-Scenes Story of the "Spatial Commerce" Development Enabled by Apple Vision Pro (Part 1)
A new project to reinvent the shopping experience kicked off in 2024. In this second phase, the fashion brand meanswhile teamed up to develop a groundbreaking application for Apple Vision Pro, where users can freely style clothes and even receive remote customer support.
What kinds of challenges and discoveries lay behind this unprecedented endeavor? We interviewed the project team in charge of planning and development to find out more.
PROFILE
Yudai Tamamura
After working on video production for games and in the 3DCG field at a major IT megaventure, he joined ZOZO Technologies Inc. (now ZOZO NEXT Inc.) in 2020. Since May 2023, he has been in his current role, promoting projects mainly in the digital fashion field within the XR/AI division.
PROFILE
Pablo Trabajos
Software engineer. Joined in 2022 and has been involved in XR projects. For this project, he developed customer service functionality utilizing video streaming and remote control, as well as UX improvements. Previously, he worked on web frontend and iOS app development.
PROFILE
Jun Ban
At ZOZO Inc., he led the development of the ZOZOTOWN iOS app as a technical lead specializing in iOS. Later, as a member of the Skeuomorphic Engineering team at ZOZO NEXT, he worked on bringing realistic textures into software and was involved in visionOS app development.
PROFILE
Yuki Ikegami
After experience in planning and pattern making at apparel brands, she joined ZOZO Technologies Inc. (now ZOZO NEXT Inc.) in 2020. In the XR/AI division, she serves as PM and handles 3D data creation for projects involving digital fashion and XR fields.
The Motivation: Building on Past Success for Even Greater Experiences
First, can you tell us how this latest project got started?
Tamamura: The first phase of this initiative began in June 2024. By November of the same year, we collaborated with the fashion brand ANREALAGE to develop a Vision Pro app for future shopping experiences and offered it to the public as a pre-order event for the brand’s new collection. The positive feedback we received from this allowed us to see new potential in the realm of Spatial Commerce.In the first round, the main focus was on letting users carefully examine a product in 3D. This second project builds on that, going one step further by adding new features to expand the experience.
Specifically, what kind of experience did you aim to expand this time?
Tamamura: This time, we added a more interactive "styling function" that lets users take a hands-on approach. Using Meanswhile’s 25A/W items featured in the collaboration, users can freely create and coordinate their own looks. This not only enables shoppers to examine individual products, but also enjoy the creative fun of planning and previewing full outfits as they consider their purchases.
Ikegami: Another major new feature is the "customer support function." Last time, staff could view the user's Vision Pro screen from beside them; this time, staff can operate on an iPad and have those actions reflected directly on the Vision Pro display that the user is seeing. This allows staff to build outfits on the spot tailored to each user and make styling suggestions within Vision Pro in real time, creating a more interactive and high-quality customer experience.
Unprecedented, Real Challenges from the Development Frontline
How did you feel as a developer when you first heard about the project?
Pablo: It was tough, but I saw it as a challenge worth overcoming. I had heard, in the early planning stages, about the idea of staff using an iPad to remotely control Vision Pro—but my first impression was, "I've never done anything like that with Vision Pro!" Even with my many years of engineering experience, I had never worked on a similar project, so I felt there were a lot of unknowns.What technical hurdle was the most difficult part of the project?
Pablo: The most difficult part was the integration between iPad and Vision Pro, specifically syncing iPad operations onto Vision Pro, which was my responsibility. It started with researching and selecting the right technologies, and understanding how to utilize Apple’s enterprise APIs and their extensions involved repeated experiments, which was honestly a struggle.For example, running these extensions places a heavy load on the system, yet there are strict memory limits—if the code exceeds around 50MB the app crashes. It was a real challenge to maintain the balance between such constraints and resource-intensive processes like video encoding.
However, since the success of this feature determined whether we could deliver one of the app’s highlights—the "customer service feature"—I wasn’t willing to say, "This can’t be done." I spent a great deal of time testing and optimizing.
The Unique Challenges of 3D Spaces—Finding the Optimal Balance through Trial & Error
Ban, you were mainly in charge of the feature development inside Vision Pro. How did you approach it?
Ban: Rather than facing specific challenges, each step confronted more fundamental issues unique to 3D development. Even if I designed a 3D object on a Mac screen and knew it would display on Vision Pro, it was very common to find, once I tested it on the real device, the outcome would differ from what I imagined.The impression of colors, as well as size and sense of distance—there were so many aspects where I wouldn’t know the real look until testing on the actual hardware. When correcting those mismatches, it was hard to determine whether tweaking a few values would suffice or if I’d need to completely remake the model. This loop of trial and error—how far back to go and what to fix—grew into quite a major effort in this project and was a real struggle.
Were there any instances where you could leverage knowledge from past projects?
Ban: My previous experience, such as conducting personal experiments and releasing the "Ambient Weaving Ⅱ" Vision Pro app—where I test-implemented textile simulations—was extremely helpful. Thanks to that, I developed a sense for efficiently changing colors and textures while maintaining performance and could quickly apply those learnings here.Also, since I had already developed a Vision Pro app before, I had a grasp on what was and wasn’t possible. This allowed me to avoid wasting time on impossible-to-realize ideas and stay focused from the beginning.
What creative efforts did you make in implementation to maximize the user experience?
Ban: Rather than focusing on specific features, I was very conscious overall about maintaining a sense of balance. For example, I repeatedly tested and adjusted the speed and movement of scene-switching animations from the moment users wear Vision Pro to when they begin their in-app experience, ensuring those transitions would feel pleasant.Also, given that Vision Pro imposes few restrictions, it’s possible to design any kind of interaction. However, keeping in mind that in the future, many different users would be interacting with it, I intentionally avoided overly unique operations, and instead stuck to standard workflows—a range where anyone can intuitively use the device.
Designing a Unified Experience that Connects MR & VR—Going Beyond Just an App
What exactly makes this app fundamentally different from apps on other devices?
Ban: Within a single app, you can seamlessly transition from Mixed Reality (MR), where information is layered over real spaces, to a fully immersive Virtual Reality (VR) shopping experience. The fact that we can deliver this entire flow in one go is something uniquely possible with Vision Pro, in my view. In the previous version, the space was entirely virtual, but this time you get to experience both MR and VR—it's a way to take full advantage of the device's distinctive features.What potential do you see in Spatial Commerce through this project?
Pablo: I hope this app experience will prove beneficial for both staff and customers. I also expect that the new customer service functionality will boost brand expressiveness. Down the line, as this feature is optimized and adopted more widely, it could also help ease the labor shortages that challenge the retail industry as a whole.Ban: As a developer, we experimented with several technologies that ultimately were not included in the final product, so it’d be exciting to try incorporating those in future updates. In terms of the overall experience, I think this Vision Pro experience provides one of the most convincing solutions currently available for the major challenge in e-commerce: buying without seeing the real thing. It would be fascinating if anyone could access this at home in the future, but for now, I’m hoping that more people will get to experience this value firsthand through demo events or in physical stores.
Finally, as developers, are there any technologies or themes you'd like to take on next?
Pablo: Going forward, I'd like to utilize technologies and languages beyond just Xcode, such as Unreal Engine, which makes richer visual expression possible. I'm also interested in linking virtual experiences to real-world spaces. For example, being able to control real lighting from within the app, spray scents, or sync physical fabric samples with in-app products via QR codes—these sorts of experiences fascinate me.
Ban: For the fashion theme that we handle, I believe there's still much room to add expressions like dynamic fabric movement. If we could drive more realistic physical simulations, it would make things even more engaging. I also want to pursue uniquely virtual experiences, such as new interfaces to quickly discover one item among a huge product lineup, or ways to dig deeper and enjoy selected products in ways not possible in real life.
For those curious to try this cutting-edge technology and get a taste of these new experiences, a demo event will be held at ( tefu ) lounge in Shimokitazawa from Friday, July 25th to Wednesday, July 30th, 2025.
Don't miss your chance to experience firsthand the future of shopping brought to life by the project team's innovations, as discussed in this interview.
Event Outline:
ZOZO NEXT "THE SPATIAL STORE" Experience the Future of Shopping with meanswhile
Dates: July 25 (Fri) – July 30 (Wed), 2025
-July 25 (Fri) 1:00 PM – 8:00 PM (last entry 7:00 PM)
-July 26 (Sat) – July 29 (Tue) 11:00 AM – 8:00 PM (last entry 7:00 PM)
-July 30 (Wed) 11:00 AM – 5:00 PM (last entry 4:00 PM)
Venue: ( tefu ) lounge Shimokitazawa (2-21-22 Kitazawa, Setagaya-ku, Tokyo, 155-0031)
Application page URL: https://zozonext.com/project/the-spatial-store
The second half will feature an interview with the project team members supporting spatial design and 3D product model development, delving into the appeal and behind-the-scenes of the future shopping experience.
How was the article?
Share
Related Articles
.jpg?w=600&fm=webp&dpr=2)
