Table of Contents for Mixed Reality and the Apple Vision Pro - Future of CGI / 3D?.:
- What is Apple Vision Pro?
- What does Spatial Computing mean?
- Meaning Advanced Perception
- Apple Vision Pro operation
- Who is the Apple Vision Pro interesting for?
- So, with Mixed Reality, is the Apple Vision Pro revolutionizing the way we work?
- No real sight - all camera and CGI
- 3D Content and Mixed Reality - User and Creator
- FAQ
With the Apple Vision Pro, the Cupertino-based company wants to launch mixed reality glasses in the near future that can also be used without Apple devices. The market launch in the USA is planned for 2024, with a few selected countries to follow. It will be a while before the glasses, which are expected to cost $3,500, are available in Germany. What can the Apple Vision Pro do? And what do tools like this mean for CGI/3D?
What is Apple Vision Pro?
If you're expecting glasses for AR, you're wrong. Apple has created what the company itself calls a spatial computer with the Vision. The Vision Pro doesn't plug into your iPhone or Mac so you can just see a little more. Rather, the Vision Pro is your new Mac. The glasses come with their own operating system and process input from multiple sensors and cameras just like what you're doing with your new spatial computer there, via an M2 and an R1 chip. What you're doing is displayed on two OLED displays (one for each eye), each with a resolution of just over 4K. This is pretty close to natural vision. But what exactly do you see there?
What does Spatial Computing mean?
Spatial computing, also known as spatial computing, is a type of information processing that uses and embeds in the physical world in which we move and interact.
This includes the use of technologies such as augmented reality (AR), virtual reality (VR), mixed reality (MR) and 3D computing to create an interface between the digital and physical worlds. In a spatial computing environment, for example, a user can navigate through a virtual landscape or interact with virtual objects as if they were physically present.
Spatial computing can have a variety of applications, including gaming, design, architecture, medicine, education, furniture/home/living and many more. It enables an immersive, interactive experience that goes far beyond what is possible with traditional two-dimensional screens.
Meaning Advanced Perception
If the Vision Pro is your new Mac, you can think of the space around you as your new display. Because what you do with your computer or the glasses, you see integrated into the space around you. So you don't need a display anymore. And you decide for yourself how big what you see is. Multiple screens, panoramas that run around you and fill the whole room - that's no problem. The glasses will show you images with a depth and size that will go beyond the walls of the room you're actually in. Nevertheless, you can always perceive what's going on in the real space around you - you are completely present.
The acoustics are also three-dimensional and include the space around you. Sensors try to create as accurate a map as possible of the surfaces, textures and materials around you. The sound is emitted via small speakers that sit in the temples of the glasses directly above your ears. The sound is emitted in a targeted manner, but is always individually adjusted to the room conditions. As a result, you also hear three-dimensionally and basically find yourself in the middle of the action. Unlike headphones with Active Noise Cancellation, you can always hear what's happening around you. You are therefore still responsive and participate in the events around you.
We won't go into the technical details here, you can get them elsewhere on the web at any time - Apple product page.
Apple Vision Pro operation
The Vision Pro does not have a keyboard, and you will search in vain for a mouse. You don't need either to work with this new type of computer. The glasses react to your eye movements. You can control apps just by fixing them. If you now press your fingers together, you select what you fixate with your eyes. Swipe, zoom in, zoom out - the gestures work flawlessly. You can use the Vision Pro without any problems while lounging on the sofa, sitting at the kitchen table or walking through the room. Thanks to the many integrated sensors and cameras, the device always knows exactly what your eyes and hands are doing.
Who is the Apple Vision Pro interesting for?
The Apple Vision Pro is not exactly inexpensive with an entry-level price of $3,500. As a nice gadget next to desktop, laptop, iPad and iPhone, you can't afford it just like that. You have to be convinced of the possibilities that this novel computer offers you. Applications will initially be more in the area of existing apps and entertainment. Nevertheless, Apple is also planning to work with the Vision Pro. Video meetings, office software, design software - it's all available. You put on the Vision Pro, press the little button on the top of the frame of the glasses and find yourself in the middle of a mixed reality. The menu immediately appears in front of you. It is remarkable that the physical pressure on the button does not cause the image to wobble. Nothing slips or jerks away, and the picture is absolutely stable.
So, with Mixed Reality, is the Apple Vision Pro revolutionizing the way we work?
Yes and no. First of all, everything doesn't look much different than usual. The apps with their user interfaces look the same as always, just standing in front of you in the room and as big as you'd like them to be right now. There are no apps that are only available here - so far. And of course it's comfortable to look a little to the right and see the three people from the video meeting on Facetime right where the sofa is, while at the same time your work screen is visible directly in front of you and a little further to the left you can see the screens of your conversation partners quite large. That's comfortable. But what is really new and unusual about it? This was already possible with several (large) screens or other hardware.
No real sight - all camera and CGI
What's really new - and this is where it gets interesting for the CGI/3D industry - is that you never really see the room around you. Or your conversation partners in Facetime. You only ever see generated images. Apple's Vision Pro is made of really heavy, good glass and an aluminum frame, to be sure. But it's not suitable for looking through. OLED displays are in front of your eyes. They're just not see-through. What you see is the image that the cameras in the glasses record around you and play back to you on the displays - in real time. The people you talk to in the meeting don't see you. They see a persona that Vision Pro has created of you, showing your facial expressions and gestures. They don't see that you're wearing the Vision Pro while you're in a video meeting with them - because the glasses are so big that they can't see your facial features and expressions.
Keyword immersion: You determine how much of your surroundings is visible behind the various open windows via the Digital Crown. Simply turn it to expand or shrink the virtual reality area. Since Siri is integrated, you can also look at the microphone and give voice commands.
So far, there are only a few applications that really take advantage of the Apple Vision Pro. Disney, for example, has created additional content that can be explored with this new computer and ties in with the familiar merchandise. The strengths of immersion are already pretty well exploited here - this has entertainment value.
The Vision Pro can do 3D. The two OLED displays show 3D as it was probably originally intended: The depth and sharpness of the content is incredible. The Vision Pro is equipped with 3D cameras so that corresponding pictures can be taken. However, these pictures are static in a certain sense: If you record a scene in front of you, you can change your viewing angle later by tilting your head slightly. But this only works on a small scale so far, you cannot completely change your viewing angle, you can neither enter the scenes nor go around to the other side.
3D Content and Mixed Reality - User and Creator
Two scenarios can be distinguished in terms of CGI / 3D and Vision Pro: First, there is an incredibly hungry, tech-savvy audience that wants to be entertained and enjoys the extreme image brilliance and image stability, depth of field and three-dimensional impact of these glasses and wants content for it. In terms of the digital spatial experience, the Apple Vision Pro is an exceedingly interesting presence, without question. This leads quite naturally to the second scenario: someone has to create the content. Because of course, it won't just be about the landscape shots from the last vacation or the next James Bond flick when more and more people acquire this spatial computer. Apple is already promoting the device by saying that the visual content of the mindfulness app can be expanded across the entire room, making relaxation exercises even more effective. These aren't images taken with cameras - they're CGI, computer generated images.
What is really new and interesting is not the product Apple Vision Pro, but the approach how Mixed Reality is understood here. The spatial computer, as Apple calls the glasses, is much more intuitive and natural to use than anything else on the market so far. The natural body feeling is less disturbed, the operation is ergonomic in the deepest sense of the word due to the naturalness of the movements, the interactions with the Vision Pro. The transition between Real Life and Virtual Reality is so natural that it poses a challenge to CGI/3D. And we gladly accept it!
Because, and this is the third scenario, of course the Vision Pro is quite interesting as a working tool for people who create CGI/3D images. The various features may not revolutionize the work with the currently available software. But they do make it much more enjoyable. When the smallest details can move from the screen into space and be resized to any size without losing their sharpness, it makes it possible to work in a completely different way. When hand-eye coordination is no longer necessary when working graphically with a mouse or drawing pencil, because integrated cameras follow the tiny movements of the pupil, then this opens up completely new possibilities.
