NIANTIC unveils a lightweight prototype AR headset powered by a Snapdragon AR2 Gen 1 processor

Pokemon GO and Pikmin Bloom maker Niantic has released a prototype model of lightweight AR glasses for outdoor use.

Equipped with a cooperating Qualcomm AR processor, it weighs about 250g and is compact enough to fold the headband.

If you wear it, the game character appears on real terrain. The real signboard is supplemented with the latest online information. A line of light is drawn on the road to guide the route. The "Real World Metaverse" as Niantic calls it, will be realized.

Niantic showed a demo video of the new AR glass prototype on the stage of the event Snapdragon Summit hosted by chip maker Qualcomm.

Niantic, a pioneer in in-game content and AR development platforms, and processor maker Qualcomm have a cooperative relationship in the development of hardware and software for AR headsets.

The "VPS" function of Niantic's AR platform Lightship will be compatible with Qualcomm's XR platform Snapdragon Spaces from 2023 according to this announcement.

Niantic's Lightship VPS is a "Visual Positioning System" function that acquires the exact position of the terminal and the surrounding environment from the camera image.

While GPS location data used in maps, etc., relies on signals from artificial satellites, so it can be off by a few meters to a few tens of meters, VPS registers points in a video captured by the camera and can determine their precise position in centimeters by matching the location data against a detailed 3D map of the world.

In addition to positional information in the sense of a map, it can also precisely recognize the relative positional relationship between the terminal and the complex terrain and three-dimensional shapes of objects in the real world, so digital information and characters can be naturally blended into the real world. It can be said that it is a key technology for augmented reality that enables.

The demo video features something like a game-like robot character flying around, extra information superimposed on billboards, and lines drawn on the road to show the route.

It's not very easy to understand what's different from smartphone AR such as Pokemon GO, but multiple people can see the same AR information and characters at the same time as if they were in the same place. It is technologically advanced that the operation is reflected in real-time by everyone, and that it is not vaguely "flat place" but is tailored to a specific signboard or distant building.

Also, in this screenshot where the robots are playing a camp game, pay attention to the flag standing on the roof above the viewpoint and the robots fighting. The expression of occlusion that hides the robot's feet is also used.

In the current simple AR of smartphones, horizontal planes and walls within the range captured by the smartphone's camera and sensors are recognized and used as a reference (pointing the camera toward a flat surface, etc.). In principle, it is difficult to match something to an invisible rooftop, or it is an expression that cannot be realized without special processing.

In the demo video, VPS already has 3D shape data for this place, and the robot is standing in the invisible part, or semantic segmentation that separates "building", "sky", "ground" etc. The expression "on the boundary line of the building" appears to be superimposed over the "empty part of".

(If you look closely, the real shape and the boundary recognized by AR are slightly misaligned. On the lower right side of the screen, at the feet of the robot being shot, the real roof spreads out in front of you. On the other hand, in AR, it is processed linearly, and the feet of the robot that should be hidden are drawn in front).

The prototype looks like a bulky pair of goggles, but it weighs about 250g and can be folded compactly.

The AR glasses prototype that Niantic unveiled is a reference design that hardware manufacturers refer to when developing their own products. Niantic itself does not sell AR glasses hardware.

The features of the first AR-only processor Snapdragon AR2 announced by Qualcomm are:

• Distributed processing architecture. 40% reduction in substrate size with multi-chip configuration, contributing to the miniaturization of wearable devices. Supports low-latency WiFi 7 and offloads processing to smartphones and PCs with low latency. Tasks that require real-time performance are distributed on the device side, and advanced processing is distributed to smartphones, etc.
• AI performance is 2.5 times higher than Snapdragon XR2. In AR, many sensors and AI processing work in parallel to recognize the real environment, so AI performance is important.
• Low power consumption. Half the XR2 ratio. Manufactured at 4nm process node

In addition to Niantic, partner companies include Lenovo, LG, Nreal, Pico, Oppo, QonoQ, Sharp, etc., and it seems that we can expect great things from each company's glasses terminals.