When connected to the Vision, mobile phone not only serves as the computing unit, but also becomes a multimodal input device, inherits human-computer interfaces from existing methods, such as the gamepad, trackpad, laser pointer, and even keyboard. Thus, we built RADA, a series of intuitive, accessible, preinstalled applications and visual representation to demonstrate the interaction and performance of this combination.
Assault is a multiplayer, room-scale, mobile AR, and FPS game. The game was designed and developed in the way of demonstrating the products' capabilities of tracking, display, and interaction with minimal user input. By simulating the gamepad on the touch screen of the phone, the intuitive interaction made it suitable for all kinds of demonstration scenarios.
The application is the default launcher when an Android phone detects the connection with Rokid Vision. It turns the phone into a 3DOF pointer with touch gesture input. By scanning apps with specialized bundle name, it automatically generates app icons with cutout layers to produce a parallax effect.
A web browser works much more than a webpage viewer nowadays, but with various web applications, it becomes an essential productivity tool. The launcher has a built-in browser to empower users to proceed with their daily activities on the web.
When the browser launches, the mobile phone turns into a trackpad with all the gesture we are familiar with. While typing is needed, the phone will launch its built-in keyboard or can be connected to a Bluetooth keyboard if possible.
This feature is part of the research project - WEAVE, an Web AR framework based on WebXR.
Read more about Project WEAVE.
