Engineers from Microsoft Research has unveiled a new virtual reality system for blind and visually impaired people. Compared to the previous version, which allowed to move the virtual stick only in the horizontal plane, in a new three brake through which the cane can be raised, lowered, as well as to attract and set almost perpendicular to the ground. The system has integrated vibration motor, which allows you to get a haptic response from a contact of the cane with surfaces, and is complemented by interaction with the environment of pre-recorded sounds. Seven of the eight visually impaired participants tested a virtual environment, handled the navigation without collision, reported on the website of the company.
In 2018, the company Microsoft first showed Canetroller — a virtual reality system for blind and visually impaired people. The main tool of this system — stick, with which user contact with the environment: it’s attached to the belt fixed on the metal rod, which moves freely. The rod is attached to the vibration motor and to the cane — sensor: collision with virtual objects a reed stop, simulating the collision of the real. In addition, this system allows you to identify some of the surface: for this contact, for example, a virtual paving slabs in the headphones, the user is sent to the corresponding sound.
The main disadvantage of Canetroller — a limited number of degrees of freedom: the cane in such a system can be moved only in the horizontal plane, which can greatly complicate navigation in large virtual space. The new navigation system, created with the participation of Mike Sinclair (Mike Sinclair), who worked on Canetroller, cane design remained roughly the same, but in addition to the brakes in a horizontal space, added two more vertical brake at the base of the metal rod and the brake on the end of the stick in place of fixing on the web.
This design allowed to significantly increase the number of degrees of freedom: the cane can be moved horizontally and vertically and also toward you. The increase in degrees of freedom not only extends the range of the interaction, but also allows, for example, put a stick perpendicular to the ground.
As in Canetroller, the new system has the ability to simulate contact with different surfaces using the cane vibration motor and pre recorded sound. Interestingly, the new system allows us to distinguish not only the surface but also the movement that is used to contact with them: for example, the user can tap or hold the virtual stick by the virtual concrete and sounds, and haptic response will be different.
To test the operation of the new system, the researchers created a complex virtual environment with a size of six by six metres: it has a few different obstacles in the form of blocks, walls or partitions, and in addition to haptic response obtained by the vibration feedback from the canes, to navigate using the sounds and their dynamics in space. Environment tested eight visually impaired people: seven of them managed to successfully complete the task (to reach the subject), without colliding with the virtual obstacles.
User experience in a virtual environment is also highly dependent on the material of the cane tip: parties that for navigation in a virtual reality used the cane with the plastic tip, often reported strange sensations when in contact with objects. This, according to scientists, due to the fact that the sound recording contact surfaces and vibration used a cane with metal tip. Those participants who got the cane with metal tip, in turn, noticed anything strange: it is, among other things, talks about the importance of selection of material for interaction with the virtual environment. More details about the new system of virtual navigation can also be found in the Preprintposted on the website of the company.
Another useful device for the blind and visually impaired smart glasses with a built-in speech recognition and voice assistant, such as those recently introduced by the Dutch company Envision: their recognition system they are embedded in Google Glass, Enterprise Edition.