The Future of Remote Collaboration: SharedNeRF

Collaborating on physical objects remotely has always been a challenge. The ability to work together on complex tasks, such as debugging hardware, when two people are not in the same room can be extremely difficult. However, a new system called SharedNeRF is changing the game when it comes to remote conferencing. Developed by Mose Sakashita, a doctoral student in information science, SharedNeRF allows the remote user to manipulate a 3D view of the scene to assist in tasks that were previously considered impossible to convey through traditional video-based systems.

SharedNeRF combines two graphics rendering techniques to create a unique experience for remote collaborators. By merging a slow and photorealistic rendering method with a faster but less precise technique, SharedNeRF offers a new way for users to experience physical spaces in real-time. The system has the potential to revolutionize the way people work together on tasks involving physical objects that require a high level of precision and detail.

SharedNeRF leverages a graphics rendering method known as a neural radiance field (NeRF). This method uses artificial intelligence to construct a 3D representation of a scene based on 2D images. The result is a highly realistic depiction of the scene, complete with reflections, transparent objects, and accurate textures. The local collaborator wears a head-mounted camera to capture the scene, which is then rendered in 3D for the remote user. This allows the remote user to view the scene from any angle and make real-time adjustments as needed.

Seven volunteers tested SharedNeRF by participating in a collaborative flower-arranging project with a partner. The results showed that when compared to standard video conferencing tools or point cloud rendering alone, five of the volunteers preferred SharedNeRF. Users appreciated the ability to independently change viewpoints, zoom in and out on the scene, and see real-time movements through point cloud rendering. The system also includes an avatar of the local collaborator’s head, giving the remote user insight into where they are looking.

While SharedNeRF is currently designed for one-on-one collaboration, the researchers behind the system see the potential for expansion to multiple users in the future. Future work will focus on improving image quality and offering a more immersive experience through virtual reality or augmented reality techniques. SharedNeRF has the potential to transform the way people collaborate remotely, making complex tasks more manageable and opening up new possibilities for remote work.