Recently, Hangzhou Qunkeng Technology has once again become the focus of the technology community with its open source spatial understanding model SpatialLM. This innovative model was specifically mentioned by Google in a research paper, marking its major breakthrough in robot training. The core function of SpatialLM is that it can allow robots to understand the geometric relationships of the physical world through ordinary videos, thus providing completely new possibilities for robot training.
SpatialLM is unique in that it can convert videos captured by mobile phones into precise three-dimensional spatial layout information. Users only need to record the scene at home with their mobile phones, and SpatialLM can generate a detailed 3D model, including the structure of the room, the location of the furniture, and the width of the channel. This technology not only greatly reduces the cost of robot training, but also significantly improves training efficiency, paving the way for the popularization and application of robot technology.
At the GTC2025 conference, CNK Technology also showed off its virtual training platform SpatialVerse. This platform combines the data generated by SpatialLM, allowing robots to train complex tasks such as obstacle avoidance and grabbing in a simulated environment, thereby achieving a complete closed loop from cognition to action. Through this system, robots can not only "see" the spatial layout, but also understand how to operate in complex environments, which provides powerful technical support for the application of robots in the real world.
The working principle of SpatialLM is based on MASt3R-SLAM technology, which disassembles video into countless frames, extracts details of objects such as sofas and tables, and builds them into a point cloud model. The model then converts this data into a structured 3D layout, recording key information about each object, such as size and position. Compared with traditional training methods, SpatialLM not only saves time and resources, but also significantly improves the robot's spatial cognitive ability, allowing it to better adapt to complex environments.
What’s unique about this technology is that it enables robots to understand and handle complex environmental changes like humans. Whether it’s everyday items in home life or tools in the workplace, SpatialLM helps robots adapt quickly and perform tasks. This capability is crucial to improving the performance of robots in real environments, especially in the current field of embodied intelligence, where many technologies are still facing difficulties in implementing them.
Through open source SpatialLM and SpatialVerse, CNK is reshaping the future of robot training, allowing it to flexibly respond to challenges in the real world. The widespread application of this technology will not only promote the further development of robotics technology, but will also bring more convenience to human life.
Project address: https://top.aibase.com/tool/spatiallm