Recently, Alibaba Tongyi Laboratory has made a major breakthrough in the field of 3D human body reconstruction and launched an innovative technology called LHM (large animated human body reconstruction model). This technology not only brings a new development direction to this field, but also greatly expands its application prospects.
Animated 3D body reconstruction from a single image has always been a challenging task, mainly due to the ambiguity of geometry, appearance and deformation separation. Most of the current latest research progress focuses on static human modeling, and these methods often rely on synthetic 3D scans for training, which largely limits their ability to generalize in real scenarios. At the same time, the optimization-based video method not only requires strict control of the capture conditions, but also has extremely intensive computing processes, making it difficult to meet the needs of practical applications.
In response to these difficulties, the LHM model came into being. The model innovatively adopts a multimodal converter architecture, and through a powerful attention mechanism, it can effectively encode human position features and image features. This architecture allows LHM to not only accurately reconstruct the geometry of the human body, but also preserve the geometric shapes and textures of the clothing in detail, presenting users with a more realistic and delicate 3D mannequin.
It is worth mentioning that LHM also proposed a head feature pyramid coding scheme. This solution further improves the model's ability to capture the details of the human head by aggregating the multi-scale characteristics of the head area, making the generated 3D human body model more realistic in the head performance. In practical applications, LHM shows amazing efficiency, able to generate reasonable animations of human bodies in just a few seconds, and no complicated post-processing is required, greatly saving time and labor costs.
After a lot of experimental verification, LHM is better than existing methods in terms of reconstruction accuracy and generalization ability. Whether in complex scenarios or under different lighting conditions, LHM can stably output high-quality 3D human body reconstruction results.
Address: https://lingtengqiu.github.io/LHM/