Tencent recently announced the open source of its latest image-to-video generation framework - HunyuanVideo-I2V. This move marks an important step for Tencent to promote the development of open source communities, especially after its successful open source HunyuanVideo, which further demonstrates its innovative capabilities in the field of artificial intelligence.
HunyuanVideo-I2V combines the most advanced video generation technology at present, which can transform static images into vivid video content, providing creators with more creative possibilities. Users only need to upload a picture and briefly describe the dynamic effect of the picture to generate a five-second short video. The feature of this model is that it not only allows the static pictures to "move", but also can be automatically matched with background sound effects, greatly enhancing the fun and attractiveness of the video.
HunyuanVideo-I2V utilizes a pre-trained multimodal large language model as a text encoder, significantly enhancing the model's ability to understand the semantic content of the input image. This means that the user input images can generate semantic image markers through the model, which are combined with video potential markers, thereby achieving a more comprehensive full attention calculation. In this way, the system can maximize the synergy between image and text modality, ensuring that the video content generated from static images is more coherent and realistic.
In order to allow more users to experience this function, the official Hunyuan AI Video website has been launched, and users can directly access the website to operate. In addition, enterprises and developers can also apply for API interfaces through Tencent Cloud to integrate this technology into their applications. This Tusheng video model is a continuation of the open source work of Hunyuan Wensheng video model. The total model parameters reach 13 billion, which is suitable for generating various types of characters and scenes, covering realistic videos, animation characters and CGI characters.
During the specific use process, users can also upload characters and enter text or audio that they want to "mouth" in their "lip-syncing". The system can make the characters in the picture "speak" or "sing". At the same time, Hunyuan has also launched the "action-driven" function, where users can generate corresponding dance videos with one click to improve the diversity and fun of creation.
It is worth mentioning that the open source Tusheng video model has been released in mainstream developer communities such as Github and HuggingFace. Developers can download related content for experimentation and development. Open source content includes model weights, inference codes, and LoRA training codes, which provide developers with more possibilities to train exclusive LoRA models on this basis.
Since open source, the popularity of HuggingFace's generation model has been increasing. In December last year, it topped the top of HuggingFace's trend list, and the number of Stars on Github has exceeded 8.9K. Many developers are also actively making plug-ins and derivative models for Hunyuanvideo, and have accumulated more than 900 derivative versions. The open source Hunyuan DiT literary graphics model earlier also performed well, with more than 1,600 derivative models.
Official website: https://video.hunyuan.tencent.com/
github: https://github.com/Tencent/HunyuanVideo-I2V
huggingface: https://huggingface.co/tencent/HunyuanVideo-I2V