```html

Improved Motion Representation in Video Models: VideoJAM Sets New Standards

Generative video models have made enormous progress in recent years. However, they still struggle to convincingly represent realistic movements, dynamics, and physical laws. A recently published paper titled "VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models" sheds light on this problem and presents a promising solution.

The authors of the study argue that the conventional pixel reconstruction method used in many generative video models creates a bias towards visual representation. This focus on image quality often comes at the expense of motion accuracy and coherence. To counteract this problem, the researchers propose VideoJAM, a novel framework that equips video generators with an effective motion prior.

VideoJAM consists of two central components. First, during the training process, the objective is expanded to predict not only the generated pixels but also their corresponding motion from a single learned representation. Second, during inference, a mechanism called "Inner-Guidance" is employed. This utilizes the motion prediction developed by the model itself as a dynamic guidance signal to steer the generation towards more coherent motion.

A notable advantage of VideoJAM is its broad applicability. The framework can be applied to various video models with minimal adjustments, without requiring changes to the training data or scaling of the model. The results of the study show that VideoJAM significantly surpasses the current state of the art in terms of motion coherence and even outperforms highly competitive proprietary models. Simultaneously, the framework also improves the perceived visual quality of the generated videos.

The researchers emphasize that visual representation and motion are complementary aspects. Through their effective integration, both the visual quality and the coherence of video generation can be improved. The results of VideoJAM suggest that considering motion priorities is key to developing more realistic and convincing generative video models.

VideoJAM was fine-tuned with only 3 million samples and yet achieves state-of-the-art results in motion generation and understanding. This indicates great potential for further improvements through more extensive training data.

The development of VideoJAM is an important step towards more realistic and expressive video generation. The ability to accurately represent motion and physics opens up new possibilities for applications in various fields, from the entertainment industry to scientific research. Future work could focus on further optimizing the framework and exploring its application in specific use cases.

Bibliographie: https://hila-chefer.github.io/videojam-paper.github.io/ https://chatpaper.com/chatpaper/zh-CN?id=4&date=1738684800&page=1 https://arxiv.org/abs/2411.08328 https://www.researchgate.net/publication/334434830_Stylizing_video_by_example https://huggingface.co/papers/2411.08328 https://arxiv.org/abs/2211.12748 https://x.com/hila_chefer https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/06030.pdf https://github.com/AlonzoLeeeooo/awesome-video-generation https://www.vdb.org/sites/default/files/2020-04/Rewind_VDB_July2009%202.pdf ```