Skip to content

MC-E/ReVideo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

21 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

ReVideo: Remake a Video with Motion and Content Control

Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, Jian Zhang

Project page arXiv


Introduction

ReVideo aims to solve the problem of local video editing. The editing target includes visual content and motion trajectory modifications.

πŸ“° New Features/Updates

  • [2024/06/26] We release the code of ReVideo.
  • [2024/05/26] Long video editing plan: We are collaborating with Open-Sora Plan team to replace SVD with Sora framework, making ReVideo suitable for long video editing. Here are some preliminary results. This initial combination is still limited in quality for long videos. In the future, we will continue to cooperate and launch high-quality long video editing models.
Generated by Open-Sora Editing Result
sora.mp4
sora_edit.mp4
- [2024/05/23] Paper and project page of **ReVideo** are available.

✏️ Todo

  • Code will be open-sourced in June

πŸ”₯πŸ”₯πŸ”₯ Main Features

Change content & Customize motion trajectoy

Input Video Editing Result
desert.mp4
desert.mp4

Change content & Keep motion trajectoy

Input Video Editing Result
woman2.mp4
woman2.mp4

Keep content & Customize motion trajectoy

Input Video Editing Result
head5.mp4
head5_2.mp4

Multi-area Editing

Input Video Editing Result
lawn2.mp4
lawn2.mp4

πŸ”§ Dependencies and Installation

pip install -r requirements.txt

⏬ Download Models

All models will be automatically downloaded. You can also choose to download manually from this url.

Since our ReVideo is trained based on Stable Video Diffusion, the usage guidelines for the model should follow the Stable Video Diffusion's NC-COMMUNITY LICENSE!

πŸ’» How to Test

You can download the testset from https://huggingface.co/Adapter/ReVideo. Inference requires at least 20GB of GPU memory for editing a 768x1344 video.

bash configs/examples/constant_motion/head6.sh

Description of the same parameters

--s_h  # The abscissa of the top left corner of the editing region
--e_h # The abscissa of the lower right corner of the editing region
--s_w # The ordinate of the top left corner of the editing region
--e_w # The ordinate of the lower right corner of the editing region
--ps_h # The abscissa of the start point
--pe_h # The abscissa of the end point
--ps_w # The ordinate of the start point
--pe_w # The ordinate of the end point
--x_bias_all # Horizontal offset of reciprocating motion
--y_bias_all # Vertical offset of reciprocating motion

Related Works

[1] https://pika.art/

[2] DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory

[3] DragAnything: Motion Control for Anything using Entity Representation

[4] AnyV2V: A Plug-and-Play Framework For Any Video-to-Video Editing Tasks

πŸ€— Acknowledgements

We appreciate the releasing code of Stable Video Diffusion.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published