Video Generation AI with ComfyUI: RIFE Edition

1 min read
Modified

RIFE is a powerful method for Frame Interpolation, commonly used to increase the frame rate and significantly improve the smoothness of generated videos. In this article, I’ll walk you through how to implement RIFE within your ComfyUI workflow.

Regarding technical detail of RIFE, since explained simply below, those interested please read here too.

動画のフレームレートを上げる技術:RIFEとそのアーキテクチャ

>-

blog.otama-playground.com

Work Procedure

1. Introduction of ComfyUI

Introduce ComfyUI by either method below. Those already introduced please update ComfyUI to latest version too.

【Stable Diffusion】ComfyUIを使って画像生成AIで遊んでみよう【導入編】

>-

blog.otama-playground.com

2. Installation of Extension

Install following extensions. (Video Helper Suite is for reading and saving of video)

GitHub - Fannovel16/ComfyUI-Frame-Interpolation: A custom node set for Video Frame Interpolation in ComfyUI.

A custom node set for Video Frame Interpolation in ComfyUI. - Fannovel16/ComfyUI-Frame-Interpolation

github.com
GitHub - Kosinkadink/ComfyUI-VideoHelperSuite: Nodes related to video workflows

Nodes related to video workflows. Contribute to Kosinkadink/ComfyUI-VideoHelperSuite development by creating an account on GitHub.

github.com

3. Create Workflow

Create RIFE VFI node, and connect like image below.

fps can be doubled by number of multiplier, but since precision, memory consumption, calculation time worsen if raising, keeping it small is safe.

Connection of RIFE VFI node
Connection of RIFE VFI node

4. Execute

Select video and execute

Generation Result

Since there was no good video lying around I used video generated before.

Before RIFE (8fps)
Before RIFE (8fps)
After RIFE (16fps)
After RIFE (16fps)

Conclusion

I think clear difference appeared in smoothness of video before and after Frame Interpolation.

Since low FPS/jerkiness concerned when generating video with AI is reduced considerably, it is one of methods that should be incorporated into workflow when wanting to generate quality video.