Nodes Browser
ComfyDeploy: How ComfyUI-IF_MemoAvatar works in ComfyUI?
What is ComfyUI-IF_MemoAvatar?
ComfyUI MemoAvatar is a talking head avatar generator using Memory-Guided Diffusion for Expressive Talking Video Generation
How to install it in ComfyDeploy?
Head over to the machine page
- Click on the "Create a new machine" button
- Select the
Editbuild steps - Add a new step -> Custom Node
- Search for
ComfyUI-IF_MemoAvatarand select it - Close the build step dialig and then click on the "Save" button to rebuild the machine
ComfyUI-IF_MemoAvatar
Memory-Guided Diffusion for Expressive Talking Video Generation
#ORIGINAL REPO MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation <br> Longtao Zheng*, Yifan Zhang*, Hanzhong Guo, Jiachun Pan, Zhenxiong Tan, Jiahao Lu, Chuanxin Tang, Bo An, Shuicheng Yan <br> Project Page | arXiv | Model
This repository contains the example inference script for the MEMO-preview model. The gif demo below is compressed. See our project page for full videos.
<div style="width: 100%; text-align: center;"> <img src="assets/demo.gif" alt="Demo GIF" style="width: 100%; height: auto;"> </div>ComfyUI-IF_MemoAvatar
Memory-Guided Diffusion for Expressive Talking Video Generation
Overview
This is a ComfyUI implementation of MEMO (Memory-Guided Diffusion for Expressive Talking Video Generation), which enables the creation of expressive talking avatar videos from a single image and audio input.
Features
- Generate expressive talking head videos from a single image
- Audio-driven facial animation
- Emotional expression transfer
- High-quality video output
https://github.com/user-attachments/assets/bfbf896d-a609-4e0f-8ed3-16ec48f8d85a
Installation
*** Xformers NOT REQUIRED BUT BETTER IF INSTALLED*** *** MAKE SURE YoU HAVE HF Token On Your environment VARIABLES ***
git clone the repo to your custom_nodes folder and then
cd ComfyUI-IF_MemoAvatar
pip install -r requirements.txt
I removed xformers from the file because it needs a particular combination of pytorch on windows to work
if you are on linux you can just run
pip install xformers
for windows users if you don't have xformers on your env
pip show xformers
follow this guide to install a good comfyui environment if you don't see any version install the latest following this free guide
Installing Triton and Sage Attention Flash Attention
Model Files
The models will automatically download to the following locations in your ComfyUI installation:
models/checkpoints/memo/
├── audio_proj/
├── diffusion_net/
├── image_proj/
├── misc/
│ ├── audio_emotion_classifier/
│ ├── face_analysis/
│ └── vocal_separator/
└── reference_net/
models/wav2vec/
models/vae/sd-vae-ft-mse/
models/emotion2vec/emotion2vec_plus_large/
Copy the faceanalisys/models models from the folder directly into faceanalisys
just until I make sure don't just move then duplicate them cos
HF will detect empty and download them every time
If you don't see a models.json or errors out create one yourself this is the content
{
"detection": [
"scrfd_10g_bnkps"
],
"recognition": [
"glintr100"
],
"analysis": [
"genderage",
"2d106det",
"1k3d68"
]
}
and a version.txt containing
0.7.3