This article explains how to use LoRA for the Flux.1 model in ComfyUI.
Previous article:
>-
What is LoRA?
Please refer to the following article for an overview of LoRA.
>-
It’s basically similar, so it is thought that it takes the form of adding LoRA dimensions somewhere in the Flux.1 model and training only there.
Steps to use LoRA for Flux.1 Model in ComfyUI
The steps up to the installation of Flux.1 are the same as the article below, so please read that.
>-
1. Download LoRA
Find it from the web.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
2. Extend Workflow
Add a LoraLoaderModelOnly node and connect it between Load CheckPoint and KSampler.
3. Generate
Select the model and generate.
Generation Results
Anime LoRA
Include anime in the prompt
An anime style portrait of a woman with long, flowing hair tied in a ponytail. She is wearing a cozy sweater, looking slightly to the side with her mouth wide open in a big, joyful laugh. A speech bubble is placed near her mouth, with the text ‘It worked with FP8!!’ in a playful and excited font.
Realism LoRA
A realistic photo of a woman with long, flowing hair tied in a ponytail. She is wearing a cozy sweater, looking slightly to the side with her mouth wide open in a big, joyful laugh. A speech bubble is placed near her mouth, with the text ‘It worked with FP8!!’ in a playful and excited font.
If I use “woman”, the wrinkles tend to get deep.
Disney LoRA
Include disney style in the prompt
A disney style image of a woman with long, flowing hair tied in a ponytail. She is wearing a cozy sweater, looking slightly to the side with her mouth wide open in a big, joyful laugh. A speech bubble is placed near her mouth, with the text ‘It worked with FP8!!’ in a playful and excited font.
Text generation failed, but the art style looks like it, so it’s good.
Conclusion
In this article, I tried LoRA for Flux.1. I tested it with the fp8 model, and I’m glad it seems to be working properly.
Actually, ControlNet for Flux.1 has also been released, but structurally my specs are probably not enough, so I’ll pass (if it’s similar to StableDiffusion, it has a structure where two DiTs are connected in parallel).
If you want to try it! If you think your environment can handle it! Please try your best while looking at the instructions in the link below.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
