Abstract
Diffusion-based image generation models have achieved great success in recent years by showing the capability of synthesizing high-quality content. However, these models contain a huge number of parameters, resulting in a significantly large model size. Saving and transferring them is a major bottleneck for various applications, especially those running on resource-constrained devices. In this work, we develop a novel weight quantization method that quantizes the UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X smaller size while exhibiting even better generation quality than the original one. Our approach includes several novel techniques, such as assigning optimal bits to each layer, initializing the quantized model for better performance, and improving the training strategy to dramatically reduce quantization error. Furthermore, we extensively evaluate our quantized model across various benchmark datasets and through human evaluation to demonstrate its superior generation quality.
Paper: https://arxiv.org/abs/2406.04333
Code: https://github.com/snap-research/BitsFusion (coming soon)
Project Page: https://snap-research.github.io/BitsFusion/
8x smaller! That’s pretty bonkers and awesome! I hope this will become the standard as it will allow the AI Horde workers to server 10x more models each and even faster (as it will cut down the loading times). I hope it doesn’t break Loras though.
Unfortunately, this won’t work with SD3 since it doesn’t use a U-Net. I wonder how many people are still training 1.5 models?
What about sdxl?
It does, so hopefully this will work on it too.
deleted by creator
Here is an alternative Piped link(s):
https://piped.video/cBMrc1cY4bs
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
