Multi-GPU batch caption with JoyCaption. JoyCaption uses Meta-Llama-3.1–8B and google/siglip-so400m-patch14–384 and a fine tuned image captioning neural network.
Link : https://www.patreon.com/posts/110613301
Link for batch caption editor : https://www.patreon.com/posts/108992085
Coding multi-gpu in Python and Torch and bitsandbytes was truly a challange.
Our APP uses JoyCaption image captioning fine tuned model.
Our APP supports bitsandbytes 4bit model loading as well even in multi GPU mode (9.5 GB VRAM)
Tested on 8x RTX A6000 (cloud) and RTX 3090 TI + RTX 3060 (my PC)
1-click to install on Windows, RunPod and Massed Compute
Excellent caption quality, automatically distributes images into each GPU, lots of features. You can resume caption with skip captioned images option.
For full details checkout screenshots
Source link
lol
