Download: tripXL.safetensors
Use this lora with a stable diffusion XL checkpoint
Also at https://civitai.com/models/119351?modelVersionId=129641
Trained with the technique described at https://civitai.com/models/117518/headshot-sdxl-lora-trained-on-sd15-output
With the https://ntcai.xyz/conceptmod/trip/ trip lora. Uses https://civitai.com/models/69651/sdprompt-prompts-from-a-huggingface-dataset wildcards.
dataset has 1450 samples
Training command
CUDA_VISIBLE_DEVICES=0 accelerate launch --num_cpu_threads_per_process=2 "sdxl_train_network.py" --enable_bucket --pretrained_model_name_or_path="/ml2/trained/ComfyUI/models/checkpoints/stable-diffusion-xl-base-1.0/sd_xl_base_1.0.safetensors" --train_data_dir="/ml2/trained/sd-scripts/data/psychedelic_trip" --resolution="1024,1024" --output_dir="/ml2/trained/ComfyUI/models/loras/Lora/sdxl" --logging_dir="./logs" --save_model_as=safetensors --output_name="trip" --network_alpha="1" --network_dim=32 --network_module=networks.lora --text_encoder_lr=0.0008 --unet_lr=0.0008 --lr_scheduler="constant" --train_batch_size="16" --max_train_steps="10000" --mixed_precision="bf16" --save_every_n_epochs="1" --save_precision="bf16" --caption_extension=".txt" --optimizer_type="Adafactor" --optimizer_args scale_parameter=False relative_step=False warmup_init=False --max_data_loader_n_workers="0" --bucket_reso_steps=64 --gradient_checkpointing --xformers --bucket_no_upscale --network_train_unet_only
Resources used
- psychedelic trip lora https://ntcai.xyz/conceptmod/trip/
- SD1.5 aniverse https://civitai.com/models/107842
- person wildcards https://civitai.com/models/98622/clonecleaner-wildcards-increase-character-consistency
- sdprompt wildcards https://civitai.com/models/69651/sdprompt-prompts-from-a-huggingface-dataset