Using runpod to train with conceptmod

By NTC

This tutorial is also on civit at: https://civitai.com/models/58873/conceptmod-tutorial-fire-updated-term-train-any-lora-with-just-text-no-data-required

Conceptmod (https://github.com/ntc-ai/conceptmod) is a technique that can change the internal latent space of trained stable diffusion models. It is based on https://erasing.baulab.info/ but has been heavily modified. Conceptmod can be used to remove, replace or emphasize concepts. This tutorial shows you how to use runpod (a gpu rental service) with conceptmod to train on your own text representations.

Use the runpod at https://runpod.io/gsc?template=8y3jhbola2&ref=xf9c949d

  1. Create a pod

I chose a 3090. It needs > 20 GB ram. Don’t encrypt your volume, container disk defaults to 5 gb and volume disk to 50 gb, these are fine click Continue

It will take a few minutes to download conceptmod, once Connect becomes enabled, you are ready to continue.

  1. Login to the web console

“Connect”, SSH or “Start web console” and connect to it.

Once you log in, it will install dependencies (take a minute) then output a welcome message.

  1. Send over your the base model checkpoint that you want to train on:

Note: be sure to use a safetensors checkpoint.

Using https://github.com/runpod/runpodctl

on local:

runpodctl send mycheckpoint.safetensors

on pod:

cd /workspace/stable-diffusion-webui/models/Stable-diffusion/ runpodctl receive

Note: You can also use scp, wget or a cloud storage attachment to transfer your model

  1. Train on your phrase (takes 3 hours for 1000 steps)

cd /workspace/sd-scripts

python3 train-scripts/train-esd.py –prompt “#|fire%{random_prompt}, fire:-0.1|fire++:guidance=2” –train_method selfattn –ckpt_path /workspace/stable-diffusion-webui/models/Stable-diffusion/mycheckpoint.safetensors

It saves every 300 steps, which is about an hour.

Selecting a phrase

Look at the models here and find one to modify: https://civitai.com/tag/conceptmod?model=58873&sort=Newest

  1. Extract the lora bash /workspace/conceptmod/docker/extract_lora.sh /workspace/stable-diffusion-webui/models/Stable-diffusion/.safetensors

The argument is your base checkpoint from (3).

  1. (optional) Test the lora in webui

Your model (and the training intermediates) will now be available in webui as a lora. Select your base model from (3) and apply the lora to figure out what the strength should be.

Freeze the seed to manually see how lora strength changes the model.

For ease of use:

cd /workspace/stable-diffusion-webui/Lora mv compvis-word_firefire%{random_prompt}-0.1-metho.safetensors fire.safetensors

  1. (optional) Create animations to show how your lora changes the images

choice a) To create an animation on your prompt:

python3 lora_anim.py -s 0.0 -e 0.7 -l “fire” -p “fire prompt”

-s is starting lora strength

-e is ending lora strength

-l “fire” is your lora

-p “fire prompt” is your prompt

choice b) To create one animation using a top 80k prompt appended with your trigger (like these previews):

python3 lora_anim.py -s 0.0 -e 0.7 -l “fire” -lp “, fire”

prompt defaults to https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts

-lp “, fire” is your trigger.

Run continuously for many videos

while true; do python3 lora_anim.py -s 0.0 -e 0.7 -l “fire” -lp “, fire”; done

Break with ctrl-c. The animations are in the v4 directory as mpv files.

Transfer your videos:

on pod:

runpodctl send v4

on local: runpodctl receive

  1. Download the lora

on pod:

runpodctl send /workspace/stable-diffusion-webui/models/Lora/

on local:

runpodctl receive

  1. Stop and terminate the pod to stop paying money

on https://runpod.io/console.pods , stop and terminate the running pod

  1. Post the lora to civitai with the tag conceptmod.

Include your training phrase for a 5 star review.

https://civitai.com/tag/conceptmod?model=58873&sort=Newest