I'm followed this guide on how to perform dataset training for stable diffusion. Although this guide mainly towards embeddings which comes in very small files and requires user to input the embedding name in prompt, it also can be used for hypernetwork training as well.

Also, applying gradient steps is important to make hypernetwork or embedding training to be accurate. But be careful not to applying it too high which can cause full garbage image output especially when training with hypernetwork and setting its strength to very low rate could be only mitigating garbage image output issues.

reddit.com/r/StableDiffusion/comments/zxkukk/detailed_guide_on_training_embeddings_on_a/