Open Issues Need Help
View All on GitHubAI Summary: The user is experiencing a `RuntimeError` while fine-tuning the Flux 1 dev model using SimpleTuner. The error, stemming from an invalid shape during latent packing, occurs despite deleting cached files and reinstalling the software. The task involves debugging the issue using the provided debug log, configuration files (`config.json` and `multidatabackend.json`), and understanding the SimpleTuner framework to identify the root cause of the shape mismatch in the latent tensors.
A general fine-tuning kit geared toward diffusion models.
AI Summary: The task is to debug an Out-of-Memory (OOM) error occurring during checkpoint saving in a diffusion model fine-tuning process using the SimpleTuner framework. The OOM happens specifically when using fp8-quanto precision for Kontext LoRA training on an RTX 4090 (24GB VRAM), while int8-quanto works without issue. The solution requires investigating why fp8-quanto causes this problem during checkpoint saving, finding a workaround to prevent the OOM, and assessing the potential impact on inference quality if switching to INT8 training for compatibility with ComfyUI.
A general fine-tuning kit geared toward diffusion models.
AI Summary: Debug a `AttributeError: 'list' object has no attribute 'shape'` occurring in the SimpleTuner project after an update, specifically within the Kontext model's `prepare_batch_conditions` function. This involves examining the `cond` variable within the function to understand why it's a list instead of an array with a shape attribute, likely due to a change in data handling after the update. The solution might involve adjusting data preprocessing or the way the `cond` variable is created and passed to the function.
A general fine-tuning kit geared toward diffusion models.
AI Summary: Debug and fix an issue with saving and restoring Exponential Moving Average (EMA) weights for the fused QKV projections feature in the SimpleTuner diffusion model fine-tuning kit. This likely involves inspecting the code responsible for saving and loading EMA weights, identifying the source of the error (e.g., incorrect file paths, data type mismatches, or serialization problems), and implementing a fix to ensure correct saving and restoration of the EMA weights.
A general fine-tuning kit geared toward diffusion models.
AI Summary: Implement automatic metadata duplication for 'reference_strict' datasets within the SimpleTuner project, leveraging the existing duplicator introduced in #1539. This involves integrating the duplicator functionality to handle metadata copying for datasets with this specific designation.
A general fine-tuning kit geared toward diffusion models.
AI Summary: The issue is a type error during HiDream LoRA fine-tuning on Apple Silicon (MPS) due to the MPS framework's lack of float64 support. The error originates in the `rope` function within the transformer, requiring a change to use `torch.float32` instead. A secondary error, a tensor size mismatch, also needs to be addressed, likely stemming from an incompatibility between the input image dimensions and the model's expected input size.
A general fine-tuning kit geared toward diffusion models.
AI Summary: The user is experiencing an error ('list' object has no attribute 'ndim') during validation while fine-tuning the Wan 2.1 model for text-to-video generation using SimpleTuner. The task is to debug this error, likely stemming from an issue with how validation images are handled or formatted within the SimpleTuner framework given the provided configuration. This involves examining the SimpleTuner codebase, particularly the validation image processing section, and potentially modifying the configuration or data preprocessing steps.
A general fine-tuning kit geared toward diffusion models.
AI Summary: The user is experiencing a warning and error message during fine-tuning of the Stable Diffusion XL model related to loading a VAE (Variational Autoencoder) from the `madebyollin/sdxl-vae-fp16-fix` repository. The error indicates that a `config.json` file is missing from the specified VAE repository, although the fine-tuning process continues. The task is to investigate why the warning and error appear despite the training continuing and determine if the missing `config.json` is causing any issues or if it's a false positive.
A general fine-tuning kit geared toward diffusion models.
AI Summary: Debug a bug in a diffusion model fine-tuning kit where validation image generation is unexpectedly still occurring despite being disabled in the settings. The issue involves investigating why the validation steps are executed even when no images are produced, potentially requiring examination of the code's logic and configuration.
A general fine-tuning kit geared toward diffusion models.