Sparsify transformers with SAEs and transcoders

1 Open Issue Need Help Last updated: Jun 24, 2025

Open Issues Need Help

View All on GitHub
AI/ML Transformer Optimization

AI Summary: Debug a KeyError in a Python library for training sparse autoencoders. The error occurs when attempting to train with multiple initialization seeds, stemming from a potential mismatch in how SAE names are handled during initialization and model forward passes. The solution likely involves correcting the naming convention for SAEs when multiple seeds are used, ensuring consistency between the `saes` dictionary keys and the hookpoint names used during the model's forward pass.

Complexity: 4/5
bug good first issue

Sparsify transformers with SAEs and transcoders

Python