pytorch

pytorch/ao

PyTorch native quantization and sparsity for training and inference

2.9K stars 549 forks 2.9K watchers Python Other
brrr cuda dtypes float8 inference llama mx pytorch quantization sparsity training transformer
View on GitHub Website
30 Open Issues Need Help Last updated: Jul 1, 2026

Open Issues Need Help

View All on GitHub
FP8 Blockwise Training Tracker 2 days ago
good first issue tracker float8 module: training moe
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Accelerate activation sparsity with activation compression 7 days ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Resolve doc build warnings 23 days ago
topic: documentation good first issue tracker
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Support for GrokAdamW 24 days ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Tensor Parallelism Support for AffineQuantizedTensor 24 days ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[low-bit optim] Add COAT optimizer 24 days ago
good first issue optimizer
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[QAT] Low-bit FSDP all-gather for QAT 24 days ago
good first issue distributed module: qat
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Unittests Migration Progress 24 days ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Refactor torchao and tests to use model architectures from torchao.testing.model_architectures 24 days ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
TorchAO ROCM tests are taking a long time 24 days ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Update documents addressing AQT workflow 24 days ago
topic: documentation good first issue module: inference
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
MoE example 28 days ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Self compressing neural networks 28 days ago
good first issue module: qat
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Quantized Training 29 days ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Broken links in first quantization example 3 months ago
topic: documentation good first issue docathon:easy
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Broken image links in docs 3 months ago
topic: documentation good first issue docathon:easy
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Clean up arglist for choose_qparams_affine_with_min_max 5 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Update fp8 per tensor linear path for better performance in B200 5 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Add FqnToConfig to torchao main doc 6 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[MPS] torchao low-bit-precision optim does not expose 'backend' argument to torch.compile 6 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Delete special handling of per tensor quant in `_choose_scale_float8` 6 months ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Filling some Int4 tensor feature gaps after tensor subclass migration 6 months ago
good first issue tracker
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Align `_choose_qparams_affine ` with `_choose_scale_float8` 7 months ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[llama] Use horizontal fusion trick from Attention for FeedForward 7 months ago
good first issue triaged
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[not user facing] Split `implements` and `implements_torch_function` 9 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Deprecate and remove subclass.py, dynamic_quant.py, and weight_only.py 9 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Migrating from AffineQuantizedTensor + Layouts to new structure of tensor subclasses 11 months ago
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
`torch_version_at_least` semantics are incorrect 11 months ago

AI Summary: The task is to correct the `torch_version_at_least` function in the TorchAO project. The current implementation incorrectly compares pre-release versions (e.g., 2.5.0.dev, 2.5.0.git) to stable releases, leading to inaccurate version checks. The fix requires adjusting the comparison logic to correctly order pre-release and stable versions (2.4.0 < 2.5.0.dev/2.5.0.git < 2.5.0).

Complexity: 3/5
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
Fix ao's weird linear registration system. 11 months ago

AI Summary: Refactor the linear registration system in the TorchAO PyTorch-native model optimization framework. The current system is inconsistent with other registration methods, making it confusing and difficult to maintain. The goal is to create a unified and cleaner registration API for linear operations, consistent with the existing @implements system, while preserving the ability to overwrite linear operations for auto-quantization.

Complexity: 4/5
good first issue
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer
[fp8 blockwise training] try using torch._scaled_mm instead of Triton kernels for fp8 gemms 11 months ago

AI Summary: This task involves optimizing the performance of fp8 blockwise training in the TorchAO project. It requires benchmarking the performance of custom Triton GEMM kernels against PyTorch's `torch._scaled_mm` for fp8 matrix multiplications within a blockwise training context. If `torch._scaled_mm` proves faster, the code needs to be updated to utilize it, accounting for potential differences in memory layout requirements.

Complexity: 4/5
good first issue float8
pytorch/ao
2.9K

PyTorch native quantization and sparsity for training and inference

Python
#brrr#cuda#dtypes#float8#inference#llama#mx#pytorch#quantization#sparsity#training#transformer