Dervish good first issues (Python)

Pre-register a test on a model that uses the convolution or expert-bank frame rules 21 days ago

help wanted experiment area: evaluation

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register Dervish against the bound-keeping Muon-family baselines 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register the ablations that were defined but never run 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register a rerun of the copy-behaviour probe with the tuned spectrum 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register a learning-rate basin re-test with a grid centred on the tuned setting 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register a non-inferiority test of Dervish against Muon with more seeds 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Pre-register and run the GPU-scale stage named in Hypothesis D001 21 days ago

help wanted experiment area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Add Muown and Muon with a spectral cap as lab baselines 21 days ago

enhancement help wanted area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Bound the LayerNorm gains and q/k biases so the attention-logit bound is a constant 21 days ago

enhancement help wanted area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Verify the CUDA and bfloat16 Newton-Schulz paths on a GPU and measure the step cost 21 days ago

help wanted area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Add a notebook that walks through the Stage D8 evidence from the run files 21 days ago

documentation good first issue area: evaluation

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Port stages E2b and E4 to E7 to run_matrices.ps1 21 days ago

enhancement good first issue area: packaging

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Align the README snippet, trainer_integration.md and the package docstring with the measured setting 21 days ago

documentation good first issue area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Add tests for the frame options that no test exercises 21 days ago

enhancement good first issue area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Add a test that a Dervish checkpoint resumes the same trajectory 21 days ago

enhancement good first issue area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Reconcile the stated singular-value range of the quintic Newton-Schulz output 21 days ago

documentation good first issue area: optimizer

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Add a CI workflow that runs the tests and examples on Linux and Windows 21 days ago

enhancement good first issue area: packaging

Cassandra-Collective/Dervish

0

A Muon-variant PyTorch optimizer whose hidden matrices have a provably bounded spectrum, with a pre-registered evaluation record.

Python

#deep-learning#muon#optimizer#pre-registration#pytorch#reproducible-research#spectral-norm#transformers

Cassandra-Collective/Dervish

Open Issues Need Help