harshitagrawal2O

harshitagrawal2O/nanolm

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

10 good first / help-wanted issues · Python · last activity Aug 22, 2026

0 stars 0 forks 0 watchers Python MIT License
attention bpe education from-scratch gpt llm nlp python test-driven-development tokenizer transformer
View on GitHub
10 Open Issues Need Help Last updated: Aug 22, 2026

Open Issues Need Help

View All on GitHub
Milestones 6-8 have no tests — write them from SPEC.md 2 months ago
help wanted good first issue no-implementation-needed
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M5 — Scaled dot-product attention, causal masking, multi-head 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M4 — Embeddings and positional encodings (learned + RoPE) 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
Build the benchmark harness skeleton 2 months ago
help wanted benchmarks no-implementation-needed
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M8 — Sampling, the PyTorch run, and perplexity 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M7 — Training loop, LR schedule, and the TensorKit run 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M6 — Transformer block and the full model 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M3 — Tokenizer benchmarking against tiktoken 2 months ago
help wanted milestone benchmarks
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M2 — Merge trie, encode/decode, byte fallback, special tokens 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer
M1 — Byte-level pretokenisation and BPE merge training 2 months ago
help wanted milestone
harshitagrawal2O/nanolm
0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python
#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer