harshitagrawal2O/nanolm

Milestones 4–8 have no tests — write them from SPEC.md about 17 hours ago

help wanted good first issue no-implementation-needed

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

Build the benchmark harness skeleton about 17 hours ago

help wanted benchmarks no-implementation-needed

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M8 — Sampling, the PyTorch run, and perplexity about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M7 — Training loop, LR schedule, and the TensorKit run about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M6 — Transformer block and the full model about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M5 — Scaled dot-product attention, causal masking, multi-head about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M4 — Embeddings and positional encodings (learned + RoPE) about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M3 — Tokenizer benchmarking against tiktoken about 17 hours ago

help wanted milestone benchmarks

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M2 — Merge trie, encode/decode, byte fallback, special tokens about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

M1 — Byte-level pretokenisation and BPE merge training about 17 hours ago

help wanted milestone

harshitagrawal2O/nanolm

0

A byte-pair-encoding tokenizer and a decoder-only transformer, from scratch in NumPy. Spec-first: 85 tests were written before any implementation.

Python

#attention#bpe#education#from-scratch#gpt#llm#nlp#python#test-driven-development#tokenizer#transformer

Open Issues Need Help