Open Issues Need Help
View All on GitHub Shrink README to landing page about 3 hours ago
good first issue W6-docs
Example notebooks about 3 hours ago
good first issue W6-docs
New-benchmark guide about 3 hours ago
good first issue W6-docs
Docs site (mkdocs-material) about 3 hours ago
help wanted W6-docs
Static leaderboard site about 3 hours ago
help wanted W5-reporting
Cost/latency/accuracy Pareto plots about 3 hours ago
help wanted W5-reporting
Vision benchmark expansion about 3 hours ago
help wanted W4-vision-depth
Image-token budget as first-class axis about 3 hours ago
help wanted W4-vision-depth
RefCOCO and grounding benchmarks about 3 hours ago
help wanted W4-vision-depth
Interleaved multi-image about 3 hours ago
help wanted W4-vision-depth
Grounding protocol (bbox output) about 3 hours ago
help wanted W4-vision-depth
Video benchmark coverage about 3 hours ago
help wanted W4-vision-depth
External manifest directory (--include-path) about 3 hours ago
good first issue W3-eval-semantics
Task groups and tags about 3 hours ago
help wanted W3-eval-semantics
System instruction flag about 3 hours ago
good first issue W3-eval-semantics
Composable filter pipeline about 3 hours ago
help wanted W3-eval-semantics
OpenAI-compatible endpoint adapter about 3 hours ago
help wanted W2-backends
Multi-GPU support via accelerate about 3 hours ago
help wanted W2-backends
Auto batch-size with OOM recovery about 3 hours ago
help wanted W2-backends
Batch inference for HuggingFace adapter about 3 hours ago
help wanted W2-backends
Add --log-samples flag about 3 hours ago
help wanted W1-reproducibility
Pin dataset revisions about 3 hours ago
help wanted W1-reproducibility
Decide on vlm_regcheck/ about 3 hours ago
help wanted W0-foundations
Clean up stray files about 3 hours ago
good first issue W0-foundations
First PyPI release with changelog about 3 hours ago
help wanted W0-foundations
CONTRIBUTING.md and templates about 3 hours ago
good first issue W0-foundations
Pre-commit hooks (ruff, mypy) about 3 hours ago
help wanted W0-foundations
Add LICENSE file about 3 hours ago
good first issue W0-foundations