Open Issues Need Help
View All on GitHub Add RamaLama as a detection provider 2 days ago
good first issue Feature Request
AlexsJones/llmfit
29.2K
Hundreds of models & providers. One command to find what runs on your hardware.
Rust
#gguf#llm#localai#mlx#skill#unsloth
Support MoE architectures and CPU/RAM offloading 5 months ago
AI Summary: The current VRAM calculation method is insufficient as it assumes dense models, leading to inaccurate estimates for MoE architectures like Mixtral, which only activate a subset of experts. It also doesn't account for the potential of CPU/RAM offloading with fast DDR5. The proposed solution involves adding MoE detection, active parameter calculation, system RAM detection, and offloading feasibility estimates.
Complexity:
4/5
enhancement help wanted
AlexsJones/llmfit
29.2K
Hundreds of models & providers. One command to find what runs on your hardware.
Rust
#gguf#llm#localai#mlx#skill#unsloth