LLMKube good first issues (Go)

[FEATURE] startupTimeout for large models instead of restating the whole startup probe 16 days ago

priority/high component/controller component/api size/small kind/feature area/runtime good first issue

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

docs: ModelRouter endpoint resolution and the embeddings/rerank routes (#1812 follow-up) 19 days ago

documentation size/small area/routing good first issue

defilantech/LLMKube

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

[BUG] ModelRouter answers 503 "no rule matched" when the request body is not valid JSON (should be 400) 20 days ago

bug size/small area/routing good first issue

defilantech/LLMKube

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

ModelRouter router-proxy: serve POST /v1/completions (IDE autocomplete / FIM clients, benchmarks) 23 days ago

enhancement size/small area/routing good first issue

defilantech/LLMKube

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

TestRemoteURL fails under a global git insteadOf rewrite; runGit drops GIT_CONFIG_* overrides 23 days ago

bug size/small good first issue

defilantech/LLMKube

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

[FEATURE] Add make lint-all target that runs golangci-lint under both GOOS=darwin and GOOS=linux 6 months ago

enhancement good first issue

defilantech/LLMKube

232

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Go

#ai#apple-silicon#autoscaling#edge-computing#gguf#gpu#homelab#inference#kubernetes#kubernetes-operator#llama-cpp#llm#local-llm#metal#mlx#multi-gpu#nvidia#self-hosted#tgi#vllm

defilantech/LLMKube

Open Issues Need Help