ggml-org/llama.cpp

Improve `cvector-generator` 12 days ago

enhancement help wanted good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Allow Filtering LLama Server Response Fields 12 days ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

GGUF convert support for Vibevoice 15 days ago

enhancement help wanted good first issue model 3rd party tts

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Graceful reasoning budget termination. Avoid mid-sentence cutoff. 18 days ago

enhancement help wanted good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

tutorials : list for llama.cpp 19 days ago

help wanted good first issue roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Llava functions compiled as extern "C" throw exceptions 19 days ago

bug good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

llama : add CLI assistant 19 days ago

enhancement good first issue roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

tts : add support for Orpheus 19 days ago

good first issue tts

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: llama-server hot swapping cvectors via API like we can do with LoRA adapters now 21 days ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Eval bug: wrong default threads in llama-bench 23 days ago

good first issue performance low severity

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Implement missing ops from backends 24 days ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Eval bug: Model loading fails on Windows when path contains non-ASCII (Cyrillic) characters 27 days ago

bug help wanted

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: server provides strutured output for response_format: json_object, but not for response_format: json_schema about 1 month ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Mapping model name to LoRA config about 1 month ago

enhancement good first issue server

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Refactor: Add more typechecking to GGUFWriter.add_key_value about 1 month ago

help wanted refactoring

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server : add "token healing" support about 1 month ago

enhancement good first issue roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

llama : combined beam search + grammar sampling strategy about 1 month ago

good first issue generation quality research 🔬 roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: support Qwen3-omni, Qwen3-TTS and MiniCPM-o about 1 month ago

enhancement help wanted good first issue Review Complexity : High tts mtmd

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server: support control vectors about 1 month ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Support for Sparse MoE models like Camelidae and Sparsetral about 1 month ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

convert.py incorrectly detects LLaMAv1 65B as a LLaMAv2 model about 1 month ago

good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

llama : integer type consistency in `llama.h` about 1 month ago

enhancement good first issue refactoring roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server: exit failure if `--embedding` is set with an incoherent `--ubatch-size` about 1 month ago

enhancement help wanted good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

examples : add configuration presets about 1 month ago

documentation enhancement help wanted good first issue examples

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

split: allow --split-max-size option about 1 month ago

enhancement help wanted good first issue split

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

feature request - disabling tokenizer in conversion / inference about 2 months ago

enhancement help wanted good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server: avoid full prompt eval when 'prompt >= ctx' about 2 months ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Installable package via winget about 2 months ago

enhancement help wanted

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Eval bug: Gemma4 fails with NotImplemented: map: filter-mapping not implemented about 2 months ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

ggml : add WebGPU backend about 2 months ago

help wanted research 🔬 roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

ggml : add ANE backend 2 months ago

help wanted research 🔬 roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Implement Hadamard Transform for GPU backends 2 months ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

ggml : add GPU support for Mamba models 2 months ago

enhancement help wanted Nvidia GPU roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Implement Fast Walsh Hadamard Transform 2 months ago

enhancement good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

llama : refactor the llm.build_xxx functions 3 months ago

good first issue refactoring roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

GGUF endianness cannot be determined from GGUF itself 3 months ago

enhancement good first issue breaking change

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Bug: Unexpected output length (Only one token response!) when set configs "-n -2 -c 256" for llama-server 3 months ago

bug good first issue low severity

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Support dots.ocr 4 months ago

enhancement help wanted model

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

CUDA non-determinism on identical requests 4 months ago

bug good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

kubernetes example 4 months ago

enhancement help wanted kubernetes

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

llama cpp server cant open to public 4 months ago

enhancement good first issue server

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Docker: Embedding Issue & possible Fix 4 months ago

bug good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

ggml : add DirectML backend 5 months ago

help wanted research 🔬 roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Study how LM Evaluation Harness works and try to implement it 5 months ago

enhancement help wanted high priority generation quality research 🔬

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: llama-finetune won't work even with 17M parameters arch:llama 5 months ago

bug help wanted Review Complexity : High

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server: doc: document the `--defrag-thold` option 5 months ago

AI Summary: This issue requests adding documentation for the newly introduced `--defrag-thold` option to the `server/README.md` file. The option was recently added via a specific pull request and needs to be documented for users to understand its purpose and usage.

Complexity: 1/5

documentation enhancement help wanted server/webui

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Eval bug: Qwen3-Coder template parsing error when tools are enabled 5 months ago

bug good first issue jinja parser

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Update Winget Package failure 6 months ago

help wanted

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: mmproj from Fleming-VL-38B "KeyError: image_mean" 6 months ago

AI Summary: The user is encountering a 'KeyError: image_mean' when attempting to convert the Fleming-VL-38B model's multimodal projection (mmproj) to GGUF format using the `convert_hf_to_gguf.py` script. This issue prevents the successful creation of the mmproj file for the specified Hugging Face model, potentially related to a warning about `trust_remote_code` not being set.

Complexity: 3/5

bug good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Eval bug: Qwen3-Next-80b crashes loading 126k tokens of context in CUDA (Vulkan is fine) 7 months ago

bug good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: convert_hf_to_gguf.py to default to the original precision 7 months ago

enhancement help wanted good first issue breaking change

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Implement MotifForCausalLM 7 months ago

enhancement help wanted model porting

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

common: download from URL, improve parallel download progress status 7 months ago

enhancement help wanted good first issue split

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server : crash when -b > -ub with embeddings 7 months ago

bug good first issue embeddings server

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

server : improvements and maintenance 7 months ago

help wanted refactoring server/webui roadmap

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: support the multi-modal Bagel model 7 months ago

enhancement help wanted model Review Complexity : High mtmd

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: Large tokens produce incorrect error messages due to integer overflow. 7 months ago

bug good first issue server low severity

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Feature Request: Choose the temp file location 7 months ago

AI Summary: The user requests a new feature for the `convert_hf_to_gguf.py` script: the ability to specify a custom directory for temporary files when the `--use-temp-file` option is active. This is motivated by insufficient disk space on their default temporary drive (C:), necessitating the use of another disk. Their attempt to override the temp directory using environment variables did not succeed.

Complexity: 2/5

enhancement help wanted good first issue

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Bug: (conversion) --mistral-format doesn't work with latest mistral-common 8 months ago

bug help wanted

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: call to llama_model_meta_val_str for array values metadata returns nothing 8 months ago

AI Summary: The issue describes a bug where the `llama_model_meta_val_str` function does not return any value when attempting to retrieve metadata that contains array values. This indicates a problem with how the function processes or serializes non-string (specifically array) data types stored in the model's metadata.

Complexity: 3/5

enhancement help wanted good first issue low severity

ggml-org/llama.cpp

118.9K

LLM inference in C/C++

C++

#ggml

Misc. bug: webui: the context usage is no longer shown on error message 8 months ago

bug good first issue server/webui low severity

ggml-org/llama.cpp

118.9K

LLM inference in C/C++