llm-d-inference-sim good first issues (Go)

Add /v1/responses/render endpoin 19 days ago

good first issue

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Typos detected by nightly scan (2 found) about 1 month ago

good first issue typo

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Remove unused blockCache.countCachedBlockPrefix about 1 month ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Typos detected by nightly scan (1 found) 2 months ago

good first issue typo

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Align repository with llm-d repo template (12 items) 4 months ago

help wanted

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Add config flag to force usage of a dummy tokenizer 5 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Echo mode: support structured content blocks in chat/completions responses 6 months ago

enhancement good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Return LoRA Model Path in root for /v1/models 6 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Add tests for metrics with gRPC 7 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Read environment variables into configuration 7 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Generate correct response IDs prefixes 11 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Print real merged configuration to log on simulator initialization about 1 year ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Add retries to connect to ZMQ about 1 year ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Support --max-model-len config parameter over 1 year ago

AI Summary: Implement a new command-line parameter, `--max-model-len`, in the vLLM simulator. This parameter will define the maximum context window size (in tokens) for the model. Requests exceeding this limit should return a 400 Bad Request error with a specific error message indicating the context length exceeded.

Complexity: 4/5

enhancement good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

llm-d/llm-d-inference-sim

Open Issues Need Help