inference-perf good first issues (Python)

Emit results.session_performance in the BR0.2 partial 12 days ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Support a template endpoint for arbitrary HTTP inference APIs 23 days ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Support the rerank API 26 days ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Support the embeddings API 29 days ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Support Custom Datasets about 1 month ago

help wanted priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

SGLang support: Incorrect metric mapping 2 months ago

kind/bug help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Improve request-level data collection when only the metrics are needed. 2 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Feature Request] Add Anthropic Messages API support 4 months ago

help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

RNG state synchronization issue across worker processes 5 months ago

kind/bug help wanted needs-triage

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add a random interval for session rate 5 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add support for VisionArena Dataset 5 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Surface vLLM prompt-cache stats (cached/uncached tokens) in lifecycle metrics 6 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Bug] Prometheus metrics not being exported in report 6 months ago

kind/bug help wanted

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Feature Request] Add command line flags to run inference-perf without config file 7 months ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

ShareGPT generation hits StopIteration 8 months ago

kind/bug good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add configurable base seed 8 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Comparitive analysis of performance metrics 8 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Consider using typing.NamedTuple instead of TypeAlias 9 months ago

good first issue priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add tests for worker concurrency 9 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

support to set an SLO threshold level and then calculate the goodput, rather than measuring the total throughput. 11 months ago

good first issue priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Handle sigint gracefully about 1 year ago

kind/feature good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Prometheus stage metric query duration incorrect about 1 year ago

kind/bug good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

need a detail description and guidance for how to setup config.yml over 1 year ago

AI Summary: Create a README file explaining how to configure the `config.yml` file for the Inference Perf benchmarking tool, covering different environment setups (model names, base URLs, API tokens, etc.), and providing clear guidance and examples for successful execution.

Complexity: 4/5

good first issue kind/support

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

kubernetes-sigs/inference-perf

Open Issues Need Help