sglang good first issues (Python)

[Kernel] Optimize triton decoding kernels for long context about 1 hour ago

good first issue help wanted high priority

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Kernel] cuDNN attention backend about 1 hour ago

enhancement good first issue help wanted high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

KeyError: 'model.layers.14.mlp.shared_expert.gate_gate_up_proj.weight' about 2 hours ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Carefully remove duplicated cotents in SGLang Docs and SGLang Cooklook 8 days ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] The Prometheus metric `avg_request_queue_latency` is not being collected 9 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Reduce constrained-decoding overhead in TP 11 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

model: support new diffusion models 13 days ago

good first issue help wanted Multi-modal new-model diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement Image Number Limitation for Image Processors 13 days ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support varied input formats for remaining VLM 18 days ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Decode Throughput Inconsistency Between bench_serving and Engine Logs 19 days ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Improve Unit Test Coverage 19 days ago

enhancement good first issue ci

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Create benchmark and dataset usage scripts for embedding models 20 days ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Request to 8-bit Quantization of Attention with SageAttention about 1 month ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Unified JIT / Precompilation Cache Directory about 1 month ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Roadmap] sglang auto tuner about 1 month ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add KV cache usage prometheus metrics about 1 month ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Overlap Spec Support about 2 months ago

enhancement good first issue help wanted high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Cascade attention kernels about 2 months ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Qwen3 Provide Ability to Turn off Thinking on server startup 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Kimi VL GPU memory usage too high 2 months ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] support `gather` instead of `all_gather` when gathering the logits 3 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Mllama ambiguous multi-image behaviour 3 months ago

good first issue help wanted Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support piggyback server load report. 3 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Reducing the time to import sglang 3 months ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add examples for running SGLang on Slurm 3 months ago

documentation good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Model] Support Unlimited OCR 3 months ago

good first issue help wanted vlm

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Performance] Remove instances of `torch.nonzero()` where appropriate 3 months ago

good first issue performance

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Refactor: TRTLLMHAAttnBackend should not inherit from FlashInferAttnBackend when reuse is small 3 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Integrate KVPress in order to try more KV cache optimization heuristics easily 3 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Tracking] Improve Multimodal CI coverage 3 months ago

good first issue performance Multi-modal ci

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support bitsandbytes in QWen2 VL 3 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Refactor common test entries of SGLang Diffusion 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Simplify `tree_speculative_sampling_target_only` 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enabling both TBO and shared experts fusion 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Currently disabling TRTLLM allreduce fusion on H20 device 4 months ago

help wanted ci

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Support spec decoding v2 for flashmla attention backend 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support more multi-modal input for VLM 5 months ago

good first issue help wanted feature Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Serving VLM VILA 5 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Detailed Break down of time spend on Launching SGLang Diffusion 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Increase server payload size limit 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] 3FS Storage backend Support HybridModel(Linear、DSA) 5 months ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[model] support qwen3-asr 5 months ago

good first issue help wanted high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Automatically truncate when the maximum tokens are exceeded instead of throwing an error 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] flashinfer gdn kernel not working 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Enhancement] Add blocking mode to /flush_cache endpoint 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Benchmark with audio input 6 months ago

good first issue help wanted Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Deprecate and remove /get_server_info endpoint 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] [Diffusion] Benchmark mix resolution 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] -inf in top_logprobs breaks FastAPI endpoint 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Log input text instead of input_ids when using openai chat apis 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Jamba 1.5 Support PLS 6 months ago

good first issue new-model

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Regression of "[PD] Add custom gpu id to device topo support #12817" 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Eagle v2 + triton: draft extend cuda graph produces NaN 7 months ago

bug good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] request for "classify" method for sglang router 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enable EPLB in Draft Models 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Compare the server-disk checksum in VAE and text encoder for update weights from disk of Diffusion models 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Lora precision error on H200 with tp_size >= 2 7 months ago

bug good first issue lora

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add arguments mapping between SGLang / vllm / trt-llm 7 months ago

documentation good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add SANA image diffusion models into SGLang 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Auto-truncation still uses full context length instead of (context_length - max_tokens) 7 months ago

good first issue inactive

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

RFC: Command-Line Interface Architecture for SGLang 7 months ago

help wanted high priority RFC

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Image generation without `base_64` return unexpected `400` 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Profile the update weights from disk API of SGLang Diffusion 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support offload and wake up of SGLang Diffusion 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Optimize getting logic of `num_reasoning_tokens` in scheduler 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Parallelize module loading to speed up server launch and refit for Diffusion models 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] [diffusion] Add SLO metric for SGL-Diffusion 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Tracking] GLM 4.5/4.6/4.7 Blackwell performance optimization 7 months ago

good first issue collaboration performance blackwell

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Roadmap for Prefill (Piecewise) CUDA Graph 7 months ago

help wanted high priority collaboration

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Optimization of DeepSeek V3.2 NVFP4 checkpoint 7 months ago

help wanted deepseek

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] test_metrics failed under 2 gpu settings (dp-2, tp-2) 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] VLM: `--mm-process-config` is currently defined but not working 7 months ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Change new allreduce_fusion api from Flashinfer 8 months ago

good first issue good second issue flashinfer

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement update_weights_from_disk for SGLang-D (Diffusion Engine) 8 months ago

good first issue reinforcement-learning diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Doc] Consolidate Diffusion Documentation into a Unified Directory 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Speculative Decoding Document failed. 8 months ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Infra] Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration 8 months ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support Ray Backend for distributed inference 8 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enable Two-Batch Overlap in Draft Models 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Xgrammar Structural Tag - Router Design 8 months ago

enhancement help wanted function-calling

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Roadmap] Lora Support 8 months ago

help wanted high priority collaboration lora

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support `abort_request` in the router 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] deepseek_v2.py Refactor 9 months ago

AI Summary: The `deepseek_v2.py` file, which is critical for several popular MoE models, has grown to over 4,000 lines of code. This issue proposes a significant refactor to improve its maintainability and extensibility. The plan involves cleaning up the existing code and extracting numerous standalone modules from its current monolithic structure.

Complexity: 4/5

good first issue deepseek Good Pro Issue good second issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] apply fixture-kit mode to `MMMUVLMMixin` 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

SGLang Cookbook Community Contribution Roadmap 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Interns-1 reasoning/tool-call parser 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Decode OOM with spec 10 months ago

good first issue Good Pro Issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Raise an error when the page size of the prefill and decode attention backend are different 10 months ago

enhancement good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Default to lower page size when not set 10 months ago

enhancement good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Task] Add priority scheduling CIs for preemption path 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support AIME25 dataset for SGLang inbuilt simple_eval 10 months ago

good first issue benchmark

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Grafana dashboard example is outdated due to /metrics API prefix change (sglang: -> sglang_) 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support PaddleOCR-VL Model 10 months ago

good first issue new-model vlm

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

diffusion: warmup 10 months ago

good first issue diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] LMCache prefix cache crashes scheduler with "Committed KV cache already freed" assertion 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add a FP8 Gemm backend for choosing FP8 gemm kernel 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Refactor `swa_radix_cache.py` 10 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement the warm-up logic for sgalng diffusion. 10 months ago

good first issue diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Rotary embedding kernel support float32 input dtype 10 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support `mrope_section` with `rope_type: "yarn"` 10 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

sgl-project/sglang

Open Issues Need Help