sgl-project/sglang

[Feature] Improve Unit Test Coverage about 9 hours ago

enhancement good first issue ci

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Overlap Spec Support 4 days ago

enhancement good first issue help wanted high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Reduce constrained-decoding overhead in TP 14 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Roadmap] sglang auto tuner 15 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Cascade attention kernels 15 days ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Qwen3 Provide Ability to Turn off Thinking on server startup 15 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Carefully remove duplicated cotents in SGLang Docs and SGLang Cooklook 18 days ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Unified JIT / Precompilation Cache Directory 20 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Kimi VL GPU memory usage too high 20 days ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

KeyError: 'model.layers.14.mlp.shared_expert.gate_gate_up_proj.weight' 26 days ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] support `gather` instead of `all_gather` when gathering the logits 30 days ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Mllama ambiguous multi-image behaviour 30 days ago

good first issue help wanted Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support piggyback server load report. 30 days ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Reducing the time to import sglang about 1 month ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add examples for running SGLang on Slurm about 1 month ago

documentation good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

model: support new diffusion models about 1 month ago

good first issue help wanted Multi-modal new-model diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Model] Support Unlimited OCR about 1 month ago

good first issue help wanted vlm

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Performance] Remove instances of `torch.nonzero()` where appropriate about 2 months ago

good first issue performance

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Refactor: TRTLLMHAAttnBackend should not inherit from FlashInferAttnBackend when reuse is small about 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Integrate KVPress in order to try more KV cache optimization heuristics easily about 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Tracking] Improve Multimodal CI coverage about 2 months ago

good first issue performance Multi-modal ci

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Create benchmark and dataset usage scripts for embedding models about 2 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support bitsandbytes in QWen2 VL about 2 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Kernel] cuDNN attention backend about 2 months ago

enhancement good first issue help wanted high priority inactive

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Refactor common test entries of SGLang Diffusion 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Simplify `tree_speculative_sampling_target_only` 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enabling both TBO and shared experts fusion 2 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Currently disabling TRTLLM allreduce fusion on H20 device 3 months ago

help wanted ci

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Support spec decoding v2 for flashmla attention backend 3 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support varied input formats for remaining VLM 3 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support more multi-modal input for VLM 3 months ago

good first issue help wanted feature Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Serving VLM VILA 3 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Detailed Break down of time spend on Launching SGLang Diffusion 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Increase server payload size limit 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] 3FS Storage backend Support HybridModel(Linear、DSA) 4 months ago

good first issue high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] The Prometheus metric `avg_request_queue_latency` is not being collected 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add KV cache usage prometheus metrics 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Kernel] Optimize triton decoding kernels for long context 4 months ago

good first issue help wanted high priority inactive

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[model] support qwen3-asr 4 months ago

good first issue help wanted high priority

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement Image Number Limitation for Image Processors 4 months ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Automatically truncate when the maximum tokens are exceeded instead of throwing an error 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] flashinfer gdn kernel not working 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Enhancement] Add blocking mode to /flush_cache endpoint 4 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Request to 8-bit Quantization of Attention with SageAttention 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Benchmark with audio input 5 months ago

good first issue help wanted Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Deprecate and remove /get_server_info endpoint 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] [Diffusion] Benchmark mix resolution 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] -inf in top_logprobs breaks FastAPI endpoint 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Log input text instead of input_ids when using openai chat apis 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Jamba 1.5 Support PLS 5 months ago

good first issue new-model

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Regression of "[PD] Add custom gpu id to device topo support #12817" 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Eagle v2 + triton: draft extend cuda graph produces NaN 5 months ago

bug good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] request for "classify" method for sglang router 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enable EPLB in Draft Models 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Compare the server-disk checksum in VAE and text encoder for update weights from disk of Diffusion models 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Lora precision error on H200 with tp_size >= 2 5 months ago

bug good first issue lora

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add arguments mapping between SGLang / vllm / trt-llm 5 months ago

documentation good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add SANA image diffusion models into SGLang 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Auto-truncation still uses full context length instead of (context_length - max_tokens) 5 months ago

good first issue inactive

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

RFC: Command-Line Interface Architecture for SGLang 5 months ago

help wanted high priority RFC

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Image generation without `base_64` return unexpected `400` 5 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Profile the update weights from disk API of SGLang Diffusion 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support offload and wake up of SGLang Diffusion 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Optimize getting logic of `num_reasoning_tokens` in scheduler 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Parallelize module loading to speed up server launch and refit for Diffusion models 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] [diffusion] Add SLO metric for SGL-Diffusion 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Tracking] GLM 4.5/4.6/4.7 Blackwell performance optimization 6 months ago

good first issue collaboration performance blackwell

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Roadmap for Prefill (Piecewise) CUDA Graph 6 months ago

help wanted high priority collaboration

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Optimization of DeepSeek V3.2 NVFP4 checkpoint 6 months ago

help wanted deepseek

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] test_metrics failed under 2 gpu settings (dp-2, tp-2) 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] VLM: `--mm-process-config` is currently defined but not working 6 months ago

good first issue Multi-modal

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Change new allreduce_fusion api from Flashinfer 6 months ago

good first issue good second issue flashinfer

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement update_weights_from_disk for SGLang-D (Diffusion Engine) 6 months ago

good first issue reinforcement-learning diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Doc] Consolidate Diffusion Documentation into a Unified Directory 6 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Speculative Decoding Document failed. 6 months ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Infra] Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration 6 months ago

documentation good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support Ray Backend for distributed inference 6 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Enable Two-Batch Overlap in Draft Models 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Xgrammar Structural Tag - Router Design 7 months ago

enhancement help wanted function-calling

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Roadmap] Lora Support 7 months ago

help wanted high priority collaboration lora

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support `abort_request` in the router 7 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] deepseek_v2.py Refactor 7 months ago

AI Summary: The `deepseek_v2.py` file, which is critical for several popular MoE models, has grown to over 4,000 lines of code. This issue proposes a significant refactor to improve its maintainability and extensibility. The plan involves cleaning up the existing code and extracting numerous standalone modules from its current monolithic structure.

Complexity: 4/5

good first issue deepseek Good Pro Issue good second issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] apply fixture-kit mode to `MMMUVLMMixin` 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

SGLang Cookbook Community Contribution Roadmap 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Interns-1 reasoning/tool-call parser 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Decode OOM with spec 8 months ago

good first issue Good Pro Issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Raise an error when the page size of the prefill and decode attention backend are different 8 months ago

enhancement good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Default to lower page size when not set 8 months ago

enhancement good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Task] Add priority scheduling CIs for preemption path 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support AIME25 dataset for SGLang inbuilt simple_eval 8 months ago

good first issue benchmark

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] Grafana dashboard example is outdated due to /metrics API prefix change (sglang: -> sglang_) 8 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support PaddleOCR-VL Model 8 months ago

good first issue new-model vlm

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

diffusion: warmup 8 months ago

good first issue diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] LMCache prefix cache crashes scheduler with "Committed KV cache already freed" assertion 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Add a FP8 Gemm backend for choosing FP8 gemm kernel 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Refactor `swa_radix_cache.py` 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Implement the warm-up logic for sgalng diffusion. 9 months ago

good first issue diffusion

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Rotary embedding kernel support float32 input dtype 9 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Feature] Support `mrope_section` with `rope_type: "yarn"` 9 months ago

good first issue help wanted

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

[Bug] `get_event_loop` is not compatible with new uvloop 9 months ago

good first issue

sgl-project/sglang

29.9K

SGLang is a high-performance serving framework for large language models and multimodal models.

Python

#attention#blackwell#cuda#deepseek#diffusion#glm#gpt-oss#inference#llama#llm#minimax#moe#qwen#qwen-image#reinforcement-learning#transformer#vlm#wan

Open Issues Need Help