[RFC]: Support ViT Full CUDA Graph (Tracker)
This RFC proposes extending CUDA graph support to the Vision Transformer (ViT) encoder in vLLM for multimodal models. Currently, the ViT encoder is processed eagerly, incurring significant overhead from numerous small kernel launches. By capturing the entire ViT forward pass into a CUDA graph, this change aims to eliminate launch costs and improve inference latency and throughput for multimodal LLMs.