v0.32.0rc3: [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718)
This release resolves a critical numerical stability issue causing NaN outputs in the Mamba2 chunk scan kernel. The fix specifically targets hardware architectures utilizing the sm90 compute capability.
Verified State Diff
Impact & Verification Analysis
Developers and researchers deploying Mamba2 models on NVIDIA Hopper-based GPUs.
Ensures functional parity and reliability for state-space models on high-performance hardware, preventing silent failures in production inference pipelines.
Full Fact Overview
The update addresses a bug in the Mamba2 implementation within vLLM, specifically affecting the chunk scan kernel when executed on NVIDIA Hopper (sm90) architecture. By correcting the underlying kernel logic, the patch ensures that state-space model (SSM) computations maintain numerical integrity during inference, preventing the generation of Not-a-Number (NaN) values that previously rendered model outputs unusable on this hardware.