v0.32.0: [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718)
This update resolves a numerical stability issue causing NaN outputs in the Mamba2 chunk scan kernel. The fix specifically targets hardware utilizing the sm90 architecture, such as NVIDIA H100 GPUs.
Verified State Diff
Impact & Verification Analysis
Developers and researchers deploying Mamba2 models on NVIDIA Hopper (sm90) GPUs.
Ensures the reliability and correctness of state space model inference on modern enterprise-grade hardware, preventing silent failures in production pipelines.
Full Fact Overview
The Mamba2 architecture relies on a specialized chunk scan kernel to handle state space model operations efficiently. On sm90 (Hopper) architectures, the previous implementation of this kernel encountered precision errors leading to NaN (Not a Number) values during computation. This patch corrects the underlying logic in the kernel execution path to ensure numerical consistency on high-performance NVIDIA hardware.