Skip to content

vllm.v1.attention.backends.mla.prefill.cpu_native

Inert MLA prefill backend for the CPU platform.

CPU's MLA attention impl (CPUMLAImpl) fully overrides forward_mha with a kernel that attends directly against the paged latent KV cache (covering both fresh prefill and cached-prefix continuation in one pass), so it never calls into a pluggable MLAPrefillBackend's run_prefill_new_tokens/ run_prefill_context_chunk. This class exists only to satisfy MLACommonMetadataBuilder's structural requirement that MLAAttention.prefill_backend be a real, cloneable object.

Classes:

CPUNativeMLAPrefillBackend

Bases: MLAPrefillBackend

Placeholder MLA prefill backend for CPU; never actually invoked.

Source code in vllm/v1/attention/backends/mla/prefill/cpu_native.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
class CPUNativeMLAPrefillBackend(MLAPrefillBackend):
    """Placeholder MLA prefill backend for CPU; never actually invoked."""

    @staticmethod
    def get_name() -> str:
        return "CPU_NATIVE"

    def run_prefill_new_tokens(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        return_softmax_lse: bool,
        out: torch.Tensor | None = None,
        output_scale: torch.Tensor | None = None,
    ) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]:
        raise AssertionError(
            "CPUNativeMLAPrefillBackend.run_prefill_new_tokens is unreachable: "
            "CPUMLAImpl.forward_mha fully overrides the dense-MHA prefill path "
            "and never calls the pluggable prefill backend."
        )

    def run_prefill_context_chunk(
        self,
        chunk: "MLACommonPrefillMetadata.ContextChunk",
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        out: torch.Tensor | None = None,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        raise AssertionError(
            "CPUNativeMLAPrefillBackend.run_prefill_context_chunk is "
            "unreachable: CPUMLAImpl.forward_mha fully overrides the "
            "dense-MHA prefill path and never calls the pluggable prefill "
            "backend."
        )