Skip to content

vllm.model_executor.models.commandr

PyTorch Cohere model.

Functions:

  • select_norm_impl –

    Returns the normalization layer class and epsilon value to use.

select_norm_impl(config)

Returns the normalization layer class and epsilon value to use. If config.rms_norm_eps is present, use RMSNorm. Otherwise default to LayerNorm with config.layer_norm_eps.

Source code in vllm/model_executor/models/commandr.py 
116
117
118
119
120
121
122
123
124
125
126
def select_norm_impl(config: CohereConfig) -> tuple[type[nn.Module], float]:
    """
    Returns the normalization layer class and epsilon value to use.
    If `config.rms_norm_eps` is present, use RMSNorm.
    Otherwise default to LayerNorm with `config.layer_norm_eps`.
    """
    rms_eps = getattr(config, "rms_norm_eps", None)
    if rms_eps is not None:
        return RMSNorm, rms_eps

    return LayerNorm, config.layer_norm_eps