Skip to content

llmcompressor.transformers.utils

Utilities for applying sparsification algorithms to Hugging Face transformers flows

is_model_ct_quantized_from_path(path)

Determine if model from path is quantized based on the config

Parameters:

Name Type Description Default
path str

path to the model or HF stub

 required

Returns:

Type Description
bool

True if config contains quantization_config from the given path
Source code in src/llmcompressor/transformers/utils/helpers.py 
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def is_model_ct_quantized_from_path(path: str) -> bool:
    """
    Determine if model from path is quantized based
    on the config

    :param path: path to the model or HF stub
    :return: True if config contains quantization_config from the given path

    """
    config = AutoConfig.from_pretrained(path)
    if config is not None:
        if (
            hasattr(config, "quantization_config")
            and config.quantization_config["quant_method"] == "compressed-tensors"
        ):
            return True
    return False