Skip to content
LLM Compressor Docs
llmcompressor.typing
Initializing search
GitHub
LLM Compressor Docs
GitHub
Home
Getting started
Getting started
Installation
Compress Your Model
Deploy with vLLM
Guides
Guides
Compression Schemes
Saving a Model
Examples
Examples
Quantizing Models with Activation-Aware Quantization (AWQ)
Quantizing Big Models with HF Accelerate
Quantizing Multimodal Audio Models
Quantizing Multimodal Vision-Language Models
int4 Weight Quantization of a 2:4 Sparse Model
fp8 Weight, Activation, and KV Cache Quantization
int4 Weight Quantization
fp8 Weight and Activation Quantization
int8 Weight and Activation Quantization
Quantizing Mixtral-8x7B-Instruct-v0.1 Model with FP8
Applying 2:4 Sparsity with Optional FP8 Quantization
Sparse Finetuning with TRL's SFTTrainer
Developer
Developer
Code of Conduct
Contributing Guide
Development Guide
API Reference
API Reference
llmcompressor
llmcompressor
llmcompressor.logger
llmcompressor.typing
llmcompressor.typing
Table of contents
typing
llmcompressor.version
args
args
llmcompressor.args.dataset_arguments
llmcompressor.args.model_arguments
llmcompressor.args.recipe_arguments
llmcompressor.args.training_arguments
llmcompressor.args.utils
core
core
llmcompressor.core.helpers
llmcompressor.core.lifecycle
llmcompressor.core.model_layer
llmcompressor.core.session
llmcompressor.core.session_functions
llmcompressor.core.state
events
events
llmcompressor.core.events.event
datasets
datasets
llmcompressor.datasets.utils
entrypoints
entrypoints
llmcompressor.entrypoints.oneshot
llmcompressor.entrypoints.train
llmcompressor.entrypoints.utils
metrics
metrics
llmcompressor.metrics.logger
utils
utils
llmcompressor.metrics.utils.frequency_manager
modifiers
modifiers
llmcompressor.modifiers.factory
llmcompressor.modifiers.interface
llmcompressor.modifiers.modifier
llmcompressor.modifiers.stage
awq
awq
llmcompressor.modifiers.awq.base
llmcompressor.modifiers.awq.mappings
distillation
distillation
output
output
llmcompressor.modifiers.distillation.output.base
utils
utils
pytorch
pytorch
llmcompressor.modifiers.distillation.utils.pytorch.kd_factory
llmcompressor.modifiers.distillation.utils.pytorch.kd_wrapper
llmcompressor.modifiers.distillation.utils.pytorch.model_wrapper
experimental
experimental
logarithmic equalization
logarithmic equalization
llmcompressor.modifiers.logarithmic_equalization.base
obcq
obcq
llmcompressor.modifiers.obcq.base
llmcompressor.modifiers.obcq.sgpt_base
llmcompressor.modifiers.obcq.sgpt_sparsify
pruning
pruning
llmcompressor.modifiers.pruning.helpers
constant
constant
llmcompressor.modifiers.pruning.constant.base
magnitude
magnitude
llmcompressor.modifiers.pruning.magnitude.base
utils
utils
pytorch
pytorch
llmcompressor.modifiers.pruning.utils.pytorch.layer_mask
llmcompressor.modifiers.pruning.utils.pytorch.mask_factory
wanda
wanda
llmcompressor.modifiers.pruning.wanda.base
llmcompressor.modifiers.pruning.wanda.wanda_sparsify
quantization
quantization
llmcompressor.modifiers.quantization.cache
llmcompressor.modifiers.quantization.calibration
gptq
gptq
llmcompressor.modifiers.quantization.gptq.base
llmcompressor.modifiers.quantization.gptq.gptq_quantize
quantization
quantization
llmcompressor.modifiers.quantization.quantization.base
llmcompressor.modifiers.quantization.quantization.mixin
smoothquant
smoothquant
llmcompressor.modifiers.smoothquant.base
llmcompressor.modifiers.smoothquant.utils
utils
utils
llmcompressor.modifiers.utils.constants
llmcompressor.modifiers.utils.hooks
llmcompressor.modifiers.utils.pytorch_helpers
observers
observers
llmcompressor.observers.base
llmcompressor.observers.helpers
llmcompressor.observers.min_max
llmcompressor.observers.mse
pipelines
pipelines
llmcompressor.pipelines.cache
llmcompressor.pipelines.registry
basic
basic
llmcompressor.pipelines.basic.pipeline
data free
data free
llmcompressor.pipelines.data_free.pipeline
independent
independent
llmcompressor.pipelines.independent.pipeline
layer sequential
layer sequential
llmcompressor.pipelines.layer_sequential.helpers
llmcompressor.pipelines.layer_sequential.pipeline
sequential
sequential
llmcompressor.pipelines.sequential.helpers
llmcompressor.pipelines.sequential.pipeline
pytorch
pytorch
model load
model load
llmcompressor.pytorch.model_load.helpers
utils
utils
llmcompressor.pytorch.utils.helpers
llmcompressor.pytorch.utils.sparsification
sparsification info
sparsification info
llmcompressor.pytorch.utils.sparsification_info.configs
llmcompressor.pytorch.utils.sparsification_info.helpers
llmcompressor.pytorch.utils.sparsification_info.module_sparsification_info
recipe
recipe
llmcompressor.recipe.base
llmcompressor.recipe.container
llmcompressor.recipe.metadata
llmcompressor.recipe.modifier
llmcompressor.recipe.recipe
llmcompressor.recipe.stage
transformers
transformers
compression
compression
llmcompressor.transformers.compression.helpers
llmcompressor.transformers.compression.quantization_format
llmcompressor.transformers.compression.sparsity_metadata_config
finetune
finetune
llmcompressor.transformers.finetune.callbacks
llmcompressor.transformers.finetune.session_mixin
llmcompressor.transformers.finetune.text_generation
llmcompressor.transformers.finetune.trainer
data
data
llmcompressor.transformers.finetune.data.base
llmcompressor.transformers.finetune.data.c4
llmcompressor.transformers.finetune.data.cnn_dailymail
llmcompressor.transformers.finetune.data.custom
llmcompressor.transformers.finetune.data.data_helpers
llmcompressor.transformers.finetune.data.evolcodealpaca
llmcompressor.transformers.finetune.data.flickr_30k
llmcompressor.transformers.finetune.data.gsm8k
llmcompressor.transformers.finetune.data.open_platypus
llmcompressor.transformers.finetune.data.peoples_speech
llmcompressor.transformers.finetune.data.ptb
llmcompressor.transformers.finetune.data.ultrachat_200k
llmcompressor.transformers.finetune.data.wikitext
sparsification
sparsification
llmcompressor.transformers.sparsification.compressed_tensors_utils
llmcompressor.transformers.sparsification.sparse_model
tracing
tracing
llmcompressor.transformers.tracing.debug
llmcompressor.transformers.tracing.gemma3
llmcompressor.transformers.tracing.idefics3
llmcompressor.transformers.tracing.llava
llmcompressor.transformers.tracing.mllama
llmcompressor.transformers.tracing.qwen2_5_vl
llmcompressor.transformers.tracing.qwen2_vl
utils
utils
llmcompressor.transformers.utils.helpers
llmcompressor.transformers.utils.preprocessing_functions
utils
utils
llmcompressor.utils.dev
llmcompressor.utils.helpers
llmcompressor.utils.metric_logging
fsdp
fsdp
llmcompressor.utils.fsdp.context
llmcompressor.utils.fsdp.helpers
pytorch
pytorch
llmcompressor.utils.pytorch.module
llmcompressor.utils.pytorch.utils
Table of contents
typing
llmcompressor.typing
Back to top