punalmanalan/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16-AutoRound

Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU — W4A16 AutoRound

This repository is a W4A16 quantized derivative of:

DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU

Original model:

https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU

This is the same underlying Qwen3.8-27B multimodal model in a lower-bit inference format.

This repository is not a new fine-tune, merge, or model branch.
The model weights were quantized from the original DavidAU checkpoint for more efficient inference on GPUs with limited VRAM.

Model

• Architecture: Qwen3.8-27B
• Parameter class: 27B
• Modality: Image + Text → Text
• Original model type: multimodal / vision-language
• Quantization: W4A16
• Weight precision: INT4
• Activation precision: 16-bit
• Group size: 128
• Quantization style: symmetric, group-wise
• Runtime format: llm_compressor / packed quantized
• Quantization algorithm: SignRound
• Quantization framework: AutoRound 0.15.0

The original model's multimodal capability is retained. The vision component was not converted into a text-only model.

Quantization

The 64 transformer language layers were quantized to:

• 4-bit integer weights
• 16-bit activations
• group size 128
• symmetric group-wise quantization

Calibration and optimization:

Setting	Value
AutoRound version	0.15.0
Dataset	NeelNanda/pile-10k
Calibration samples	128
Optimization iterations	200
Sequence length	2048
Batch size	1
Gradient accumulation	8
Torch compile	enabled on GPU
Disk streaming	enabled
Quantization device map	GPU

Head, Embedding and MTP preparation

The language-model output head, embeddings, and MTP components were prepared separately from the main 4-bit transformer body.

LM head

• INT8
• group size 128
• symmetric
• packed quantized representation

Token embeddings

• INT8
• group size 128
• symmetric
• packed quantized representation

MTP modules

The MTP linear components were quantized separately and preserved in the final model package.

The final package also includes the required packed draft LM head and:

mtp_draft_vocab_ids.pt

for MTP/speculative decoding setups that support this format.

Original model lineage

Base model:

Qwen/Qwen3.8-27B

Original fine-tuned model:

DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU

This repository:

punalmanalan/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16-AutoRound

Relationship:

Qwen/Qwen3.8-27B
        ↓
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
        ↓
W4A16 AutoRound quantization
        ↓
This repository

Vision / image input

The original model is an Image-Text-to-Text model.

This quantized repository preserves the multimodal model structure and is not intended to be treated as a text-only checkpoint.

The primary Hugging Face task is therefore:

image-text-to-text

Example high-level Transformers usage:

from transformers import pipeline

pipe = pipeline(
    "image-text-to-text",
    model="punalmanalan/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16-AutoRound"
)

For runtimes that support the underlying multimodal architecture, image and text inputs can be supplied through the processor/model interface.

Runtime support for vision input depends on the inference framework and its support for this quantized format.

Context length

The original model supports a native context length of 262,144 tokens.

This quantized package was prepared for long-context inference and can be used with serving configurations above the native window only when the selected runtime and RoPE/context configuration explicitly support it.

Do not assume that increasing max_model_len automatically makes every runtime configuration safe or performant.

Intended use

This quantization is intended for efficient local inference, particularly on consumer GPUs where the original BF16 checkpoint is expensive to load.

Typical use cases include:

• coding
• reasoning
• creative writing
• instruction following
• conversational workloads
• image understanding, where supported by the serving runtime
• MTP/speculative decoding, where supported

Accuracy / quality

Quantization necessarily changes the numerical representation of the model weights, so this checkpoint is not mathematically identical to the original BF16 model.

This repository was prepared specifically to reduce that loss while providing substantially lower memory usage.

No claim is made here that the quantized checkpoint is benchmark-identical to BF16. Users should evaluate both versions on their own workload when exact quality comparisons are important.

Inference

This package is intended primarily for inference frameworks that support:

• Qwen3.8 multimodal models
• compressed-tensors
• packed W4A16 quantization
• MTP/speculative decoding where applicable

It was prepared and tested around a patched vLLM 0.27.1 serving stack with MTP support.

The exact runtime configuration used for this repository is documented separately in the associated deployment project.

File contents

The repository contains:

• 67 model shards
• model_extra_tensors.safetensors
• model.safetensors.index.json
• config.json
• tokenizer files
• generation configuration
• chat template
• MTP draft vocabulary IDs
• packed MTP tensors

Important

This repository is a quantization of the original DavidAU model.

It is not a new fine-tune.

It is not a new merge.

It is not the BIGBANG model.

It is not the NOESIS model.

It is the original:

DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU

converted to a W4A16 AutoRound inference format.

Credits

Original model:

DavidAU

Original model page:

https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU

Quantization and packaging:

Punal Manalan

Quantization tooling:

Intel AutoRound

Calibration dataset:

NeelNanda/pile-10k