Instructions to use meatloafer/efficientnet-b0-sub4bit-mixed-precision with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Keras
How to use meatloafer/efficientnet-b0-sub4bit-mixed-precision with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://meatloafer/efficientnet-b0-sub4bit-mixed-precision") - LiteRT
How to use meatloafer/efficientnet-b0-sub4bit-mixed-precision with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
EfficientNet-B0 Sub-4-Bit Mixed-Precision Recovery & LiteRT Deployment Artifacts
Official model checkpoints, 82-layer dynamic-programming (DP) bit-allocation profiles, exported LiteRT (.tflite) deployment FlatBuffers, and SHA-256 verification manifests for the paper:
Why Sub-4-Bit Vision CNNs Collapseβand Where Mixed-Precision Recovery Stops
Michael Schroeder (2026)
Overview
While full-integer 8-bit quantization (INT8) on EfficientNet-B0 is nearly lossless ($72.81%$ vs. $72.86%$ top-1 on the 50,000-image ImageNet-1k validation split, $-70.7%$ artifact size, and $-32.2%$ median CPU latency), uniform 4-bit ($W_4$) and 3-bit ($W_3$) weight quantization degrade accuracy to $23.03%$ and $0.30%$.
This repository releases the checkpoints, per-layer precision profiles, and exported LiteRT FlatBuffers that isolate the three physical mechanisms of sub-4-bit collapse and their recovery boundaries:
- Early $3\times 3$ Depthwise Spatial Sensitivity: Protecting just the first $0.34%$ of model weights ($13,952$ parameters, $14\text{ KB}$) at 8-bit while holding the remaining $99.66%$ at 4-bit restores top-1 accuracy from $23.03% \rightarrow 65.95%$ ($+42.92\text{ pp}$).
- Per-Channel MSE Grid Clipping $\times$ BatchNorm Recalibration Interaction: Searching optimal per-channel clipping thresholds $\alpha^* \in [0.50, 1.00]$ and recalibrating BatchNorm statistics on 1,000 class-balanced training images lifts a 3.55-bit W8/W4/W3 model from $51.15% \rightarrow 64.09%$ ($+12.94\text{ pp}$).
- 82-Layer Dynamic-Programming Bit Allocation ($\text{DP}_{0.5}$, $3.56$ Mean Bits) + 1-Epoch QAT: Solving a knapsack dynamic program across all 82 weighted layers at ${W_3, W_4, W_8}$ reaches $67.94%$ post-training accuracy at $3.56$ mean bits ($2.35\text{ MB}$ weight budget), and one epoch of QAT followed by 1,000-image post-QAT BatchNorm recalibration reaches **$68.98%$**βrecovering $+68.68$ percentage points over uniform $W_3$ ($0.30%$) while establishing a $\sim 3.9$-point accuracy floor below FP32 ($72.86%$).
- The Hardware Reality Gap: Real-world LiteRT CPU benchmarks included here show that full-integer INT8 (
litert/model_int8.tflite, 6.00 MB, 6.65 ms) dominates FP32 (9.81 ms) on both size and speed, whereas sub-8-bit formats (litert/model_dynamic_wi4.tflite, 3.59 MB, 15.43 ms) execute slower on standard CPUs without native sub-byte depthwise kernels due to runtime bit-unpacking overhead.
Included Artifacts
1. Keras Checkpoints (weights/)
| File | Precision Profile | Mean Bits | Weight Budget | Top-1 (%) | Top-5 (%) | File Size | SHA-256 Prefix |
|---|---|---|---|---|---|---|---|
weights/efficientnet_b0_fp32_reference_e86.weights.h5 |
FP32 Reference Baseline (B0 A4, e86) | 32.00 | 20.43 MB | 72.864% | 91.110% | 41.13 MB | 4d11eb8259c4 |
weights/efficientnet_b0_dp05_3.56bit_qat.weights.h5 |
$\text{DP}_{0.5}$ ($W_8/W_4/W_3$) + MSE Clip + 1-Ep QAT | 3.56 | 2.35 MB | 68.984% | 88.684% | 41.13 MB | 1d6430b8725f |
2. Exported LiteRT / TFLite Deployment FlatBuffers (litert/)
| File | Runtime Format | Operator Breakdown | Artifact Size | Top-1 (%) | CPU p50 Latency | SHA-256 Prefix |
|---|---|---|---|---|---|---|
litert/model_int8.tflite |
Full-Integer INT8 ($\star$ Pareto Sweet Spot) | 82 native INT8 SIMD conv/dw/dense ops | 5.90 MiB (6.00 MB) | 72.806% | 6.65 ms ($-32.2%$) | ed409b344bdf |
litert/model_dynamic_wi4.tflite |
Mixed 4-Bit Weights (WI4) | 66 INT4 conv/dense + 16 FP32 depthwise ops | 3.54 MiB (3.59 MB) | 63.268% | 15.43 ms ($+47.9%$) | 4f6092d7003a |
litert/model_float32.tflite |
Float32 Reference Baseline | 82 native FP32 SIMD conv/dw/dense ops | 20.15 MiB (20.43 MB) | 72.864% | 9.81 ms ($1.00\times$) | 2f1b1cc2071c |
3. Dynamic-Programming Bit-Allocation Profile (profiles/)
profiles/dp05_3.56bit_mse_clip_profile.json: Exact 82-layer ${W_3, W_4, W_8}$ bit-width assignments, per-channel MSE clipping scales ($\alpha^*$), and distortion ledger for the $3.56$-bit $\text{DP}_{0.5}$ profile (sha256: 1c9c86e2171e...).
Repository Structure
.
βββ README.md
βββ SHA256SUMS.txt # Exact SHA-256 checksums for all artifacts
βββ artifacts_manifest.json # Machine-readable metadata & checksum ledger
βββ code/
β βββ models.py # EfficientNet-B0 ReLU6 model builder
β βββ quantization.py # Per-channel symmetric & MSE-clipped quantizers
β βββ batchnorm_recalibration.py # 1,000-image population BatchNorm recalibration
β βββ adaptive_mixed_precision_qat.py # Layer-wise mixed-precision STE QAT wrapper
βββ litert/
β βββ model_int8.tflite # 6.00 MB Full-Integer INT8 FlatBuffer (72.81% Top-1)
β βββ model_dynamic_wi4.tflite # 3.59 MB Mixed 4-Bit WI4 FlatBuffer (63.27% Top-1)
β βββ model_float32.tflite # 20.43 MB Float32 Reference FlatBuffer (72.86% Top-1)
βββ profiles/
β βββ dp05_3.56bit_mse_clip_profile.json # 82-layer DP_0.5 bit-allocation & MSE clipping profile
βββ weights/
βββ efficientnet_b0_fp32_reference_e86.weights.h5
βββ efficientnet_b0_dp05_3.56bit_qat.weights.h5
Quickstart
1. Run Full-Integer INT8 Inference with LiteRT / TFLite
import numpy as np
import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="litert/model_int8.tflite", num_threads=4)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()[0]
output_details = interpreter.get_output_details()[0]
# Preprocess a 224x224 RGB image normalized with ImageNet mean/std
dummy_img = np.zeros((1, 224, 224, 3), dtype=np.float32)
interpreter.set_tensor(input_details["index"], dummy_img)
interpreter.invoke()
logits = interpreter.get_tensor(output_details["index"])
2. Load the Keras Checkpoints & $\text{DP}_{0.5}$ Mixed-Precision Profile
import json
from code.models import build_efficientnet
# Load FP32 reference checkpoint (72.864% Top-1)
model = build_efficientnet(variant="b0", num_classes=1000)
model.load_weights("weights/efficientnet_b0_fp32_reference_e86.weights.h5")
# Inspect the 82-layer 3.56-bit DP_0.5 bit allocation
with open("profiles/dp05_3.56bit_mse_clip_profile.json") as f:
dp_profile = json.load(f)
Verify file integrity at any time with:
sha256sum -c SHA256SUMS.txt
Citation
@article{schroeder2026sub4bit,
title = {Why Sub-4-Bit Vision CNNs Collapse---and Where Mixed-Precision Recovery Stops},
author = {Schroeder, Michael},
year = {2026}
}
- Downloads last month
- 21
Dataset used to train meatloafer/efficientnet-b0-sub4bit-mixed-precision
Evaluation results
- Top-1 Accuracy (FP32 Reference Checkpoint, 20.43 MB) on ImageNet-1k (50,000-image validation split)validation set self-reported72.864
- Top-1 Accuracy (Full-Integer INT8 LiteRT FlatBuffer, 6.00 MB, 6.65 ms CPU) on ImageNet-1k (50,000-image validation split)validation set self-reported72.806
- Top-1 Accuracy (DP_0.5 3.56-Bit Mixed-Precision + 1-Ep QAT + BN Recalibration, 2.35 MB) on ImageNet-1k (50,000-image validation split)validation set self-reported68.984
- Top-1 Accuracy (DP_0.5 3.56-Bit Mixed-Precision PTQ + MSE Clipping + BN Recalibration, 2.35 MB) on ImageNet-1k (50,000-image validation split)validation set self-reported67.940
- Top-1 Accuracy (Mixed 4-Bit WI4 LiteRT FlatBuffer, 3.59 MB) on ImageNet-1k (50,000-image validation split)validation set self-reported63.268