Notes and experiments on low-bit quantization, efficient LLM inference, and numerical formats for neural networks.
A walkthrough of block-scaled 4-bit floating point (MXFP4/MXFP6), how the shared exponent works, and where accuracy is lost versus INT4.
Second-order error compensation for post-training quantization — how the layer-wise Hessian shapes rounding decisions, and practical notes on calibration.
Why a small fraction of channels carry outsized magnitude, and how mixed-precision or channel reordering keeps them from dominating quantization error.
Post-training quantization · Low-bit numerical formats (INT4/INT8, FP4/FP6, MX) · KV-cache compression · Efficient LLM serving · Kernel-level performance
LowBit Lab collects working notes on making large models cheaper to run — quantization algorithms, numerical formats, and inference systems. Content here is exploratory and updated as experiments progress.