pith. sign in

arxiv: 2209.05433 · v2 · pith:KZGL6CQJnew · submitted 2022-09-12 · 💻 cs.LG

FP8 Formats for Deep Learning

Paulius Micikevicius , Dusan Stosic , Neil Burgess , Marius Cornea , Pradeep Dubey , Richard Grisenthwaite , Sangwon Ha , Alexander Heinecke
show 7 more authors
Patrick Judd John Kamalu Naveen Mellempudi Stuart Oberman Mohammad Shoeybi Michael Siu Hao Wu
This is my paper

Pith reviewed 2026-05-15 09:42 UTC · model grok-4.3

classification 💻 cs.LG
keywords FP88-bit floating pointdeep learning trainingquantizationlarge language modelsCNNRNNTransformer
0
0 comments X

The pith

FP8 with E4M3 and E5M2 encodings matches 16-bit training accuracy on large language and image models without hyperparameter changes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes an 8-bit floating point format for deep learning to speed up training and inference beyond current 16-bit standards. It defines two encodings, E4M3 and E5M2, with E4M3 extending the dynamic range by omitting infinities. Experiments show this format achieves the same result quality as 16-bit training across CNNs, RNNs, and Transformers, including models with up to 175 billion parameters, while keeping all other settings the same. It also works for post-training quantization of models that resist integer quantization.

Core claim

We propose an 8-bit floating point (FP8) binary interchange format consisting of two encodings - E4M3 (4-bit exponent and 3-bit mantissa) and E5M2 (5-bit exponent and 2-bit mantissa). While E5M2 follows IEEE 754 conventions for representation of special values, E4M3's dynamic range is extended by not representing infinities and having only one mantissa bit-pattern for NaNs. We demonstrate the efficacy of the FP8 format on a variety of image and language tasks, effectively matching the result quality achieved by 16-bit training sessions. Our study covers the main modern neural network architectures - CNNs, RNNs, and Transformer-based models, leaving all the hyperparameters unchanged from the

What carries the argument

The FP8 format consisting of E4M3 and E5M2 encodings that balance range and precision for neural network training and inference.

If this is right

  • FP8 training matches 16-bit accuracy on CNNs, RNNs, and Transformers without changing hyperparameters.
  • The format supports post-training quantization for language models that resist int8 quantization.
  • Accuracy is preserved for models up to 175 billion parameters.
  • FP8 enables acceleration of both training and inference beyond 16-bit formats.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Adopting FP8 could halve the memory and compute costs for large AI model training compared to 16-bit formats.
  • The format's design may serve as a template for even lower precision formats in future deep learning hardware.
  • Integration into standard processors would allow seamless switching from 16-bit to FP8 in existing workflows.
  • Further validation on additional tasks like reinforcement learning could extend the applicability.

Load-bearing premise

That the chosen E4M3 and E5M2 encodings will preserve accuracy across all tasks and model scales without any hyperparameter retuning or task-specific adjustments.

What would settle it

Observing a significant accuracy drop when training a 175B parameter language model in FP8 compared to 16-bit, with all other training settings unchanged, would falsify the main claim.

read the original abstract

FP8 is a natural progression for accelerating deep learning training inference beyond the 16-bit formats common in modern processors. In this paper we propose an 8-bit floating point (FP8) binary interchange format consisting of two encodings - E4M3 (4-bit exponent and 3-bit mantissa) and E5M2 (5-bit exponent and 2-bit mantissa). While E5M2 follows IEEE 754 conventions for representatio of special values, E4M3's dynamic range is extended by not representing infinities and having only one mantissa bit-pattern for NaNs. We demonstrate the efficacy of the FP8 format on a variety of image and language tasks, effectively matching the result quality achieved by 16-bit training sessions. Our study covers the main modern neural network architectures - CNNs, RNNs, and Transformer-based models, leaving all the hyperparameters unchanged from the 16-bit baseline training sessions. Our training experiments include large, up to 175B parameter, language models. We also examine FP8 post-training-quantization of language models trained using 16-bit formats that resisted fixed point int8 quantization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper proposes two 8-bit floating-point interchange formats, E4M3 (4-bit exponent, 3-bit mantissa, no infinities, single NaN pattern) and E5M2 (5-bit exponent, 2-bit mantissa, IEEE 754 conventions), for deep-learning training and inference. It reports that these formats match FP16 accuracy on CNNs, RNNs, and Transformer-based models (including language models up to 175B parameters) when all optimizer, learning-rate, batch-size, and loss-scaling hyperparameters are left unchanged from the 16-bit baselines; it also examines FP8 post-training quantization on models resistant to int8.

Significance. If the reported matching accuracy holds across the claimed scales and architectures, the work provides a concrete, immediately usable path to accelerate both training and inference on hardware that supports FP8, with direct relevance to scaling large models while preserving quality and without requiring hyperparameter retuning.

minor comments (3)
  1. [Abstract] Abstract: the efficacy claim would be stronger if it included one or two concrete accuracy numbers (e.g., top-1 on ImageNet or perplexity on a language-modeling benchmark) rather than the general statement of 'matching the result quality.'
  2. [Section 3] Section 3 (format definitions): a small table comparing the dynamic range and precision of E4M3/E5M2 against FP16 and bfloat16 would help readers quickly assess why the chosen encodings are expected to suffice for the reported tasks.
  3. [Experiments] Experimental sections: while the paper states that hyperparameters were left unchanged, it would be useful to list the exact loss-scaling factors used for each model family to allow exact reproduction.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of our work and the recommendation for minor revision. We appreciate the recognition of the practical relevance of the proposed FP8 formats for both training and inference at scale.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper defines explicit FP8 encodings (E4M3 and E5M2) with fixed bit allocations and special-value rules, then reports direct empirical accuracy matches to 16-bit baselines on CNNs, RNNs, and Transformers (including 175B models) using identical hyperparameters. No mathematical derivations, fitted parameters, or predictions appear; all load-bearing claims are observational results from controlled experiments rather than quantities that reduce to the inputs by construction. No self-citations are invoked to justify uniqueness or force the format choice. The work is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The formats are defined by fixed bit allocations with one deviation from IEEE 754; no numerical fitting or new physical entities are introduced.

axioms (1)
  • standard math Standard rules for floating-point representation and rounding apply except for special values in E4M3
    Invoked when defining E4M3 without infinities and single NaN pattern
invented entities (1)
  • E4M3 encoding without infinities no independent evidence
    purpose: Extend dynamic range for activations and gradients in deep learning
    Introduced to better cover the numerical range encountered in neural network training

pith-pipeline@v0.9.0 · 5551 in / 1249 out tokens · 34536 ms · 2026-05-15T09:42:41.400605+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 59 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Rigel: Reverse-Engineering the Metal 4.1 Tensor Compute Path on the Apple M4 Max GPU

    cs.CL 2026-06 accept novelty 8.0

    Rigel reverse-engineers the Metal 4.1 tensor compute path on M4 Max, finding fp8 matmul2d is emulated on GPU shader cores at 0.94x fp16 throughput with an 8x8 fragment layout and no ANE involvement.

  2. What Limits Does Quantization Place on Dense Top-$k$ Retrieval? A Theoretical Study

    cs.IR 2026-06 unverdicted novelty 8.0

    Quantization forces Bd = Ω(k ln N) for perfect top-k retrieval realizability, plus a B* = O(ln ln N) threshold below which no d works under uniform scalar quantization.

  3. Bit-Accurate Modeling of GPU Matrix Multiply-Accumulate Units: Demystifying Numerical Discrepancy and Accuracy

    cs.AR 2025-11 accept novelty 8.0

    The authors derive the first bit-accurate arithmetic models for matrix multiply-accumulate operations on ten GPU architectures spanning NVIDIA Volta to Blackwell and AMD CDNA1 to CDNA3.

  4. Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads

    cs.AI 2026-06 unverdicted novelty 7.0

    The paper delivers the first systems characterization of agent memory, with a four-axis taxonomy, phase-aware profiler, evaluation of ten systems on two benchmarks, and ten design recommendations.

  5. Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning

    cs.LG 2026-06 unverdicted novelty 7.0 partial

    IEEE P3109 defines a family of adjustable low-precision floating-point formats for ML with decoding to extended reals, multiple rounding modes, block operations, kappa-approximation for approximations, and mechanical ...

  6. Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations

    cs.LG 2026-05 unverdicted novelty 7.0

    Floating-point neural networks achieve universal representability for practical activations like ReLU, sigmoid, and tanh under arbitrary reduction orders and bounded ulp errors in activations via a new distinguishabil...

  7. LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

    cs.CV 2026-05 unverdicted novelty 7.0

    LongLive-2.0 delivers an NVFP4 parallel infrastructure that enables direct training of long multi-shot autoregressive diffusion video models and achieves up to 2.15x training and 1.84x inference speedups on Blackwell ...

  8. AIS: Adaptive Importance Sampling for Quantized RL

    stat.ML 2026-05 unverdicted novelty 7.0

    AIS adaptively corrects non-stationary policy gradient bias in quantized LLM RL, matching BF16 performance while retaining 1.5-2.76x FP8 rollout speedup.

  9. The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures

    cs.DC 2026-05 unverdicted novelty 7.0

    Power capping is illusory in LLM decode as memory-bound operation leaves power headroom untouched on 700 W GPUs, while SM clock locking saves up to 32% energy and three DVFS classes appear across attention types.

  10. HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model

    cs.CL 2026-05 unverdicted novelty 7.0

    Hebatron is the first open-weight Hebrew MoE LLM adapted from Nemotron-3, reaching 73.8% on Hebrew reasoning benchmarks while activating only 3B parameters per pass and supporting 65k-token context.

  11. TransDot: An Area-efficient Reconfigurable Floating-Point Unit for Trans-Precision Dot-Product Accumulation for FPGA AI Engines

    cs.AR 2026-05 unverdicted novelty 7.0

    TransDot unifies SIMD FMA and trans-precision DPA in one reconfigurable FPU, achieving 2x FP16, 4x FP8, and 8x FP4 throughput with FP32 accumulation plus 1.46x to 2.92x area efficiency gains over the FPnew baseline.

  12. ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs

    cs.AR 2026-03 unverdicted novelty 7.0

    ENEC delivers 3.43X higher throughput than DietGPU and 1.12X better compression ratio than nvCOMP for lossless model weight compression on Ascend NPUs, yielding up to 6.3X end-to-end inference speedup.

  13. Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention

    cs.LG 2025-10 unverdicted novelty 7.0

    Low-precision Flash Attention fails due to similar low-rank attention representations combined with biased rounding errors that accumulate and corrupt weight updates; a minimal fix to reduce rounding bias stabilizes training.

  14. DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling

    cs.LG 2025-09 unverdicted novelty 7.0

    DPQuant uses epoch-wise probabilistic layer rotation and DP loss sensitivity to quantize only a changing subset of layers, reducing accuracy degradation from quantization noise in DP-SGD and delivering up to 2.21x thr...

  15. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

    cs.LG 2024-07 accept novelty 7.0

    FlashAttention-3 achieves 1.5-2x speedup on H100 GPUs for attention, reaching 740 TFLOPs/s (75% utilization) in FP16 and near 1.2 PFLOPs/s in FP8 while cutting numerical error by 2.6x versus baseline FP8 attention.

  16. OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters

    cs.DC 2026-07 unverdicted novelty 6.0

    OmniPilot combines conformal quantile regression with OOD detection to rank LLM serving configurations on mixed GPUs, reporting 6.2% MAPE throughput prediction and 95% top-1 accuracy on 460 benchmark runs while abstai...

  17. HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

    cs.LG 2026-06 unverdicted novelty 6.0

    HyperQuant unifies Hadamard transform, optimal lattice quantization, and entropy coding to outperform prior schemes on LLM weight and KV cache quantization down to 1.7 bits per scalar while preserving quality on a 19B...

  18. Rethinking Shrinkage Bias in LLM FP4 Pretraining: Geometric Origin, Systemic Impact, and UFP4 Recipe

    cs.AI 2026-06 unverdicted novelty 6.0

    E2M1 FP4 has inherent shrinkage bias from asymmetric bin geometry that accumulates and destabilizes training; UFP4 with uniform E1M2/INT4 grids and selective RHT/stochastic rounding reduces BF16-relative degradation i...

  19. Achieving Cloud-Grade SLOs for Local Mixture-of-Experts Inference through CPU-GPU Hybrid Design

    cs.DC 2026-06 unverdicted novelty 6.0

    A CPU-GPU hybrid design with stream-loading prefill, expert parallelism, and disaggregation achieves cloud SLOs for local MoE inference on dual-socket CPUs and consumer GPUs.

  20. dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats

    cs.LG 2026-06 unverdicted novelty 6.0

    dMX is a differentiable mixed-precision framework that learns per-layer MXFP bit-width assignments for LLMs and outperforms KL-based heuristics on perplexity and zero-shot accuracy under bit-width budgets.

  21. Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models

    cs.CV 2026-05 unverdicted novelty 6.0

    RBDC trains wide vision models by recursive block-diagonal coupling of narrower pre-trained models, reducing training FLOPs by 30% at similar ImageNet accuracy for DeiT and ResNet while outperforming model growth baselines.

  22. Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

    cs.LG 2026-05 unverdicted novelty 6.0

    MXFP4 quantization error decomposes into scale bias, deadzone truncation, and grid noise; macro-block scaling, outlier fallback, and adaptive quantization noise recover BF16 accuracy to within 0.7% and 3.0% on tested models.

  23. Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

    cs.LG 2026-05 unverdicted novelty 6.0

    MXFP4 error decomposes into scale bias, deadzone truncation, and grid noise that each dominate distinct RL failure modes, with macro-block scaling, outlier fallback, and adaptive noise recovering or exceeding BF16 per...

  24. Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

    cs.LG 2026-05 unverdicted novelty 6.0

    MXFP4 quantization error decomposes into scale bias, deadzone truncation, and grid noise; mode-targeted corrections recover BF16 accuracy within 0.7% on Qwen2.5-3B and exceed it by 1.0% on Qwen3-30B-A3B.

  25. Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory

    cs.CV 2026-05 unverdicted novelty 6.0

    IAMFlow is a training-free identity-aware memory system that tracks entities via LLM global ID assignment and VLM frame verification to reduce identity drift in narrative long video generation from shifting prompts.

  26. Search Your Block Floating Point Scales!

    cs.LG 2026-05 unverdicted novelty 6.0

    ScaleSearch optimizes block floating point scales via fine-grained search to cut quantization error by 27% for NVFP4, improving PTQ by up to 15 points on MATH500 for Qwen3-8B and attention PPL by 0.77 on Llama 3.1 70B.

  27. ShardTensor: Domain Parallelism for Scientific Machine Learning

    cs.DC 2026-05 unverdicted novelty 6.0

    ShardTensor is a domain-parallelism system for SciML that enables flexible scaling of extreme-resolution spatial datasets by removing the constraint of batch size one per device.

  28. FalconGEMM: Surpassing Hardware Peaks with Lower-Complexity Matrix Multiplication

    cs.DC 2026-05 unverdicted novelty 6.0

    FalconGEMM is a framework with deployment, execution, and decision modules that makes lower-complexity matrix multiplication practical, outperforming standard GEMM libraries by 7.59-17.85% and competitors like AlphaTe...

  29. FalconGEMM: Surpassing Hardware Peaks with Lower-Complexity Matrix Multiplication

    cs.DC 2026-05 unverdicted novelty 6.0

    FalconGEMM delivers a framework with deployment, group-parallel execution, and analytical decision modules that makes lower-complexity matrix multiplication practical, beating cuBLAS and similar libraries by 7.59-17.8...

  30. Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization

    stat.ML 2026-05 unverdicted novelty 6.0

    Spectral analysis of activations and gradients provides new diagnostics that link batch size to representation geometry, early covariance tails to token efficiency, and spectral shifts to learning dynamics in decoder-...

  31. ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parameters

    cs.CV 2026-05 unverdicted novelty 6.0

    ViTok-v2 is a 5B-parameter native-resolution image autoencoder using NaFlex and DINOv3 loss that matches or exceeds prior tokenizers at 256p and outperforms them at 512p and above while advancing the Pareto frontier i...

  32. Neural-Network-Based Variational Method in Nuclear Density Functional Theory: Application to the Extended Thomas-Fermi Model

    nucl-th 2026-04 unverdicted novelty 6.0

    Neural networks represent densities in a variational extended Thomas-Fermi model, yielding binding energies within 0.5% of prior ETF results and reproducing nuclear pasta phases.

  33. Neural-Network-Based Variational Method in Nuclear Density Functional Theory: Application to the Extended Thomas-Fermi Model

    nucl-th 2026-04 unverdicted novelty 6.0

    Neural networks parametrize nuclear densities and are variationally optimized to solve the extended Thomas-Fermi model, reproducing binding energies within 0.5% and pasta structures.

  34. StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models

    cs.LG 2026-04 unverdicted novelty 6.0

    StoSignSGD resolves SignSGD divergence on non-smooth objectives via structural stochasticity, matching optimal convex rates and improving non-convex bounds while delivering 1.44-2.14x speedups in FP8 LLM pretraining.

  35. LLM-PRISM: Characterizing Silent Data Corruption from Permanent GPU Faults in LLM Training

    cs.AR 2026-04 unverdicted novelty 6.0

    LLMs resist low-frequency permanent GPU faults but certain datapaths and precision formats trigger catastrophic training divergence even at moderate fault rates.

  36. STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training

    cs.LG 2026-04 unverdicted novelty 6.0

    STQuant dynamically allocates quantization bits for optimizer states in multimodal model training, reducing memory by 84.4% to an average 5.1 bits while preserving quality on GPT-2 and ViT.

  37. AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation

    cs.LG 2026-04 unverdicted novelty 6.0

    AdaHOP applies pattern-aware Hadamard transforms and selective outlier extraction to enable from-scratch MXFP4 training of LLMs at BF16 quality with up to 3.6X memory compression and 1.46X speedup.

  38. BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models

    cs.LG 2025-12 unverdicted novelty 6.0

    BOOST delivers 1.46-2.27x end-to-end speedups for low-rank bottleneck LLMs by redesigning tensor parallelism around the bottleneck structure plus supporting optimizations.

  39. P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formats

    cs.AR 2025-11 unverdicted novelty 6.0

    P3-LLM delivers 4.9x average speedup over HBM-PIM for edge LLM inference by pairing hybrid-format quantization with iso-area-optimized low-precision PIM compute units and operator fusion.

  40. P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S=2^8

    cs.AR 2026-06 unverdicted novelty 5.0

    Forward KV iteration in FP8 attention produces P-collapse under attention sink; reverse iteration with S=256 removes it and is optimal among bit-exact scales.

  41. GNMR: Runtime Stability Control for Low-Precision Large Language Model Training

    cs.LG 2026-05 unverdicted novelty 5.0

    GNMR is a gradient-norm-based controller that maps local stability signals to budgeted recovery actions to stabilize low-precision LLM training while preserving quality.

  42. Stochastic Rounding Increases Small Singular Values

    math.NA 2026-05 unverdicted novelty 5.0

    Stochastic rounding lifts clusters of small singular values even in constant aspect ratio matrices, extending its role as a spectral regularizer.

  43. Training-Inference Kernel Contracts: Bounding Divergence in Post-Training and Deployment

    cs.LG 2026-05 unverdicted novelty 5.0

    Introduces kernel contracts framework with derived bounds on divergence from logit drift to reward drift, specialized for RL post-training under support and norm assumptions.

  44. StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k

    cs.LG 2026-05 accept novelty 5.0

    Chunked streaming top-k enables CSA indexer execution at 1M sequence length with 6.21 GB peak memory and >=0.998 recall on synthetic V4-shaped inputs.

  45. TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

    cs.DC 2026-04 unverdicted novelty 5.0

    TACO compresses tensor-parallel intermediate tensors with an adaptive FP8 scheme and fused kernels, yielding up to 1.87X throughput gains on GPT and Qwen models with near-lossless accuracy.

  46. SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipelining

    cs.LG 2026-02 conditional novelty 5.0

    SnapMLA achieves up to 1.91x higher throughput in long-output MLA decoding using FP8 quantization and specialized kernels while keeping benchmark quality near the BF16 baseline.

  47. Balancing FP8 Computation Accuracy and Efficiency on Digital CIM via Shift-Aware On-the-fly Aligned-Mantissa Bitwidth Prediction

    cs.AR 2026-02 unverdicted novelty 5.0

    A 28nm digital CIM accelerator for FP8 uses on-the-fly shift-aware bitwidth prediction, FIFO alignment, and scalable MACs to reach 20.4 TFLOPS/W and 2.8x better efficiency than prior work while supporting variable man...

  48. NVIDIA Nemotron 3: Efficient and Open Intelligence

    cs.CL 2025-12 unverdicted novelty 5.0

    NVIDIA releases the Nemotron 3 model family with hybrid Mamba-Transformer architecture, LatentMoE, NVFP4 training, MTP layers, and multi-environment RL post-training for reasoning and agentic tasks.

  49. OISMA: On-the-fly In-memory Stochastic Multiplication Architecture for Matrix-Multiplication Workloads

    cs.AR 2025-08 unverdicted novelty 5.0

    OISMA is an in-memory computing design using quasi-stochastic bent-pyramid computing to convert memory reads into multiplications, demonstrated in a 4-kB RRAM array with 0.789 TOPS/W at 50 MHz in 180-nm technology and...

  50. Analysis of Floating-Point Matrix Multiplication Computed via Integer Arithmetic

    math.NA 2025-06 unverdicted novelty 5.0

    Error analysis and cost estimator for recasting floating-point matrix multiplication as accumulated integer products on mixed-precision hardware.

  51. NVILA: Efficient Frontier Visual Language Models

    cs.CV 2024-12 unverdicted novelty 5.0

    NVILA improves on VILA with a scale-then-compress visual token strategy and full-lifecycle efficiency optimizations, matching or exceeding leading VLMs on image and video benchmarks while reducing training cost 1.9-5....

  52. $\text{Log}_\text{b}$Quant: Quantizing Language Models in Logarithmic Space

    cs.CL 2026-07 unverdicted novelty 4.0

    Log_b Quant is an adjustable-base logarithmic quantization technique that outperforms tensor-wise asymmetric linear quantization at 4-bit precision on language model benchmarks while providing memory savings.

  53. GoldenFloat: A Phi-Derived Static-Split Floating-Point Family from GF4 to GF1024 with a Lucas-Exact Integer Identity

    cs.AR 2026-06 unverdicted novelty 4.0

    GoldenFloat introduces a phi-derived rule for setting exponent and fraction widths across floating-point formats from 4 to 1024 bits, backed by open RTL generator, Lucas-exact accumulator, and FPGA implementation.

  54. OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning

    cs.CV 2026-05 unverdicted novelty 4.0

    OSP-Next reports 83.73% VBench score and up to 2.27x speedup via hybrid sparse attention, SSP parallelism, HiF8 quantization, and Mix-GRPO on diffusion transformers.

  55. Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training

    cs.LG 2026-05 unverdicted novelty 4.0

    Identifies amax saturation and catastrophic forgetting in HiF8 W8A8 QAT for OpenPangu-Embedded-1B and mitigates them with 64-step max-window DTS and 500-step BF16 warmup at lr=1e-5 to achieve under 0.6% benchmark drops.

  56. StatQAT: Statistical Quantizer Optimization for Deep Networks

    stat.ML 2026-05 unverdicted novelty 4.0

    A statistical error analysis framework yields iterative and analytic quantizers that improve accuracy and stability when incorporated into quantization-aware training for integer and floating-point formats.

  57. HiFloat4 Format for Language Model Pre-training on Ascend NPUs

    cs.LG 2026-04 unverdicted novelty 4.0

    HiFloat4 FP4 with stabilization techniques trains dense and MoE language models on Ascend NPUs at relative error within 1% of full-precision baselines.

  58. Mellum2 Technical Report

    cs.CL 2026-05 unverdicted novelty 3.0

    Mellum 2 is a 12B MoE model with 2.5B active parameters, trained on 10.6T tokens with MoE, GQA, SWA, and MTP, then post-trained into Instruct and Thinking variants, claimed competitive with 4B-14B models at 2.5B compute.

  59. Rewriting TTS Inference Economics: Lightning V2 on Tenstorrent Achieves 4x Lower Cost Than NVIDIA L40S

    eess.AS 2026-03 unverdicted novelty 3.0

    Lightning V2 achieves 4x lower on-prem accelerator cost for TTS inference on Tenstorrent hardware than NVIDIA L40S at equivalent throughput and production audio fidelity.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages · cited by 55 Pith papers · 7 internal anchors

  1. [1]

    Michael J. Anderson, Benny Chen, Stephen Chen, Summer Deng, Jordan Fix, Michael Gschwind, Aravind Kalaiah, Changkyu Kim, Jaewon Lee, Jason Liang, Haixin Liu, Yinghai Lu, Jack Montgomery, Arun Moorthy, Nadathur Satish, Sam Naghshineh, Avinash Nayak, Jongsoo Park, Chris Petersen, Martin Schatz, Narayanan Sundaram, Bangsheng Tang, Peter Tang, Amy Yang, Jieca...

    work page arXiv 2021

  2. [2]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott...

    work page 1901

  3. [3]

    Bfloat16 processing for neural networks

    Neil Burgess, Jelena Milanovic, Nigel Stephens, Konstantinos Monachopoulos, and David Mansell. Bfloat16 processing for neural networks. In Martin Langhammer Sylvie Boldo, editor, 26th IEEE Symposium on Computer Arithmetic, ARITH 2019, Kyoto, Japan, June 10-12, 2019 , pages 88–91. IEEE, 2017

    work page 2019

  4. [4]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradb...

    work page 2022

  5. [5]

    Binaryconnect: Training deep neural networks with binary weights during propagations

    Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in neural information processing systems , 28, 2015. 7

    work page 2015

  6. [6]

    The Pile: An 800GB Dataset of Diverse Text for Language Modeling

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  7. [7]

    Binarized neural networks

    Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks. Advances in neural information processing systems , 29, 2016

    work page 2016

  8. [8]

    Rethinking floating point for deep learning

    Jeff Johnson. Rethinking floating point for deep learning. CoRR, abs/1811.01721, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  9. [9]

    Dhiraj D. Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja V ooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. A study of BFLOAT16 fo...

    work page internal anchor Pith review Pith/arXiv arXiv 1905

  10. [10]

    FP8 quantization: The power of the exponent

    Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. FP8 quantization: The power of the exponent. arXiv, 2208.09225, 2022

    work page arXiv 2022

  11. [11]

    Lee, Daisuke Miyashita, Elaina Chai, Boris Murmann, and S

    Edward H. Lee, Daisuke Miyashita, Elaina Chai, Boris Murmann, and S. Simon Wong. Lognet: Energy-efficient neural networks using logarithmic computation. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5-9, 2017 , pages 5900–5904. IEEE, 2017

    work page 2017

  12. [12]

    Mixed precision training with 8-bit floating point, 2019

    Naveen Mellempudi, Sudarshan Srinivasan, Dipankar Das, and Bharat Kaul. Mixed precision training with 8-bit floating point, 2019

    work page 2019

  13. [13]

    Mixed precision training: theory and prac- tice, 2018

    Paulius Micikevicius. Mixed precision training: theory and prac- tice, 2018. https://on-demand.gputechconf.com/gtc/2018/presentation/ s8923-training-neural-networks-with-mixed-precision-theory-and-practice.pdf , Accessed on 2022-09-11

    work page 2018

  14. [14]

    Mixed Precision Training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F. Diamos, Erich Elsen, David García, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. Mixed precision training.arxiv, 1710.03740, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  15. [15]

    Convolutional Neural Networks using Logarithmic Data Representation

    Daisuke Miyashita, Edward H. Lee, and Boris Murmann. Convolutional neural networks using logarithmic data representation. CoRR, abs/1603.01025, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  16. [16]

    8-bit numerical formats for deep neural networks,

    Badreddine Noune, Philip Jones, Daniel Justus, Dominic Masters, and Carlo Luschi. 8-bit numerical formats for deep neural networks. arXiv preprint arXiv:2206.02915, 2022

    work page arXiv 2022

  17. [17]

    Xnor-net: Imagenet classification using binary convolutional neural networks

    Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016 , pages 525–542, Cham, 2016. Springer International Publishing

    work page 2016

  18. [18]

    Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

    Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. Using deepspeed and megatron to ...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  19. [19]

    Accelerating ai training with nvidia tf32 tensor cores, 2021

    Dusan Stosic and Paulius Micikevicius. Accelerating ai training with nvidia tf32 tensor cores, 2021. https:// developer.nvidia.com/blog/accelerating-ai-training-with-tf32-tensor-cores/ , Accessed on 2022-09-4

    work page 2021

  20. [20]

    Hybrid 8-bit floating point (hfp8) training and inference for deep neural networks

    Xiao Sun, Jungwook Choi, Chia-Yu Chen, Naigang Wang, Swagath Venkataramani, Vijayalakshmi (Viji) Srini- vasan, Xiaodong Cui, Wei Zhang, and Kailash Gopalakrishnan. Hybrid 8-bit floating point (hfp8) training and inference for deep neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neu...

    work page 2019

  21. [21]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, volume 139, pages 10347–10357, July 2021

    work page 2021

  22. [22]

    Training deep neural networks with 8-bit floating point numbers

    Naigang Wang, Jungwook Choi, Daniel Brand, Chia-Yu Chen, and Kailash Gopalakrishnan. Training deep neural networks with 8-bit floating point numbers. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018

    work page 2018

  23. [23]

    arXiv preprint arXiv:2004.09602 , year=

    Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quantization for deep learning inference: Principles and empirical evaluation. arxiv, 2004.09602, 2020. 8

    work page arXiv 2004

  24. [24]

    Lq-nets: Learned quantization for highly accurate and compact deep neural networks

    Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. Lq-nets: Learned quantization for highly accurate and compact deep neural networks. European conference on computer vision (ECCV) , pages 365–382, 2018

    work page 2018

  25. [25]

    Opt: Open pre-trained transformer language models, 2022

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022

    work page 2022

  26. [26]

    DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint, 1606.06160, 2016. 9

    work page internal anchor Pith review Pith/arXiv arXiv 2016