EP Scaling in MoE Inference: Why Prefill and Decode Diverge
Expert parallelism changes weight reuse, token replication, and load imbalance. A simple cost model explains why prefill favors small EP groups while decode benefits from wider ones.
Writing
Writing
Notes on machine learning systems, GPU kernels, and efficient inference.
DeepEP Dispatch: Four Designs for Unknown Receive Counts
Compare four DeepEP dispatch designs through buffer ownership, completion signals, and network traffic—and connect those choices to communication latency.
Push or Pull? Communication Design in DeepEP and Mega MoE
The direction of a transfer follows who knows its final address. DeepEP and Mega MoE illustrate how that choice shapes buffering, execution order, and GPU resource use.
Fusing Indexer Scoring and Exact Top-k for DeepSeek-V3.2
An H20 case study in overlapping Indexer scoring with exact selection: sampled filtering, warp specialization, performance attribution, and end-to-end validation.
Profiling Ozaki II: The Cost Behind Fast INT8 GEMMs
Fast Tensor Core GEMMs can expose a slower preprocessing path. Trace Ozaki II from modular arithmetic to H20 instructions, measured stage costs, and optimization opportunities.
Sparse Attention from Algorithm to GPU: NSA, DSA, and Discovery
Follow sparse attention from selection rules to GPU tiles, then examine why discovering the selected tokens can dominate a sparse attention pipeline.
