ResearchLMSYS·10d ago·3 min readSummaryListen 0:42
Accelerating Long-Context and Agentic Inference with NVFP4 KV CacheNVFP4 KV cache accelerates long-context and agentic inference.
News Hub
What actually shipped in agent engineering, pulled from the labs, arXiv and Hacker News.
See who we follow51 articles
ResearchLMSYS·16d ago·3 min readSummaryListen 0:53
SGLang and Miles Add Day-0 Support for DeepSeek-V4.1SGLang and Miles add day-zero support for DeepSeek-V4.1.
ResearchLMSYS·28d ago·3 min read
Running DeepSeek-V4-Flash and Kimi-K3 on Consumer Hardware with SSD Expert PackDeepSeek-V4-Flash and Kimi-K3 models can run on consumer hardware using SSD Expert Pack.
ResearchLMSYS·30d ago·3 min readSummary
MiniMax-H3 on 8×H200: 1.95× Lossless, Up to 6.24× at 0.76MiniMax-H3 achieves 1.95x lossless speedup on 8 H200 GPUs, scaling to 6.24x at reduced precision.
ResearchLMSYS·31d ago·3 min readSummary
Qwen3.8-Flash-Next: Day-0 Support in SGLangSGLang provides day-zero support for Qwen3.8-Flash-Next model.
ResearchLMSYS·36d ago·3 min readSummaryListen 0:51
Chasing the Batch-1 Floor: Ling-3.0-flash Speculative Decode on BlackwellLing-3.0-flash uses speculative decoding on Blackwell to optimize batch-1 inference performance.
ResearchLMSYS·37d ago·3 min readSummaryListen 0:45
Mooncake for Miles: From Fragmented Rollout Data to Efficient Bulk I/OMooncake converts fragmented rollout data into efficient bulk I/O operations.
ResearchLMSYS·38d ago·3 min readSummaryListen 0:47
Pushing the Limits of Serving DeepSeek-V4-ProLMSYS examined serving performance limits for DeepSeek-V4-Pro models.
ResearchLMSYS·39d ago·3 min readSummaryListen 0:47
Miles v0.1: Production-level Post-trainingLMSYS released Miles v0.1 for production-level post-training of language models.
ResearchLMSYS·40d ago·3 min readSummaryListen 0:48
Advanced CUDA Graph Techniques in InferenceAdvanced CUDA Graph Techniques in Inference discusses optimization methods for model inference.
ResearchLMSYS·45d ago·3 min readSummaryListen 0:52
SGLang and Miles Add Day-0 Support for Qwen3.8SGLang and Miles add day-0 support for Qwen3.8 model.
ResearchLMSYS·46d ago·3 min readSummaryListen 0:45
SGLang Adds Day-0 Support for NVIDIA Nemotron 3.5 LightningSGLang added immediate support for NVIDIA Nemotron 3.5 Lightning.
ResearchLMSYS·50d ago·3 min readSummaryListen 0:47
HPC-Ops × SGLang: High-Performance Attention, Router GEMM, and MoE Kernels from Tencent HunyuanTencent Hunyuan contributes high-performance attention, router GEMM, and MoE kernels to SGLang.
ResearchLMSYS·53d ago·3 min readSummaryListen 0:40
SpecForge v0.3.0: a Unified Disaggregated and Colocated Speculative Decoding Stack, and New Open SpecBundle Draft ModelsSpecForge v0.3.0 releases a unified disaggregated and colocated speculative decoding stack with new open SpecBundle draft models.
ResearchLMSYS·59d ago·3 min readSummaryListen 0:43
Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in MilesEnd-to-end reinforcement learning with 8-bit and 4-bit formats on Blackwell hardware.
ResearchLMSYS·60d ago·3 min readSummaryListen 0:54
Toward a Cleaner Quantization Stack in SGLangSGLang improves quantization stack implementation.
ResearchLMSYS·74d ago·3 min read
Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 WeeksSGLang achieved 500 tokens per second serving GLM5.2 NVFP4 agentic workloads in two weeks.
ResearchLMSYS·82d ago·3 min read
DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length VerificationDSpark implements speculative decoding with confidence-driven variable-length verification in SGLang.
ResearchLMSYS·86d ago·3 min read
Agent-Assisted SGLang Development: An Initial ExplorationAgent-assisted development tools explored for SGLang.
ResearchLMSYS·101d ago·3 min read
Optimizing Ling-2.6-1T on TPU with SGLang-JAX: Hiding MoE Data Movement Behind Compute with One Pallas KernelLing-2.6-1T optimized on TPU with SGLang-JAX hides MoE data movement behind compute.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.