TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

In 30 Sekunden

Was passiert ist

TensorRT Edge-LLM completed MLPerf Edge Agentic benchmark 6.4x faster than reference, running Qwen3.6-27B on Jetson AGX Thor at 52.33 tokens per second.

Warum es zählt

Matters for engineers deploying multi-turn AI agents on edge devices like robots and vehicles where long-context inference and tool calling efficiency are critical.

Achtung

Result uses specialized techniques including NVFP4 quantization, tree-based multi-token prediction, and KV cache reuse that require specific hardware and careful configuration to replicate.