EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities

In 30 Sekunden

Was passiert ist

EngineAgent automates deploying new LLM models to mobile inference engines by tracking cross-layer dependencies and hardware capabilities across conversion, compilation, and execution.

Warum es zählt

Mobile engineers adding model support to on-device inference systems need to coordinate changes across multiple abstraction layers and debug failures spanning host and device execution.

Achtung

The system was tested on only twelve tasks across three models and four engine-backend pairs; generalization to other hardware platforms and inference engines remains unproven.