Agentic Design

Patterns
🤖

Machine Learning Model-Based Routing(MLMR)

A specialized routing approach that employs discriminative models (classifiers) fine-tuned on labeled data to make routing decisions, encoding routing logic directly in model weights rather than prompts, enabling sub-10ms inference for high-volume agentic AI systems requiring deterministic and explainable routing decisions

Complexity: highRouting

🎯 30-Second Overview

Pattern: Fine-tuned discriminative model encoding routing logic in learned weights

Why: Enables ultra-fast (<10ms) routing decisions with high accuracy after supervised training

Key Insight: Routing logic embedded in model parameters, not in prompts - inference without generation

⚡ Quick Implementation

1Label Data:Create training corpus with routing labels
2Train Model:Fine-tune classifier on labeled examples
3Embed Logic:Encode routing in model weights
4Deploy:Serve model for real-time routing
5Monitor:Track accuracy, drift, and performance
Example: query → classifier → {support: 0.92, sales: 0.05, billing: 0.03} → route_to_support

📋 Do's & Don'ts

✅Use supervised fine-tuning with domain-specific labeled data
✅Start with smaller models (BERT-base) for lower latency
✅Implement confidence thresholds for routing decisions
✅Monitor class distribution and retrain on drift
✅Use synthetic data generation from LLMs to augment training set
❌Use generative models for real-time routing decisions
❌Deploy without fallback mechanisms for low-confidence predictions
❌Ignore class imbalance in training data
❌Skip A/B testing against baseline routing methods
❌Neglect explainability for critical routing decisions

🚦 When to Use

Use When

  • • High-volume routing with labeled training data
  • • Need sub-10ms routing latency
  • • Clear routing categories/classes
  • • Regulatory requirements for deterministic decisions

Avoid When

  • • Limited labeled data (<1000 examples)
  • • Constantly evolving routing rules
  • • Need interpretable routing logic
  • • Small-scale applications

📊 Key Metrics

Routing Accuracy
F1 score per route class
Latency
P50/P95/P99 inference time
Model Drift
Distribution shift detection
Confidence Calibration
ECE (Expected Calibration Error)
Coverage
% queries above confidence threshold
Cost Efficiency
Inference cost per 1M requests

💡 Top Use Cases

Intent Classification: customer_query → {support: 0.89, sales: 0.08, info: 0.03}
Ticket Routing: issue_description → {technical_L1: 0.72, technical_L2: 0.25, billing: 0.03}
Language Detection: multilingual_text → {en: 0.95, es: 0.03, fr: 0.02}
Priority Triage: request → {urgent: 0.91, normal: 0.07, low: 0.02}
Department Assignment: email → {hr: 0.88, legal: 0.10, finance: 0.02}

References & Further Reading

Deepen your understanding with these curated resources

Academic Papers

Fine-Tuning Language Models for Classification Tasks (Howard & Ruder, 2018)

BERT: Pre-training of Deep Bidirectional Transformers (Devlin et al., 2019)

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (Reimers & Gurevych, 2019)

On Calibration of Modern Neural Networks (Guo et al., 2017)

Implementation Guides

Hugging Face Text Classification Fine-tuning Guide

Google Cloud AutoML Text Classification

AWS SageMaker Multi-class Classification

FastAPI + ONNX for Production ML Routing

Tools & Libraries

SetFit - Efficient Few-shot Learning for Classification

AutoGluon - AutoML for Text Classification

Scikit-learn - Traditional ML Classifiers

ONNX Runtime - High-performance Inference

Community & Best Practices

MLOps Community - Model Deployment Best Practices

Google ML Crash Course - Text Classification

Papers with Code - Text Classification Benchmarks

Model Monitoring and Drift Detection Guide

Contribute to this collection

Know a great resource? Submit a pull request to add it.

Contribute

Patterns

closed

Loading...

Built by Kortexya