Agentic DesignPatterns and tools for production AI agents

Loading...

🔄

Feedback Chaining

Implements iterative improvement cycles where outputs are evaluated and fed back as inputs until quality criteria are met, proven to enhance output quality through convergence-based refinement - particularly effective for creative and optimization tasks

Complexity: mediumPrompt Chaining

Loading technique guide…

🔄

Feedback Chaining

Complexity: mediumPrompt Chaining

Loading technique guide…

Patterns

closed

Loading...

Loading Agentic Design…

Agentic DesignPatterns and tools for production AI agents

Expert help Pricing

Design Patterns & Techniques

Loading...

🔄

Feedback Chaining

Complexity: mediumPrompt Chaining

Core Mechanism

Feedback chaining runs an iterative loop: generate → evaluate → improve, repeating until a target quality threshold or stop condition is met. An evaluator (rules, rubric, or model-as-judge) produces specific feedback that drives focused revisions, enabling convergence to higher quality outputs.

🔄

Flow

Looped iterations

🧪

Evaluation

Rubrics & scoring

🎯

Convergence

Stop when target met

🧭

Control

Quality gates

Workflow / Steps

Define quality criteria and evaluation rubrics with specific, measurable thresholds.
Generate initial output using appropriate prompting and generation strategies.
Evaluate output against criteria using rules, rubrics, or model-as-judge approach.
Collect specific, actionable feedback focusing on areas needing improvement.
Revise output based on feedback, targeting identified weaknesses and gaps.
Repeat evaluation-revision loop until quality threshold met or max iterations reached.
Finalize output with quality score and iteration history for transparency.

Best Practices

Use a lightweight evaluator where possible; reserve strongest model for final acceptance.

Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.

Set max iterations and early-stop on small deltas to prevent infinite loops and control costs.

Design structured feedback formats to guide focused revisions rather than general critiques.

Implement confidence thresholds to determine when additional iterations provide diminishing returns.

Cache evaluation results and reuse across similar tasks to improve efficiency.

Monitor convergence patterns to optimize evaluation criteria and stopping conditions.

When NOT to Use

Simple tasks where the first attempt typically meets quality requirements.
Real-time applications with strict latency constraints that cannot afford iteration cycles.
Tasks lacking clear evaluation criteria or measurable quality thresholds.
Resource-constrained environments where multiple generation rounds are prohibitive.
Cases where human evaluation is required and automated feedback is insufficient.

Common Pitfalls

Vague or inconsistent evaluation criteria leading to poor feedback quality.
Infinite loops when stop conditions are poorly calibrated or missing.
Excessive iteration costs without proportional quality improvements.
Feedback that is too generic to guide effective revisions.
Over-reliance on automated evaluation without human validation of final outputs.
Poor state management causing loss of important context between iterations.

Key Features

Iterative generate-evaluate-improve loop with quality convergence tracking

Flexible evaluation strategies: rules, rubrics, or model-as-judge approaches

Structured feedback collection with actionable improvement guidance

Configurable stopping conditions with max iterations and convergence thresholds

History management with context preservation and truncation strategies

Quality scoring and transparency with iteration tracking and rationale logging

KPIs / Success Metrics

Quality gain per iteration: Δ score per iteration until convergence.
Convergence iterations: Average iterations to reach target or stop.
Judge consistency: Inter-rater or self-consistency of evaluator.
Target attainment rate: Share of runs achieving target threshold.
Cost per improvement point: Tokens per +1 score or per acceptance.
Early stop rate: Runs halted due to small deltas or max-iter cap.

Token / Resource Usage

Total cost ≈ Σ over iterations of (generator tokens + evaluator tokens + revision tokens). Expect diminishing returns; optimize for early convergence.
Use a lightweight evaluator where possible; reserve strongest model for final acceptance.
Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.
Set max iterations and early-stop on small deltas to prevent cost explosion.
Cache evaluation results and feedback patterns to reduce redundant processing.
Monitor cost-per-quality-point to optimize iteration strategies and stopping thresholds.

Best Use Cases

Content creation requiring iterative refinement for quality and style.
Code generation with compilation, testing, and improvement cycles.
Academic writing with peer review simulation and revision processes.
Creative tasks where multiple perspectives and refinements add value.
Quality assurance scenarios requiring systematic improvement until standards are met.
Complex problem-solving where initial attempts need debugging and enhancement.

References & Further Reading

Deepen your understanding with these curated resources

Academic Papers

Self-Refine: Iterative Refinement with Self-Feedback (Madaan et al., 2023)

Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)

Training Language Models to Follow Instructions with Human Feedback (Ouyang et al., 2022)

Implementation Guides

LangChain Self-Critique and Refinement Chains

OpenAI Iterative Prompting Best Practices

Anthropic Constitutional AI Implementation

Tools & Libraries

LangChain feedback and critique chain implementations

OpenAI function calling for structured evaluation

Custom evaluation frameworks and rubric systems

Community & Discussions

LangChain Community - Feedback loop patterns

r/MachineLearning - Self-refinement discussions

OpenAI Community - Iterative improvement strategies

Loading technique guide…

🔄

Feedback Chaining

Complexity: mediumPrompt Chaining

Core Mechanism

🔄

Flow

Looped iterations

🧪

Evaluation

Rubrics & scoring

🎯

Convergence

Stop when target met

🧭

Control

Quality gates

Workflow / Steps

Define quality criteria and evaluation rubrics with specific, measurable thresholds.
Generate initial output using appropriate prompting and generation strategies.
Evaluate output against criteria using rules, rubrics, or model-as-judge approach.
Collect specific, actionable feedback focusing on areas needing improvement.
Revise output based on feedback, targeting identified weaknesses and gaps.
Repeat evaluation-revision loop until quality threshold met or max iterations reached.
Finalize output with quality score and iteration history for transparency.

Best Practices

Use a lightweight evaluator where possible; reserve strongest model for final acceptance.

Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.

Set max iterations and early-stop on small deltas to prevent infinite loops and control costs.

Design structured feedback formats to guide focused revisions rather than general critiques.

Implement confidence thresholds to determine when additional iterations provide diminishing returns.

Cache evaluation results and reuse across similar tasks to improve efficiency.

Monitor convergence patterns to optimize evaluation criteria and stopping conditions.

When NOT to Use

Simple tasks where the first attempt typically meets quality requirements.
Real-time applications with strict latency constraints that cannot afford iteration cycles.
Tasks lacking clear evaluation criteria or measurable quality thresholds.
Resource-constrained environments where multiple generation rounds are prohibitive.
Cases where human evaluation is required and automated feedback is insufficient.

Common Pitfalls

Vague or inconsistent evaluation criteria leading to poor feedback quality.
Infinite loops when stop conditions are poorly calibrated or missing.
Excessive iteration costs without proportional quality improvements.
Feedback that is too generic to guide effective revisions.
Over-reliance on automated evaluation without human validation of final outputs.
Poor state management causing loss of important context between iterations.

Key Features

Iterative generate-evaluate-improve loop with quality convergence tracking

Flexible evaluation strategies: rules, rubrics, or model-as-judge approaches

Structured feedback collection with actionable improvement guidance

Configurable stopping conditions with max iterations and convergence thresholds

History management with context preservation and truncation strategies

Quality scoring and transparency with iteration tracking and rationale logging

KPIs / Success Metrics

Quality gain per iteration: Δ score per iteration until convergence.
Convergence iterations: Average iterations to reach target or stop.
Judge consistency: Inter-rater or self-consistency of evaluator.
Target attainment rate: Share of runs achieving target threshold.
Cost per improvement point: Tokens per +1 score or per acceptance.
Early stop rate: Runs halted due to small deltas or max-iter cap.

Token / Resource Usage

Total cost ≈ Σ over iterations of (generator tokens + evaluator tokens + revision tokens). Expect diminishing returns; optimize for early convergence.
Use a lightweight evaluator where possible; reserve strongest model for final acceptance.
Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.
Set max iterations and early-stop on small deltas to prevent cost explosion.
Cache evaluation results and feedback patterns to reduce redundant processing.
Monitor cost-per-quality-point to optimize iteration strategies and stopping thresholds.

Best Use Cases

Content creation requiring iterative refinement for quality and style.
Code generation with compilation, testing, and improvement cycles.
Academic writing with peer review simulation and revision processes.
Creative tasks where multiple perspectives and refinements add value.
Quality assurance scenarios requiring systematic improvement until standards are met.
Complex problem-solving where initial attempts need debugging and enhancement.

References & Further Reading

Deepen your understanding with these curated resources

Academic Papers

Self-Refine: Iterative Refinement with Self-Feedback (Madaan et al., 2023)

Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)

Training Language Models to Follow Instructions with Human Feedback (Ouyang et al., 2022)

Implementation Guides

LangChain Self-Critique and Refinement Chains

OpenAI Iterative Prompting Best Practices

Anthropic Constitutional AI Implementation

Tools & Libraries

LangChain feedback and critique chain implementations

OpenAI function calling for structured evaluation

Custom evaluation frameworks and rubric systems

Community & Discussions

LangChain Community - Feedback loop patterns

r/MachineLearning - Self-refinement discussions

OpenAI Community - Iterative improvement strategies

Loading technique guide…

Patterns

closed

Design Patterns & Techniques

Loading...

Core Mechanism

🔄

Flow

Looped iterations

🧪

Evaluation

Rubrics & scoring

🎯

Convergence

Stop when target met

🧭

Control

Quality gates

Workflow / Steps

Define quality criteria and evaluation rubrics with specific, measurable thresholds.
Generate initial output using appropriate prompting and generation strategies.
Evaluate output against criteria using rules, rubrics, or model-as-judge approach.
Collect specific, actionable feedback focusing on areas needing improvement.
Revise output based on feedback, targeting identified weaknesses and gaps.
Repeat evaluation-revision loop until quality threshold met or max iterations reached.
Finalize output with quality score and iteration history for transparency.

Best Practices

Use a lightweight evaluator where possible; reserve strongest model for final acceptance.

Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.

Set max iterations and early-stop on small deltas to prevent infinite loops and control costs.

Design structured feedback formats to guide focused revisions rather than general critiques.

Implement confidence thresholds to determine when additional iterations provide diminishing returns.

Cache evaluation results and reuse across similar tasks to improve efficiency.

Monitor convergence patterns to optimize evaluation criteria and stopping conditions.

When NOT to Use

Simple tasks where the first attempt typically meets quality requirements.
Real-time applications with strict latency constraints that cannot afford iteration cycles.
Tasks lacking clear evaluation criteria or measurable quality thresholds.
Resource-constrained environments where multiple generation rounds are prohibitive.
Cases where human evaluation is required and automated feedback is insufficient.

Common Pitfalls

Vague or inconsistent evaluation criteria leading to poor feedback quality.
Infinite loops when stop conditions are poorly calibrated or missing.
Excessive iteration costs without proportional quality improvements.
Feedback that is too generic to guide effective revisions.
Over-reliance on automated evaluation without human validation of final outputs.
Poor state management causing loss of important context between iterations.

Key Features

Iterative generate-evaluate-improve loop with quality convergence tracking

Flexible evaluation strategies: rules, rubrics, or model-as-judge approaches

Structured feedback collection with actionable improvement guidance

Configurable stopping conditions with max iterations and convergence thresholds

History management with context preservation and truncation strategies

Quality scoring and transparency with iteration tracking and rationale logging

KPIs / Success Metrics

Quality gain per iteration: Δ score per iteration until convergence.
Convergence iterations: Average iterations to reach target or stop.
Judge consistency: Inter-rater or self-consistency of evaluator.
Target attainment rate: Share of runs achieving target threshold.
Cost per improvement point: Tokens per +1 score or per acceptance.
Early stop rate: Runs halted due to small deltas or max-iter cap.

Token / Resource Usage

Total cost ≈ Σ over iterations of (generator tokens + evaluator tokens + revision tokens). Expect diminishing returns; optimize for early convergence.
Use a lightweight evaluator where possible; reserve strongest model for final acceptance.
Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.
Set max iterations and early-stop on small deltas to prevent cost explosion.
Cache evaluation results and feedback patterns to reduce redundant processing.
Monitor cost-per-quality-point to optimize iteration strategies and stopping thresholds.

Best Use Cases

Content creation requiring iterative refinement for quality and style.
Code generation with compilation, testing, and improvement cycles.
Academic writing with peer review simulation and revision processes.
Creative tasks where multiple perspectives and refinements add value.
Quality assurance scenarios requiring systematic improvement until standards are met.
Complex problem-solving where initial attempts need debugging and enhancement.

References & Further Reading

Deepen your understanding with these curated resources

Academic Papers

Self-Refine: Iterative Refinement with Self-Feedback (Madaan et al., 2023)

Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)

Training Language Models to Follow Instructions with Human Feedback (Ouyang et al., 2022)

Implementation Guides

LangChain Self-Critique and Refinement Chains

OpenAI Iterative Prompting Best Practices

Anthropic Constitutional AI Implementation

Tools & Libraries

LangChain feedback and critique chain implementations

OpenAI function calling for structured evaluation

Custom evaluation frameworks and rubric systems

Community & Discussions

LangChain Community - Feedback loop patterns

r/MachineLearning - Self-refinement discussions

OpenAI Community - Iterative improvement strategies

Core Mechanism

🔄

Flow

Looped iterations

🧪

Evaluation

Rubrics & scoring

🎯

Convergence

Stop when target met

🧭

Control

Quality gates

Workflow / Steps

Define quality criteria and evaluation rubrics with specific, measurable thresholds.
Generate initial output using appropriate prompting and generation strategies.
Evaluate output against criteria using rules, rubrics, or model-as-judge approach.
Collect specific, actionable feedback focusing on areas needing improvement.
Revise output based on feedback, targeting identified weaknesses and gaps.
Repeat evaluation-revision loop until quality threshold met or max iterations reached.
Finalize output with quality score and iteration history for transparency.

Best Practices

Use a lightweight evaluator where possible; reserve strongest model for final acceptance.

Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.

Set max iterations and early-stop on small deltas to prevent infinite loops and control costs.

Design structured feedback formats to guide focused revisions rather than general critiques.

Implement confidence thresholds to determine when additional iterations provide diminishing returns.

Cache evaluation results and reuse across similar tasks to improve efficiency.

Monitor convergence patterns to optimize evaluation criteria and stopping conditions.

When NOT to Use

Simple tasks where the first attempt typically meets quality requirements.
Real-time applications with strict latency constraints that cannot afford iteration cycles.
Tasks lacking clear evaluation criteria or measurable quality thresholds.
Resource-constrained environments where multiple generation rounds are prohibitive.
Cases where human evaluation is required and automated feedback is insufficient.

Common Pitfalls

Vague or inconsistent evaluation criteria leading to poor feedback quality.
Infinite loops when stop conditions are poorly calibrated or missing.
Excessive iteration costs without proportional quality improvements.
Feedback that is too generic to guide effective revisions.
Over-reliance on automated evaluation without human validation of final outputs.
Poor state management causing loss of important context between iterations.

Key Features

Iterative generate-evaluate-improve loop with quality convergence tracking

Flexible evaluation strategies: rules, rubrics, or model-as-judge approaches

Structured feedback collection with actionable improvement guidance

Configurable stopping conditions with max iterations and convergence thresholds

History management with context preservation and truncation strategies

Quality scoring and transparency with iteration tracking and rationale logging

KPIs / Success Metrics

Quality gain per iteration: Δ score per iteration until convergence.
Convergence iterations: Average iterations to reach target or stop.
Judge consistency: Inter-rater or self-consistency of evaluator.
Target attainment rate: Share of runs achieving target threshold.
Cost per improvement point: Tokens per +1 score or per acceptance.
Early stop rate: Runs halted due to small deltas or max-iter cap.

Token / Resource Usage

Total cost ≈ Σ over iterations of (generator tokens + evaluator tokens + revision tokens). Expect diminishing returns; optimize for early convergence.
Use a lightweight evaluator where possible; reserve strongest model for final acceptance.
Truncate history aggressively: pass distilled state (facts, constraints, last diff) instead of full transcripts.
Set max iterations and early-stop on small deltas to prevent cost explosion.
Cache evaluation results and feedback patterns to reduce redundant processing.
Monitor cost-per-quality-point to optimize iteration strategies and stopping thresholds.

Best Use Cases

Content creation requiring iterative refinement for quality and style.
Code generation with compilation, testing, and improvement cycles.
Academic writing with peer review simulation and revision processes.
Creative tasks where multiple perspectives and refinements add value.
Quality assurance scenarios requiring systematic improvement until standards are met.
Complex problem-solving where initial attempts need debugging and enhancement.

References & Further Reading

Deepen your understanding with these curated resources

Academic Papers

Self-Refine: Iterative Refinement with Self-Feedback (Madaan et al., 2023)

Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)

Training Language Models to Follow Instructions with Human Feedback (Ouyang et al., 2022)

Implementation Guides

LangChain Self-Critique and Refinement Chains

OpenAI Iterative Prompting Best Practices

Anthropic Constitutional AI Implementation

Tools & Libraries

LangChain feedback and critique chain implementations

OpenAI function calling for structured evaluation

Custom evaluation frameworks and rubric systems

Community & Discussions

LangChain Community - Feedback loop patterns

r/MachineLearning - Self-refinement discussions

OpenAI Community - Iterative improvement strategies