Agentic Systems • Developer Skills • Practical Patterns

Agentic System Design: The Next Mandatory Skill for Developers

Last updated: 7 October 2025

Agentic systems extend beyond prompt-in, text-out. They maintain goals, deliberate across steps, call tools and APIs, check results, and iterate. As IBM’s Dr. Maryam Ashoori notes, modern AI agents take actions on your behalf and should provide transparency into their reasoning steps and tool interactions.^[8]

1) Foundations: What Agentic System Design Actually Means

1.1 Core Properties

• Autonomy: Operates with minimal human oversight, within explicit guardrails.
• Goal Direction: Keeps a clear notion of “done” and how to measure it.
• Tool-Centric Action: Uses APIs, retrieval, file systems, schedulers, and webhooks to affect the world.
• Memory: Stores and recalls facts, decisions, and context—episodic + semantic.
• Transparency: Logs plans, tools, and outcomes for explainability and audit.
• Safety: Policy checks, permissions, fallbacks, and human-in-the-loop at risk boundaries.

1.2 Glossary

• Planner: component that decomposes goals into steps.
• Executor: component that calls tools/APIs to complete steps.
• Critic/Verifier: component that inspects outputs against specs/policies.
• Memory: vector/graph stores + summaries used to ground decisions.
• Arbiter: component that resolves conflicts between agents or paths.

1.3 Capability Map

2) Pattern Catalog: Reliable Ways to Orchestrate Agents

2.1 Planner → Executor → Critic (PEC)

A simple, dependable backbone: a planner decomposes tasks, an executor calls tools, and a critic checks results against specs and policies before progressing.

2.2 Master–Worker

A coordinator assigns sub-tasks to specialist workers (retrieval, analysis, writing, QA). Good for pipelines with clear stages and SLAs.

2.3 Peer-to-Peer

Agents negotiate roles and exchange partial solutions; useful in exploratory or creative tasks where a single plan is hard to define upfront.

2.4 Hierarchical Arbitration

A tree of decision makers escalates when the lower level can’t prove a safe or correct result. Attach human hand-offs at the top tiers.

2.5 Reflex + Deliberate Hybrid

Fast heuristic responses for simple cases, reflective planning for complex situations—minimizes latency without sacrificing reliability.

3) Memory & Context: What to Remember, What to Forget

3.1 Memory Types

• Episodic: what happened this run (tools, results, exceptions).
• Semantic: facts and summaries reusable across runs.
• Vector: embeddings for search/retrieval.
• Graph: entities and relationships (customers → subscriptions → invoices).

3.2 Retention Rules

• Keep just enough: summarize old conversations; store trace IDs, not raw PII.
• Define TTLs and data owners; align with privacy law and client contracts.
• Make memory writes deliberate—attach reasons and provenance.

3.3 Retrieval Playbook

1. Begin with the minimum viable context (MVC) for the step.
2. Use tool selection prompts that request evidence, not just answers.
3. Cache grounded facts with source pointers for re-use.

4) Tooling & Frameworks You’ll Meet

• Microsoft AutoGen — collaborative multi-agent interactions and conversation orchestration.
• LangGraph — graph-structured stateful flows and branching control.
• Semantic Kernel — connect models to app logic with planners and skills.
• CrewAI — role-based teams, tool use, and task delegation.

5) Evaluation: From “Seems Smart” to “Proves It”

5.1 Offline Evaluation

• Golden Sets: Curate tasks with known-good outputs and policy assertions.
• Counterfactuals: Test robustness to subtly altered inputs.
• Spec-Based Checks: Assertions like “table must include total with two-decimal currency.”
• Safety Tests: Injection strings, role-confusion prompts, adversarial data.

5.2 Live Evaluation

• Shadow Mode: Run the agent without taking action; compare against human.
• Canary Releases: Gradually lift traffic share.
• A/B Tests: Evaluate policy changes with business KPIs.

5.3 Metrics That Matter

• Task completion rate, tool success rate, escalation rate.
• Hallucination rate (measured via critic assertions and human audits).
• Cost and latency budgets per task; SLOs per outcome.
• Safety events and near misses; time to detect and contain.

// Tiny example: spec assertion pseudo-code
assert(hasTable(output));
assert(sumColumn("Amount") === sourceTruthTotal);
assert(noPII(output));
assert(policy_passed === true);

6) Reliability: Make It Work on Tuesday at 3 A.M.

6.1 Failure-First Design

1. Explicit timeouts, retries, circuit breakers, and dead-letter queues.
2. Idempotent tool calls; safe rollback; compensating actions.
3. Self-check prompts for high-risk steps; quorum checks for critical outputs.

6.2 Observability

• Trace IDs across plan → tools → outputs; link to logs and metrics.
• Replay harness: re-run a task with a different policy/model for comparison.
• Cost accounting per step and per outcome (to guide optimizations).

7) Security, Safety & Governance

Provenance & Human Contribution. For regulated workflows, add signed evidence of the human create→edit→review→approve chain and cryptographically link it to outputs. This improves auditability and trust in agentic pipelines.

8) Org Design & ROI: From Demo to Durable Value

8.1 Crawl → Walk → Run

1. Crawl: One outcome-agent, a few tools, shadow mode. Define KPIs.
2. Walk: Canary traffic, human approvals on risk steps, replay and eval harness.
3. Run: SLOs + policy automation, provenance, cost guardrails, incident runbooks.

8.2 Buy vs. Build

• Buy: time to value, compliance support, vendor roadmap.
• Build: deep integration, custom logic, cost control at scale.
• Hybrid: bought core + custom evaluators and domain tools.

8.3 ROI Patterns

• Reduce time-to-approval; lower audit hours per sample; raise automated policy pass-rate.
• Deflect tickets; shorten resolution time; increase first-contact resolution.
• Increase throughput for content/analysis while keeping quality above threshold.

9) Case Snapshots

• Klarna: AI assistant handles ~two-thirds of customer chats (≈700 FTE eq.), with faster resolution times.^[5]
• Morgan Stanley: GPT-4 knowledge assistant supports advisors with contextual answers and source links.^[6]

10) Anti-Patterns to Avoid

• Tool Sprawl: dozens of tools with no permission model → security incidents.
• Prompt-Only “Agents”: no state, no metrics, no SLOs → fragile behavior.
• Invisible Memory: silent writes; no retention rules → data surprises.
• Over-Orchestration: complex multi-agent webs without evidence of need.
• No Replay: can’t reproduce incidents; can’t regress-test policy changes.

11) Your First Week: A Concrete Plan

Days 1–2: Environment

• Python 3.10+, Docker, Git; model SDK; tracing (OTel-compatible); secrets manager.
• LangGraph / CrewAI; a vector store; simple policy store.

Days 3–5: Outcome-Agent (Weekly KPI Report)

• Retrieve data → draft → self-check → human approve → send.
• Store artifacts and links; log plan, tools, costs, and latency.
• Add policy assertions (no PII in emails, totals match source).

Days 6–7: Evaluate, Harden, Document

• Golden tests; adversarial inputs; timeouts and retries.
• Replay two runs with different policies; compare metrics.
• Write a runbook and an “on-call at 3 A.M.” playbook.

12) Operations Runbook (Copy-Paste)

Pre-flight

• ✅ Tool scopes defined; secrets set; rate limits configured.
• ✅ Guardrails (PII redaction, jailbreak checks) active.
• ✅ Observability connected (trace IDs, spans, cost tracking).

During

• 👀 Monitor KPIs: task completion rate, tool success, latency budget.
• 🧯 On error: retry policy → fallback path → human-approve.
• 🧪 Sample outputs against assertions; record near-misses.

Post

• 📝 Append run to audit log; capture incident learnings.
• 🔁 Update prompts, policies, and tests based on failures.
• 💰 Review cost/latency vs. targets; tune caching/batching.

13) Developer Toolkit

Books Affiliate

Affiliate links: Amazon purchases may earn TechLifeFuture a small commission at no extra cost to you.

• Generative AI with LangChain — view on Amazon.
• Docker Deep Dive — view on Amazon.

Courses & Paths

14) FAQ

Is agentic design just “better prompting”?
No—prompting is one ingredient. Agentic design adds planning, tools, memory, arbitration, evaluation, and governance.

Do I need multi-agent sets from day one?
Start with a single outcome-agent plus a few tools. Add specialists when bottlenecks become clear.

How do I measure “done”?
Define outcome KPIs (e.g., “weekly report sent with 0 policy violations”) and compare against a human baseline.

What about long-horizon autonomy claims?
Treat them as research signals. METR’s time-horizon work is useful, but avoid hard forecasts—measure your system directly.^[4]

What’s the fastest way to get value?
Target repetitive, policy-bound workflows that already have a clear “definition of done.”

15) Appendices & Templates

A. Outcome Spec Template

Outcome: "Weekly KPI Report emailed to exec list by 09:00 Mon (AEST)"
Inputs: CRM API, Billing DB, Analytics export (last 7 days)
Constraints: No PII in email body; totals must reconcile to sources
KPIs: Delivered on time; zero policy violations; variance <= 0.5%
SLOs: p95 latency < 120s; p99 cost < $0.75/run
Approval: Human approver on first 5 runs; auto-approve if 5/5 pass

B. Policy Assertions (Spec-Based Checks)

assert(output.contains("Summary Table"))
assert(sum(output.column("Revenue")) == source.billing.total_last_7_days)
assert(noPII(output))
assert(links.all_valid)
assert(policy_violations == 0)

C. Incident Runbook (Excerpt)

Trigger: KPI report missing by 09:00
1) Check scheduler logs (job fired?)
2) Replay last successful run (compare tool latency)
3) Inspect dead-letter queue (payloads, error types)
4) If API rate limit: backoff + token bucket adjust
5) If policy failure: open human approval, annotate cause
6) Postmortem within 24h; add regression test

D. Change Control Checklist

• ☑ Update eval set; run offline suite; record metrics deltas.
• ☑ Canary 10% traffic; watch safety events and escalations.
• ☑ Update runbook and versioned policy docs.
• ☑ Communicate change window to stakeholders.

E. Sample Policy JSON (Minimal)

{
  "allow_tools": ["crm.read", "billing.read", "email.send"],
  "deny_tools": ["email.bulk_send"],
  "max_cost_usd": 0.75,
  "pii_scan": true,
  "approval_required": ["email.send"]
}

F. Minimal LangGraph-Style Pseudocode

start -> retrieve_data -> draft_report -> self_check -> human_approve? -> send_email -> end
                        \-> fail -> incident_log -> end

Disclosures And Editorial Standards

References

1. McKinsey (2024). The State of AI in 2024 — ~65% of organizations report gen-AI use in at least one function. mckinsey.com
2. Stanford HAI (2025). AI Index Report — 2024 private AI investment in the U.S. (~$109.1B) and global gen-AI investment (~$33.9B). aiindex.stanford.edu
3. Boston Consulting Group (Oct 2024). AI Adoption in 2024: 74% of Companies Struggle to Achieve and Scale Value (press release). bcg.com
4. METR (2025). Measuring Model Time Horizon — framing long-task completion ability; avoid over-specific forecasts. metr.org
5. Klarna (2024/25). AI assistant performance (≈two-thirds of chats; ~700 FTE equivalent). prnewswire.com
6. Morgan Stanley (Sept 2023). Wealth Management launches GPT-4-powered assistant. morganstanley.com
7. Educative. Agentic System Design. educative.io/courses/agentic-ai-systems
8. IBM Think / Watsonx (2025). Dr. Maryam Ashoori on agent transparency and actions. ibm.com
9. Educative. Build AI Agents & Multi-Agent Systems with CrewAI. educative.io/courses/build-ai-agents-and-multi-agent-systems-with-crewai
10. Educative. Unleash the Power of LLMs Using LangChain. educative.io/courses/langchain-llm
11. Educative. Fundamentals of RAG with LangChain. educative.io/courses/rag-llm
12. Educative. Generative AI Essentials. educative.io/courses/generative-ai-essentials
13. Educative. Skill Path: Become an Agentic AI Expert. educative.io/path/become-an-agentic-ai-expert
14. OECD. AI Principles. oecd.ai
15. EU Council (2024–2025). AI Act adoption timeline. consilium.europa.eu

Tags: Agentic Systems System Design AI Engineering LangGraph AutoGen CrewAI