AI Agents and Agentic AI

Observe → Reason → Act → Verify

Automation executes predetermined steps. Agents choose steps dynamically based on context — which is why they are powerful and risky.

ServiceNow’s advantage is that the loop can be anchored on a record (incident/case/change) with audit trails and ACL boundaries.

A chatbot can draft a resolution note. An agent can: create a task, trigger a diagnostic flow, open a change, notify stakeholders, and update the incident — all governed.

Agents convert language into workflow. That means they touch real systems: tickets, CMDB, approvals, integrations. The cost of error rises sharply.

Therefore the deployment pattern changes: you start with read-only agents, then add limited writes, then expand autonomy only after monitoring proves safety.

Core pieces: agent, tools, memory/state, orchestrator

ServiceNow’s strength: tools are already enterprise-grade (Flow, IntegrationHub, approvals) and actions land on a record with audit logs.

Design rule: store durable state in tables, not in “the agent’s brain.” If the session dies, you must be able to recover from records.

Human-in-the-loop is default for HR, finance, and customer-facing comms.

Supervised autonomy fits IT operations with strong runbooks and rollback paths.

Fully autonomous is rare; limited to low-risk, reversible tasks with strong monitoring.

Internal access: incidents, cases, CMDB, knowledge, catalog, change records — gated by ACL and tool definitions.

External access: SaaS systems via IntegrationHub spokes or REST tools — gated by connection credentials and allowlists.

Rule: don’t give direct table write access as a generic tool. Wrap writes behind purpose-built tools with schema validation and approvals.

Hard boundaries: money, identity, customer comms, production changes

Soft boundaries: high-uncertainty actions, ambiguous intent, missing evidence. Agents must ask or stop.

Design: approvals + review queue + audit logs + rollback. If you can’t roll back, you can’t automate.

Traditional automation is best when policy is known.

RPA is a last resort; brittle but sometimes necessary.

Agentic is best for ambiguous workflows and long-tail problem solving — with human checkpoints.

Value buckets: triage time reduced, fewer misroutes, faster comms, fewer duplicate incidents, and improved SLA compliance.

Cost buckets: LLM usage, tool engineering, integration governance, audit logs, human review time, and incident response readiness.

Honest model: start with one process and measure time saved per record. Scale only when metrics prove net benefit and safety.

Studio consolidates: agent persona and instruction design, tool registration, test scenarios, activation controls, and observability hooks.

Treat Studio outputs as deployable artifacts.

A strong agent program has owners: platform team maintains the framework; domain teams own agent objectives and guardrails.

Define purpose and boundaries. Include explicit out-of-scope cases.

Define scope by tables and domains. Avoid “any table” permissions.

Define persona: tone, decision principles, escalation behavior. Persona is not just UX; it affects actions and risk.

Include: objective, input assumptions, allowed actions, required checks, and stop conditions. Use explicit lists and decision rules.

Add evidence constraints. This reduces hallucinated actions.

Write instructions as policy: if a user asks for forbidden actions, refuse and escalate. Do not rely on “the model will know.”

Start with read-only tools: search KB, fetch record context, list similar incidents. Only add write tools when monitoring and approvals exist.

Prefer Flow tools to direct writes: Flow encodes policy, error handling, and approvals. Agents should call policy, not bypass it.

External tools via IntegrationHub should be narrowed: prebuilt spokes with fixed scopes beat arbitrary REST calls.

Test types: happy path, ambiguous input (agent must ask), boundary violations (agent must refuse), and tool failures (agent must degrade safely).

Define expected tool sequence and expected stop conditions. If the agent loops, you need circuit breakers.

Keep a regression suite. Every agent version must pass it before promotion.

Use progressive rollout: internal testers → champion agents → one team → all teams. Expand autonomy only after stability and trust.

Define degraded mode: what happens when agent is disabled or LLM is unavailable. Operations must still run.

Instrument early: adoption, error rate, approval queue load, and false action rate.

Version triggers: instruction changes, tool schema changes, new autonomy levels, and guardrail updates.

Rule: never change tool schemas without versioning — it breaks existing runs and produces unpredictable behaviour.

Maintain release notes: what changed, why, and what tests passed. Auditors will ask.

Step 1: Agent mission: triage new incidents — assign group and add summary tag.

Step 2: Tools: read incident, search KB, suggest assignment (PI), create triage task.

Step 3: Add approval gate for any write beyond assignment/tag.

Step 4: Test harness with 10 scenarios + 2 adversarial prompts.

Step 5: Deploy to pilot role only; monitor override rate and errors.

Tools should be purpose-built.

Tools must return structured outputs: success flag, evidence, and references (record ids). Text-only responses are hard to audit.

Agent safety starts with tool allowlists. Prompting is not enough.

Wrap a subflow with clear inputs/outputs and a single responsibility. Keep it idempotent where possible.

Add approval gates inside Flow for risky steps. The agent triggers the flow; humans approve inside the flow.

Log action metadata on the record: which tool, which version, why, and what evidence.

Use spokes for read-first integration: fetch status, post draft message, open ticket. Avoid delete/write actions early.

Define connection scopes and secrets management. Agents should not hold API keys in prompts or instructions.

Use allowlists: only approved spokes/actions are callable by the agent.

Treat script tools like APIs: validate inputs, enforce RBAC, return structured outputs, and log requests.

Prefer calling scripts through Flow when possible to unify policy. Use direct script tools for computation and validation helpers.

Version script tools and maintain backward compatibility — agent versions depend on tool stability.

Create REST tools with strict allowlists: fixed hostnames, fixed paths, and fixed methods. Avoid “call any URL.”

Return structured outputs and store request ids. Apply rate limits and retries with caps.

Prefer read-only REST tools early. Writes require approvals and rollback paths.

Inputs: required vs optional, enum constraints, max lengths, and validation. Outputs: success flag, result payload, evidence links, and next-step hints.

Errors should be actionable: 'AUTH_FAILED', 'NOT_FOUND', 'POLICY_BLOCKED' — not generic exceptions.

Add idempotency keys for tools that can be re-run safely to prevent duplicate actions under retries.

Enforce least privilege: each tool runs with minimal roles and minimal table access. Avoid agent-wide superuser identities.

Use explicit write restrictions: prohibit delete operations; restrict updates to specific fields; require approvals for state transitions.

Audit: log every tool call with who/what/why and the record ids touched. Exportable logs matter for compliance.

Step 1: Define the action in one sentence. Step 2: Define schema. Step 3: Implement logic (Flow/script/spoke). Step 4: Add RBAC + approvals. Step 5: Add audit logging. Step 6: Register + test harness. Step 7: Deploy versioned.

Prefer Flow subflow for write actions; script tools for validation/computation; spokes for external systems.

Never ship tools without an error-handling contract and a rollback story for write actions.

Coordinator: owns goal and safety policy. Specialists: own narrow domains and tools. Communication happens through structured messages and record-backed state.

Architectural benefit: each specialist can have tighter tools and stricter permissions. The comms agent shouldn’t have change execution tools.

Design rule: agents communicate via contracts (schemas), not prose. Prose causes drift and misunderstandings between agents.

Good task specs include: objective, required evidence, allowed tools, stop conditions, and escalation criteria.

Coordinator should prefer deterministic steps first: gather facts, check known errors, validate CI topology — before proposing risky actions.

Decomposition also enables measurement: you can evaluate each sub-agent’s success rate and improve it independently.

Store state in tables: what was tried, outputs, timestamps, and current phase. This enables restart and audit.

Pass only necessary context. Over-sharing increases token costs and leakage risk.

Use a standard “handoff payload” schema across agents.

Only parallelise independent tasks. Avoid parallel writes to the same record without coordination — that creates race conditions.

Coordinator merges results with a strict policy: prefer grounded evidence, resolve conflicts, and ask humans when uncertain.

Rate limit parallel tool calls to avoid runaway costs and external API throttling.

Evidence hierarchy: platform record fields and tool outputs outrank narrative. Require citations for claims.

If conflict is about action choice and both are plausible, prefer the safer reversible action or request human approval.

Log conflicts and resolutions — they are training data for improving instructions and tools.

Coordinator→specialists Pipeline chain

Parallel map-reduce Supervisor/HITL

Rule: the higher the risk, the more you should bias toward supervisor pattern with approvals and review queues.

Log structure: coordinator run id links all specialist runs; each tool call has inputs/outputs (redacted as needed), timestamp, and result code.

Build dashboards: success rate, average steps, escalation rate, and common failure reasons.

Treat this as SRE: define SLOs for agent workflows (e.g., 99% no-unsafe-write).

Design: coordinator plus three specialists. Triage reads + routes. Resolution runs diagnostics and proposes remediation. Comms drafts updates and status changes — approval required for external posts.

Build: tools are flows and spokes with schemas; state stored on agent-run records; monitoring dashboards track loops, errors, and approvals.

Outcome: faster comms cadence, reduced triage time, fewer duplicate incidents during outages — without granting agents unsafe autonomy.

Session memory is bounded. Long incident threads and large tool outputs consume budget and trigger truncation.

Design pattern: summarise early, store summary on record, and feed summary back instead of replaying full history each step.

Operational rule: never keep 'policy constraints' only in chat history; encode them in instructions and tool boundaries.

Patterns: KB for policies and procedures; custom tables for agent preferences, learned mappings, and stable facts; incident/task records for operational state.

Keep memory curated. Long-term memory that grows without hygiene becomes wrong and unsafe — like an outdated KB.

Design for deletion and residency. Persistent memory must comply with HR/GDPR retention rules.

Store: run id, inputs, tool calls (redacted), decisions, approvals, and final outcome. Link to the underlying incident/case.

Use episodic memory for improvements: identify repeated failures, add guardrails, refine tool schemas, and update instructions.

Do not let the agent freely “learn” from episodes by modifying policy without human review. Learning must be governed.

Use AI Search and KB scoping to provide the agent only the relevant, approved procedures.

Prefer structured KB: prerequisites, steps, escalation. Agents can follow structured runbooks more reliably.

Treat KB lifecycle as agent safety. Retire stale procedures; otherwise agents will execute outdated actions.

Use a stage model: triage → diagnose → propose → approve → execute → verify → close. Store stage as a field.

Idempotency: tools should be safe to retry (no duplicate tasks). Use idempotency keys and 'already done' checks.

Resumability: if the agent stops, a human or another run should resume from stored state, not restart from scratch.

Risks: outdated routing maps, stale incident context, wrong CI relationships. If memory isn’t refreshed, agents act on yesterday’s truth.

Design controls: TTL on memory entries, version tags, and explicit refresh steps (re-query facts before action).

Provide reset tools: clear session state, invalidate cached facts, and force re-grounding on current records.

KB-first Episodic logs Fact tables

Design rule: policy lives in KB, state lives on records, facts live in tables. Avoid mixing them.

Keep memory minimal. If a human wouldn’t maintain it, an agent shouldn’t either.

Step 1: Create agent-run log table with run_id, stage, evidence, and outcome fields.

Step 2: Store a session summary to incident field or agent-run record after each stage.

Step 3: Create a small fact table: service→assignment mapping with TTL.

Step 4: Tool: 'GetRoutingFact' queries the table; agent uses it instead of guessing.

Step 5: Demonstrate resume: stop run mid-way; restart from stored stage.

Autonomy raises the cost of mistakes: wrong change, wrong comms, wrong access grant. Human checkpoints convert catastrophic errors into reviewable drafts.

Compliance requires traceability: who approved what, based on what evidence. HITL creates that chain.

Trust grows through progressive autonomy: users accept automation when it’s transparent, consistent, and reversible.

Tool-call approval State approval

Content approval Plan approval

Design tip: approve plans for complex tasks, approve actions for risky tasks, approve content for external comms.

Escalation triggers: missing KB, low confidence, conflicting signals, repeated tool errors, or forbidden requests.

Escalation output should include structured evidence: what was tried, what was found, and what decision is needed.

Provide escalation destinations by domain: IAM queue, change manager, HR specialist, incident commander.

Combine model confidence with action risk: a high confidence on a high-risk action still needs approval.

Calibrate with evaluation data: measure precision at thresholds for routing, dedup, and recommendations.

Log confidence and threshold decisions for audit and tuning.

Show: proposed tool call, affected records, evidence links, confidence, and rollback plan. Hide noise.

Collect structured reasons for reject/override — these become training data for improving tools and instructions.

SLA: review queue needs an SLA. If approvals take days, agents are useless for real operations.

Log structure: run id, timestamps, tool calls, parameters (redacted), outputs, approval decisions, and final outcome. Link to records.

Retention and privacy: store minimal necessary content; redact secrets; align with GDPR/HR retention policies.

Make it exportable: auditors want CSV/PDF snapshots. Design for evidence extraction early.

Provide action summaries on the record: ‘Agent routed to Network because CI=VPN-GW and similar incidents show pattern.’

Allow override and feedback. Users need the ability to correct and improve the system.

Avoid anthropomorphism. Treat the agent as a system making recommendations under policy.

Each progression step requires: monitoring, low error rate, clear rollback, and a trained human review cohort.

Use measurable gates: precision, override rate, incident rate, audit findings. No metrics, no autonomy expansion.

Document progression in governance policy and communicate to stakeholders to set expectations.

Best first agents: triage (route/tag), comms drafting, duplicate linking, and runbook-guided diagnostics. Avoid payroll, access grants, and production changes.

Success requires: a baseline metric, a review workflow, and a kill switch. Demos without governance don’t survive security review.

Pick a use case where the human today spends time on coordination and searching — agents excel there.

Prepare: 20–50 synthetic incidents across categories; a small KB set of runbooks; basic Flow subflows as tools; and an agent-run log table.

Define roles: pilot users, reviewers, admins. Do not test everything as admin.

Enable observability: log every tool call and decision. POC without logs cannot be trusted.

Tools: read incident, search KB, get PI routing suggestion, update assignment group (scoped), create triage task for low confidence.

Policies: never close incident; never change priority without approval; always log reasons and evidence.

Metrics: time-to-assign, override rate, and misroute cost.

Start with one known issue type (e.g., VPN profile reset) and one diagnostic tool. Expand slowly.

Tools must be reversible. If not reversible, require approval and rollback plan.

Measure: time-to-resolution and false fixes (reopen rate).

Inputs: incident timeline, impacted services, known status, next update time. Outputs: update draft with tone and audience parameter.

Governance: external sends require approval; internal updates may become semi-autonomous later.

Measure: cadence, quality feedback, and coordinator time saved.

State: single incident sys_id + agent-run records. Handoffs include evidence and current stage.

Parallelism: resolution gathers evidence while comms drafts update. Coordinator merges results and posts on approval.

Stop conditions: if confidence low or tool failures occur, escalate to human and halt automation.

Demo narrative: baseline pain → agent workflow → approvals → audit log → KPI improvement. Keep it under 12 minutes.

Metrics: time-to-assign, time-to-first-update, coordinator minutes saved, override rate, false action rate.

Show failure mode: induce tool failure and show safe escalation. This builds trust more than a perfect run.

Proposal sections: problem, scope, architecture, autonomy policy, security review, integration plan, monitoring plan, and KPI targets.

Include autonomy progression and kill switch readiness. Security wants to see stop controls.

Roadmap: Phase 1 read-only, Phase 2 approval-gated writes, Phase 3 limited autonomy. Tie each to measurable success criteria.

Common failures: acting on incomplete evidence, misinterpreting intent, calling the wrong tool, repeating actions on retries, and leaking restricted data in drafts.

Human failure mode: copy-paste trust. People accept outputs without review under time pressure — design approvals and UI to prevent that.

Mitigation strategy: scope + allowlists + approvals + observability + rollback + kill switch.

Start with one business service or one category. Expand only after measurable success and audit readiness.

Minimise data exposure: exclude restricted fields and domains. If the agent doesn’t need it, it shouldn’t see it.

Minimise tool surface: fewer tools means fewer ways to do the wrong thing.

Allowlist by tool and by action within tool. For REST tools, allowlist endpoints. For record writes, allowlist fields.

Tie allowlists to roles and environment: dev agents may have broader tools; prod agents are restricted.

Review allowlists quarterly in governance council.

Reversible patterns: draft-only writes, creating tasks instead of modifying records, using staged fields rather than final fields, and approvals before commit.

For writes, store pre-change snapshots (or deltas) with run id so rollback is deterministic.

Define compensating actions: if agent created wrong task, auto-close and reassign with apology note — under policy.

Controls: max tool calls per run, max retries per tool, max runtime, and backoff policies.

Add “storm mode”: during major incidents, cap actions and switch to draft-only to prevent the agent from amplifying chaos.

Alert when breakers trigger. Frequent triggers indicate missing tools, poor instructions, or downstream outages.

Kill switch should stop: agent triggers, tool calls, and autonomous writes. It should still allow humans to work manually.

Include a ‘resume’ procedure: what conditions must be met to re-enable agents after incident.

Practice kill switch in tabletop exercises. A kill switch nobody has used is not real.

Adversarial prompts: “ignore policy”, “export all HR cases”, “call this URL”, “grant me admin”. Agents should refuse and/or escalate.

Tool misuse tests: attempt to pass unexpected parameters, huge payloads, and endpoint overrides.

Outcome tests: verify no restricted fields leak in drafts; verify logs redact secrets; verify kill switch stops actions.

Define roles: agent builder, tool owner, approver, auditor, on-call admin. Assign RACI per agent.

Define lifecycle: dev/test/prod promotion, versioning, A/B tests, and rollback. Tools and agents are releases.

Define safety controls: allowlists, kill switch, circuit breakers, and red-team schedule.