AO Agentic Operations
MODULE 03 / 10
All modules ↑
BUILD · TARGET L3 · 5 days
03

Deterministic orchestration

Operate a read-only diagnostic state machine that retries safely, records state and pauses for approval.

Build an inspectable AWS workflow for steps that should not be delegated to open-ended reasoning.

Prerequisites

  • Modules 01–02
  • Step Functions and Lambda at L2
  • IAM least privilege at L2
01 / CONCEPT LESSONS

Build the mental model first.

01

State as evidence

Named states make transitions, retries and failures inspectable.

02

Idempotency

A retry must not duplicate an effect. Use stable execution identifiers and conditional writes.

03

Timeout and retry budgets

Retries consume time and money; distinguish transient from permanent failure.

04

Compensation

When atomic rollback is impossible, record completed effects and explicitly compensate.

05

Approval task tokens

Pause with sufficient context, expiration and an auditable identity.

02 / SYSTEM FLOW

Trace the complete path.

Alert eventValidateCollect evidenceClassifyApproval waitRecommendVerifyClose / escalate

For every arrow, the engineer must name the data, identity, trust boundary, failure mode and evidence produced.

03 / GUIDED LAB

Build it in bounded steps.

AI and documentation are allowed. The learner owns every generated artifact and must preserve the stated safety boundaries.

  1. 01Define the input and output schema.
  2. 02Implement read-only evidence collectors.
  3. 03Add bounded retries and per-state timeouts.
  4. 04Add approval with expiry and escalation.
  5. 05Persist execution correlation and result.
  6. 06Replay the same event and prove no duplicate effect.
04 / INCIDENT SIMULATION

A retry creates duplicate tickets and the workflow never exits.

Mentor injects

  • Unbounded retry on a permanent 403
  • Ticket call lacks idempotency key
  • Approval wait has no expiry

Required evidence

  • Execution history
  • Duplicate side-effect proof
  • Corrected retry classification and idempotency design

Investigation loop

Symptom → impact → hypothesis → evidence → test → root cause → containment → correction → verification.

05 / DESIGN CHALLENGE

Start with a blank page.

Compare Standard and Express workflows for the pilot. Include volume, duration, auditability and cost.

06 / VERIFY & SUBMIT

Verification checklist

Submission package

  • State diagram
  • Workflow definition
  • IAM policy explanation
  • Failure test log
  • Operations runbook

Review questions

  • Which states genuinely need a model?
  • What happens after a worker times out?
  • How is duplicate delivery handled?
  • Where is compensation required?
  • How does an operator stop execution?
07 / MENTOR-ONLY GUIDANCE

Inject a permanent permission failure. The engineer must distinguish it from a transient error rather than increasing retries.

SCORING1 · cannot explain2 · explains with gaps3 · implements and verifies4 · designs and handles trade-offs

Record the achieved L-level only from observed evidence. Course completion and certification do not automatically change the skill matrix.