Task statement 2.2 · Applications and integration

2.2 · Systems Life Cycle

In Shelfwise, the course example, the stock agent went live once but keeps changing: the engineers edit the stock rules, Anthropic releases and retires models, and the Agent SDK ships new versions. The systems life cycle says which checks each change must pass before it reaches staff.

Why it's tested

The tempting view treats launch as the finish line and later changes as small patches. An engineer who swaps the model ID in production because a newer model is cheaper skips the tests that guarded the first release, so the first sign of a changed answer is a complaint from a shop. The edit was one line, but it changed behaviour as much as a release, because a Claude application's behaviour lives in the model and the prompt as well as the code.

Start hereThree terms this section adds

deprecated model
A model that still works but is no longer recommended. Anthropic names a replacement and sets a retirement date.
retired model
A model that is no longer available. Every request to it fails.
staged rollout
Releasing a change to a small share of traffic first, then widening it, with the old version ready for rollback.

KnowledgePhases, and the order they run in

Systems life cycle management splits a system's life into phases, each with its own concerns and checks. ISO/IEC/IEEE 15288, the international standard for system life cycle processes, spans conception, development, production, utilisation, support and retirement. The exam guide uses four verbs for that span: develop, implement, operate and maintain. Development builds and tests the system, implementation puts it into service, operation runs it for its users, and maintenance keeps it working as its surroundings change.

A sequential model, often called waterfall, runs each phase once and in order, so maintenance only patches what was built. An iterative model repeats the loop in small increments, and each change passes back through development and implementation. ISO/IEC/IEEE 15288 prescribes neither, and says its processes can be applied iteratively and concurrently. A Claude application leans iterative, because the model underneath it changes on a schedule the team does not control, so a model replacement is a new release, not a patch.

KnowledgeWhat each phase means for a Claude application

In development, the prompt, the tool definitions and the evals are artefacts, versioned and reviewed like code. An eval is a set of test inputs with expected answers, graded automatically, and Anthropic's guidance is to design evals that mirror the real task distribution, edge cases included. Shelfwise's eval set is 150 recorded stock questions, each with the expected count and feed timestamp or "no record", and it later shows whether a new model or prompt still passes.

Implementation puts a tested version into service. Since its first model change, Shelfwise keeps the model ID and the prompt version in configuration rather than code (2.6), so a release or a rollback changes a setting. A staged rollout sends the new version to part of the traffic first, which limits the damage if the evals missed something. A gradual ramp also helps, because usage that rises too fast can trigger 429 errors from the API's acceleration limits.

Operation runs the service and notices problems before its users do. The Usage and Cost Admin API reports the organisation's token counts (Usage API) and costs (Cost API) by model or workspace, usually within five minutes, and it needs an Admin API credential rather than a workspace key. On call, errors are triaged by code: a 429 means the service hit its own rate limit (2.1), while a 529 means the API is overloaded across all users, so the incident lies outside Shelfwise's code. Anthropic's status page, status.claude.com, lists active incidents and capacity notices. Logging each response's request-id lets the team quote it to support.

Maintenance keeps the service working while its surroundings change. Anthropic's model deprecations page gives each model a state: active, legacy, deprecated or retired. A deprecated model gets a recommended replacement and a retirement date, and publicly released models get at least 60 days' notice by email. After that date its requests fail, so the replacement must pass the evals and a rollout first. Dependencies need the same upkeep: version 1.0 of the Python SDK removed three deprecated sampling parameters, so code that still passes them raises a TypeError.

Interactive · Shelfwise's life cycle, and where a change re-enters

Shelfwise's life cycle as four phases read from top to bottom. Develop: prompt, tools and stock rules versioned in git, and an eval set of 150 recorded questions. Implement: model ID and prompt version in configuration, and a staged rollout to the Leeds branch first. Operate: the Usage and Cost Admin API and alerts on 429 and 529 errors, with request IDs logged. Maintain: the model deprecations page and SDK upgrades. An arrow on the left returns from Maintain to Develop: a model, prompt or SDK change re-enters development. 1 · DEVELOP Prompt, tools and stock rules versioned in git Eval set: 150 recorded questions, graded automatically evals pass 2 · IMPLEMENT Model ID and prompt version held in config Staged rollout: Leeds branch first, then every branch rollout complete 3 · OPERATE Usage and Cost Admin API: tokens and cost by model On call: 429 = our limit, 529 = platform; request-id logged notice, drift or new version 4 · MAINTAIN Deprecations page: model state and retirement date Model, prompt and SDK updates planned as releases CHANGE RE-ENTERS DEVELOP Time runs downward through one release; the dashed arrow starts the next one.

Step through the phases of one Shelfwise release.

1 · Develop: the prompt, tools and stock rules are versioned artefacts, and the eval set of 150 recorded questions decides whether a version is ready.

2 · Implement: a version that passed the evals goes into service through configuration, on the Leeds branch first, so a rollback is a settings change.

3 · Operate: the team watches tokens and cost by model, and triages errors on call. A 429 is Shelfwise's own limit; a 529 is overload across all users.

4 · Maintain: a deprecation notice, drifting answers or a new SDK version starts a change. It goes back to Develop and runs the whole loop again.

Read downward through one release: blue is development, amber implementation, green operation and ember maintenance. The dashed arrow sends a model, prompt or SDK change back through development instead of patching it in place.

SkillReplacing the model behind Shelfwise

Shelfwise's first model change went wrong. After a deprecation notice named a replacement for its model, an engineer edited the model ID in the code on a Friday and redeployed the service, so every branch got it at once:

# Friday: model ID edited in code, service redeployed for every branch
Saturday → staff report counts with no feed timestamp
# cause: the change skipped develop (no eval run) and implement (no stage)
eval set re-run → 141/150 pass, 9 answers omit the timestamp
# fix: model ID moved to config; stock rules updated in git; evals 150/150
Leeds only for a week → mean cost per answer under $0.01, then every branch

The change was maintenance, but it altered which model answers, so it went back through development and implementation. The eval set found the broken functional requirement in minutes instead of on a busy Saturday, and the Leeds week confirmed the mean cost per answer: the Cost API's figure for the new model, less the tagged eval runs, divided by the answers logged. Rolling back would have stopped the complaints, but the next model change would fail the same way, because nothing tested a change before every branch received it.

Ask one question: does the change alter which model answers or what Claude receives, such as the model, the prompt, the tools or the SDK? Yes → it is a new release: run the evals, roll it out in stages and keep the old version for rollback. No → handle it in operation: monitor, alert and adjust the configuration.
Takeaway
  • Develop, implement, operate and maintain each have their own checks; in a Claude application, maintenance keeps returning work to development.
  • A deprecated model still answers until its retirement date, and after it every request fails.
  • Watch usage and cost through the Usage and Cost Admin API; a 429 is your own limit, a 529 is platform overload.

check yourself

0 / 3 answered

A bank's support assistant has a revised prompt that passes the team's eval suite, and the assistant handles about 40,000 conversations a day. If the evals missed a problem, the bank wants as few customers as possible to meet it, and it wants to undo the change without waiting for a new deploy.Which release approach best meets both conditions?

Select one.

A logistics firm's invoice-classification service has run unchanged for 14 months, and since Monday every request it sends fails. A newer service on a later model, using the same API key and workspace, is unaffected. Request volume matches last week, and Anthropic's emails go to a shared inbox nobody has read for months.What is the most likely root cause?

Select one.

A retailer's five product teams share one Claude organisation, and each team has its own workspace. Finance wants each team's spend in dollars once a day. Engineering wants an alert within about ten minutes when any team's token use jumps, without adding code to the teams' services.Which TWO set-ups best meet both needs?

Select two.

sources

  1. ISO, "ISO/IEC/IEEE 15288:2023 Systems and software engineering — System life cycle processes": the full life cycle from conception, development, production, utilisation and support to retirement; processes applied iteratively and concurrently; no specific life cycle model prescribed. Checked 2026-10-04.
  2. Claude Platform Docs, "Model deprecations": active, legacy, deprecated and retired; a replacement and retirement date for deprecated models; requests to retired models fail; at least 60 days' notice, by email; test replacements well before retirement; the Python SDK v1.0 parameter removal. Checked 2026-10-04.
  3. Claude Platform Docs, "Define success criteria and build evaluations": task-specific evals that mirror the real task distribution, edge cases included, graded automatically. Checked 2026-10-04.
  4. Claude Platform Docs, "Usage and Cost API": usage and cost by model, workspace and API key; Admin API credentials, not workspace keys; data typically within 5 minutes; usage buckets of a minute, an hour or a day, and cost by day only. Checked 2026-10-04.
  5. Claude Platform Docs, "Rate limits": acceleration limits and ramping traffic up gradually; the Usage page's rate-limit charts. Checked 2026-10-04.
  6. Claude Platform Docs, "Claude API errors": 529 overloaded_error during high traffic across all users; the request-id header to quote to support. Checked 2026-10-04.
  7. Claude Code Docs, "Error reference": check status.claude.com for active incidents (500) and capacity notices (529); a 529 is not your usage limit. Checked 2026-10-04.
  8. Shelfwise's 150-question eval set, its config-held model ID and prompt version, the Leeds-first rollout, the move to the replacement model named in a deprecation notice and the 141/150 result are this course's example details.