Task statement 2.2 · Applications and integration
2.2 · Systems Life Cycle
In Shelfwise, the course example, the stock agent went live once but keeps changing: the engineers edit the stock rules, Anthropic releases and retires models, and the Agent SDK ships new versions. The systems life cycle says which checks each change must pass before it reaches staff.
share
The tempting view treats launch as the finish line and later changes as small patches. An engineer who swaps the model ID in production because a newer model is cheaper skips the tests that guarded the first release, so the first sign of a changed answer is a complaint from a shop. The edit was one line, but it changed behaviour as much as a release, because a Claude application's behaviour lives in the model and the prompt as well as the code.
Start hereThree terms this section adds
- deprecated model
- A model that still works but is no longer recommended. Anthropic names a replacement and sets a retirement date.
- retired model
- A model that is no longer available. Every request to it fails.
- staged rollout
- Releasing a change to a small share of traffic first, then widening it, with the old version ready for rollback.
KnowledgePhases, and the order they run in
Systems life cycle management splits a system's life into phases, each with its own concerns and checks. ISO/IEC/IEEE 15288, the international standard for system life cycle processes, spans conception, development, production, utilisation, support and retirement. The exam guide uses four verbs for that span: develop, implement, operate and maintain. Development builds and tests the system, implementation puts it into service, operation runs it for its users, and maintenance keeps it working as its surroundings change.
A sequential model, often called waterfall, runs each phase once and in order, so maintenance only patches what was built. An iterative model repeats the loop in small increments, and each change passes back through development and implementation. ISO/IEC/IEEE 15288 prescribes neither, and says its processes can be applied iteratively and concurrently. A Claude application leans iterative, because the model underneath it changes on a schedule the team does not control, so a model replacement is a new release, not a patch.
KnowledgeWhat each phase means for a Claude application
In development, the prompt, the tool definitions and the evals are artefacts, versioned and reviewed like code. An eval is a set of test inputs with expected answers, graded automatically, and Anthropic's guidance is to design evals that mirror the real task distribution, edge cases included. Shelfwise's eval set is 150 recorded stock questions, each with the expected count and feed timestamp or "no record", and it later shows whether a new model or prompt still passes.
Implementation puts a tested version into service. Since its first model change, Shelfwise keeps the model ID and the prompt version in configuration rather than code (2.6), so a release or a rollback changes a setting. A staged rollout sends the new version to part of the traffic first, which limits the damage if the evals missed something. A gradual ramp also helps, because usage that rises too fast can trigger 429 errors from the API's acceleration limits.
Operation runs the service and notices problems before its users do. The Usage and Cost Admin API reports the organisation's token counts (Usage API) and costs (Cost API) by model or workspace, usually within five minutes, and it needs an Admin API credential rather than a workspace key. On call, errors are triaged by code: a 429 means the service hit its own rate limit (2.1), while a 529 means the API is overloaded across all users, so the incident lies outside Shelfwise's code. Anthropic's status page, status.claude.com, lists active incidents and capacity notices. Logging each response's request-id lets the team quote it to support.
Maintenance keeps the service working while its surroundings change. Anthropic's model deprecations page gives each model a state: active, legacy, deprecated or retired. A deprecated model gets a recommended replacement and a retirement date, and publicly released models get at least 60 days' notice by email. After that date its requests fail, so the replacement must pass the evals and a rollout first. Dependencies need the same upkeep: version 1.0 of the Python SDK removed three deprecated sampling parameters, so code that still passes them raises a TypeError.
Interactive · Shelfwise's life cycle, and where a change re-enters
Step through the phases of one Shelfwise release.
1 · Develop: the prompt, tools and stock rules are versioned artefacts, and the eval set of 150 recorded questions decides whether a version is ready.
2 · Implement: a version that passed the evals goes into service through configuration, on the Leeds branch first, so a rollback is a settings change.
3 · Operate: the team watches tokens and cost by model, and triages errors on call. A 429 is Shelfwise's own limit; a 529 is overload across all users.
4 · Maintain: a deprecation notice, drifting answers or a new SDK version starts a change. It goes back to Develop and runs the whole loop again.
SkillReplacing the model behind Shelfwise
Shelfwise's first model change went wrong. After a deprecation notice named a replacement for its model, an engineer edited the model ID in the code on a Friday and redeployed the service, so every branch got it at once:
# Friday: model ID edited in code, service redeployed for every branch Saturday → staff report counts with no feed timestamp # cause: the change skipped develop (no eval run) and implement (no stage) eval set re-run → 141/150 pass, 9 answers omit the timestamp # fix: model ID moved to config; stock rules updated in git; evals 150/150 Leeds only for a week → mean cost per answer under $0.01, then every branch
The change was maintenance, but it altered which model answers, so it went back through development and implementation. The eval set found the broken functional requirement in minutes instead of on a busy Saturday, and the Leeds week confirmed the mean cost per answer: the Cost API's figure for the new model, less the tagged eval runs, divided by the answers logged. Rolling back would have stopped the complaints, but the next model change would fail the same way, because nothing tested a change before every branch received it.
- Develop, implement, operate and maintain each have their own checks; in a Claude application, maintenance keeps returning work to development.
- A deprecated model still answers until its retirement date, and after it every request fails.
- Watch usage and cost through the Usage and Cost Admin API; a 429 is your own limit, a 529 is platform overload.