Task statement 7.2 · Security and safety
7.2 · Guardrails and Safe Deployment
In Shelfwise, the course example, Claude can only look up stock counts, yet a release must still settle what staff may ask, what Claude may touch, what data leaves the shop and who may change the service. Guardrails hold those lines, and the strongest hold whatever Claude decides.
share
A capable engineer often assumes safety arrives with the model, because Claude refuses harmful requests and Anthropic enforces its Usage Policy. That leaves the product's own rules unwritten and the deployer's duties undone, such as telling users they face an AI. The same instinct grants broad permissions to avoid permission prompts, so one mistaken tool call reaches everything.
KnowledgeContent policy: Anthropic's floor and your own rules
Content policy works at two levels. Anthropic's Usage Policy applies to anyone who can submit inputs, so it binds your users as well as you. Its High-Risk Use Case Requirements cover output that advises or decides about people in areas such as legal, healthcare, finance and employment: a qualified professional reviews it before it reaches them, and they are told AI helped. Every consumer-facing chatbot must also disclose, at least at the start of each session, that users are talking to an AI.
Your own content policy sits on top: narrower rules for what your product accepts and says. The moderation guide builds them as categories, from examples to flag and examples to allow. Claude's built-in safety behaviour follows the Usage Policy rather than your scope, so a system prompt states your policy while an input screen and an output check enforce it.
More precise than the guide. Anthropic's pages have no "content policy" (as of 2026-10-05), only the Usage Policy, also called the Acceptable Use Policy, and the moderation guide's categories. Exam answer: content policy means the rules on what an application may accept and produce, with the Usage Policy as the floor and the product's stricter rules enforced by its own guardrails.
KnowledgeGuardrail layering
Guardrail layering is defence in depth for an AI application: independent controls sit along a request's path, so a mistake or an attack that one misses, another catches. A request meets an input screen, the system prompt, tool permissions, a sandbox or container, human approval for consequential actions, an output check and, afterwards, monitoring. The input screen is the harmlessness screen of 7.1, which classifies the request before the main conversation sees it. Claude follows the system prompt most of the time. Every other layer, including a hook (7.3) and the output check (6.3), is enforced by code, the operating system or a person, whatever Claude decides. Monitoring closes the loop, finding what got through so that the other layers can be refined (7.1).
Interactive · which layers can stop each failure
Choose a failure to see which layers can stop it.
Off-policy question: a user asks for something your content policy rules out. The input screen can refuse it before Claude sees it, the system prompt tells Claude how to refuse, the output check can replace a reply that slipped through, and monitoring finds users who keep trying. Of these four, only the system prompt depends on Claude.
Unwanted write call: Claude, mistaken or misled, calls a tool that changes something. Tool permissions can remove the tool or deny the call, and a hook can block it; human approval puts a person before the side effect; monitoring shows it afterwards. A line in the system prompt cannot promise it never happens.
Command sends a file out: a shell command tries to upload a credentials file to an unknown host. A deny rule matches only the command as written, so a reworded command can pass, and an approval prompt holds only if the person reads the command closely. The sandbox's network isolation blocks the connection whatever the command does, so it is the layer that holds. Monitoring shows the attempt.
KnowledgeSecure by design: least privilege, access and privacy
Secure-by-design principles are choices made in the design, before an incident forces them. Least privilege, in NIST's words, restricts each user or process to the minimum access its tasks need. In the Agent SDK, allowed_tools only pre-approves the tools it names (1.2), and an unlisted tool falls through to the permission mode, so under bypassPermissions a list of ["Read"] still lets Claude run Bash. A tools list or a bare-name disallowed_tools entry removes a tool instead. Permission rules judge a command before it runs, so a sandbox or container adds a boundary that the operating system enforces, with both filesystem and network isolation, because either alone leaves a way out. A proxy outside that boundary adds credentials to requests, so the agent never sees them.
Identity and access management applies the same idea to people and services. Claude Platform workspaces separate projects or environments, each with its own members, API keys, and rate and spend limits, and workspace roles run from Workspace User, limited to the playground, to Workspace Admin. In Claude Code, managed settings, which outrank every other settings file (2.6), can turn off bypassPermissions for the whole organisation.
Privacy by design means sending Claude only the data a task needs and knowing where that data is kept. Anthropic never trains on retained API data without your express permission, and under zero data retention (ZDR), enabled per organisation, it stores no prompts or outputs after the response. ZDR does not block the features it excludes, so a Files API upload, a batch or code execution keeps its data under that feature's own policy. Flagged content can be kept for up to two years, even under ZDR. 7.1 covers personal data in prompts.
SkillWorked example: a public stock checker, reviewed on paper
The owners asked whether customers could check stock on the shops' website, and the team reviewed the idea on paper. The first draft put the staff service behind a public page unchanged:
# draft: the staff service behind a public page workspace "shelfwise", 100,000 ITPM cap public traffic could starve staff answers no opening line about AI a consumer chatbot must disclose it questions go straight to the agent users may now be adversaries # revised, then parked: own workspace and limits, an AI notice, a screen first
Each gap came from an assumption that held only for staff, who are trusted, know Shelfwise is an AI and share the workspace's 100,000 input tokens a minute with nobody hostile. The revision gave the public page its own workspace and limits, a disclosure in every chat's first message and a Haiku-family screen before the agent. Least privilege needed no change, since the public version would offer only stock_lookup and never the parked stock_reserve. The screen's extra request on every answer helped park the idea.
- The Usage Policy binds your users too, and a consumer-facing chatbot must disclose at each session's start that users are talking to an AI.
- In the Agent SDK,
allowed_toolspre-approves without restricting; removing a tool takes atoolslist or a bare-namedisallowed_toolsentry. - ZDR does not block the features it excludes, such as the Files API, which keep their data under their own retention policies.