Task statement 7.2 · Security and safety

7.2 · Guardrails and Safe Deployment

In Shelfwise, the course example, Claude can only look up stock counts, yet a release must still settle what staff may ask, what Claude may touch, what data leaves the shop and who may change the service. Guardrails hold those lines, and the strongest hold whatever Claude decides.

Why it's tested

A capable engineer often assumes safety arrives with the model, because Claude refuses harmful requests and Anthropic enforces its Usage Policy. That leaves the product's own rules unwritten and the deployer's duties undone, such as telling users they face an AI. The same instinct grants broad permissions to avoid permission prompts, so one mistaken tool call reaches everything.

KnowledgeContent policy: Anthropic's floor and your own rules

Content policy works at two levels. Anthropic's Usage Policy applies to anyone who can submit inputs, so it binds your users as well as you. Its High-Risk Use Case Requirements cover output that advises or decides about people in areas such as legal, healthcare, finance and employment: a qualified professional reviews it before it reaches them, and they are told AI helped. Every consumer-facing chatbot must also disclose, at least at the start of each session, that users are talking to an AI.

Your own content policy sits on top: narrower rules for what your product accepts and says. The moderation guide builds them as categories, from examples to flag and examples to allow. Claude's built-in safety behaviour follows the Usage Policy rather than your scope, so a system prompt states your policy while an input screen and an output check enforce it.

More precise than the guide. Anthropic's pages have no "content policy" (as of 2026-10-05), only the Usage Policy, also called the Acceptable Use Policy, and the moderation guide's categories. Exam answer: content policy means the rules on what an application may accept and produce, with the Usage Policy as the floor and the product's stricter rules enforced by its own guardrails.

KnowledgeGuardrail layering

Guardrail layering is defence in depth for an AI application: independent controls sit along a request's path, so a mistake or an attack that one misses, another catches. A request meets an input screen, the system prompt, tool permissions, a sandbox or container, human approval for consequential actions, an output check and, afterwards, monitoring. The input screen is the harmlessness screen of 7.1, which classifies the request before the main conversation sees it. Claude follows the system prompt most of the time. Every other layer, including a hook (7.3) and the output check (6.3), is enforced by code, the operating system or a person, whatever Claude decides. Monitoring closes the loop, finding what got through so that the other layers can be refined (7.1).

Interactive · which layers can stop each failure

Seven guardrail layers in the order a request meets them, from top to bottom. 1, input screen: a small model classifies the request; enforced by your code. 2, system prompt: states the policy and how to refuse; followed by Claude, so it is guidance. 3, tool permissions: which tools exist and which need approval; enforced by the harness. 4, sandbox or container: which files and hosts a command can reach; enforced by the operating system. 5, human approval: a person confirms each side effect. 6, output check: the reply is checked before anyone sees it; enforced by your code. 7, monitoring: finds what got through and acts on repeat offenders; done by your team afterwards. An off-policy question can be stopped by layers 1, 2, 6 and 7. An unwanted write call can be stopped by layers 3, 5 and 7. A shell command that sends a file to an unknown host can be stopped by layers 3, 4, 5 and 7, and only layer 4 holds whatever the command does. One request, top to bottom ORDER A REQUEST MEETS THEM 1 · input screen A SMALL MODEL CLASSIFIES THE REQUEST YOUR CODE 2 · system prompt STATES THE POLICY AND HOW TO REFUSE CLAUDE: GUIDANCE 3 · tool permissions WHICH TOOLS EXIST, WHICH NEED APPROVAL THE HARNESS 4 · sandbox or container WHICH FILES AND HOSTS A COMMAND REACHES THE OS 5 · human approval A PERSON CONFIRMS EACH SIDE EFFECT A PERSON 6 · output check THE REPLY IS CHECKED BEFORE ANYONE SEES IT YOUR CODE 7 · monitoring FINDS WHAT GOT THROUGH, ACTS ON REPEAT OFFENDERS YOUR TEAM, LATER Green: enforced whatever Claude decides. Amber: guidance Claude follows. Blue: after the fact.

Choose a failure to see which layers can stop it.

Off-policy question: a user asks for something your content policy rules out. The input screen can refuse it before Claude sees it, the system prompt tells Claude how to refuse, the output check can replace a reply that slipped through, and monitoring finds users who keep trying. Of these four, only the system prompt depends on Claude.

Unwanted write call: Claude, mistaken or misled, calls a tool that changes something. Tool permissions can remove the tool or deny the call, and a hook can block it; human approval puts a person before the side effect; monitoring shows it afterwards. A line in the system prompt cannot promise it never happens.

Command sends a file out: a shell command tries to upload a credentials file to an unknown host. A deny rule matches only the command as written, so a reworded command can pass, and an approval prompt holds only if the person reads the command closely. The sandbox's network isolation blocks the connection whatever the command does, so it is the layer that holds. Monitoring shows the attempt.

Rows are layers in request order. A failure lights the layers that can stop it, and each chip names what enforces that layer.

KnowledgeSecure by design: least privilege, access and privacy

Secure-by-design principles are choices made in the design, before an incident forces them. Least privilege, in NIST's words, restricts each user or process to the minimum access its tasks need. In the Agent SDK, allowed_tools only pre-approves the tools it names (1.2), and an unlisted tool falls through to the permission mode, so under bypassPermissions a list of ["Read"] still lets Claude run Bash. A tools list or a bare-name disallowed_tools entry removes a tool instead. Permission rules judge a command before it runs, so a sandbox or container adds a boundary that the operating system enforces, with both filesystem and network isolation, because either alone leaves a way out. A proxy outside that boundary adds credentials to requests, so the agent never sees them.

Identity and access management applies the same idea to people and services. Claude Platform workspaces separate projects or environments, each with its own members, API keys, and rate and spend limits, and workspace roles run from Workspace User, limited to the playground, to Workspace Admin. In Claude Code, managed settings, which outrank every other settings file (2.6), can turn off bypassPermissions for the whole organisation.

Privacy by design means sending Claude only the data a task needs and knowing where that data is kept. Anthropic never trains on retained API data without your express permission, and under zero data retention (ZDR), enabled per organisation, it stores no prompts or outputs after the response. ZDR does not block the features it excludes, so a Files API upload, a batch or code execution keeps its data under that feature's own policy. Flagged content can be kept for up to two years, even under ZDR. 7.1 covers personal data in prompts.

SkillWorked example: a public stock checker, reviewed on paper

The owners asked whether customers could check stock on the shops' website, and the team reviewed the idea on paper. The first draft put the staff service behind a public page unchanged:

# draft: the staff service behind a public page
workspace "shelfwise", 100,000 ITPM cap   public traffic could starve staff answers
no opening line about AI                  a consumer chatbot must disclose it
questions go straight to the agent        users may now be adversaries
# revised, then parked: own workspace and limits, an AI notice, a screen first

Each gap came from an assumption that held only for staff, who are trusted, know Shelfwise is an AI and share the workspace's 100,000 input tokens a minute with nobody hostile. The revision gave the public page its own workspace and limits, a disclosure in every chat's first message and a Haiku-family screen before the agent. Least privilege needed no change, since the public version would offer only stock_lookup and never the parked stock_reserve. The screen's extra request on every answer helped park the idea.

For each harm you must prevent, ask: which layer stops it, and does that layer hold whatever Claude decides? Yes → it is enforced, so it can carry the requirement. Only the system prompt stops it → add an enforced layer, and keep the prompt as guidance.
Takeaway
  • The Usage Policy binds your users too, and a consumer-facing chatbot must disclose at each session's start that users are talking to an AI.
  • In the Agent SDK, allowed_tools pre-approves without restricting; removing a tool takes a tools list or a bare-name disallowed_tools entry.
  • ZDR does not block the features it excludes, such as the Files API, which keep their data under their own retention policies.

check yourself

0 / 3 answered

A data team's Python Agent SDK worker summarises each CSV export as it lands in a shared folder. No one is present to answer prompts, so it sets permission_mode="bypassPermissions" and allowed_tools=["Read", "Grep"], expecting Claude to read and nothing more. Today's log shows that Claude ran a Bash command that deleted a scratch folder in the worker's working directory.What is the most likely root cause?

Select one.

A consultancy's organisation has a zero data retention (ZDR) arrangement. Its Messages API app must let Claude write and run Python to analyse clients' confidential sales spreadsheets, which each request carries. The firm's rule is that no client data may stay stored at Anthropic once a response returns. A developer proposes enabling the code execution tool.Which approach best meets the firm's rule?

Select one.

A support platform runs an Agent SDK agent in a Docker container. It parses customers' uploaded log bundles with Bash. Security requires that, even if Claude is misled or mistaken, nothing leaves the container except calls to the Claude API and the ticket API, and that the agent can neither change the uploaded logs nor read any other host directory.Which TWO controls best meet both requirements?

Select two.

sources

  1. Anthropic, "Usage Policy" (effective 2025-09-15): applies to anyone who can submit inputs; Universal Usage Standards; High-Risk Use Case Requirements, human-in-the-loop and disclosure; consumer-facing chatbots disclose AI at the start of each session. Checked 2026-10-05.
  2. Claude Platform Docs, "Content moderation": moderation categories built from examples to flag and to allow; built-in safety behaviour may flag content regardless of the prompt; review the Acceptable Use Policy first. Checked 2026-10-05.
  3. Claude Platform Docs, "Mitigate jailbreaks and prompt injections": harmlessness screens with a lightweight model and structured outputs; ethical boundaries in the system prompt; throttling or banning repeat offenders; least privilege; continuous monitoring; layering strategies. Checked 2026-10-05.
  4. Claude Code Docs, "Securely deploying AI agents": defense in depth; least privilege for filesystem, network, credentials and capabilities; --network none with a proxy outside the container; read-only mounts; the credential-injecting proxy. Checked 2026-10-05.
  5. Claude Code Docs, "Configure permissions": allowed_tools and disallowed_tools; a bare-name deny removes the tool definition; allowed_tools does not constrain bypassPermissions. And "Configure permissions" (Claude Code): managed settings that no other level can override; disableBypassPermissionsMode. And "Security": a deny rule matches the command as written. Checked 2026-10-05.
  6. Claude Code Docs, "Configure the sandboxed Bash tool": an OS-enforced boundary around shell commands and their child processes; file tools, MCP servers and hooks outside it; both filesystem and network isolation needed. And Anthropic Engineering, "Making Claude Code more secure and autonomous with sandboxing": the two boundaries and why each needs the other. Checked 2026-10-05.
  7. Claude Platform Docs, "Workspaces": separating projects, environments or teams; workspace-scoped API keys; per-workspace rate and spend limits; the workspace roles. Checked 2026-10-05.
  8. Claude Platform Docs, "API and data retention": no training on retained data without express permission; ZDR per organisation, nothing stored at rest after the response; features marked "No" are not blocked; Files API, batch and code execution retention; flagged content kept up to 2 years. Checked 2026-10-05.
  9. NIST CSRC Glossary, "least privilege" and "defense-in-depth": minimum access needed for assigned tasks; layered countermeasures so that attacks missed by one are caught by another. Checked 2026-10-05.
  10. The public stock checker, its draft and its revision are this course's additions to Shelfwise, a proposal reviewed on paper and not built.