Repository Public
1# CSAM guidance23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45{"OpenAI developed this resource with expert input from the "}6 {", "}7 {", the "}8 {", and the "}9 {"."}10 111213## Build with child safety in mind1415OpenAI has clear child safety expectations for developers:16You are responsible for ensuring that your users use OpenAI services in17compliance with applicable laws, including laws that criminalize child sexual18abuse and exploitation. Never use OpenAI services to exploit, endanger, or19sexualize anyone under the age of 18. See the OpenAI .2021Online child sexual exploitation and abuse affects a range of products and22services, including those that don't target children. OpenAI wants to help23developers understand what actions to consider taking to address this abuse.2425From the earliest possible stage, consider how people could misuse your product.26Start early so child safety safeguards can scale with you, rather than becoming27something you try to retrofit into an already complex product or system.28Developer teams and organizations of all sizes should assess how people could29misuse their products for a range of harms, including child sexual abuse material30(CSAM), grooming, sexual extortion, the sexualization of children, livestreamed31abuse, and trafficking—especially if their products support messaging, content32uploads, image editing, livestreaming, discovery, or payments.3334This resource focuses on CSAM and offers developers practical guidance for35protecting children.3637## Where to start3839It can be difficult to know where to start. The right solutions and40implementation paths depend on your organization's size, maturity, and available41resources.4243The following checklist is a good starting point for addressing CSAM. The44important thing is to begin addressing the risk: Don't wait until you have every45tool or step complete before taking action.4647## Prevent4849Set clear rules for your product or service, and establish mechanisms for50hearing from your users about their experiences.5152- **Set clear rules.** Prohibit child sexual exploitation and abuse in your terms53 of service, acceptable use policy, or community guidelines. Learn more from54 or the Tech Coalition's free55 for expert guidance and practical tools, including a resource on external56 standards that prohibit online child sexual exploitation and abuse.57- **Make reporting available to your users.** Give users a visible way to flag58 potentially harmful content or behavior, and route those concerns to a monitored queue59 or location with enough information to make policy decisions. For more60 guidance, see the Australian .61- **Track uploads and users through safety identifiers.** In your product or service,62 associate every upload with a user. Sending63 [safety identifiers with supported OpenAI64 requests](https://developers.openai.com/api/docs/guides/safety-best-practices#implement-safety-identifiers)65 can help OpenAI monitor and detect abuse. This can help OpenAI provide your team with more66 actionable feedback if OpenAI detects policy violations in your67 product or service. Safety identifiers can also help your team respond to abuse68 faster. They create a stable way to trace activity back to an individual end69 user and reduce the chance that one user's misuse disrupts access for your70 broader organization. Use a unique string to represent each user. To protect71 privacy, hash email addresses or usernames before sending them to avoid72 disclosing personally identifiable information. The direct Images API uses a73 different parameter for this purpose: Send the same stable identifier as74 `user` for [image75 generation](https://developers.openai.com/api/reference/resources/images/methods/generate) and [image76 edits](https://developers.openai.com/api/reference/resources/images/methods/edit).7778When you are ready to do more, consider other prevention measures:7980- Add prevention-focused blocking or refusals for CSAM-related keywords81 or URLs. Learn more from the .82- Provide warnings or messages to users who are attempting to engage in83 CSAM-related behavior. Learn more from .84- If your service is available to children and a user tells you that sexual85 images or videos of them are on your platform, direct them to support services86 such as the IWF and NSPCC's program, which87 enables children in the UK to report images and videos for removal or88 blocking, or NCMEC's service.89- If you become aware that a child is facing immediate or imminent harm:90 - Refer the case to emergency services, such as local law enforcement.91 - Give the child information on how to contact emergency services.9293## Detect9495Identify potentially violative content or conduct for review and action.9697Use accessible CSAM detection tools:9899- **Perceptual hash matching.** If your product supports uploading, storing, or100 processing visual media, consider implementing perceptual hash matching. This101 technology creates a digital fingerprint of an image or video and compares it102 with fingerprints of known CSAM, helping to identify known103 material even after someone resizes, compresses, or otherwise modifies a file.104 You need access to both hash-matching technology, such as PhotoDNA, and CSAM105 hash lists, which are repositories of known CSAM hashes. Not all services106 provide both.107 - **Recommended hash-matching technologies:**108 - offers purpose-built detection for known CSAM in images and videos.109 - The Tech Coalition offers eligible companies sublicenses to through its free110 .111 - .112 - YouTube's technology113 provides hash matching to identify known CSAM in videos.114 - **Recommended CSAM hash lists:**115 - NCMEC offers lists of known CSAM, exploitative content, and generative AI116 CSAM. Contact its [Electronic Service Provider team](mailto:espteam@ncmec.org).117 - IWF offers , a118 hash-matching service for eligible small businesses and startups that119 requires minimal technical expertise to integrate.120- **Novel CSAM detection classifiers.** These tools can detect unknown or unseen121 CSAM.122 - offers123 classifiers intended to identify potentially novel CSAM in images and124 videos, as well as relevant text-based exploitation signals.125 - Google's uses AI to126 classify images and videos and assign their review priority. The higher the127 priority assigned by the classifier, the more likely the media contains128 CSAM.129- **Bad actor detection and enforcement.** Using available identifiers and130 information—such as usernames, email addresses, and device IDs—consider131 permanently banning or otherwise disrupting accounts that people have used to132 violate, or attempt to violate, CSAM policies. Watch for repeat offenders who133 attempt to circumvent enforcement.134135## Respond and report136137Make sure your product and team can take appropriate action when you become138aware of CSAM on your service.139140- **Register and prepare to report CSAM to the authorities.** or your . Consider what makes a report to NCMEC141 actionable and how it can support child safeguarding. Include as much142 information as possible to help route the report to the appropriate143 jurisdiction and identify the suspect. NCMEC's includes IP addresses, device IDs, and other data.144145Local laws and reporting obligations may vary by jurisdiction.146147- **Keep usable records and identifiers.** When you make a report, maintain148 documentation of the incident and any associated data that could help identify149 violative actors, so you can respond to requests from law enforcement.150- **Write a response playbook.** Define who reviews reports, how to escalate151 urgent cases internally and externally, what actions to take against users152 responsible for violations, and who can make those decisions. This can help you establish enforcement operations.153- Consider establishing a network of trusted expert reporters, including154 organizations such as IWF and other hotlines, that can use their expertise to155 flag CSAM cases for you.156- **Train and support the people involved.** It's important for humans to be in157 the loop. Make sure reviewers, support teams, and on-call staff understand your158 policy, escalation path, and the limits of any automated system.159- **Use the tools you need to moderate content or respond to abuse.** Other160 tools can help you address child safety risks and harms:161 - The [Moderation API](https://developers.openai.com/api/docs/guides/moderation) detects potentially harmful162 content in text and images. Learn more about the163 [`omni-moderation-latest`](https://developers.openai.com/api/docs/models/omni-moderation-latest) model.164 This isn't a substitute for dedicated CSAM detection. It still includes a165 `sexual/minors` category covering sexual content involving people166 under 18; this category is text-only. Use results to:167 - Block or filter content.168 - Send content for human review.169 - Intervene on an account.170 - Add friction to repeated misuse and apply product- or service-specific171 enforcement.172 - Consider other moderation tools that could help. For example, is an open-source review console for173 triaging potential policy violations in text, multimedia, and profiles. It174 supports human and automated review, takes a wellness-oriented approach for175 reviewers, and enables end-to-end moderation workflows, including176 NCMEC CyberTipline reporting.177178When you are ready to do more, consider other response and reporting179measures:180181- **Support the people involved in tackling CSAM.** Organizations should invest182 in training, support, and a well-being program for CSAM reviewers. Read183 .184- **Get specialist support.** You don't need a large trust and safety team to185 start. The Tech Coalition offers ways for companies to build stronger child186 safety systems:187 - is a free capacity-building program designed especially for startups and188 small and midsize platforms, while remaining open to companies of all sizes.189 It provides practical resources, guidance, and support to help companies190 establish strong child safety foundations. Eligible companies can also apply191 for a PhotoDNA sublicense through Pathways.192 - provides tailored consulting and implementation support for companies193 seeking more hands-on help to strengthen their child safety programs and194 respond to specific risks.195 - enables companies to take part in the industry's global collaborative196 response to online child sexual exploitation and abuse, engage with peers,197 share expertise, and contribute to collective action. Contact the [Tech198 Coalition team](mailto:md@technologycoalition.org) for an initial199 consultation.200 - Developers can use the Tech Coalition's for further guidance.201202### Scale safeguards203204The right controls and safeguards depend on the product, its development stage205and maturity, its users and features, the regions in which it operates, and its206available resources.207208Products with greater risk exposure—for example, those that support209livestreaming, image generation or editing, file storage, or private210connections—should consider implementing and strengthening safeguards such as:211212- Product risk assessments before launch and whenever high-risk features change.213 See .214- Layered detection appropriate to the service, which may include hash matching,215 image or video classifiers, text signals, keyword detection, and URL blocking.216- Human review of high-confidence or high-severity signals, using tools that217 protect reviewer well-being and limit unnecessary exposure to harmful218 material.219- Rate limits, account controls, and abuse monitoring that make repeat misuse220 harder.221- Regular testing and measurement so you can find gaps, track outcomes, and222 improve your controls.223224These recommendations are a starting point, not legal advice or a universal225standard of care. Adapt them to your service, risk profile, and applicable law.