Amazon Comprehend Trust and Safety

Detect undesired content in text
Get started with Toxicity Detection
Get started with Prompt Safety Classification

Amazon Comprehend’s Trust and Safety features help organizations moderate text content. Amazon Comprehend toxicity detection API is an ML-powered capability that identifies toxic content by classifying user generated or machine generated text across seven categories including sexual harassment, hate speech, threat, abuse, profanity, insult, and graphic. Amazon Comprehend prompt safety classifier enables moderation of generative AI input prompts to prevent inappropriate use of generative AI applications. Lastly, Comprehend PII detect API can prevent PII data leak by redacting all personal information from generative AI output.","id":"collection-text-media#overview-1-prod-amazon-comprehend-trust-safety","customSortOrder":"1","heading":"Why Amazon Comprehend Trust and Safety?"},"metadata":{"tags":[{"id":"GLOBAL#product#comprehend-trust-safety","name":"Comprehend trust Safety","namespaceId":"GLOBAL#product","description":"Comprehend trust Safety","metadata":{}}]}}]},"metadata":{"auth":{},"pagination":{"empty":false,"present":true},"testAttributes":{}},"context":{"page":{"pageUrl":"https://aws.amazon.com/comprehend/trust-and-safety/"},"environment":{"stage":"prod","region":"us-east-1"},"sdkVersion":"1.0.120"},"refMap":{"manifest.js":"47864e0370","rt-text-media-collection.js":"2ff4ed8f57","rt-text-media-collection.css":"4cb26045f9","rt-text-media-collection.css.js":"fb8a6a6554","rt-text-media-collection.rtl.css":"f31a3175e6","rt-text-media-collection.rtl.css.js":"77897591ba"},"settings":{"templateMappings":{"hyperlinkText":"hyperlinkText","hyperlinkUrl":"hyperlinkUrl","heading":"heading","mediaAltText":"mediaAltText","mediaPosition":"mediaPosition","mediaUrl":"mediaUrl","subheader":"subheader","bodyContent":"bodyContent","videoOverlayTitle":"videoOverlayTitle","videoThumbnailUrl":"videoThumbnailUrl","videoPlayButtonText":"videoPlayButtonText","dark":"dark"}}}

Why Amazon Comprehend Trust and Safety?

Amazon Comprehend’s Trust and Safety features help organizations moderate text content. Amazon Comprehend toxicity detection API is an ML-powered capability that identifies toxic content by classifying user generated or machine generated text across seven categories including sexual harassment, hate speech, threat, abuse, profanity, insult, and graphic. Amazon Comprehend prompt safety classifier enables moderation of generative AI input prompts to prevent inappropriate use of generative AI applications. Lastly, Comprehend PII detect API can prevent PII data leak by redacting all personal information from generative AI output.

Benefits of Amazon Comprehend Trust and Safety

Quickly and accurately moderate large volume of text and keep your online platforms free from inappropriate content.
Customize the moderation thresholds in API responses to suit your application needs.
Deploy Comprehend APIs through Langchain to moderate input and output of LLMs.

Amazon Comprehend toxicity detection classifies text content and provides a confidence score (0 to 1) for the following seven categories: sexual harassment, hate speech, violence/threat, abuse, profanity, insult, and graphic."},"metadata":{"tags":[{"id":"GLOBAL#pattern#overview","name":"Overview","namespaceId":"GLOBAL#pattern","description":"Overview","metadata":{}},{"id":"GLOBAL#product#comprehend-trust-safety","name":"Comprehend trust Safety","namespaceId":"GLOBAL#product","description":"Comprehend trust Safety","metadata":{}}]}},{"fields":{"useCaseTitle":"Moderate generative AI prompt","id":"product-use-cases#usecase-2-prod-overview-amazon-comprehend-trust-safety","customSortOrder":"2","useCaseSummary":"

Prompt safety classifier provides a confidence score (0 to 1) for the input prompt to be safe or not."},"metadata":{"tags":[{"id":"GLOBAL#pattern#overview","name":"Overview","namespaceId":"GLOBAL#pattern","description":"Overview","metadata":{}},{"id":"GLOBAL#product#comprehend-trust-safety","name":"Comprehend trust Safety","namespaceId":"GLOBAL#product","description":"Comprehend trust Safety","metadata":{}}]}},{"fields":{"useCaseTitle":"Prevent PII data leaks","id":"product-use-cases#usecase-3-prod-overview-amazon-comprehend-trust-safety","customSortOrder":"3","useCaseSummary":"

Comprehend PII detect can mask upto 22 universal PII entities like address, age, credit card number etc. and up to 14 country specific entities like US social security number, CA health number, Passport number etc."},"metadata":{"tags":[{"id":"GLOBAL#pattern#overview","name":"Overview","namespaceId":"GLOBAL#pattern","description":"Overview","metadata":{}},{"id":"GLOBAL#product#comprehend-trust-safety","name":"Comprehend trust Safety","namespaceId":"GLOBAL#product","description":"Comprehend trust Safety","metadata":{}}]}}]},"metadata":{"auth":{},"pagination":{"empty":false,"present":true},"testAttributes":{}},"context":{"page":{"pageUrl":"https://aws.amazon.com/comprehend/trust-and-safety/"},"environment":{"stage":"prod","region":"us-east-1"},"sdkVersion":"1.0.120"},"refMap":{"manifest.js":"3daf852ae2","rt-qa-sampler.css":"6682628699","rt-qa-sampler.js":"68f95da020","rt-qa-sampler.css.js":"088fba180d","rt-qa-sampler.rtl.css":"ebe31daae4","rt-qa-sampler.rtl.css.js":"17e3041e3d"},"settings":{"templateMappings":{"patternHeading":"patternHeading","patternSubheading":"patternSubheading","topic":"useCaseTitle","content":"useCaseSummary"}}}

Use cases

Amazon Comprehend toxicity detection classifies text content and provides a confidence score (0 to 1) for the following seven categories: sexual harassment, hate speech, violence/threat, abuse, profanity, insult, and graphic.

Prompt safety classifier provides a confidence score (0 to 1) for the input prompt to be safe or not.

Comprehend PII detect can mask upto 22 universal PII entities like address, age, credit card number etc. and up to 14 country specific entities like US social security number, CA health number, Passport number etc.