Repository Public
1---2title: Customer support agent3url: https://platform.claude.com/docs/en/about-claude/use-case-guides/customer-support-chat4description: Build a customer support chatbot with Claude that answers product questions, stays on topic, and generates quotes through tool use.5---67## Prerequisites89To follow this guide, you need:1011* A Claude API key (set as the `ANTHROPIC_API_KEY` environment variable)12* Python 3.10 or later1314Install the required packages:1516```bash17pip install anthropic streamlit python-dotenv18```1920## Before building with Claude2122### Decide whether to use Claude for support chat2324Here are some key indicators that you should employ an LLM like Claude to automate portions of your customer support process:2526<AccordionGroup>27 <Accordion title="High volume of repetitive queries">28 Claude excels at handling a large number of similar questions efficiently, freeing up human agents for more complex issues.29 </Accordion>3031 <Accordion title="Need for quick information synthesis">32 Claude can quickly retrieve, process, and combine information from vast knowledge bases, while human agents may need time to research or consult multiple sources.33 </Accordion>3435 <Accordion title="24/7 availability requirement">36 Claude can provide round-the-clock support without fatigue, whereas staffing human agents for continuous coverage can be costly and challenging.37 </Accordion>3839 <Accordion title="Rapid scaling during peak periods">40 Claude can handle sudden increases in query volume without the need for hiring and training additional staff.41 </Accordion>4243 <Accordion title="Consistent brand voice">44 You can instruct Claude to consistently represent your brand's tone and values, whereas human agents may vary in their communication styles.45 </Accordion>46</AccordionGroup>4748Some considerations for choosing Claude over other LLMs:4950* You prioritize natural, nuanced conversation: Claude's sophisticated language understanding allows for more natural, context-aware conversations that feel more human-like than chats with other LLMs.51* You often receive complex and open-ended queries: Claude can handle a wide range of topics and inquiries without generating canned responses or requiring extensive programming of permutations of user utterances.52* You need scalable multilingual support: Claude's multilingual capabilities allow it to engage in conversations in over 200 languages without the need for separate chatbots or extensive translation processes for each supported language.5354### Define your ideal chat interaction5556Outline an ideal customer interaction to define how and when you expect the customer to interact with Claude. This outline will help to determine the technical requirements of your solution.5758Here is an example chat interaction for car insurance customer support:5960* **Customer:** Initiates support chat experience61 * **Claude:** Warmly greets customer and initiates conversation6263* **Customer:** Asks about insurance for their new electric car64 * **Claude:** Provides relevant information about electric vehicle coverage6566* **Customer:** Asks questions related to unique needs for electric vehicle insurances67 * **Claude:** Responds with accurate and informative answers and provides links to the sources6869* **Customer:** Asks off-topic questions unrelated to insurance or cars70 * **Claude:** Clarifies it does not discuss unrelated topics and steers the user back to car insurance7172* **Customer:** Expresses interest in an insurance quote7374 * **Claude:** Ask a set of questions to determine the appropriate quote, adapting to their responses75 * **Claude:** Sends a request to use the quote generation API tool along with necessary information collected from the user76 * **Claude:** Receives the response information from the API tool use, synthesizes the information into a natural response, and presents the provided quote to the user7778* **Customer:** Asks follow up questions7980 * **Claude:** Answers follow up questions as needed81 * **Claude:** Guides the customer to the next steps in the insurance process and closes out the conversation8283<Tip>84 In the real example that you write for your own use case, you might find it useful to write out the actual words in this interaction so that you can also get a sense of the ideal tone, response length, and level of detail you want Claude to have.85</Tip>8687### Break the interaction into unique tasks8889Customer support chat is a collection of multiple different tasks, from question answering to information retrieval to taking action on requests, wrapped up in a single customer interaction. Before you start building, break down your ideal customer interaction into every task you want Claude to be able to perform. This ensures you can prompt and evaluate Claude for every task, and gives you a good sense of the range of interactions you need to account for when writing test cases.9091<Tip>92 Customers sometimes find it helpful to visualize this as an interaction flowchart of possible conversation inflection points depending on user requests.93</Tip>9495Here are the key tasks associated with the example insurance interaction:96971. Greeting and general guidance9899 * Warmly greet the customer and initiate conversation100 * Provide general information about the company and interaction1011022. Product information103104 * Provide information about electric vehicle coverage105 <Note>106 This will require that Claude have the necessary information in its context, and might imply that a 107108 [RAG integration](https://platform.claude.com/cookbook/capabilities-retrieval-augmented-generation-guide)109110 is necessary.111 </Note>112 * Answer questions related to unique electric vehicle insurance needs113 * Answer follow-up questions about the quote or insurance details114 * Offer links to sources when appropriate1151163. Conversation management117118 * Stay on topic (car insurance)119 * Redirect off-topic questions back to relevant subjects1201214. Quote generation122123 * Ask appropriate questions to determine quote eligibility124 * Adapt questions based on customer responses125 * Submit collected information to quote generation API126 * Present the provided quote to the customer127128### Establish success criteria129130Work with your support team to [define success criteria and write detailed evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests) with measurable benchmarks and goals.131132Here are criteria and benchmarks that can be used to evaluate how successfully Claude performs the defined tasks:133134<AccordionGroup>135 <Accordion title="Query comprehension accuracy">136 This metric evaluates how accurately Claude understands customer inquiries across various topics. Measure this by reviewing a sample of conversations and assessing whether Claude has the correct interpretation of customer intent, critical next steps, what successful resolution looks like, and more. Aim for a comprehension accuracy of 95% or higher.137 </Accordion>138139 <Accordion title="Response relevance">140 This assesses how well Claude's response addresses the customer's specific question or issue. Evaluate a set of conversations and rate the relevance of each response (using LLM-based grading for scale). Target a relevance score of 90% or above.141 </Accordion>142143 <Accordion title="Response accuracy">144 Assess the correctness of general company and product information provided to the user, based on the information provided to Claude in context. Target 100% accuracy in this introductory information.145 </Accordion>146147 <Accordion title="Citation provision relevance">148 Track the frequency and relevance of links or sources offered. Target providing relevant sources in 80% of interactions where additional information could be beneficial.149 </Accordion>150151 <Accordion title="Topic adherence">152 Measure how well Claude stays on topic, such as the topic of car insurance in the example implementation. Aim for 95% of responses to be directly related to car insurance or the customer's specific query.153 </Accordion>154155 <Accordion title="Content generation effectiveness">156 Measure how successful Claude is at determining when to generate informational content and how relevant that content is. For example, in this implementation, you would be determining how well Claude understands when to generate a quote and how accurate that quote is. Target 100% accuracy, as this is vital information for a successful customer interaction.157 </Accordion>158159 <Accordion title="Escalation efficiency">160 This measures Claude's ability to recognize when a query needs human intervention and escalate appropriately. Track the percentage of correctly escalated conversations versus those that should have been escalated but weren't. Aim for an escalation accuracy of 95% or higher.161 </Accordion>162</AccordionGroup>163164Here are criteria and benchmarks that can be used to evaluate the business impact of employing Claude for support:165166<AccordionGroup>167 <Accordion title="Sentiment maintenance">168 This assesses Claude's ability to maintain or improve customer sentiment throughout the conversation. Use sentiment analysis tools to measure sentiment at the beginning and end of each conversation. Aim for maintained or improved sentiment in 90% of interactions.169 </Accordion>170171 <Accordion title="Deflection rate">172 The percentage of customer inquiries successfully handled by the chatbot without human intervention. Typically aim for 70-80% deflection rate, depending on the complexity of inquiries.173 </Accordion>174175 <Accordion title="Customer satisfaction score">176 A measure of how satisfied customers are with their chatbot interaction. Usually done through post-interaction surveys. Aim for a CSAT score of 4 out of 5 or higher.177 </Accordion>178179 <Accordion title="Average handle time">180 The average time it takes for the chatbot to resolve an inquiry. This varies widely based on the complexity of issues, but generally, aim for a lower AHT compared to human agents.181 </Accordion>182</AccordionGroup>183184## How to implement Claude as a customer service agent185186### Choose the right Claude model187188The choice of model depends on the trade-offs between cost, accuracy, and response time.189190For customer support chat, Claude Opus 5 is well suited to balance intelligence, latency, and cost, including the most complex support scenarios that require deep reasoning across long, multi-step conversations. However, for instances where you have conversation flow with multiple prompts including RAG, tool use, or long-context prompts, Claude Haiku 4.5 may be more suitable to optimize for latency.191192### Build a strong prompt193194Using Claude for customer support requires Claude having enough direction and context to respond appropriately, while having enough flexibility to handle a wide range of customer inquiries.195196Start by writing the elements of a strong prompt, beginning with a system prompt. Create a file called `config.py` and add each of the following blocks to it:197198```python199IDENTITY = """You are Eva, a friendly and knowledgeable AI assistant for Acme Insurance200Company. Your role is to warmly welcome customers and provide information on201Acme's insurance offerings, which include car insurance and electric car202insurance. You can also help customers get quotes for their insurance needs."""203```204205<Tip>206 While you may be tempted to put all your information inside a system prompt as a way to separate instructions from the user conversation, Claude actually works best with the bulk of its prompt content written inside the first 207208 `User`209210 turn (with the only exception being role prompting). Read more at 211212 [Giving Claude a role with a system prompt](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices#give-claude-a-role)213214 .215</Tip>216217It's best to break down complex prompts into subsections and write one part at a time. For each task, you might find greater success by following a step-by-step process to define the parts of the prompt Claude would need to do the task well. For this car insurance customer support example, you'll be writing piecemeal all the parts for a prompt starting with the "Greeting and general guidance" task. This also makes debugging your prompt easier as you can more quickly adjust individual parts of the overall prompt.218219```python220STATIC_GREETINGS_AND_GENERAL = """221<static_context>222Acme Auto Insurance: Your Trusted Companion on the Road223224About:225At Acme Insurance, we understand that your vehicle is more than just a mode of transportation—it's your ticket to life's adventures.226Since 1985, we've been crafting auto insurance policies that give drivers the confidence to explore, commute, and travel with peace of mind.227Whether you're navigating city streets or embarking on cross-country road trips, Acme is there to protect you and your vehicle.228Our innovative auto insurance policies are designed to adapt to your unique needs, covering everything from fender benders to major collisions.229With Acme's award-winning customer service and swift claim resolution, you can focus on the joy of driving while we handle the rest.230We're not just an insurance provider—we're your co-pilot in life's journeys.231Choose Acme Auto Insurance and experience the assurance that comes with superior coverage and genuine care. Because at Acme, we don't just232insure your car—we fuel your adventures on the open road.233234Note: We also offer specialized coverage for electric vehicles, ensuring that drivers of all car types can benefit from our protection.235236Acme Insurance offers the following products:237- Car insurance238- Electric car insurance239- Two-wheeler insurance240241Business hours: Monday-Friday, 9 AM - 5 PM EST242Customer service number: 1-800-123-4567243</static_context>244"""245```246247Then do the same for your car insurance and electric car insurance information.248249```python250STATIC_CAR_INSURANCE = """251<static_context>252Car Insurance Coverage:253Acme's car insurance policies typically cover:2541. Liability coverage: Pays for bodily injury and property damage you cause to others.2552. Collision coverage: Pays for damage to your car in an accident.2563. Comprehensive coverage: Pays for damage to your car from non-collision incidents.2574. Medical payments coverage: Pays for medical expenses after an accident.2585. Uninsured/underinsured motorist coverage: Protects you if you're hit by a driver with insufficient insurance.259260Optional coverages include:261- Rental car reimbursement262- Roadside assistance263- New car replacement264</static_context>265"""266267STATIC_ELECTRIC_CAR_INSURANCE = """268<static_context>269Electric Car Insurance:270Our specialized electric car insurance goes beyond traditional auto coverage,271offering tailored protection for your high-tech investment. In addition to standard coverages, we offer:272- Safeguarding expensive battery systems273- Covering home charging equipment274- Adapting to the rapidly evolving world of electric vehicles275276Whether you're driving a Rivian, Nissan Leaf, Chevrolet Bolt,277or any other electric vehicle, our knowledgeable agents are ready to craft a policy278that gives you the confidence to enjoy every electron-powered mile.279</static_context>280"""281```282283Now that you have your static content, add at least 4-5 sample "good" interactions to guide Claude's responses. These examples should be representative of your ideal customer interaction and can include elements such as guardrails and tool calls.284285```python286EXAMPLES = """287Here are a few examples of how you can interact with customers:288289<example 1>290H: Hi, do you offer commercial insurance for small businesses?291292A: Ah! Unfortunately, we don't offer commercial insurance at this time.293However, we do provide a range of personal insurance products including car insurance and electric car insurance.294Would you like to know more about these offerings?295</example 1>296297<example 2>298H: Hi, what are your business hours?299300A: Our business hours are Monday to Friday, 9 AM to 5 PM EST.301If you have any questions or need assistance, feel free to reach out during those times.302How else can I assist you today?303</example 2>304305<example 3>306H: I'm shopping for car insurance. What does Acme offer?307308A: Great choice! Acme's car insurance policies typically include:309- Liability coverage: Protects you if you're at fault in an accident310- Collision coverage: Pays for damage to your car in an accident311- Comprehensive coverage: Covers non-collision incidents like theft or vandalism312- Medical payments coverage: Pays for medical expenses after an accident313- Uninsured/underinsured motorist coverage: Protects you from drivers with insufficient insurance314315We also offer optional coverages like rental car reimbursement, roadside assistance, and new car replacement.316Would you like more details on any of these coverages, or do you have specific questions about insuring your vehicle?317</example 3>318319<example 4>320H: I'd like to hear about your electric car insurance.321322A: Certainly! Our electric car insurance is specially designed to meet the unique needs of electric vehicle owners.323In addition to all the standard coverages we offer for regular cars, our electric car policies include:324325- Battery coverage: Protection against damage or degradation of your vehicle's expensive battery system326- Tech package: Covers repairs or replacement of advanced electronics and onboard computer systems327- Charging station coverage: Protects both home and portable charging equipment328- Cybersecurity protection: Coverage against hacking or digital threats329- Specialized roadside assistance: Includes flatbed towing to protect the vehicle's undercarriage330331We cover a wide range of electric vehicles, including popular models from Tesla, Rivian, Nissan, Chevrolet, and more.332Is there a specific electric vehicle you're looking to insure, or would you like more details on any of these coverages?333</example 4>334335<example 5>336H: I'd like to get a quote for my car insurance.337338A: Certainly! I'd be happy to help you get a quote for your car insurance.339To provide you with an accurate quote, I'll need to collect some information about your vehicle and the primary driver.340Let's start with the basics:3413421. What is the make and model of your vehicle?3432. What year was it manufactured?3443. Approximately how many miles have you driven?3454. What is the age of the primary driver?346347Once you provide this information, I'll use our quoting tool to generate a personalized insurance quote for you.348</example 5>349"""350```351352You will also want to include any important instructions outlining dos and don'ts for how Claude should interact with the customer. This may draw from brand guardrails or support policies.353354```python355ADDITIONAL_GUARDRAILS = """Please adhere to the following guardrails:3561. Only provide information about insurance types listed in our offerings.3572. If asked about an insurance type we don't offer, politely state358that we don't provide that service.3593. Do not speculate about future product offerings or company plans.3604. Don't make promises or enter into agreements it's not authorized to make.361You only provide information and guidance.3625. Do not mention any competitor's products or services.363"""364```365366Now combine all these sections into a single string to use as your prompt.367368```python369TASK_SPECIFIC_INSTRUCTIONS = " ".join(370 [371 STATIC_GREETINGS_AND_GENERAL,372 STATIC_CAR_INSURANCE,373 STATIC_ELECTRIC_CAR_INSURANCE,374 EXAMPLES,375 ADDITIONAL_GUARDRAILS,376 ]377)378```379380### Add dynamic and agentic capabilities with tool use381382Claude is capable of taking actions and retrieving information dynamically using client-side tool use functionality. Start by listing any external tools or APIs the prompt should use.383384For this example, start with one tool for calculating the quote.385386<Tip>387 As a reminder, this tool will not perform the actual calculation, it will just signal to the application that a tool should be used with whatever arguments specified.388</Tip>389390Add the model name, the tool definition, and a stub implementation to `config.py`:391392```python393import time394395MODEL = "claude-opus-5"396397TOOLS = [398 {399 "name": "get_quote",400 "description": "Calculate the insurance quote based on user input. Returned value is per month premium.",401 "input_schema": {402 "type": "object",403 "properties": {404 "make": {"type": "string", "description": "The make of the vehicle."},405 "model": {"type": "string", "description": "The model of the vehicle."},406 "year": {407 "type": "integer",408 "description": "The year the vehicle was manufactured.",409 },410 "mileage": {411 "type": "integer",412 "description": "The mileage on the vehicle.",413 },414 "driver_age": {415 "type": "integer",416 "description": "The age of the primary driver.",417 },418 },419 "required": ["make", "model", "year", "mileage", "driver_age"],420 },421 }422]423424425def get_quote(make, model, year, mileage, driver_age):426 """Returns the premium per month in USD"""427 # You can call an http endpoint or a database to get the quote.428 # Here, we simulate a delay of 1 seconds and return a fixed quote of 100.429 time.sleep(1)430 return 100431```432433### Deploy your prompts434435It's hard to know how well your prompt works without deploying it in a test production setting and [running evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests). Build a small application using the prompt, the Anthropic SDK, and Streamlit for a user interface.436437In a file called `chatbot.py` (or the equivalent module in your language), set up the ChatBot class, which will encapsulate the interactions with the Anthropic SDK.438439The class should have two main methods: one that calls the API to generate a message, and one that processes each incoming user input.440441<CodeGroup exclude="shell">442 ```python Python443 # In your chatbot.py, import these from the config.py you wrote above:444 # from config import IDENTITY, TOOLS, MODEL, get_quote445 from anthropic import Anthropic446 from dotenv import load_dotenv447448 load_dotenv()449450451 class ChatBot:452 def __init__(self, session_state):453 self.anthropic = Anthropic()454 self.session_state = session_state455456 def generate_message(457 self,458 messages,459 max_tokens,460 ):461 try:462 response = self.anthropic.messages.create(463 model=MODEL,464 system=IDENTITY,465 max_tokens=max_tokens,466 messages=messages,467 tools=TOOLS,468 )469 return response470 except Exception as e:471 return {"error": str(e)}472473 def process_user_input(self, user_input):474 self.session_state.messages.append({"role": "user", "content": user_input})475476 response_message = self.generate_message(477 messages=self.session_state.messages,478 max_tokens=2048,479 )480481 if "error" in response_message:482 return f"An error occurred: {response_message['error']}"483484 if response_message.content[-1].type == "tool_use":485 tool_use = response_message.content[-1]486 func_name = tool_use.name487 func_params = tool_use.input488 tool_use_id = tool_use.id489490 result = self.handle_tool_use(func_name, func_params)491 self.session_state.messages.append(492 {"role": "assistant", "content": response_message.content}493 )494 self.session_state.messages.append(495 {496 "role": "user",497 "content": [498 {499 "type": "tool_result",500 "tool_use_id": tool_use_id,501 "content": f"{result}",502 }503 ],504 }505 )506507 follow_up_response = self.generate_message(508 messages=self.session_state.messages,509 max_tokens=2048,510 )511512 if "error" in follow_up_response:513 return f"An error occurred: {follow_up_response['error']}"514515 response_text = next(516 (block.text for block in follow_up_response.content if block.type == "text"),517 None,518 )519 if response_text is None:520 raise Exception("An error occurred: Unexpected response type")521 self.session_state.messages.append(522 {"role": "assistant", "content": response_text}523 )524 return response_text525526 text_block = next(527 (block for block in response_message.content if block.type == "text"), None528 )529 if text_block is not None:530 response_text = text_block.text531 self.session_state.messages.append(532 {"role": "assistant", "content": response_text}533 )534 return response_text535536 raise Exception("An error occurred: Unexpected response type")537538 def handle_tool_use(self, func_name, func_params):539 if func_name == "get_quote":540 premium = get_quote(**func_params)541 return f"Quote generated: ${premium:.2f} per month"542543 raise Exception("An unexpected tool was used")544 ```545546 ```typescript TypeScript547 import Anthropic from "@anthropic-ai/sdk";548549 class ChatBot {550 // IDENTITY, MODEL, TOOLS, and getQuote mirror the config.py values defined551 // earlier in this guide (shown in Python).552 readonly anthropic = new Anthropic();553 readonly messages: Anthropic.MessageParam[] = [];554555 async generateMessage(556 messages: Anthropic.MessageParam[],557 maxTokens: number558 ): Promise<Anthropic.Message> {559 return this.anthropic.messages.create({560 model: MODEL,561 system: IDENTITY,562 max_tokens: maxTokens,563 messages,564 tools: TOOLS565 });566 }567568 async processUserInput(userInput: string): Promise<string> {569 this.messages.push({ role: "user", content: userInput });570571 const responseMessage = await this.generateMessage(this.messages, 2048);572573 const lastBlock = responseMessage.content.at(-1);574 if (lastBlock?.type === "tool_use") {575 const toolResult = this.handleToolUse(lastBlock.name, lastBlock.input);576577 this.messages.push({ role: "assistant", content: responseMessage.content });578 this.messages.push({579 role: "user",580 content: [{ type: "tool_result", tool_use_id: lastBlock.id, content: toolResult }]581 });582583 const followUpResponse = await this.generateMessage(this.messages, 2048);584585 const followUpBlock = followUpResponse.content.find(586 (block): block is Anthropic.TextBlock => block.type === "text"587 );588 if (!followUpBlock) {589 throw new Error("An error occurred: Unexpected response type");590 }591 this.messages.push({ role: "assistant", content: followUpBlock.text });592 return followUpBlock.text;593 }594595 const firstBlock = responseMessage.content.find(596 (block): block is Anthropic.TextBlock => block.type === "text"597 );598 if (firstBlock) {599 this.messages.push({ role: "assistant", content: firstBlock.text });600 return firstBlock.text;601 }602603 throw new Error("An error occurred: Unexpected response type");604 }605606 handleToolUse(toolName: string, toolInput: unknown): string {607 if (toolName === "get_quote") {608 // The SDK types tool_use.input as unknown; narrow it to the get_quote schema.609 if (610 toolInput === null ||611 typeof toolInput !== "object" ||612 !("make" in toolInput) || typeof toolInput.make !== "string" ||613 !("model" in toolInput) || typeof toolInput.model !== "string" ||614 !("year" in toolInput) || typeof toolInput.year !== "number" ||615 !("mileage" in toolInput) || typeof toolInput.mileage !== "number" ||616 !("driver_age" in toolInput) || typeof toolInput.driver_age !== "number"617 ) {618 throw new Error("An error occurred: Unexpected tool input");619 }620 const { make, model: vehicleModel, year, mileage, driver_age: driverAge } = toolInput;621622 const premium = getQuote(make, vehicleModel, year, mileage, driverAge);623 return `Quote generated: $${premium.toFixed(2)} per month`;624 }625626 throw new Error("An unexpected tool was used");627 }628 }629 ```630631 ```csharp C#632 using System.Text.Json;633 using Anthropic;634 using Anthropic.Models.Messages;635636 // Config.Model, Config.Identity, Config.Tools, and Config.GetQuote mirror637 // the config.py values defined earlier in this guide (shown in Python).638 public class ChatBot639 {640 private readonly AnthropicClient _anthropic = new();641642 public List<MessageParam> Messages { get; } = [];643644 public async Task<Message> GenerateMessage(List<MessageParam> messages, long maxTokens) =>645 await _anthropic.Messages.Create(646 new MessageCreateParams647 {648 Model = Config.Model,649 System = Config.Identity,650 MaxTokens = maxTokens,651 Messages = messages,652 Tools = Config.Tools,653 }654 );655656 public async Task<string> ProcessUserInput(string userInput)657 {658 Messages.Add(new() { Role = Role.User, Content = userInput });659660 var responseMessage = await GenerateMessage(Messages, maxTokens: 2048);661662 if (responseMessage.Content[^1].TryPickToolUse(out var toolUse))663 {664 var toolResult = HandleToolUse(toolUse.Name, toolUse.Input);665666 Messages.Add(new()667 {668 Role = Role.Assistant,669 Content = responseMessage.Content670 .Select(contentBlock => new ContentBlockParam(contentBlock.Json))671 .ToList(),672 });673 Messages.Add(new()674 {675 Role = Role.User,676 Content = new List<ContentBlockParam>677 {678 new ToolResultBlockParam { ToolUseID = toolUse.ID, Content = toolResult },679 },680 });681682 var followUpResponse = await GenerateMessage(Messages, maxTokens: 2048);683684 foreach (var block in followUpResponse.Content)685 {686 if (block.TryPickText(out var followUpText))687 {688 Messages.Add(new() { Role = Role.Assistant, Content = followUpText.Text });689 return followUpText.Text;690 }691 }692693 throw new InvalidOperationException("An error occurred: Unexpected response type");694 }695696 foreach (var block in responseMessage.Content)697 {698 if (block.TryPickText(out var textBlock))699 {700 Messages.Add(new() { Role = Role.Assistant, Content = textBlock.Text });701 return textBlock.Text;702 }703 }704705 throw new InvalidOperationException("An error occurred: Unexpected response type");706 }707708 public string HandleToolUse(string funcName, IReadOnlyDictionary<string, JsonElement> funcParams)709 {710 if (funcName == "get_quote")711 {712 var premium = Config.GetQuote(713 funcParams["make"].GetString()!,714 funcParams["model"].GetString()!,715 funcParams["year"].GetInt64(),716 funcParams["mileage"].GetInt64(),717 funcParams["driver_age"].GetInt64()718 );719 return $"Quote generated: ${premium:F2} per month";720 }721722 throw new ArgumentException("An unexpected tool was used");723 }724 }725 ```726727 ```go Go728 import (729 "context"730 "encoding/json"731 "fmt"732733 "github.com/anthropics/anthropic-sdk-go"734 )735736 // ChatBot wraps the Anthropic client and the conversation history. The737 // identity, model, tools, and getQuote values it uses mirror the config.py738 // definitions earlier in this guide (shown in Python).739 type ChatBot struct {740 client anthropic.Client741 messages []anthropic.MessageParam742 }743744 func NewChatBot() *ChatBot {745 return &ChatBot{client: anthropic.NewClient()}746 }747748 func (bot *ChatBot) GenerateMessage(ctx context.Context, messages []anthropic.MessageParam, maxTokens int64) (*anthropic.Message, error) {749 return bot.client.Messages.New(ctx, anthropic.MessageNewParams{750 Model: model,751 System: []anthropic.TextBlockParam{{Text: identity}},752 MaxTokens: maxTokens,753 Messages: messages,754 Tools: tools,755 })756 }757758 func (bot *ChatBot) ProcessUserInput(ctx context.Context, userInput string) (string, error) {759 bot.messages = append(bot.messages, anthropic.NewUserMessage(anthropic.NewTextBlock(userInput)))760761 response, err := bot.GenerateMessage(ctx, bot.messages, 2048)762 if err != nil {763 return "", err764 }765766 lastBlock := response.Content[len(response.Content)-1]767 if toolUse, ok := lastBlock.AsAny().(anthropic.ToolUseBlock); ok {768 result, err := bot.HandleToolUse(toolUse.Name, toolUse.Input)769 if err != nil {770 return "", err771 }772773 bot.messages = append(bot.messages,774 response.ToParam(),775 anthropic.NewUserMessage(anthropic.NewToolResultBlock(toolUse.ID, result, false)),776 )777778 followUp, err := bot.GenerateMessage(ctx, bot.messages, 2048)779 if err != nil {780 return "", err781 }782783 for _, block := range followUp.Content {784 if textBlock, ok := block.AsAny().(anthropic.TextBlock); ok {785 bot.messages = append(bot.messages, anthropic.NewAssistantMessage(anthropic.NewTextBlock(textBlock.Text)))786 return textBlock.Text, nil787 }788 }789 return "", fmt.Errorf("an error occurred: unexpected response type")790 }791792 for _, block := range response.Content {793 if textBlock, ok := block.AsAny().(anthropic.TextBlock); ok {794 bot.messages = append(bot.messages, anthropic.NewAssistantMessage(anthropic.NewTextBlock(textBlock.Text)))795 return textBlock.Text, nil796 }797 }798799 return "", fmt.Errorf("an error occurred: unexpected response type")800 }801802 func (bot *ChatBot) HandleToolUse(toolName string, toolInput json.RawMessage) (string, error) {803 if toolName != "get_quote" {804 return "", fmt.Errorf("an unexpected tool was used: %s", toolName)805 }806807 var input struct {808 Make string `json:"make"`809 Model string `json:"model"`810 Year int `json:"year"`811 Mileage int `json:"mileage"`812 DriverAge int `json:"driver_age"`813 }814 if err := json.Unmarshal(toolInput, &input); err != nil {815 return "", err816 }817 premium := getQuote(input.Make, input.Model, input.Year, input.Mileage, input.DriverAge)818 return fmt.Sprintf("Quote generated: $%.2f per month", premium), nil819 }820821 ```822823 ```java Java824 import com.anthropic.client.AnthropicClient;825 import com.anthropic.client.okhttp.AnthropicOkHttpClient;826 import com.anthropic.core.JsonValue;827 import com.anthropic.models.messages.ContentBlock;828 import com.anthropic.models.messages.ContentBlockParam;829 import com.anthropic.models.messages.Message;830 import com.anthropic.models.messages.MessageCreateParams;831 import com.anthropic.models.messages.MessageParam;832 import com.anthropic.models.messages.ToolResultBlockParam;833 import com.anthropic.models.messages.ToolUseBlock;834835 // IDENTITY, MODEL, TOOLS, and getQuote mirror the config.py values defined836 // earlier in this guide (shown in Python).837 class ChatBot {838 final AnthropicClient anthropic;839 final List<MessageParam> messages;840841 ChatBot() {842 // Reads the API key from the ANTHROPIC_API_KEY environment variable843 this.anthropic = AnthropicOkHttpClient.fromEnv();844 this.messages = new ArrayList<>();845 }846847 Message generateMessage(List<MessageParam> messages, long maxTokens) {848 return anthropic.messages().create(MessageCreateParams.builder()849 .model(MODEL)850 .system(IDENTITY)851 .maxTokens(maxTokens)852 .messages(messages)853 .tools(TOOLS)854 .build());855 }856857 String processUserInput(String userInput) {858 messages.add(MessageParam.builder()859 .role(MessageParam.Role.USER)860 .content(userInput)861 .build());862863 Message responseMessage = generateMessage(messages, 2048);864865 List<ContentBlock> content = responseMessage.content();866 ContentBlock lastBlock = content.getLast();867 if (lastBlock.isToolUse()) {868 ToolUseBlock toolUse = lastBlock.asToolUse();869 Map<String, JsonValue> toolInput =870 (Map<String, JsonValue>) toolUse._input().asObject().orElseThrow();871 String result = handleToolUse(toolUse.name(), toolInput);872873 messages.add(MessageParam.builder()874 .role(MessageParam.Role.ASSISTANT)875 .contentOfBlockParams(content.stream().map(ContentBlock::toParam).toList())876 .build());877 messages.add(MessageParam.builder()878 .role(MessageParam.Role.USER)879 .contentOfBlockParams(List.of(ContentBlockParam.ofToolResult(880 ToolResultBlockParam.builder()881 .toolUseId(toolUse.id())882 .content(result)883 .build())))884 .build());885886 Message followUpResponse = generateMessage(messages, 2048);887888 ContentBlock followUpBlock = followUpResponse.content().stream()889 .filter(ContentBlock::isText)890 .findFirst()891 .orElseThrow(() -> new IllegalStateException("An error occurred: Unexpected response type"));892 String responseText = followUpBlock.asText().text();893 messages.add(MessageParam.builder()894 .role(MessageParam.Role.ASSISTANT)895 .content(responseText)896 .build());897 return responseText;898 } else {899 ContentBlock textBlock = content.stream()900 .filter(ContentBlock::isText)901 .findFirst()902 .orElseThrow(() -> new IllegalStateException("An error occurred: Unexpected response type"));903 String responseText = textBlock.asText().text();904 messages.add(MessageParam.builder()905 .role(MessageParam.Role.ASSISTANT)906 .content(responseText)907 .build());908 return responseText;909 }910 }911912 String handleToolUse(String funcName, Map<String, JsonValue> funcParams) {913 return switch (funcName) {914 case "get_quote" -> {915 double premium = getQuote(916 funcParams.get("make").asStringOrThrow(),917 funcParams.get("model").asStringOrThrow(),918 ((Number) funcParams.get("year").asNumber().orElseThrow()).longValue(),919 ((Number) funcParams.get("mileage").asNumber().orElseThrow()).longValue(),920 ((Number) funcParams.get("driver_age").asNumber().orElseThrow()).longValue());921 yield "Quote generated: $%.2f per month".formatted(premium);922 }923 default -> throw new IllegalArgumentException("An unexpected tool was used");924 };925 }926 }927 ```928929 ```php PHP930 use Anthropic\Client;931 use Anthropic\Messages\Message;932 use Anthropic\Messages\MessageParam;933 use Anthropic\Messages\TextBlock;934 use Anthropic\Messages\ToolResultBlockParam;935 use Anthropic\Messages\ToolUseBlock;936937 class ChatBot938 {939 // MODEL, IDENTITY, TOOLS, and get_quote() mirror the config.py values940 // defined earlier in this guide (shown in Python).941942 /** @var list<MessageParam> */943 public private(set) array $messages = [];944945 public function __construct(946 private readonly Client $anthropic = new Client(),947 ) {}948949 /**950 * @param list<MessageParam> $messages951 */952 public function generateMessage(array $messages, int $maxTokens): Message953 {954 return $this->anthropic->messages->create(955 model: MODEL,956 system: IDENTITY,957 maxTokens: $maxTokens,958 messages: $messages,959 tools: TOOLS,960 );961 }962963 public function processUserInput(string $userInput): string964 {965 $this->messages[] = MessageParam::with(role: 'user', content: $userInput);966967 $responseMessage = $this->generateMessage($this->messages, maxTokens: 2048);968969 $content = $responseMessage->content;970 $lastBlock = array_last($content);971972 if ($lastBlock instanceof ToolUseBlock) {973 $toolResult = $this->handleToolUse($lastBlock->name, $lastBlock->input);974975 $this->messages[] = MessageParam::with(role: 'assistant', content: $content);976 $this->messages[] = MessageParam::with(977 role: 'user',978 content: [979 ToolResultBlockParam::with(toolUseID: $lastBlock->id, content: $toolResult),980 ],981 );982983 $followUpResponse = $this->generateMessage($this->messages, maxTokens: 2048);984985 $firstBlock = array_find(986 $followUpResponse->content,987 static fn ($block): bool => $block instanceof TextBlock,988 );989 if (!$firstBlock instanceof TextBlock) {990 throw new RuntimeException('An error occurred: Unexpected response type');991 }992993 $this->messages[] = MessageParam::with(role: 'assistant', content: $firstBlock->text);994995 return $firstBlock->text;996 }997998 $firstBlock = array_find($content, static fn ($block): bool => $block instanceof TextBlock);999 if ($firstBlock instanceof TextBlock) {1000 $this->messages[] = MessageParam::with(role: 'assistant', content: $firstBlock->text);10011002 return $firstBlock->text;1003 }10041005 throw new RuntimeException('An error occurred: Unexpected response type');1006 }10071008 /**1009 * @param array<string, mixed> $funcParams1010 */1011 private function handleToolUse(string $funcName, array $funcParams): string1012 {1013 if ($funcName === 'get_quote') {1014 $premium = get_quote(...$funcParams);10151016 return sprintf('Quote generated: $%.2f per month', $premium);1017 }10181019 throw new RuntimeException('An unexpected tool was used');1020 }1021 }1022 ```10231024 ```ruby Ruby1025 # IDENTITY, MODEL, TOOLS, and get_quote mirror the config.py values defined1026 # earlier in this guide (shown in Python).1027 require "anthropic"10281029 class ChatBot1030 attr_reader :messages10311032 def initialize1033 @anthropic = Anthropic::Client.new1034 @messages = []1035 end10361037 def generate_message(messages, max_tokens)1038 @anthropic.messages.create(1039 model: MODEL,1040 system_: IDENTITY,1041 max_tokens:,1042 messages:,1043 tools: TOOLS1044 )1045 end10461047 def process_user_input(user_input)1048 @messages << {role: "user", content: user_input}10491050 response_message = generate_message(@messages, 2048)10511052 case response_message.content1053 in [*, Anthropic::ToolUseBlock => tool_use]1054 result = handle_tool_use(tool_use.name, tool_use.input)1055 @messages << {role: "assistant", content: response_message.content}1056 @messages << {1057 role: "user",1058 content: [{type: "tool_result", tool_use_id: tool_use.id, content: result}]1059 }10601061 follow_up_response = generate_message(@messages, 2048)10621063 case follow_up_response.content1064 in [*, Anthropic::TextBlock => text_block, *]1065 @messages << {role: "assistant", content: text_block.text}1066 text_block.text1067 else1068 raise "An error occurred: Unexpected response type"1069 end1070 in [*, Anthropic::TextBlock => text_block, *]1071 @messages << {role: "assistant", content: text_block.text}1072 text_block.text1073 else1074 raise "An error occurred: Unexpected response type"1075 end1076 end10771078 def handle_tool_use(tool_name, tool_input)1079 raise "An unexpected tool was used" unless tool_name == "get_quote"10801081 premium = get_quote(**tool_input)1082 format("Quote generated: $%.2f per month", premium)1083 end1084 end1085 ```1086</CodeGroup>10871088### Build your user interface10891090Test deploying this code with Streamlit using a main method. This `main()` function sets up a Streamlit-based chat interface. Streamlit is a Python framework, so this part of the walkthrough is shown in Python only; the ChatBot class above is the piece you can port to any language.10911092Do this in a file called `app.py`10931094```python1095import streamlit as st1096from chatbot import ChatBot1097from config import TASK_SPECIFIC_INSTRUCTIONS109810991100def main():1101 st.title("Chat with Eva, Acme Insurance Company's Assistant🤖")11021103 if "messages" not in st.session_state:1104 st.session_state.messages = [1105 {"role": "user", "content": TASK_SPECIFIC_INSTRUCTIONS},1106 {"role": "assistant", "content": "Understood"},1107 ]11081109 chatbot = ChatBot(st.session_state)11101111 # Display user and assistant messages skipping the first two1112 for message in st.session_state.messages[2:]:1113 # ignore tool use blocks1114 if isinstance(message["content"], str):1115 with st.chat_message(message["role"]):1116 st.markdown(message["content"])11171118 if user_msg := st.chat_input("Type your message here..."):1119 st.chat_message("user").markdown(user_msg)11201121 with st.chat_message("assistant"):1122 with st.spinner("Eva is thinking..."):1123 response_placeholder = st.empty()1124 full_response = chatbot.process_user_input(user_msg)1125 response_placeholder.markdown(full_response)112611271128if __name__ == "__main__":1129 main()1130```11311132Run the program with:11331134```bash1135streamlit run app.py1136```11371138### Evaluate your prompts11391140Prompting often requires testing and optimization for it to be production ready. To determine the readiness of your solution, evaluate the chatbot performance using a systematic process combining quantitative and qualitative methods. Creating a [strong empirical evaluation](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests#build-evaluations) based on your defined success criteria will allow you to optimize your prompts.11411142### Improve performance11431144In complex scenarios, it may be helpful to consider additional strategies to improve performance beyond standard [prompt engineering techniques](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview) & [guardrail implementation strategies](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations). Here are some common scenarios:11451146#### Reduce long context latency with RAG11471148When dealing with large amounts of static and dynamic context, including all information in the prompt can lead to high costs, slower response times, and reaching context window limits. In this scenario, implementing Retrieval Augmented Generation (RAG) techniques can improve performance and efficiency.11491150By using [embedding models like Voyage](https://platform.claude.com/docs/en/build-with-claude/embeddings) to convert information into vector representations, you can create a more scalable and responsive system. This approach allows for dynamic retrieval of relevant information based on the current query, rather than including all possible context in every prompt.11511152Implementing RAG for support use cases has been shown to increase accuracy, reduce response times, and reduce API costs in systems with extensive context requirements. See the [RAG recipe](https://platform.claude.com/cookbook/capabilities-retrieval-augmented-generation-guide) for a worked example.11531154#### Integrate real-time data with tool use11551156When dealing with queries that require real-time information, such as account balances or policy details, embedding-based RAG approaches are not sufficient. Instead, tool use can enhance your chatbot's ability to provide accurate, real-time responses. For example, you can use tool use to look up customer information, retrieve order details, and cancel orders on behalf of the customer.11571158This approach, [outlined in the tool use: customer service agent recipe](https://platform.claude.com/cookbook/tool-use-customer-service-agent), lets you integrate live data into Claude's responses and provide a more personalized and efficient customer experience.11591160#### Strengthen input and output guardrails11611162When deploying a chatbot, especially in customer service scenarios, it's important to prevent risks associated with misuse, out-of-scope queries, and inappropriate responses. While Claude is inherently resilient to such scenarios, here are additional steps to strengthen your chatbot guardrails:11631164* [Reduce hallucination](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations): Implement fact-checking mechanisms and [citations](https://platform.claude.com/cookbook/misc-using-citations) to ground responses in provided information.1165* Cross-check information: Verify that the agent's responses align with your company's policies and known facts.1166* Avoid contractual commitments: Ensure the agent doesn't make promises or enter into agreements it's not authorized to make.1167* [Mitigate jailbreaks](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks): Use methods like harmlessness screens and input validation to prevent users from exploiting model vulnerabilities, aiming to generate inappropriate content.1168* Avoid mentioning competitors: Implement a competitor mention filter to maintain brand focus and not mention any competitor's products or services.1169* [Increase output consistency](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/increase-consistency): Prevent Claude from changing style or going out of character, even during long, complex interactions.1170* Remove Personally Identifiable Information (PII): Unless explicitly required and authorized, strip out any PII from responses.11711172#### Reduce perceived response time with streaming11731174When dealing with potentially lengthy responses, implementing streaming can improve user engagement and satisfaction. In this scenario, users receive the answer progressively instead of waiting for the entire response to be generated.11751176Here is how to implement streaming:117711781. Use the [Anthropic Streaming API](https://platform.claude.com/docs/en/build-with-claude/streaming) to support streaming responses.11792. Set up your frontend to handle incoming chunks of text.11803. Display each chunk as it arrives, simulating real-time typing.11814. Implement a mechanism to save the full response, allowing users to view it if they navigate away and return.11821183In some cases, streaming enables the use of more advanced models with higher base latencies, as the progressive display mitigates the impact of longer processing times.11841185#### Scale your chatbot11861187As the complexity of your chatbot grows, your application architecture can evolve to match. Before you add further layers to your architecture, consider the following less exhaustive options:11881189* Ensure that you are making the most out of your prompts and optimizing through prompt engineering. Use the [prompt engineering guides](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview) to write the most effective prompts.1190* Add additional [tools](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview) to the prompt (which can include [prompt chains](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices#chain-complex-prompts)) and see if you can achieve the functionality required.11911192If your chatbot handles incredibly varied tasks, you may want to consider adding a [separate intent classifier](https://platform.claude.com/cookbook/capabilities-classification-guide) to route the initial customer query. For the existing application, this would involve creating a decision tree that would route customer queries through the classifier and then to specialized conversations (with their own set of tools and system prompts). Note, this method requires an additional call to Claude that can increase latency.11931194### Integrate Claude into your support workflow11951196While these examples have focused on Python functions callable within a Streamlit environment, deploying Claude for real-time support chatbot requires an API service.11971198Here's how you can approach this:119912001. Create an API wrapper: Develop a simple API wrapper around your classification function. For example, you can use Flask API or Fast API to wrap your code into a HTTP Service. Your HTTP service could accept the user input and return the Assistant response in its entirety. Thus, your service could have the following characteristics:12011202 * Server-Sent Events (SSE): SSE allows for real-time streaming of responses from the server to the client. This provides a smooth, interactive experience when working with LLMs.1203 * Caching: Implementing caching can improve response times and reduce unnecessary API calls.1204 * Context retention: Maintaining context when a user navigates away and returns is important for continuity in conversations.120512062. Build a web interface: Implement a user-friendly web UI for interacting with the Claude-powered agent.12071208## Next steps12091210<CardGroup cols={2}>1211 <Card title="Tool use" icon="wrench" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview">1212 Give Claude access to your APIs so it can take action on behalf of customers.1213 </Card>12141215 <Card title="Develop tests" icon="check" href="https://platform.claude.com/docs/en/test-and-evaluate/develop-tests">1216 Build evaluations to measure your support agent against the success criteria you defined.1217 </Card>12181219 <Card title="Streaming" icon="bolt" href="https://platform.claude.com/docs/en/build-with-claude/streaming">1220 Stream responses so customers see answers as they generate.1221 </Card>12221223 <Card title="Prompt engineering" icon="lightbulb" href="https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview">1224 Refine your system prompt and examples for better task performance.1225 </Card>1226</CardGroup>1227