SPB Git

Repository Public

customer-support-chat.md 53.9 KB 1,227 lines markdown

1---2title: Customer support agent3url: https://platform.claude.com/docs/en/about-claude/use-case-guides/customer-support-chat4description: Build a customer support chatbot with Claude that answers product questions, stays on topic, and generates quotes through tool use.5---67## Prerequisites89To follow this guide, you need:1011* A Claude API key (set as the `ANTHROPIC_API_KEY` environment variable)12* Python 3.10 or later1314Install the required packages:1516```bash17pip install anthropic streamlit python-dotenv18```1920## Before building with Claude2122### Decide whether to use Claude for support chat2324Here are some key indicators that you should employ an LLM like Claude to automate portions of your customer support process:2526<AccordionGroup>27  <Accordion title="High volume of repetitive queries">28    Claude excels at handling a large number of similar questions efficiently, freeing up human agents for more complex issues.29  </Accordion>3031  <Accordion title="Need for quick information synthesis">32    Claude can quickly retrieve, process, and combine information from vast knowledge bases, while human agents may need time to research or consult multiple sources.33  </Accordion>3435  <Accordion title="24/7 availability requirement">36    Claude can provide round-the-clock support without fatigue, whereas staffing human agents for continuous coverage can be costly and challenging.37  </Accordion>3839  <Accordion title="Rapid scaling during peak periods">40    Claude can handle sudden increases in query volume without the need for hiring and training additional staff.41  </Accordion>4243  <Accordion title="Consistent brand voice">44    You can instruct Claude to consistently represent your brand's tone and values, whereas human agents may vary in their communication styles.45  </Accordion>46</AccordionGroup>4748Some considerations for choosing Claude over other LLMs:4950* You prioritize natural, nuanced conversation: Claude's sophisticated language understanding allows for more natural, context-aware conversations that feel more human-like than chats with other LLMs.51* You often receive complex and open-ended queries: Claude can handle a wide range of topics and inquiries without generating canned responses or requiring extensive programming of permutations of user utterances.52* You need scalable multilingual support: Claude's multilingual capabilities allow it to engage in conversations in over 200 languages without the need for separate chatbots or extensive translation processes for each supported language.5354### Define your ideal chat interaction5556Outline an ideal customer interaction to define how and when you expect the customer to interact with Claude. This outline will help to determine the technical requirements of your solution.5758Here is an example chat interaction for car insurance customer support:5960* **Customer:** Initiates support chat experience61  * **Claude:** Warmly greets customer and initiates conversation6263* **Customer:** Asks about insurance for their new electric car64  * **Claude:** Provides relevant information about electric vehicle coverage6566* **Customer:** Asks questions related to unique needs for electric vehicle insurances67  * **Claude:** Responds with accurate and informative answers and provides links to the sources6869* **Customer:** Asks off-topic questions unrelated to insurance or cars70  * **Claude:** Clarifies it does not discuss unrelated topics and steers the user back to car insurance7172* **Customer:** Expresses interest in an insurance quote7374  * **Claude:** Ask a set of questions to determine the appropriate quote, adapting to their responses75  * **Claude:** Sends a request to use the quote generation API tool along with necessary information collected from the user76  * **Claude:** Receives the response information from the API tool use, synthesizes the information into a natural response, and presents the provided quote to the user7778* **Customer:** Asks follow up questions7980  * **Claude:** Answers follow up questions as needed81  * **Claude:** Guides the customer to the next steps in the insurance process and closes out the conversation8283<Tip>84  In the real example that you write for your own use case, you might find it useful to write out the actual words in this interaction so that you can also get a sense of the ideal tone, response length, and level of detail you want Claude to have.85</Tip>8687### Break the interaction into unique tasks8889Customer support chat is a collection of multiple different tasks, from question answering to information retrieval to taking action on requests, wrapped up in a single customer interaction. Before you start building, break down your ideal customer interaction into every task you want Claude to be able to perform. This ensures you can prompt and evaluate Claude for every task, and gives you a good sense of the range of interactions you need to account for when writing test cases.9091<Tip>92  Customers sometimes find it helpful to visualize this as an interaction flowchart of possible conversation inflection points depending on user requests.93</Tip>9495Here are the key tasks associated with the example insurance interaction:96971. Greeting and general guidance9899   * Warmly greet the customer and initiate conversation100   * Provide general information about the company and interaction1011022. Product information103104   * Provide information about electric vehicle coverage105     <Note>106       This will require that Claude have the necessary information in its context, and might imply that a 107108       [RAG integration](https://platform.claude.com/cookbook/capabilities-retrieval-augmented-generation-guide)109110        is necessary.111     </Note>112   * Answer questions related to unique electric vehicle insurance needs113   * Answer follow-up questions about the quote or insurance details114   * Offer links to sources when appropriate1151163. Conversation management117118   * Stay on topic (car insurance)119   * Redirect off-topic questions back to relevant subjects1201214. Quote generation122123   * Ask appropriate questions to determine quote eligibility124   * Adapt questions based on customer responses125   * Submit collected information to quote generation API126   * Present the provided quote to the customer127128### Establish success criteria129130Work with your support team to [define success criteria and write detailed evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests) with measurable benchmarks and goals.131132Here are criteria and benchmarks that can be used to evaluate how successfully Claude performs the defined tasks:133134<AccordionGroup>135  <Accordion title="Query comprehension accuracy">136    This metric evaluates how accurately Claude understands customer inquiries across various topics. Measure this by reviewing a sample of conversations and assessing whether Claude has the correct interpretation of customer intent, critical next steps, what successful resolution looks like, and more. Aim for a comprehension accuracy of 95% or higher.137  </Accordion>138139  <Accordion title="Response relevance">140    This assesses how well Claude's response addresses the customer's specific question or issue. Evaluate a set of conversations and rate the relevance of each response (using LLM-based grading for scale). Target a relevance score of 90% or above.141  </Accordion>142143  <Accordion title="Response accuracy">144    Assess the correctness of general company and product information provided to the user, based on the information provided to Claude in context. Target 100% accuracy in this introductory information.145  </Accordion>146147  <Accordion title="Citation provision relevance">148    Track the frequency and relevance of links or sources offered. Target providing relevant sources in 80% of interactions where additional information could be beneficial.149  </Accordion>150151  <Accordion title="Topic adherence">152    Measure how well Claude stays on topic, such as the topic of car insurance in the example implementation. Aim for 95% of responses to be directly related to car insurance or the customer's specific query.153  </Accordion>154155  <Accordion title="Content generation effectiveness">156    Measure how successful Claude is at determining when to generate informational content and how relevant that content is. For example, in this implementation, you would be determining how well Claude understands when to generate a quote and how accurate that quote is. Target 100% accuracy, as this is vital information for a successful customer interaction.157  </Accordion>158159  <Accordion title="Escalation efficiency">160    This measures Claude's ability to recognize when a query needs human intervention and escalate appropriately. Track the percentage of correctly escalated conversations versus those that should have been escalated but weren't. Aim for an escalation accuracy of 95% or higher.161  </Accordion>162</AccordionGroup>163164Here are criteria and benchmarks that can be used to evaluate the business impact of employing Claude for support:165166<AccordionGroup>167  <Accordion title="Sentiment maintenance">168    This assesses Claude's ability to maintain or improve customer sentiment throughout the conversation. Use sentiment analysis tools to measure sentiment at the beginning and end of each conversation. Aim for maintained or improved sentiment in 90% of interactions.169  </Accordion>170171  <Accordion title="Deflection rate">172    The percentage of customer inquiries successfully handled by the chatbot without human intervention. Typically aim for 70-80% deflection rate, depending on the complexity of inquiries.173  </Accordion>174175  <Accordion title="Customer satisfaction score">176    A measure of how satisfied customers are with their chatbot interaction. Usually done through post-interaction surveys. Aim for a CSAT score of 4 out of 5 or higher.177  </Accordion>178179  <Accordion title="Average handle time">180    The average time it takes for the chatbot to resolve an inquiry. This varies widely based on the complexity of issues, but generally, aim for a lower AHT compared to human agents.181  </Accordion>182</AccordionGroup>183184## How to implement Claude as a customer service agent185186### Choose the right Claude model187188The choice of model depends on the trade-offs between cost, accuracy, and response time.189190For customer support chat, Claude Opus 5 is well suited to balance intelligence, latency, and cost, including the most complex support scenarios that require deep reasoning across long, multi-step conversations. However, for instances where you have conversation flow with multiple prompts including RAG, tool use, or long-context prompts, Claude Haiku 4.5 may be more suitable to optimize for latency.191192### Build a strong prompt193194Using Claude for customer support requires Claude having enough direction and context to respond appropriately, while having enough flexibility to handle a wide range of customer inquiries.195196Start by writing the elements of a strong prompt, beginning with a system prompt. Create a file called `config.py` and add each of the following blocks to it:197198```python199IDENTITY = """You are Eva, a friendly and knowledgeable AI assistant for Acme Insurance200Company. Your role is to warmly welcome customers and provide information on201Acme's insurance offerings, which include car insurance and electric car202insurance. You can also help customers get quotes for their insurance needs."""203```204205<Tip>206  While you may be tempted to put all your information inside a system prompt as a way to separate instructions from the user conversation, Claude actually works best with the bulk of its prompt content written inside the first 207208  `User`209210   turn (with the only exception being role prompting). Read more at 211212  [Giving Claude a role with a system prompt](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices#give-claude-a-role)213214  .215</Tip>216217It's best to break down complex prompts into subsections and write one part at a time. For each task, you might find greater success by following a step-by-step process to define the parts of the prompt Claude would need to do the task well. For this car insurance customer support example, you'll be writing piecemeal all the parts for a prompt starting with the "Greeting and general guidance" task. This also makes debugging your prompt easier as you can more quickly adjust individual parts of the overall prompt.218219```python220STATIC_GREETINGS_AND_GENERAL = """221<static_context>222Acme Auto Insurance: Your Trusted Companion on the Road223224About:225At Acme Insurance, we understand that your vehicle is more than just a mode of transportation—it's your ticket to life's adventures.226Since 1985, we've been crafting auto insurance policies that give drivers the confidence to explore, commute, and travel with peace of mind.227Whether you're navigating city streets or embarking on cross-country road trips, Acme is there to protect you and your vehicle.228Our innovative auto insurance policies are designed to adapt to your unique needs, covering everything from fender benders to major collisions.229With Acme's award-winning customer service and swift claim resolution, you can focus on the joy of driving while we handle the rest.230We're not just an insurance provider—we're your co-pilot in life's journeys.231Choose Acme Auto Insurance and experience the assurance that comes with superior coverage and genuine care. Because at Acme, we don't just232insure your car—we fuel your adventures on the open road.233234Note: We also offer specialized coverage for electric vehicles, ensuring that drivers of all car types can benefit from our protection.235236Acme Insurance offers the following products:237- Car insurance238- Electric car insurance239- Two-wheeler insurance240241Business hours: Monday-Friday, 9 AM - 5 PM EST242Customer service number: 1-800-123-4567243</static_context>244"""245```246247Then do the same for your car insurance and electric car insurance information.248249```python250STATIC_CAR_INSURANCE = """251<static_context>252Car Insurance Coverage:253Acme's car insurance policies typically cover:2541. Liability coverage: Pays for bodily injury and property damage you cause to others.2552. Collision coverage: Pays for damage to your car in an accident.2563. Comprehensive coverage: Pays for damage to your car from non-collision incidents.2574. Medical payments coverage: Pays for medical expenses after an accident.2585. Uninsured/underinsured motorist coverage: Protects you if you're hit by a driver with insufficient insurance.259260Optional coverages include:261- Rental car reimbursement262- Roadside assistance263- New car replacement264</static_context>265"""266267STATIC_ELECTRIC_CAR_INSURANCE = """268<static_context>269Electric Car Insurance:270Our specialized electric car insurance goes beyond traditional auto coverage,271offering tailored protection for your high-tech investment. In addition to standard coverages, we offer:272- Safeguarding expensive battery systems273- Covering home charging equipment274- Adapting to the rapidly evolving world of electric vehicles275276Whether you're driving a Rivian, Nissan Leaf, Chevrolet Bolt,277or any other electric vehicle, our knowledgeable agents are ready to craft a policy278that gives you the confidence to enjoy every electron-powered mile.279</static_context>280"""281```282283Now that you have your static content, add at least 4-5 sample "good" interactions to guide Claude's responses. These examples should be representative of your ideal customer interaction and can include elements such as guardrails and tool calls.284285```python286EXAMPLES = """287Here are a few examples of how you can interact with customers:288289<example 1>290H: Hi, do you offer commercial insurance for small businesses?291292A: Ah! Unfortunately, we don't offer commercial insurance at this time.293However, we do provide a range of personal insurance products including car insurance and electric car insurance.294Would you like to know more about these offerings?295</example 1>296297<example 2>298H: Hi, what are your business hours?299300A: Our business hours are Monday to Friday, 9 AM to 5 PM EST.301If you have any questions or need assistance, feel free to reach out during those times.302How else can I assist you today?303</example 2>304305<example 3>306H: I'm shopping for car insurance. What does Acme offer?307308A: Great choice! Acme's car insurance policies typically include:309- Liability coverage: Protects you if you're at fault in an accident310- Collision coverage: Pays for damage to your car in an accident311- Comprehensive coverage: Covers non-collision incidents like theft or vandalism312- Medical payments coverage: Pays for medical expenses after an accident313- Uninsured/underinsured motorist coverage: Protects you from drivers with insufficient insurance314315We also offer optional coverages like rental car reimbursement, roadside assistance, and new car replacement.316Would you like more details on any of these coverages, or do you have specific questions about insuring your vehicle?317</example 3>318319<example 4>320H: I'd like to hear about your electric car insurance.321322A: Certainly! Our electric car insurance is specially designed to meet the unique needs of electric vehicle owners.323In addition to all the standard coverages we offer for regular cars, our electric car policies include:324325- Battery coverage: Protection against damage or degradation of your vehicle's expensive battery system326- Tech package: Covers repairs or replacement of advanced electronics and onboard computer systems327- Charging station coverage: Protects both home and portable charging equipment328- Cybersecurity protection: Coverage against hacking or digital threats329- Specialized roadside assistance: Includes flatbed towing to protect the vehicle's undercarriage330331We cover a wide range of electric vehicles, including popular models from Tesla, Rivian, Nissan, Chevrolet, and more.332Is there a specific electric vehicle you're looking to insure, or would you like more details on any of these coverages?333</example 4>334335<example 5>336H: I'd like to get a quote for my car insurance.337338A: Certainly! I'd be happy to help you get a quote for your car insurance.339To provide you with an accurate quote, I'll need to collect some information about your vehicle and the primary driver.340Let's start with the basics:3413421. What is the make and model of your vehicle?3432. What year was it manufactured?3443. Approximately how many miles have you driven?3454. What is the age of the primary driver?346347Once you provide this information, I'll use our quoting tool to generate a personalized insurance quote for you.348</example 5>349"""350```351352You will also want to include any important instructions outlining dos and don'ts for how Claude should interact with the customer. This may draw from brand guardrails or support policies.353354```python355ADDITIONAL_GUARDRAILS = """Please adhere to the following guardrails:3561. Only provide information about insurance types listed in our offerings.3572. If asked about an insurance type we don't offer, politely state358that we don't provide that service.3593. Do not speculate about future product offerings or company plans.3604. Don't make promises or enter into agreements it's not authorized to make.361You only provide information and guidance.3625. Do not mention any competitor's products or services.363"""364```365366Now combine all these sections into a single string to use as your prompt.367368```python369TASK_SPECIFIC_INSTRUCTIONS = " ".join(370    [371        STATIC_GREETINGS_AND_GENERAL,372        STATIC_CAR_INSURANCE,373        STATIC_ELECTRIC_CAR_INSURANCE,374        EXAMPLES,375        ADDITIONAL_GUARDRAILS,376    ]377)378```379380### Add dynamic and agentic capabilities with tool use381382Claude is capable of taking actions and retrieving information dynamically using client-side tool use functionality. Start by listing any external tools or APIs the prompt should use.383384For this example, start with one tool for calculating the quote.385386<Tip>387  As a reminder, this tool will not perform the actual calculation, it will just signal to the application that a tool should be used with whatever arguments specified.388</Tip>389390Add the model name, the tool definition, and a stub implementation to `config.py`:391392```python393import time394395MODEL = "claude-opus-5"396397TOOLS = [398    {399        "name": "get_quote",400        "description": "Calculate the insurance quote based on user input. Returned value is per month premium.",401        "input_schema": {402            "type": "object",403            "properties": {404                "make": {"type": "string", "description": "The make of the vehicle."},405                "model": {"type": "string", "description": "The model of the vehicle."},406                "year": {407                    "type": "integer",408                    "description": "The year the vehicle was manufactured.",409                },410                "mileage": {411                    "type": "integer",412                    "description": "The mileage on the vehicle.",413                },414                "driver_age": {415                    "type": "integer",416                    "description": "The age of the primary driver.",417                },418            },419            "required": ["make", "model", "year", "mileage", "driver_age"],420        },421    }422]423424425def get_quote(make, model, year, mileage, driver_age):426    """Returns the premium per month in USD"""427    # You can call an http endpoint or a database to get the quote.428    # Here, we simulate a delay of 1 seconds and return a fixed quote of 100.429    time.sleep(1)430    return 100431```432433### Deploy your prompts434435It's hard to know how well your prompt works without deploying it in a test production setting and [running evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests). Build a small application using the prompt, the Anthropic SDK, and Streamlit for a user interface.436437In a file called `chatbot.py` (or the equivalent module in your language), set up the ChatBot class, which will encapsulate the interactions with the Anthropic SDK.438439The class should have two main methods: one that calls the API to generate a message, and one that processes each incoming user input.440441<CodeGroup exclude="shell">442  ```python Python443  # In your chatbot.py, import these from the config.py you wrote above:444  # from config import IDENTITY, TOOLS, MODEL, get_quote445  from anthropic import Anthropic446  from dotenv import load_dotenv447448  load_dotenv()449450451  class ChatBot:452      def __init__(self, session_state):453          self.anthropic = Anthropic()454          self.session_state = session_state455456      def generate_message(457          self,458          messages,459          max_tokens,460      ):461          try:462              response = self.anthropic.messages.create(463                  model=MODEL,464                  system=IDENTITY,465                  max_tokens=max_tokens,466                  messages=messages,467                  tools=TOOLS,468              )469              return response470          except Exception as e:471              return {"error": str(e)}472473      def process_user_input(self, user_input):474          self.session_state.messages.append({"role": "user", "content": user_input})475476          response_message = self.generate_message(477              messages=self.session_state.messages,478              max_tokens=2048,479          )480481          if "error" in response_message:482              return f"An error occurred: {response_message['error']}"483484          if response_message.content[-1].type == "tool_use":485              tool_use = response_message.content[-1]486              func_name = tool_use.name487              func_params = tool_use.input488              tool_use_id = tool_use.id489490              result = self.handle_tool_use(func_name, func_params)491              self.session_state.messages.append(492                  {"role": "assistant", "content": response_message.content}493              )494              self.session_state.messages.append(495                  {496                      "role": "user",497                      "content": [498                          {499                              "type": "tool_result",500                              "tool_use_id": tool_use_id,501                              "content": f"{result}",502                          }503                      ],504                  }505              )506507              follow_up_response = self.generate_message(508                  messages=self.session_state.messages,509                  max_tokens=2048,510              )511512              if "error" in follow_up_response:513                  return f"An error occurred: {follow_up_response['error']}"514515              response_text = next(516                  (block.text for block in follow_up_response.content if block.type == "text"),517                  None,518              )519              if response_text is None:520                  raise Exception("An error occurred: Unexpected response type")521              self.session_state.messages.append(522                  {"role": "assistant", "content": response_text}523              )524              return response_text525526          text_block = next(527              (block for block in response_message.content if block.type == "text"), None528          )529          if text_block is not None:530              response_text = text_block.text531              self.session_state.messages.append(532                  {"role": "assistant", "content": response_text}533              )534              return response_text535536          raise Exception("An error occurred: Unexpected response type")537538      def handle_tool_use(self, func_name, func_params):539          if func_name == "get_quote":540              premium = get_quote(**func_params)541              return f"Quote generated: ${premium:.2f} per month"542543          raise Exception("An unexpected tool was used")544  ```545546  ```typescript TypeScript547  import Anthropic from "@anthropic-ai/sdk";548549  class ChatBot {550    // IDENTITY, MODEL, TOOLS, and getQuote mirror the config.py values defined551    // earlier in this guide (shown in Python).552    readonly anthropic = new Anthropic();553    readonly messages: Anthropic.MessageParam[] = [];554555    async generateMessage(556      messages: Anthropic.MessageParam[],557      maxTokens: number558    ): Promise<Anthropic.Message> {559      return this.anthropic.messages.create({560        model: MODEL,561        system: IDENTITY,562        max_tokens: maxTokens,563        messages,564        tools: TOOLS565      });566    }567568    async processUserInput(userInput: string): Promise<string> {569      this.messages.push({ role: "user", content: userInput });570571      const responseMessage = await this.generateMessage(this.messages, 2048);572573      const lastBlock = responseMessage.content.at(-1);574      if (lastBlock?.type === "tool_use") {575        const toolResult = this.handleToolUse(lastBlock.name, lastBlock.input);576577        this.messages.push({ role: "assistant", content: responseMessage.content });578        this.messages.push({579          role: "user",580          content: [{ type: "tool_result", tool_use_id: lastBlock.id, content: toolResult }]581        });582583        const followUpResponse = await this.generateMessage(this.messages, 2048);584585        const followUpBlock = followUpResponse.content.find(586          (block): block is Anthropic.TextBlock => block.type === "text"587        );588        if (!followUpBlock) {589          throw new Error("An error occurred: Unexpected response type");590        }591        this.messages.push({ role: "assistant", content: followUpBlock.text });592        return followUpBlock.text;593      }594595      const firstBlock = responseMessage.content.find(596        (block): block is Anthropic.TextBlock => block.type === "text"597      );598      if (firstBlock) {599        this.messages.push({ role: "assistant", content: firstBlock.text });600        return firstBlock.text;601      }602603      throw new Error("An error occurred: Unexpected response type");604    }605606    handleToolUse(toolName: string, toolInput: unknown): string {607      if (toolName === "get_quote") {608        // The SDK types tool_use.input as unknown; narrow it to the get_quote schema.609        if (610          toolInput === null ||611          typeof toolInput !== "object" ||612          !("make" in toolInput) || typeof toolInput.make !== "string" ||613          !("model" in toolInput) || typeof toolInput.model !== "string" ||614          !("year" in toolInput) || typeof toolInput.year !== "number" ||615          !("mileage" in toolInput) || typeof toolInput.mileage !== "number" ||616          !("driver_age" in toolInput) || typeof toolInput.driver_age !== "number"617        ) {618          throw new Error("An error occurred: Unexpected tool input");619        }620        const { make, model: vehicleModel, year, mileage, driver_age: driverAge } = toolInput;621622        const premium = getQuote(make, vehicleModel, year, mileage, driverAge);623        return `Quote generated: $${premium.toFixed(2)} per month`;624      }625626      throw new Error("An unexpected tool was used");627    }628  }629  ```630631  ```csharp C#632  using System.Text.Json;633  using Anthropic;634  using Anthropic.Models.Messages;635636  // Config.Model, Config.Identity, Config.Tools, and Config.GetQuote mirror637  // the config.py values defined earlier in this guide (shown in Python).638  public class ChatBot639  {640      private readonly AnthropicClient _anthropic = new();641642      public List<MessageParam> Messages { get; } = [];643644      public async Task<Message> GenerateMessage(List<MessageParam> messages, long maxTokens) =>645          await _anthropic.Messages.Create(646              new MessageCreateParams647              {648                  Model = Config.Model,649                  System = Config.Identity,650                  MaxTokens = maxTokens,651                  Messages = messages,652                  Tools = Config.Tools,653              }654          );655656      public async Task<string> ProcessUserInput(string userInput)657      {658          Messages.Add(new() { Role = Role.User, Content = userInput });659660          var responseMessage = await GenerateMessage(Messages, maxTokens: 2048);661662          if (responseMessage.Content[^1].TryPickToolUse(out var toolUse))663          {664              var toolResult = HandleToolUse(toolUse.Name, toolUse.Input);665666              Messages.Add(new()667              {668                  Role = Role.Assistant,669                  Content = responseMessage.Content670                      .Select(contentBlock => new ContentBlockParam(contentBlock.Json))671                      .ToList(),672              });673              Messages.Add(new()674              {675                  Role = Role.User,676                  Content = new List<ContentBlockParam>677                  {678                      new ToolResultBlockParam { ToolUseID = toolUse.ID, Content = toolResult },679                  },680              });681682              var followUpResponse = await GenerateMessage(Messages, maxTokens: 2048);683684              foreach (var block in followUpResponse.Content)685              {686                  if (block.TryPickText(out var followUpText))687                  {688                      Messages.Add(new() { Role = Role.Assistant, Content = followUpText.Text });689                      return followUpText.Text;690                  }691              }692693              throw new InvalidOperationException("An error occurred: Unexpected response type");694          }695696          foreach (var block in responseMessage.Content)697          {698              if (block.TryPickText(out var textBlock))699              {700                  Messages.Add(new() { Role = Role.Assistant, Content = textBlock.Text });701                  return textBlock.Text;702              }703          }704705          throw new InvalidOperationException("An error occurred: Unexpected response type");706      }707708      public string HandleToolUse(string funcName, IReadOnlyDictionary<string, JsonElement> funcParams)709      {710          if (funcName == "get_quote")711          {712              var premium = Config.GetQuote(713                  funcParams["make"].GetString()!,714                  funcParams["model"].GetString()!,715                  funcParams["year"].GetInt64(),716                  funcParams["mileage"].GetInt64(),717                  funcParams["driver_age"].GetInt64()718              );719              return $"Quote generated: ${premium:F2} per month";720          }721722          throw new ArgumentException("An unexpected tool was used");723      }724  }725  ```726727  ```go Go728  import (729  	"context"730  	"encoding/json"731  	"fmt"732733  	"github.com/anthropics/anthropic-sdk-go"734  )735736  // ChatBot wraps the Anthropic client and the conversation history. The737  // identity, model, tools, and getQuote values it uses mirror the config.py738  // definitions earlier in this guide (shown in Python).739  type ChatBot struct {740  	client   anthropic.Client741  	messages []anthropic.MessageParam742  }743744  func NewChatBot() *ChatBot {745  	return &ChatBot{client: anthropic.NewClient()}746  }747748  func (bot *ChatBot) GenerateMessage(ctx context.Context, messages []anthropic.MessageParam, maxTokens int64) (*anthropic.Message, error) {749  	return bot.client.Messages.New(ctx, anthropic.MessageNewParams{750  		Model:     model,751  		System:    []anthropic.TextBlockParam{{Text: identity}},752  		MaxTokens: maxTokens,753  		Messages:  messages,754  		Tools:     tools,755  	})756  }757758  func (bot *ChatBot) ProcessUserInput(ctx context.Context, userInput string) (string, error) {759  	bot.messages = append(bot.messages, anthropic.NewUserMessage(anthropic.NewTextBlock(userInput)))760761  	response, err := bot.GenerateMessage(ctx, bot.messages, 2048)762  	if err != nil {763  		return "", err764  	}765766  	lastBlock := response.Content[len(response.Content)-1]767  	if toolUse, ok := lastBlock.AsAny().(anthropic.ToolUseBlock); ok {768  		result, err := bot.HandleToolUse(toolUse.Name, toolUse.Input)769  		if err != nil {770  			return "", err771  		}772773  		bot.messages = append(bot.messages,774  			response.ToParam(),775  			anthropic.NewUserMessage(anthropic.NewToolResultBlock(toolUse.ID, result, false)),776  		)777778  		followUp, err := bot.GenerateMessage(ctx, bot.messages, 2048)779  		if err != nil {780  			return "", err781  		}782783  		for _, block := range followUp.Content {784  			if textBlock, ok := block.AsAny().(anthropic.TextBlock); ok {785  				bot.messages = append(bot.messages, anthropic.NewAssistantMessage(anthropic.NewTextBlock(textBlock.Text)))786  				return textBlock.Text, nil787  			}788  		}789  		return "", fmt.Errorf("an error occurred: unexpected response type")790  	}791792  	for _, block := range response.Content {793  		if textBlock, ok := block.AsAny().(anthropic.TextBlock); ok {794  			bot.messages = append(bot.messages, anthropic.NewAssistantMessage(anthropic.NewTextBlock(textBlock.Text)))795  			return textBlock.Text, nil796  		}797  	}798799  	return "", fmt.Errorf("an error occurred: unexpected response type")800  }801802  func (bot *ChatBot) HandleToolUse(toolName string, toolInput json.RawMessage) (string, error) {803  	if toolName != "get_quote" {804  		return "", fmt.Errorf("an unexpected tool was used: %s", toolName)805  	}806807  	var input struct {808  		Make      string `json:"make"`809  		Model     string `json:"model"`810  		Year      int    `json:"year"`811  		Mileage   int    `json:"mileage"`812  		DriverAge int    `json:"driver_age"`813  	}814  	if err := json.Unmarshal(toolInput, &input); err != nil {815  		return "", err816  	}817  	premium := getQuote(input.Make, input.Model, input.Year, input.Mileage, input.DriverAge)818  	return fmt.Sprintf("Quote generated: $%.2f per month", premium), nil819  }820821  ```822823  ```java Java824  import com.anthropic.client.AnthropicClient;825  import com.anthropic.client.okhttp.AnthropicOkHttpClient;826  import com.anthropic.core.JsonValue;827  import com.anthropic.models.messages.ContentBlock;828  import com.anthropic.models.messages.ContentBlockParam;829  import com.anthropic.models.messages.Message;830  import com.anthropic.models.messages.MessageCreateParams;831  import com.anthropic.models.messages.MessageParam;832  import com.anthropic.models.messages.ToolResultBlockParam;833  import com.anthropic.models.messages.ToolUseBlock;834835  // IDENTITY, MODEL, TOOLS, and getQuote mirror the config.py values defined836  // earlier in this guide (shown in Python).837  class ChatBot {838      final AnthropicClient anthropic;839      final List<MessageParam> messages;840841      ChatBot() {842          // Reads the API key from the ANTHROPIC_API_KEY environment variable843          this.anthropic = AnthropicOkHttpClient.fromEnv();844          this.messages = new ArrayList<>();845      }846847      Message generateMessage(List<MessageParam> messages, long maxTokens) {848          return anthropic.messages().create(MessageCreateParams.builder()849                  .model(MODEL)850                  .system(IDENTITY)851                  .maxTokens(maxTokens)852                  .messages(messages)853                  .tools(TOOLS)854                  .build());855      }856857      String processUserInput(String userInput) {858          messages.add(MessageParam.builder()859                  .role(MessageParam.Role.USER)860                  .content(userInput)861                  .build());862863          Message responseMessage = generateMessage(messages, 2048);864865          List<ContentBlock> content = responseMessage.content();866          ContentBlock lastBlock = content.getLast();867          if (lastBlock.isToolUse()) {868              ToolUseBlock toolUse = lastBlock.asToolUse();869              Map<String, JsonValue> toolInput =870                      (Map<String, JsonValue>) toolUse._input().asObject().orElseThrow();871              String result = handleToolUse(toolUse.name(), toolInput);872873              messages.add(MessageParam.builder()874                      .role(MessageParam.Role.ASSISTANT)875                      .contentOfBlockParams(content.stream().map(ContentBlock::toParam).toList())876                      .build());877              messages.add(MessageParam.builder()878                      .role(MessageParam.Role.USER)879                      .contentOfBlockParams(List.of(ContentBlockParam.ofToolResult(880                              ToolResultBlockParam.builder()881                                      .toolUseId(toolUse.id())882                                      .content(result)883                                      .build())))884                      .build());885886              Message followUpResponse = generateMessage(messages, 2048);887888              ContentBlock followUpBlock = followUpResponse.content().stream()889                      .filter(ContentBlock::isText)890                      .findFirst()891                      .orElseThrow(() -> new IllegalStateException("An error occurred: Unexpected response type"));892              String responseText = followUpBlock.asText().text();893              messages.add(MessageParam.builder()894                      .role(MessageParam.Role.ASSISTANT)895                      .content(responseText)896                      .build());897              return responseText;898          } else {899              ContentBlock textBlock = content.stream()900                      .filter(ContentBlock::isText)901                      .findFirst()902                      .orElseThrow(() -> new IllegalStateException("An error occurred: Unexpected response type"));903              String responseText = textBlock.asText().text();904              messages.add(MessageParam.builder()905                      .role(MessageParam.Role.ASSISTANT)906                      .content(responseText)907                      .build());908              return responseText;909          }910      }911912      String handleToolUse(String funcName, Map<String, JsonValue> funcParams) {913          return switch (funcName) {914              case "get_quote" -> {915                  double premium = getQuote(916                          funcParams.get("make").asStringOrThrow(),917                          funcParams.get("model").asStringOrThrow(),918                          ((Number) funcParams.get("year").asNumber().orElseThrow()).longValue(),919                          ((Number) funcParams.get("mileage").asNumber().orElseThrow()).longValue(),920                          ((Number) funcParams.get("driver_age").asNumber().orElseThrow()).longValue());921                  yield "Quote generated: $%.2f per month".formatted(premium);922              }923              default -> throw new IllegalArgumentException("An unexpected tool was used");924          };925      }926  }927  ```928929  ```php PHP930  use Anthropic\Client;931  use Anthropic\Messages\Message;932  use Anthropic\Messages\MessageParam;933  use Anthropic\Messages\TextBlock;934  use Anthropic\Messages\ToolResultBlockParam;935  use Anthropic\Messages\ToolUseBlock;936937  class ChatBot938  {939      // MODEL, IDENTITY, TOOLS, and get_quote() mirror the config.py values940      // defined earlier in this guide (shown in Python).941942      /** @var list<MessageParam> */943      public private(set) array $messages = [];944945      public function __construct(946          private readonly Client $anthropic = new Client(),947      ) {}948949      /**950       * @param list<MessageParam> $messages951       */952      public function generateMessage(array $messages, int $maxTokens): Message953      {954          return $this->anthropic->messages->create(955              model: MODEL,956              system: IDENTITY,957              maxTokens: $maxTokens,958              messages: $messages,959              tools: TOOLS,960          );961      }962963      public function processUserInput(string $userInput): string964      {965          $this->messages[] = MessageParam::with(role: 'user', content: $userInput);966967          $responseMessage = $this->generateMessage($this->messages, maxTokens: 2048);968969          $content = $responseMessage->content;970          $lastBlock = array_last($content);971972          if ($lastBlock instanceof ToolUseBlock) {973              $toolResult = $this->handleToolUse($lastBlock->name, $lastBlock->input);974975              $this->messages[] = MessageParam::with(role: 'assistant', content: $content);976              $this->messages[] = MessageParam::with(977                  role: 'user',978                  content: [979                      ToolResultBlockParam::with(toolUseID: $lastBlock->id, content: $toolResult),980                  ],981              );982983              $followUpResponse = $this->generateMessage($this->messages, maxTokens: 2048);984985              $firstBlock = array_find(986                  $followUpResponse->content,987                  static fn ($block): bool => $block instanceof TextBlock,988              );989              if (!$firstBlock instanceof TextBlock) {990                  throw new RuntimeException('An error occurred: Unexpected response type');991              }992993              $this->messages[] = MessageParam::with(role: 'assistant', content: $firstBlock->text);994995              return $firstBlock->text;996          }997998          $firstBlock = array_find($content, static fn ($block): bool => $block instanceof TextBlock);999          if ($firstBlock instanceof TextBlock) {1000              $this->messages[] = MessageParam::with(role: 'assistant', content: $firstBlock->text);10011002              return $firstBlock->text;1003          }10041005          throw new RuntimeException('An error occurred: Unexpected response type');1006      }10071008      /**1009       * @param array<string, mixed> $funcParams1010       */1011      private function handleToolUse(string $funcName, array $funcParams): string1012      {1013          if ($funcName === 'get_quote') {1014              $premium = get_quote(...$funcParams);10151016              return sprintf('Quote generated: $%.2f per month', $premium);1017          }10181019          throw new RuntimeException('An unexpected tool was used');1020      }1021  }1022  ```10231024  ```ruby Ruby1025  # IDENTITY, MODEL, TOOLS, and get_quote mirror the config.py values defined1026  # earlier in this guide (shown in Python).1027  require "anthropic"10281029  class ChatBot1030    attr_reader :messages10311032    def initialize1033      @anthropic = Anthropic::Client.new1034      @messages = []1035    end10361037    def generate_message(messages, max_tokens)1038      @anthropic.messages.create(1039        model: MODEL,1040        system_: IDENTITY,1041        max_tokens:,1042        messages:,1043        tools: TOOLS1044      )1045    end10461047    def process_user_input(user_input)1048      @messages << {role: "user", content: user_input}10491050      response_message = generate_message(@messages, 2048)10511052      case response_message.content1053      in [*, Anthropic::ToolUseBlock => tool_use]1054        result = handle_tool_use(tool_use.name, tool_use.input)1055        @messages << {role: "assistant", content: response_message.content}1056        @messages << {1057          role: "user",1058          content: [{type: "tool_result", tool_use_id: tool_use.id, content: result}]1059        }10601061        follow_up_response = generate_message(@messages, 2048)10621063        case follow_up_response.content1064        in [*, Anthropic::TextBlock => text_block, *]1065          @messages << {role: "assistant", content: text_block.text}1066          text_block.text1067        else1068          raise "An error occurred: Unexpected response type"1069        end1070      in [*, Anthropic::TextBlock => text_block, *]1071        @messages << {role: "assistant", content: text_block.text}1072        text_block.text1073      else1074        raise "An error occurred: Unexpected response type"1075      end1076    end10771078    def handle_tool_use(tool_name, tool_input)1079      raise "An unexpected tool was used" unless tool_name == "get_quote"10801081      premium = get_quote(**tool_input)1082      format("Quote generated: $%.2f per month", premium)1083    end1084  end1085  ```1086</CodeGroup>10871088### Build your user interface10891090Test deploying this code with Streamlit using a main method. This `main()` function sets up a Streamlit-based chat interface. Streamlit is a Python framework, so this part of the walkthrough is shown in Python only; the ChatBot class above is the piece you can port to any language.10911092Do this in a file called `app.py`10931094```python1095import streamlit as st1096from chatbot import ChatBot1097from config import TASK_SPECIFIC_INSTRUCTIONS109810991100def main():1101    st.title("Chat with Eva, Acme Insurance Company's Assistant🤖")11021103    if "messages" not in st.session_state:1104        st.session_state.messages = [1105            {"role": "user", "content": TASK_SPECIFIC_INSTRUCTIONS},1106            {"role": "assistant", "content": "Understood"},1107        ]11081109    chatbot = ChatBot(st.session_state)11101111    # Display user and assistant messages skipping the first two1112    for message in st.session_state.messages[2:]:1113        # ignore tool use blocks1114        if isinstance(message["content"], str):1115            with st.chat_message(message["role"]):1116                st.markdown(message["content"])11171118    if user_msg := st.chat_input("Type your message here..."):1119        st.chat_message("user").markdown(user_msg)11201121        with st.chat_message("assistant"):1122            with st.spinner("Eva is thinking..."):1123                response_placeholder = st.empty()1124                full_response = chatbot.process_user_input(user_msg)1125                response_placeholder.markdown(full_response)112611271128if __name__ == "__main__":1129    main()1130```11311132Run the program with:11331134```bash1135streamlit run app.py1136```11371138### Evaluate your prompts11391140Prompting often requires testing and optimization for it to be production ready. To determine the readiness of your solution, evaluate the chatbot performance using a systematic process combining quantitative and qualitative methods. Creating a [strong empirical evaluation](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests#build-evaluations) based on your defined success criteria will allow you to optimize your prompts.11411142### Improve performance11431144In complex scenarios, it may be helpful to consider additional strategies to improve performance beyond standard [prompt engineering techniques](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview) & [guardrail implementation strategies](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations). Here are some common scenarios:11451146#### Reduce long context latency with RAG11471148When dealing with large amounts of static and dynamic context, including all information in the prompt can lead to high costs, slower response times, and reaching context window limits. In this scenario, implementing Retrieval Augmented Generation (RAG) techniques can improve performance and efficiency.11491150By using [embedding models like Voyage](https://platform.claude.com/docs/en/build-with-claude/embeddings) to convert information into vector representations, you can create a more scalable and responsive system. This approach allows for dynamic retrieval of relevant information based on the current query, rather than including all possible context in every prompt.11511152Implementing RAG for support use cases has been shown to increase accuracy, reduce response times, and reduce API costs in systems with extensive context requirements. See the [RAG recipe](https://platform.claude.com/cookbook/capabilities-retrieval-augmented-generation-guide) for a worked example.11531154#### Integrate real-time data with tool use11551156When dealing with queries that require real-time information, such as account balances or policy details, embedding-based RAG approaches are not sufficient. Instead, tool use can enhance your chatbot's ability to provide accurate, real-time responses. For example, you can use tool use to look up customer information, retrieve order details, and cancel orders on behalf of the customer.11571158This approach, [outlined in the tool use: customer service agent recipe](https://platform.claude.com/cookbook/tool-use-customer-service-agent), lets you integrate live data into Claude's responses and provide a more personalized and efficient customer experience.11591160#### Strengthen input and output guardrails11611162When deploying a chatbot, especially in customer service scenarios, it's important to prevent risks associated with misuse, out-of-scope queries, and inappropriate responses. While Claude is inherently resilient to such scenarios, here are additional steps to strengthen your chatbot guardrails:11631164* [Reduce hallucination](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations): Implement fact-checking mechanisms and [citations](https://platform.claude.com/cookbook/misc-using-citations) to ground responses in provided information.1165* Cross-check information: Verify that the agent's responses align with your company's policies and known facts.1166* Avoid contractual commitments: Ensure the agent doesn't make promises or enter into agreements it's not authorized to make.1167* [Mitigate jailbreaks](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks): Use methods like harmlessness screens and input validation to prevent users from exploiting model vulnerabilities, aiming to generate inappropriate content.1168* Avoid mentioning competitors: Implement a competitor mention filter to maintain brand focus and not mention any competitor's products or services.1169* [Increase output consistency](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/increase-consistency): Prevent Claude from changing style or going out of character, even during long, complex interactions.1170* Remove Personally Identifiable Information (PII): Unless explicitly required and authorized, strip out any PII from responses.11711172#### Reduce perceived response time with streaming11731174When dealing with potentially lengthy responses, implementing streaming can improve user engagement and satisfaction. In this scenario, users receive the answer progressively instead of waiting for the entire response to be generated.11751176Here is how to implement streaming:117711781. Use the [Anthropic Streaming API](https://platform.claude.com/docs/en/build-with-claude/streaming) to support streaming responses.11792. Set up your frontend to handle incoming chunks of text.11803. Display each chunk as it arrives, simulating real-time typing.11814. Implement a mechanism to save the full response, allowing users to view it if they navigate away and return.11821183In some cases, streaming enables the use of more advanced models with higher base latencies, as the progressive display mitigates the impact of longer processing times.11841185#### Scale your chatbot11861187As the complexity of your chatbot grows, your application architecture can evolve to match. Before you add further layers to your architecture, consider the following less exhaustive options:11881189* Ensure that you are making the most out of your prompts and optimizing through prompt engineering. Use the [prompt engineering guides](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview) to write the most effective prompts.1190* Add additional [tools](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview) to the prompt (which can include [prompt chains](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices#chain-complex-prompts)) and see if you can achieve the functionality required.11911192If your chatbot handles incredibly varied tasks, you may want to consider adding a [separate intent classifier](https://platform.claude.com/cookbook/capabilities-classification-guide) to route the initial customer query. For the existing application, this would involve creating a decision tree that would route customer queries through the classifier and then to specialized conversations (with their own set of tools and system prompts). Note, this method requires an additional call to Claude that can increase latency.11931194### Integrate Claude into your support workflow11951196While these examples have focused on Python functions callable within a Streamlit environment, deploying Claude for real-time support chatbot requires an API service.11971198Here's how you can approach this:119912001. Create an API wrapper: Develop a simple API wrapper around your classification function. For example, you can use Flask API or Fast API to wrap your code into a HTTP Service. Your HTTP service could accept the user input and return the Assistant response in its entirety. Thus, your service could have the following characteristics:12011202   * Server-Sent Events (SSE): SSE allows for real-time streaming of responses from the server to the client. This provides a smooth, interactive experience when working with LLMs.1203   * Caching: Implementing caching can improve response times and reduce unnecessary API calls.1204   * Context retention: Maintaining context when a user navigates away and returns is important for continuity in conversations.120512062. Build a web interface: Implement a user-friendly web UI for interacting with the Claude-powered agent.12071208## Next steps12091210<CardGroup cols={2}>1211  <Card title="Tool use" icon="wrench" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview">1212    Give Claude access to your APIs so it can take action on behalf of customers.1213  </Card>12141215  <Card title="Develop tests" icon="check" href="https://platform.claude.com/docs/en/test-and-evaluate/develop-tests">1216    Build evaluations to measure your support agent against the success criteria you defined.1217  </Card>12181219  <Card title="Streaming" icon="bolt" href="https://platform.claude.com/docs/en/build-with-claude/streaming">1220    Stream responses so customers see answers as they generate.1221  </Card>12221223  <Card title="Prompt engineering" icon="lightbulb" href="https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview">1224    Refine your system prompt and examples for better task performance.1225  </Card>1226</CardGroup>1227
Files main
Menu

Theme