Repository Public
1# Predicted Outputs23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45**Predicted Outputs** enable you to speed up API responses from [Chat Completions](https://developers.openai.com/api/reference/resources/chat) when many of the output tokens are known ahead of time. This is most common when you are regenerating a text or code file with minor modifications. You can provide your prediction using the [`prediction` request parameter in Chat Completions](https://developers.openai.com/api/reference/resources/chat#chat-create-prediction).67Predicted Outputs are available today using the latest `gpt-4o`, `gpt-4o-mini`, `gpt-4.1`, `gpt-4.1-mini`, and `gpt-4.1-nano` models. Read on to learn how to use Predicted Outputs to reduce latency in your applications.89## Code refactoring example1011Predicted Outputs are particularly useful for regenerating text documents and code files with small modifications. Let's say you want the [GPT-4o model](https://developers.openai.com/api/docs/models#gpt-4o) to refactor a piece of JavaScript code, and convert the `username` property of the `User` class to be `email` instead:1213```javascript14class User {15 firstName = "";16 lastName = "";17 username = "";18}1920export default User;21```222324Most of the file will be unchanged, except for line 4 above. If you use the current text of the code file as your prediction, you can regenerate the entire file with lower latency. These time savings add up quickly for larger files.2526Below is an example of using the `prediction` parameter in our SDKs to predict that the final output of the model will be very similar to our original code file, which we use as the prediction text.2728Refactor a JavaScript class with a Predicted Output2930```javascript31import OpenAI from "openai";3233const code = `34class User {35 firstName = "";36 lastName = "";37 username = "";38}3940export default User;41`.trim();4243const openai = new OpenAI();4445const refactorPrompt = `46Replace the "username" property with an "email" property. Respond only47with code, and with no markdown formatting.48`;4950const completion = await openai.chat.completions.create({51 model: "gpt-4.1",52 messages: [53 {54 role: "user",55 content: refactorPrompt,56 },57 {58 role: "user",59 content: code,60 },61 ],62 store: true,63 prediction: {64 type: "content",65 content: code,66 },67});6869// Inspect returned data70console.log(completion);71console.log(completion.choices[0].message.content);72```7374```python75from openai import OpenAI7677code = """78class User {79 firstName = "";80 lastName = "";81 username = "";82}8384export default User;85""".strip()8687refactor_prompt = """88Replace the "username" property with an "email" property. Respond only89with code, and with no markdown formatting.90"""9192client = OpenAI()9394completion = client.chat.completions.create(95 model="gpt-4.1",96 messages=[97 {"role": "user", "content": refactor_prompt},98 {"role": "user", "content": code},99 ],100 prediction={"type": "content", "content": code},101)102103print(completion)104print(completion.choices[0].message.content)105```106107```go108package main109110import (111 "context"112 "fmt"113 "strings"114115 "github.com/openai/openai-go/v3"116 "github.com/openai/openai-go/v3/shared"117)118119func main() {120 client := openai.NewClient()121 code := strings.TrimSpace(`122class User {123 firstName = "";124 lastName = "";125 username = "";126}127128export default User;129`)130 refactorPrompt := strings.TrimSpace(`131Replace the "username" property with an "email" property. Respond only132with code, and with no markdown formatting.133`)134 completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{135 Model: shared.ChatModelGPT4_1,136 Messages: []openai.ChatCompletionMessageParamUnion{137 openai.UserMessage(refactorPrompt),138 openai.UserMessage(code),139 },140 Store: openai.Bool(true),141 Prediction: openai.ChatCompletionPredictionContentParam{142 Content: openai.ChatCompletionPredictionContentContentUnionParam{OfString: openai.String(code)},143 },144 })145 if err != nil {146 panic(err)147 }148 fmt.Println(completion.Choices[0].Message.Content)149}150```151152```java153import com.openai.client.OpenAIClient;154import com.openai.client.okhttp.OpenAIOkHttpClient;155import com.openai.models.chat.completions.ChatCompletionCreateParams;156import com.openai.models.chat.completions.ChatCompletionPredictionContent;157158String code =159 """160 class User {161 firstName: string = "";162 lastName: string = "";163 username: string = "";164 }165166 export default User;167 """;168String refactorPrompt =169 "Replace the \"username\" property with an \"email\" property. "170 + "Respond only with code, and with no markdown formatting.";171172ChatCompletionCreateParams params =173 ChatCompletionCreateParams.builder()174 .model("gpt-4.1")175 .addUserMessage(refactorPrompt)176 .addUserMessage(code)177 .prediction(ChatCompletionPredictionContent.builder().content(code).build())178 .store(true)179 .build();180181client.chat().completions().create(params).choices().stream()182 .flatMap(choice -> choice.message().content().stream())183 .forEach(System.out::println);184```185186```csharp187using OpenAI.Chat;188#pragma warning disable OPENAI001189190string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;191string model = "gpt-4.1";192ChatClient client = new(model, key);193194string code =195 """196 class User {197 firstName = "";198 lastName = "";199 username = "";200 }201202 export default User;203 """;204ChatCompletionOptions options = new()205{206 OutputPrediction = ChatOutputPrediction.CreateStaticContentPrediction(code),207};208ChatCompletion completion = await client.CompleteChatAsync(209 [210 new UserChatMessage(211 "Replace the username property with an email property. Respond only with code, and with no markdown formatting."212 ),213 new UserChatMessage(code),214 ],215 options216);217218Console.WriteLine(completion.Content[0].Text);219```220221```ruby222require "openai"223224client = OpenAI::Client.new225code = <<~CODE226 class User {227 firstName: string = "";228 lastName: string = "";229 username: string = "";230 }231232 export default User;233CODE234refactor_prompt = <<~PROMPT235 Replace the "username" property with an "email" property. Respond only236 with code, and with no markdown formatting.237PROMPT238completion = client.chat.completions.create(239 model: "gpt-4.1",240 messages: [241 {242 role: :user,243 content: refactor_prompt244 },245 {246 role: :user,247 content: code248 }249 ],250 prediction: {251 type: :content,252 content: code253 },254 store: true255)256257puts(completion.choices.fetch(0).message.content)258```259260```bash261curl https://api.openai.com/v1/chat/completions \262 -H "Content-Type: application/json" \263 -H "Authorization: Bearer $OPENAI_API_KEY" \264 -d '{265 "model": "gpt-4.1",266 "messages": [267 {268 "role": "user",269 "content": "Replace the username property with an email property. Respond only with code, and with no markdown formatting."270 },271 {272 "role": "user",273 "content": "$CODE_CONTENT_HERE"274 }275 ],276 "prediction": {277 "type": "content",278 "content": "$CODE_CONTENT_HERE"279 }280 }'281```282283284In addition to the refactored code, an abridged model response without the `choices` field contains usage data like this:285286```json287{288 "id": "chatcmpl-xxx",289 "object": "chat.completion",290 "created": 1786652188,291 "model": "gpt-4.1-2025-04-14",292 "usage": {293 "prompt_tokens": 59,294 "completion_tokens": 24,295 "total_tokens": 83,296 "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 },297 "completion_tokens_details": {298 "reasoning_tokens": 0,299 "audio_tokens": 0,300 "accepted_prediction_tokens": 14,301 "rejected_prediction_tokens": 2302 }303 },304 "system_fingerprint": "fp_6ddb4f7408"305}306```307308Note both the `accepted_prediction_tokens` and `rejected_prediction_tokens` in the `usage` object. In this example, 14 tokens from the prediction were used to speed up the response, while 2 were rejected.309310Note that any rejected tokens are still billed like other completion tokens311 generated by the API, so Predicted Outputs can introduce higher costs for your312 requests.313314## Streaming example315316The latency gains of Predicted Outputs are even greater when you use streaming for API responses. Here is an example of the same code refactoring use case, but using streaming in the OpenAI SDKs instead.317318Predicted Outputs with streaming319320```javascript321import OpenAI from "openai";322323const code = `324class User {325 firstName = "";326 lastName = "";327 username = "";328}329330export default User;331`.trim();332333const openai = new OpenAI();334335const refactorPrompt = `336Replace the "username" property with an "email" property. Respond only337with code, and with no markdown formatting.338`;339340const completion = await openai.chat.completions.create({341 model: "gpt-4.1",342 messages: [343 {344 role: "user",345 content: refactorPrompt,346 },347 {348 role: "user",349 content: code,350 },351 ],352 store: true,353 prediction: {354 type: "content",355 content: code,356 },357 stream: true,358});359360// Inspect returned data361for await (const chunk of completion) {362 process.stdout.write(chunk.choices[0]?.delta?.content || "");363}364```365366```python367from openai import OpenAI368369code = """370class User {371 firstName = "";372 lastName = "";373 username = "";374}375376export default User;377""".strip()378379refactor_prompt = """380Replace the "username" property with an "email" property. Respond only381with code, and with no markdown formatting.382"""383384client = OpenAI()385386stream = client.chat.completions.create(387 model="gpt-4.1",388 messages=[389 {"role": "user", "content": refactor_prompt},390 {"role": "user", "content": code},391 ],392 prediction={"type": "content", "content": code},393 stream=True,394)395396for chunk in stream:397 if chunk.choices[0].delta.content is not None:398 print(chunk.choices[0].delta.content, end="")399```400401```go402package main403404import (405 "context"406 "fmt"407 "strings"408409 "github.com/openai/openai-go/v3"410 "github.com/openai/openai-go/v3/shared"411)412413func main() {414 client := openai.NewClient()415 code := strings.TrimSpace(`416class User {417 firstName = "";418 lastName = "";419 username = "";420}421422export default User;423`)424 refactorPrompt := strings.TrimSpace(`425Replace the "username" property with an "email" property. Respond only426with code, and with no markdown formatting.427`)428 stream := client.Chat.Completions.NewStreaming(context.Background(), openai.ChatCompletionNewParams{429 Model: shared.ChatModelGPT4_1,430 Messages: []openai.ChatCompletionMessageParamUnion{431 openai.UserMessage(refactorPrompt),432 openai.UserMessage(code),433 },434 Store: openai.Bool(true),435 Prediction: openai.ChatCompletionPredictionContentParam{436 Content: openai.ChatCompletionPredictionContentContentUnionParam{OfString: openai.String(code)},437 },438 })439 for stream.Next() {440 if len(stream.Current().Choices) > 0 {441 fmt.Print(stream.Current().Choices[0].Delta.Content)442 }443 }444 if err := stream.Err(); err != nil {445 panic(err)446 }447}448```449450```java451import com.openai.client.OpenAIClient;452import com.openai.client.okhttp.OpenAIOkHttpClient;453import com.openai.core.http.StreamResponse;454import com.openai.models.chat.completions.ChatCompletionChunk;455import com.openai.models.chat.completions.ChatCompletionCreateParams;456import com.openai.models.chat.completions.ChatCompletionPredictionContent;457458String code =459 """460 class User {461 firstName: string = "";462 lastName: string = "";463 username: string = "";464 }465466 export default User;467 """;468String refactorPrompt =469 "Replace the \"username\" property with an \"email\" property. "470 + "Respond only with code, and with no markdown formatting.";471472ChatCompletionCreateParams params =473 ChatCompletionCreateParams.builder()474 .model("gpt-4.1")475 .addUserMessage(refactorPrompt)476 .addUserMessage(code)477 .prediction(ChatCompletionPredictionContent.builder().content(code).build())478 .store(true)479 .build();480481try (StreamResponse<ChatCompletionChunk> stream =482 client.chat().completions().createStreaming(params)) {483 stream.stream()484 .flatMap(chunk -> chunk.choices().stream())485 .flatMap(choice -> choice.delta().content().stream())486 .forEach(System.out::print);487}488```489490```csharp491using OpenAI.Chat;492#pragma warning disable OPENAI001493494string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;495string model = "gpt-4.1";496ChatClient client = new(model, key);497498string code =499 """500 class User {501 firstName = "";502 lastName = "";503 username = "";504 }505506 export default User;507 """;508ChatCompletionOptions options = new()509{510 OutputPrediction = ChatOutputPrediction.CreateStaticContentPrediction(code),511};512513await foreach (514 StreamingChatCompletionUpdate update in client.CompleteChatStreamingAsync(515 [516 new UserChatMessage(517 "Replace the username property with an email property. Respond only with code, and with no markdown formatting."518 ),519 new UserChatMessage(code),520 ],521 options522 )523)524{525 foreach (ChatMessageContentPart part in update.ContentUpdate)526 {527 Console.Write(part.Text);528 }529}530```531532```ruby533require "openai"534535client = OpenAI::Client.new536code = <<~CODE537 class User {538 firstName: string = "";539 lastName: string = "";540 username: string = "";541 }542543 export default User;544CODE545refactor_prompt = <<~PROMPT546 Replace the "username" property with an "email" property. Respond only547 with code, and with no markdown formatting.548PROMPT549stream = client.chat.completions.stream(550 model: "gpt-4.1",551 messages: [552 {553 role: :user,554 content: refactor_prompt555 },556 {557 role: :user,558 content: code559 }560 ],561 prediction: {562 type: :content,563 content: code564 },565 store: true566)567568stream.text.each { |text| print(text) }569```570571572## Position of predicted text in response573574When providing prediction text, your prediction can appear anywhere within the generated response, and still provide latency reduction for the response. Let's say your predicted text is the simple [Hono](https://hono.dev/) server shown below:575576```javascript577import { serve } from "@hono/node-server";578import { serveStatic } from "@hono/node-server/serve-static";579import { Hono } from "hono";580581const app = new Hono();582583app.get("/api", (c) => {584 return c.text("Hello Hono!");585});586587// You will need to build the client code first: `pnpm run ui:build`.588app.use(589 "/*",590 serveStatic({591 rewriteRequestPath: (path) => `./dist${path}`,592 })593);594595const port = 3000;596console.log(`Server is running on port ${port}`);597598serve({599 fetch: app.fetch,600 port,601});602```603604605You could prompt the model to regenerate the file with a prompt like:606607```608Add a get route to this application that responds with609the text "hello world". Generate the entire application610file again with this route added, and with no other611markdown formatting.612```613614The response to the prompt might look something like this:615616```javascript617import { serve } from "@hono/node-server";618import { serveStatic } from "@hono/node-server/serve-static";619import { Hono } from "hono";620621const app = new Hono();622623app.get("/api", (c) => {624 return c.text("Hello Hono!");625});626627app.get("/hello", (c) => {628 return c.text("hello world");629});630631// You will need to build the client code first: `pnpm run ui:build`.632app.use(633 "/*",634 serveStatic({635 rewriteRequestPath: (path) => `./dist${path}`,636 })637);638639const port = 3000;640console.log(`Server is running on port ${port}`);641642serve({643 fetch: app.fetch,644 port,645});646```647648649An abridged model response without the `choices` field would still show accepted prediction tokens, even though the prediction text appeared both before and after the new content added to the response:650651```json652{653 "id": "chatcmpl-xxx",654 "object": "chat.completion",655 "created": 1731014771,656 "model": "gpt-4o-2024-08-06",657 "usage": {658 "prompt_tokens": 203,659 "completion_tokens": 159,660 "total_tokens": 362,661 "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 },662 "completion_tokens_details": {663 "reasoning_tokens": 0,664 "audio_tokens": 0,665 "accepted_prediction_tokens": 60,666 "rejected_prediction_tokens": 0667 }668 },669 "system_fingerprint": "fp_9ee9e968ea"670}671```672673This time, there were no rejected prediction tokens, because the entire content of the file we predicted was used in the final response. Nice! 🔥674675## Limitations676677When using Predicted Outputs, you should consider the following factors and limitations.678679- Predicted Outputs are only supported with the GPT-4o, GPT-4o-mini, GPT-4.1, GPT-4.1-mini, and GPT-4.1-nano series of models.680- When providing a prediction, any tokens provided that are not part of the final completion are still charged at completion token rates. See the [`rejected_prediction_tokens` property of the `usage` object](https://developers.openai.com/api/reference/resources/chat#chat/object-usage) to see how many tokens are not used in the final response.681- The following [API parameters](https://developers.openai.com/api/reference/resources/chat) are not supported when using Predicted Outputs:682 - `n`: values higher than 1 are not supported683 - `logprobs`: not supported684 - `presence_penalty`: values greater than 0 are not supported685 - `frequency_penalty`: values greater than 0 are not supported686 - `audio`: Predicted Outputs are not compatible with [audio inputs and outputs](https://developers.openai.com/api/docs/guides/audio)687 - `modalities`: Only `text` modalities are supported688 - `max_completion_tokens`: not supported689 - `tools`: Function calling is not currently supported with Predicted Outputs