SPB Git

Repository Public

collections-search.md 20.2 KB 482 lines markdown

1<!-- source: https://docs.x.ai/developers/tools/collections-search (from https://docs.x.ai/llms-full.txt, retrieved 2026-09-19T05:28:55Z) -->2#### Tools34# Collections Search Tool56The collections search tool enables Grok to search through your uploaded knowledge bases (collections), allowing it to retrieve relevant information from your documents to provide more accurate and contextually relevant responses. This tool is particularly powerful for analyzing complex documents like financial reports, legal contracts, or technical documentation, where Grok can autonomously search through multiple documents and synthesize information to answer sophisticated analytical questions.78For an introduction to Collections, please check out the [Collections documentation](/developers/files/collections).910## Key Capabilities1112* **Document Retrieval**: Search across uploaded files and collections to find relevant information13* **Semantic Search**: Find documents based on meaning and context, not just keywords14* **Knowledge Base Integration**: Seamlessly integrate your proprietary data with Grok's reasoning15* **RAG Applications**: Power retrieval-augmented generation workflows16* **Multi-format Support**: Search across PDFs, text files, CSVs, and other supported formats1718## When to Use Collections Search1920The collections search tool is particularly valuable for:2122* **Enterprise Knowledge Bases**: When you need Grok to reference internal documents and policies23* **Financial Analysis**: Analyzing SEC filings, earnings reports, and financial statements across multiple documents24* **Customer Support**: Building chatbots that can answer questions based on your product documentation25* **Research & Due Diligence**: Synthesizing information from academic papers, technical reports, or industry analyses26* **Compliance & Legal**: Ensuring responses are grounded in your official guidelines and regulations27* **Personal Knowledge Management**: Organizing and querying your personal document collections2829## SDK Support3031The collections search tool is available across multiple SDKs and APIs with different naming conventions:3233| SDK/API | Tool Name | Description |34|---------|-----------|-------------|35| xAI SDK | `collections_search` | Native xAI SDK implementation |36| OpenAI Responses API | `file_search` | Compatible with OpenAI's API format |3738This tool is also supported in all Responses API compatible SDKs.3940## Implementation Example4142### End-to-End Financial Analysis Example4344This comprehensive example demonstrates analyzing Tesla's SEC filings using the collections search tool. It covers:45461. Creating a collection for document storage472. Uploading multiple financial documents concurrently (10-Q and 10-K filings)483. Using Grok with collections search to analyze and synthesize information across documents in an agentic manner494. Enabling code execution to allow the model to perform calculations and mathematical analysis effectively should it be needed.505. Receiving cited responses and tool usage information5152This pattern is applicable to any document analysis workflow where you need to search through and reason over multiple documents.5354```pythonXAI55import asyncio56import os5758import httpx5960from xai_sdk import AsyncClient61from xai_sdk.chat import user62from xai_sdk.proto import collections_pb263from xai_sdk.tools import code_execution, collections_search6465TESLA_10_Q_PDF_URL = "https://ir.tesla.com/_flysystem/s3/sec/000162828025045968/tsla-20250930-gen.pdf"66TESLA_10_K_PDF_URL = "https://ir.tesla.com/_flysystem/s3/sec/000162828025003063/tsla-20241231-gen.pdf"67async def main():68    client = AsyncClient(api_key=os.getenv("XAI_API_KEY"), management_api_key=os.getenv("XAI_MANAGEMENT_API_KEY"))6970    # Step 1: Create a collection for Tesla SEC filings71    response = await client.collections.create("tesla-sec-filings")72    print(f"Created collection: {response.collection_id}")7374    # Step 2: Upload documents to the collection concurrently75    async def upload_document(76        url: str, name: str, collection_id: str, http_client: httpx.AsyncClient77    ) -> None:78        pdf_response = await http_client.get(url, timeout=30.0)79        pdf_content = pdf_response.content8081        print(f"Uploading {name} document to collection")82        response = await client.collections.upload_document(83            collection_id=collection_id,84            name=name,85            data=pdf_content,86        )8788        # Poll until document is processed and ready for search89        response = await client.collections.get_document(response.file_metadata.file_id, collection_id)90        print(f"Waiting for document {name} to be processed")91        while response.status != collections_pb2.DOCUMENT_STATUS_PROCESSED:92            await asyncio.sleep(3)93            response = await client.collections.get_document(response.file_metadata.file_id, collection_id)9495        print(f"Document {name} processed")9697    # Upload both documents concurrently98    async with httpx.AsyncClient() as http_client:99        await asyncio.gather(100            upload_document(TESLA_10_Q_PDF_URL, "tesla-10-Q-2024.pdf", response.collection_id, http_client),101            upload_document(TESLA_10_K_PDF_URL, "tesla-10-K-2024.pdf", response.collection_id, http_client),102        )103104    # Step 3: Create a chat with collections search enabled105    chat = client.chat.create(106        model="grok-4.6",  # Use a reasoning model for better analysis107        tools=[108            collections_search(109                collection_ids=[response.collection_id],110            ),111            code_execution(),112        ],113        include=["verbose_streaming"],114    )115116    # Step 4: Ask a complex analytical question that requires searching multiple documents117    chat.append(118        user(119            "How many consumer vehicles did Tesla produce in total in 2024 and 2025? "120            "Show your working and cite your sources."121        )122    )123124    # Step 5: Stream the response and display reasoning progress125    is_thinking = True126    async for response, chunk in chat.stream():127        # View server-side tool calls as they happen128        for tool_call in chunk.tool_calls:129            print(f"\\nCalling tool: {tool_call.function.name} with arguments: {tool_call.function.arguments}")130        if response.usage.reasoning_tokens and is_thinking:131            print(f"\\rThinking... ({response.usage.reasoning_tokens} tokens)", end="", flush=True)132        if chunk.content and is_thinking:133            print("\\n\\nFinal Response:")134            is_thinking = False135        if chunk.content and not is_thinking:136            print(chunk.content, end="", flush=True)137        latest_response = response138139    # Step 6: Review citations and tool usage140    print("\\n\\nCitations:")141    print(latest_response.citations)142    print("\\n\\nUsage:")143    print(latest_response.usage)144    print(latest_response.server_side_tool_usage)145    print("\\n\\nTool Calls:")146    print(latest_response.tool_calls)147if __name__ == "__main__":148    asyncio.run(main())149```150151```pythonOpenAISDK152import os153from openai import OpenAI154155# Using OpenAI SDK with xAI API (requires pre-created collection)156api_key = os.getenv("XAI_API_KEY")157client = OpenAI(158    api_key=api_key,159    base_url="https://api.x.ai/v1",160)161162# Note: You must create the collection and upload documents first using either the xAI console (console.x.ai) or the xAI SDK163# The collection_id below should be replaced with your actual collection ID164response = client.responses.create(165    model="grok-4.6",166    input=[167        {168            "role": "user",169            "content": "How many consumer vehicles did Tesla produce in total in 2024 and 2025? Show your working and cite your sources.",170        },171    ],172    tools=[173        {174            "type": "file_search",175            "vector_store_ids": ["your_collection_id_here"],  # Replace with actual collection ID176            "max_num_results": 10177        },178        {"type": "code_interpreter"},  # Enable code execution for calculations179    ],180)181182print(response)183```184185```javascriptAISDK186import { createOpenAI } from '@ai-sdk/openai';187import { streamText } from 'ai';188189const openai = createOpenAI({190  baseURL: 'https://api.x.ai/v1',191  apiKey: process.env.XAI_API_KEY,192});193194const result = streamText({195  model: openai('grok-4.6'),196  prompt: 'What documents do you have access to?',197  tools: {198    file_search: openai.tools.fileSearch({199      vectorStoreIds: ['your-vector-store-id'],200      maxNumResults: 5,201    }),202  },203});204```205206```pythonRequests207import os208import requests209210# Using raw requests (requires pre-created collection)211url = "https://api.x.ai/v1/responses"212headers = {213    "Content-Type": "application/json",214    "Authorization": f"Bearer {os.getenv('XAI_API_KEY')}"215}216payload = {217    "model": "grok-4.6",218    "input": [219        {220            "role": "user",221            "content": "How many consumer vehicles did Tesla produce in total in 2024 and 2025? Show your working and cite your sources."222        }223    ],224    "tools": [225        {226            "type": "file_search",227            "vector_store_ids": ["your_collection_id_here"],  # Replace with actual collection ID228            "max_num_results": 10,229        },230        {"type": "code_interpreter"}  # Enable code execution for calculations231    ]232}233response = requests.post(url, headers=headers, json=payload)234print(response.json())235```236237```bash238# Using curl (requires pre-created collection)239curl https://api.x.ai/v1/responses \\240  -H "Content-Type: application/json" \\241  -H "Authorization: Bearer $XAI_API_KEY" \\242  -d '{243  "model": "grok-4.6",244  "input": [245    {246      "role": "user",247      "content": "How many consumer vehicles did Tesla produce in total in 2024 and 2025? Show your working and cite your sources."248    }249  ],250  "tools": [251    {252      "type": "file_search",253      "vector_store_ids": ["your_collection_id_here"],254      "max_num_results": 10255    },256    {257      "type": "code_interpreter"258    }259  ]260}'261```262263## Example Output264265When you run the Python xAI SDK example above, you'll see output like this showing the complete workflow from collection creation to the final analyzed response:266267```output268Created collection: collection_3be0eec8-ee8e-4a18-a9d4-fb70a3150d64269Uploading tesla-10-K-2024.pdf document to collection270Uploading tesla-10-Q-2024.pdf document to collection271Waiting for document tesla-10-K-2024.pdf to be processed272Waiting for document tesla-10-Q-2024.pdf to be processed273Document tesla-10-Q-2024.pdf processed274Document tesla-10-K-2024.pdf processed275276Thinking... (199 tokens)277Calling tool: collections_search with arguments: {"query":"Tesla total vehicle production 2024","limit":10}278Thinking... (370 tokens)279Calling tool: collections_search with arguments: {"query":"Tesla total vehicles produced 2024","limit":5}280Thinking... (370 tokens)281Calling tool: collections_search with arguments: {"query":"Tesla vehicles produced 2025 Q1 Q2 Q3","limit":5}282Thinking... (477 tokens)283Calling tool: collections_search with arguments: {"query":"Tesla production and deliveries 2024 all quarters","limit":10}284Thinking... (477 tokens)285Calling tool: collections_search with arguments: {"query":"Tesla production and deliveries 2025 Q1 Q2 Q3","limit":10}286Thinking... (569 tokens)287Calling tool: collections_search with arguments: {"query":"Tesla total production 2024 full year consumer vehicles","limit":5}288Thinking... (776 tokens)289Calling tool: collections_search with arguments: {"query":"Tesla annual vehicle production 2024 total","limit":5}290Thinking... (1148 tokens)291Calling tool: collections_search with arguments: {"query":"Tesla Q4 2024 production and deliveries","limit":5}292Thinking... (1240 tokens)293Calling tool: collections_search with arguments: {"query":"Tesla Q1 2024 vehicle production","limit":3}294Thinking... (1240 tokens)295Calling tool: collections_search with arguments: {"query":"Tesla Q2 2024 vehicle production","limit":3}296Thinking... (1240 tokens)297Calling tool: collections_search with arguments: {"query":"Tesla Q3 2024 vehicle production","limit":3}298Thinking... (1240 tokens)299Calling tool: collections_search with arguments: {"query":"Tesla Q4 2024 vehicle production","limit":3}300Thinking... (2804 tokens)301Calling tool: collections_search with arguments: {"query":"Tesla vehicle production nine months 2024","limit":5}302Thinking... (3352 tokens)303304Final Response:305### Working306307To answer this question, I used the collections_search tool to query Tesla's official 308financial filings (10-K and 10-Q reports) for production numbers. Tesla reports vehicle 309production and deliveries quarterly, and consumer vehicles refer to passenger vehicles 310like Model 3, Model Y, Model S, Model X, and Cybertruck (excluding Tesla Semi or other 311non-consumer products).312313#### Step 1: 2024 Production314Based on Tesla's official quarterly production and delivery reports (aggregated from SEC 315filings and press releases referenced in the collections), Tesla produced **1,773,443 316consumer vehicles in 2024**.317  - Q1 2024: 433,371 produced318  - Q2 2024: 410,831 produced319  - Q3 2024: 469,796 produced320  - Q4 2024: 459,445 produced321322#### Step 2: 2025 Production323The Q3 2025 10-Q filing explicitly states: "In 2025, we produced approximately 1,220,000 324consumer vehicles [...] through the third quarter."325  - This is the sum of Q1, Q2, and Q3 2025 production326  - Q4 2025 data is not available as of November 13, 2025327328#### Step 3: Total for 2024 and 2025329- 2024 full year: 1,773,443330- 2025 (through Q3): 1,220,000331- **Total: 2,993,443 consumer vehicles**332333Citations:334['collections://collection_3be0eec8-ee8e-4a18-a9d4-fb70a3150d64/files/file_d4d1a968-9037-4caa-8eca-47a1563f28ab', 335 'collections://collection_3be0eec8-ee8e-4a18-a9d4-fb70a3150d64/files/file_ff41a42e-6cdc-4ca1-918a-160644d52704']336337Usage:338completion_tokens: 1306339prompt_tokens: 383265340total_tokens: 387923341prompt_text_tokens: 383265342reasoning_tokens: 3352343cached_prompt_text_tokens: 177518344345{'SERVER_SIDE_TOOL_COLLECTIONS_SEARCH': 13}346Tool Calls:347... (omitted for brevity)348```349350### Understanding Collections Citations351352When using the collections search tool, citations follow a special URI format that uniquely identifies the source documents:353354```355collections://collection_id/files/file_id356```357358For example:359360```361collections://collection_3be0eec8-ee8e-4a18-a9d4-fb70a3150d64/files/file_d4d1a968-9037-4caa-8eca-47a1563f28ab362```363364**Format Breakdown:**365366* **`collections://`**: Protocol identifier indicating this is a collection-based citation367* **`collection_id`**: The unique identifier of the collection that was searched (e.g., `collection_3be0eec8-ee8e-4a18-a9d4-fb70a3150d64`)368* **`files/`**: Path segment indicating file-level reference369* **`file_id`**: The unique identifier of the specific document file that was referenced (e.g., `file_d4d1a968-9037-4caa-8eca-47a1563f28ab`)370371These citations represent all the documents from your collections that Grok referenced during its search and analysis. Each citation points to a specific file within a collection, allowing you to trace back exactly which uploaded documents contributed to the final response.372373### Key Observations3743751. **Autonomous Search Strategy**: Grok autonomously performs 13 different searches across the documents, progressively refining queries to find specific quarterly and annual production data.3763772. **Reasoning Process**: The output shows reasoning tokens accumulating (199 → 3,352 tokens), demonstrating how the model thinks through the problem before generating the final response.3783793. **Cited Sources**: All information is grounded in the uploaded documents with specific file citations, ensuring transparency and verifiability.3803814. **Structured Analysis**: The final response breaks down the methodology, shows calculations, and clearly states assumptions and limitations (e.g., Q4 2025 data not yet available).3823835. **Token Efficiency**: Notice the high number of cached prompt tokens (177,518) - this demonstrates how the collections search tool efficiently reuses context across multiple queries.384385## Combining Collections Search with Web Search/X-Search386387One of the most powerful patterns is combining the collections search tool with web search/x-search to answer questions that require both your internal knowledge base and real-time external information. This enables sophisticated analysis that grounds responses in your proprietary data while incorporating current market intelligence, news, and public sentiment.388389### Example: Internal Data + Market Intelligence390391Building on the Tesla example above, let's analyze how market analysts view Tesla's performance based on the production numbers from our internal documents:392393```pythonXAI394import asyncio395396import httpx397398from xai_sdk import AsyncClient399from xai_sdk.chat import user400from xai_sdk.proto import collections_pb2401from xai_sdk.tools import code_execution, collections_search, web_search, x_search402403# ... (collection creation and document upload same as before)404405async def hybrid_analysis(client: AsyncClient, collection_id: str, model: str) -> None:406    # Enable collections search, web search, and code execution407    chat = client.chat.create(408        model=model,409        tools=[410            collections_search(411                collection_ids=[collection_id],412            ),413            web_search(),  # Enable web search for external data414            x_search(),  # Enable x-search for external data415            code_execution(),  # Enable code execution for calculations416        ],417        include=["verbose_streaming"],418    )419420    # Ask a question that requires both internal and external information421    chat.append(422        user(423            "Based on Tesla's actual production figures in my documents (collection), what is the "424            "current market and analyst sentiment on their 2024-2025 vehicle production performance?"425        )426    )427428    is_thinking = True429    async for response, chunk in chat.stream():430        for tool_call in chunk.tool_calls:431            print(f"\\nCalling tool: {tool_call.function.name} with arguments: {tool_call.function.arguments}")432        if response.usage.reasoning_tokens and is_thinking:433            print(f"\\rThinking... ({response.usage.reasoning_tokens} tokens)", end="", flush=True)434        if chunk.content and is_thinking:435            print("\\n\\nFinal Response:")436            is_thinking = False437        if chunk.content and not is_thinking:438            print(chunk.content, end="", flush=True)439        latest_response = response440441    print("\\n\\nCitations:")442    print(latest_response.citations)443    print("\\n\\nTool Usage:")444    print(latest_response.server_side_tool_usage)445```446447### How It Works448449When you provide both `collections_search()` and `web_search()`/`x_search()` tools, Grok autonomously determines the optimal search strategy:4504511. **Internal Analysis First**: Searches your uploaded Tesla SEC filings to extract actual production numbers4522. **External Context Gathering**: Performs web/x-search searches to find analyst reports, market sentiment, and production expectations4533. **Synthesis**: Combines both data sources to provide a comprehensive analysis comparing actual performance against market expectations4544. **Cited Sources**: Returns citations from both your internal documents (using `collections://` URIs) and external web sources (using `https://` URLs)455456### Example Output Pattern457458```output459Thinking... (201 tokens)460Calling tool: collections_search with arguments: {"query":"Tesla vehicle production figures 2024 2025","limit":20}461Thinking... (498 tokens)462Calling tool: collections_search with arguments: {"query":"Tesla quarterly vehicle production and deliveries 2024 2025","limit":20}463Thinking... (738 tokens)464Calling tool: web_search with arguments: {"query":"Tesla quarterly vehicle production and deliveries 2024 2025","num_results":10}465Thinking... (738 tokens)466Calling tool: web_search with arguments: {"query":"market and analyst sentiment Tesla vehicle production performance 2024 2025","num_results":10}467Thinking... (1280 tokens)468469Final Response 470... (omitted for brevity)471```472473### Use Cases for Hybrid Search474475This pattern is valuable for:476477* **Market Analysis**: Compare internal financial data with external market sentiment and competitor performance478* **Competitive Intelligence**: Analyze your product performance against industry reports and competitor announcements479* **Compliance Verification**: Cross-reference internal policies with current regulatory requirements and industry standards480* **Strategic Planning**: Ground business decisions in both proprietary data and real-time market conditions481* **Customer Research**: Combine internal customer data with external reviews, social sentiment, and market trends482
Files main
Menu

Theme