SPB Git forge

spb/doc-api

Public
2commits 1branches 0releases
15.7 MBsize
maindefault branch
13 days agolast push
Python 88.3% TypeScript 7.6% Shell 4.1%
12.1 KB

# Gemini — File Search (tools[].fileSearch + fileSearchStores API, Preview)

Status: DOCUMENTED + LIVE_VERIFIED (2026-09-18 run — see "Live verification" section at the end)

Sources:

Last verified: 2026-09-18 (docs only)

# 1. Concepts and lifecycle

text
 file bytes ──uploadToFileSearchStore (LRO)──▶ FileSearchStore ──▶ Document ──▶ Chunks + embeddings
 Files API File (48 h) ──importFile (LRO)────▶      │
                                                     └── generateContent tools[].fileSearch ──▶ groundingMetadata.retrievedContext
  • FileSearchStore: persistent container (no TTL; deleted only by you or model deprecation). Names are globally scoped, server-generated: fileSearchStores/<displayName-slug>-<12 random chars>.
  • Document: one ingested file; state STATE_PENDING → STATE_ACTIVE | STATE_FAILED; up to 20 customMetadata.
  • Embedding model fixed at store creation: default text model (gemini-embedding-001 per guide) or models/gemini-embedding-2 for multimodal (PNG/JPEG ≤ 4K×4K).
  • Semantic search at query time; retrieved chunks are injected as context and cited in groundingMetadata.

# 2. Endpoints

Method Path Body / params Returns SDK (python / node)
POST /v1beta/fileSearchStores {displayName?, embeddingModel?} FileSearchStore client.file_search_stores.create(config={'display_name':…,'embedding_model':…}) / ai.fileSearchStores.create({config})
GET /v1beta/fileSearchStores pageSize (≤10), pageToken {fileSearchStores[], nextPageToken} .list()
GET /v1beta/fileSearchStores/{id} — FileSearchStore .get(name=…)
DELETE /v1beta/fileSearchStores/{id} force (bool) {} .delete(name=…, config={'force': True})
POST /upload/v1beta/fileSearchStores/{id}:uploadToFileSearchStore (media URI; also /resumable/upload/v1beta/…; metadata-only /v1beta/…:uploadToFileSearchStore) multipart/resumable: metadata {displayName?, mimeType?, customMetadata[]?, chunkingConfig?} + bytes (≤100 MB, any MIME) Operation (fileSearchStores/{id}/upload/operations/{op}) .upload_to_file_search_store(file=…, file_search_store_name=…, config={…}) / ai.fileSearchStores.uploadToFileSearchStore({file, fileSearchStoreName, config})
POST /v1beta/fileSearchStores/{id}:importFile {fileName: "files/…", customMetadata[]?, chunkingConfig?} Operation (fileSearchStores/{id}/operations/{op}) .import_file(file_search_store_name=…, file_name=…, custom_metadata=[…]) / ai.fileSearchStores.importFile({…})
GET /v1beta/fileSearchStores/{id}/documents pageSize (≤20), pageToken {documents[], nextPageToken} .documents.list(parent=…)
GET /v1beta/fileSearchStores/{id}/documents/{doc} — Document .documents.get(name=…)
DELETE /v1beta/fileSearchStores/{id}/documents/{doc} force {} .documents.delete(name=…, config={'force': True})
GET /v1beta/fileSearchStores/{id}/operations/{op} — Operation client.operations.get(op) / ai.operations.get({operation})
GET /v1beta/fileSearchStores/{id}/upload/operations/{op} — Operation same
GET /v1beta/fileSearchStores/{id}/media/{blob} (guide shows /v1/) — image bytes of a cited mediaId .download_media(media_id=…) / ai.fileSearchStores.downloadMedia(id) — not in discovery (UNVERIFIED)

Delete semantics: force=false (default) → FAILED_PRECONDITION if the store has Documents / the Document has Chunks; force=true cascades.

# 3. Upload protocols (REST)

Resumable (guide example): POST https://generativelanguage.googleapis.com/upload/v1beta/fileSearchStores/{id}:uploadToFileSearchStore with headers X-Goog-Upload-Protocol: resumable, X-Goog-Upload-Command: start, X-Goog-Upload-Header-Content-Length: <bytes>, X-Goog-Upload-Header-Content-Type: <mime>, JSON body {"displayName":"sample.txt"} → read X-Goog-Upload-URL from response headers → POST <upload url> with X-Goog-Upload-Offset: 0, X-Goog-Upload-Command: upload, finalize, --data-binary @file → Operation JSON. Simple/multipart: multipart/related body on the same /upload/v1beta/ URI (discovery mediaUpload.protocols.simple.multipart: true). Poll GET /v1beta/<operation.name> until done: true.

# 4. Chunking, metadata, filters

Item Shape
chunkingConfig.whiteSpaceConfig {maxTokensPerChunk (words; ≤512 recommended → ≈2,560 tokens), maxOverlapTokens} — only algorithm exposed
customMetadata[] {key, stringValue | numericValue | stringListValue{values[]}}; ≤20 per Document
tools[].fileSearch.metadataFilter string, AIP-160 list-filter syntax (google.aip.dev/160), e.g. author = "Robert Graves", year > 1930; operators (Condition enum): LESS, LESS_EQUAL, EQUAL, GREATER_EQUAL, GREATER, NOT_EQUAL, INCLUDES, EXCLUDES (string lists)
tools[].fileSearch.topK integer, REST reference only
tools[].fileSearch.fileSearchStoreNames[] required; several stores allowed

# 5. Query response

json
"groundingMetadata": {
  "groundingChunks": [{"retrievedContext": {"title": "sample.txt", "uri": "...", "text": "chunk text",
      "fileSearchStore": "fileSearchStores/my-store-123", "pageNumber": 3,
      "mediaId": "fileSearchStores/my-store-123/media/BlobId-456",
      "customMetadata": [{"key": "author", "stringValue": "Robert Graves"}, {"key": "year", "numericValue": 1934}]}}],
  "groundingSupports": [{"segment": {"startIndex": 0, "endIndex": 50, "text": "..."}, "groundingChunkIndices": [0]}]
}

pageNumber for paged documents (PDF); mediaId for image chunks (persistent across searches). With includeServerSideToolInvocations: toolCall{toolType: FILE_SEARCH} (no user-visible args). Structured outputs + File Search: Gemini 3.

# 6. Supported file types (summary)

Application: dart, ecmascript, json, ms-java, msword, pdf, sql, typescript, vnd.curl, vnd.dart, vnd.ibm.secure-container, vnd.jupyter, vnd.ms-excel, vnd.oasis.opendocument.text, OOXML pptx/xlsx/docx/dotx, x-csh, x-hwp(-v5), x-latex, x-php, x-powershell, x-sh, x-shellscript, x-tex, x-zsh, xml, zip. Text: ~150 text/* types incl. plain, html, css, csv, tsv, markdown, javascript, jsx, tsx, x-python, x-java, x-c, x-go, x-rust, yaml, xml… (full list in the guide). Images (PNG, JPEG) only with models/gemini-embedding-2. Audio and video not supported.

# 7. Limits and tiers

Limit Value
Max file / document size 100 MB (discovery maxSize 104857600)
Total store size per project Free 1 GB · Tier 1 10 GB · Tier 2 100 GB · Tier 3 1 TB (backend ≈3× input incl. embeddings)
Recommended store size < 20 GB for retrieval latency
displayName ≤512 chars; store id ≤40 chars
Live API not supported
Tool combinations cannot be combined with Google Search, URL context, etc.; Gemini 3 can combine with function calling

# 8. Pricing

Item Price
Indexing (embedding at ingest) $0.15 / 1M tokens (pricing tools table; the gemini-embedding-2 table lists $0.20 standard / $0.10 batch text input — inconsistent)
Storage free
Query-time embedding free
Retrieved chunks billed as regular input tokens of the generating model
Free tier free of charge

# 9. Supported models

Guide table: gemini-3.8/3.7/3.6/3.5-flash, 3.5-flash-lite, 3.1-pro-preview, 3.1-flash-lite, 3-flash-preview, 2.5-pro, 2.5-flash-lite. Model pages add gemini-2.5-flash, gemini-3-pro-preview, 3.1-flash-lite-preview, robotics ER 2; gemini-3.1-pro-preview page says "Supported (AI Studio only)".

# 10. Examples

bash
# create store
curl -s -X POST "https://generativelanguage.googleapis.com/v1beta/fileSearchStores" \
  -H "x-goog-api-key: $GEMINI_API_KEY" -H "Content-Type: application/json" \
  -d '{"displayName":"atlas-test","embeddingModel":"models/gemini-embedding-2"}'
# query
curl -s "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash-lite:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"What is in the uploaded file?"}]}],
       "tools":[{"fileSearch":{"fileSearchStoreNames":["fileSearchStores/STORE_ID"],"metadataFilter":"author = \"X\""}}]}'
python
store = client.file_search_stores.create(config={"display_name": "atlas-test"})
op = client.file_search_stores.upload_to_file_search_store(file="sample.txt", file_search_store_name=store.name,
        config={"display_name": "sample.txt", "custom_metadata": [{"key": "author", "string_value": "X"}]})
while not op.done: time.sleep(3); op = client.operations.get(op)
r = client.models.generate_content(model="gemini-3.5-flash-lite", contents="What is in the file?",
        config=types.GenerateContentConfig(tools=[types.Tool(file_search=types.FileSearch(file_search_store_names=[store.name]))]))
print(r.candidates[0].grounding_metadata.grounding_chunks)
client.file_search_stores.delete(name=store.name, config={"force": True})
ts
const store = await ai.fileSearchStores.create({ config: { displayName: 'atlas-test' } });
let op = await ai.fileSearchStores.uploadToFileSearchStore({ file: 'sample.txt', fileSearchStoreName: store.name, config: { displayName: 'sample.txt' } });
while (!op.done) op = await ai.operations.get({ operation: op });
const r = await ai.models.generateContent({ model: 'gemini-3.5-flash-lite', contents: 'What is in the file?',
  config: { tools: [{ fileSearch: { fileSearchStoreNames: [store.name] } }] } });
await ai.fileSearchStores.delete({ name: store.name, config: { force: true } });

# Live verification (2026-09-18)

LIVE_VERIFIED end-to-end (tmp-live/gemini-tools/g*.json, examples/gemini/file-search/file_search_lifecycle.py, file_search.ts, tests/gemini/test_file_search.py):

Step Call Observed
create POST /v1beta/fileSearchStores {displayName} 200 {name:"fileSearchStores/atlasprobe20260918-…", displayName, createTime, updateTime, embeddingModel:"models/gemini-embedding-001"}
upload (resumable) POST /upload/v1beta/{store}:uploadToFileSearchStore headers X-Goog-Upload-Protocol: resumable, -Command: start, JSON {displayName, customMetadata, chunkingConfig} 200 + X-Goog-Upload-URL (+ X-Goog-Upload-Control-URL, -Chunk-Granularity, -Status)
finalize POST <upload url> X-Goog-Upload-Command: upload, finalize, 200 bytes 200 Operation{name:".../upload/operations/…", response} (done omitted)
poll GET /v1beta/{operation} done:true, response{@type, documentName, mimeType, parent, sizeBytes}
documents GET …/documents, GET …/documents/{doc} {name, displayName, mimeType, sizeBytes, state:"STATE_ACTIVE", customMetadata, createTime, updateTime}
query tools:[{fileSearch:{fileSearchStoreNames:[…], metadataFilter:"kind=probe"}}] correct answer; groundingMetadata.groundingChunks[{retrievedContext{title, text, fileSearchStore}}] + groundingSupports in 2 of 3 runs, absent once (only usageMetadata.toolUsePromptTokenCount 834–985 proved retrieval)
store get / list GET /v1beta/{store}, GET /v1beta/fileSearchStores?pageSize=5 activeDocumentsCount:"1", sizeBytes:"200"; {fileSearchStores:[…]}
delete document DELETE /v1beta/{doc} 400 FAILED_PRECONDITION "Cannot delete non-empty Document"; with ?force=true → 200
delete store DELETE /v1beta/{store}?force=true 200 {}

Not exercised: importFile, /operations/{op} poller for imports, media/{blob} download.