Skip to content

Chat files & indexing

Every file in a user’s library is stored first and indexed in the background: files uploaded in the chat, files your client-side tools deliver, files an MCP tool writes with put_file, and files the assistant generates. Indexing reads the file’s text so the assistant can read it with read_file, and so file_search can return passages from it. Each file reports a status that says how far this got and, when something went wrong, why. How the assistant then reads a file is in How the assistant reads files.

This page is for integrators building on the widget or the files API. The upload limit, quota and retention an administrator sets are in File storage limits, quota & retention.

Once an upload is accepted, the file is processing and a background job indexes it:

  1. Read. The hub reads the stored file as a stream. Plain text, Markdown, JSON and CSV are decoded as they are read. PDFs are read page by page and Excel workbooks sheet by sheet. A Word document is read in one piece. Other formats are read by their own parsers; see Supported formats.
  2. Store the text. The extracted text is written to the file store, next to the file, as the file’s stored text. There is no length cap: a long document is stored and indexed in full.
  3. Split. The text is cut into overlapping chunks. Each chunk records where it came from: the PDF page, the sheet, row range and A1 range, or the line range. Spreadsheet and CSV chunks also record their column headers.
  4. Embed. Chunks are embedded and written in batches, so memory use doesn’t grow with the file’s size.

The new index becomes searchable only when the whole run completes. Until then, any earlier index of the same file keeps serving search, so re-indexing a file never leaves a gap. A run that fails leaves the earlier index in place.

Indexing runs in the hub’s background job queue. Temporary failures, such as an embedding provider’s rate limit or a storage error, are retried with backoff. Permanent ones, such as a damaged file, settle at once. At most two indexing jobs run at once per library (a user’s library, an anonymous session, or a knowledge base), so one user uploading a hundred files doesn’t hold up everyone else.

DioscHub reads text from documents (PDF, Word, PowerPoint .pptx, OpenDocument, EPUB, RTF, mail), tables (Excel .xlsx, OpenDocument spreadsheets, CSV, TSV) and text, code and markup files. Images are not indexed: an image is ready, a vision model can view it, and file_search doesn’t return it. Media, archives and unknown types are stored without being read, and executables are refused at upload. The full list, and how an administrator adds types of their own, is in File types, kinds & parser limits.

Every file is in one of six states. The hub also derives two flags from the state: canRead (the assistant can read the file) and canSearch (file_search can return passages from it). Read the flags instead of working them out from the state yourself.

statusWhat it meanscanReadcanSearch
processingBeing indexed.Only when text from an earlier run is storedNo
readyRead and indexed. An image is ready but not indexed.YesYes. No for an image.
search_unavailableThe text was read and stored, but the search index wasn’t built.YesNo
no_textThe file was read and holds no text, for example a scanned PDF with no text layer.NoNo
stored_onlyDioscHub doesn’t read this kind of file, or its content doesn’t match its type. It is stored and can be downloaded.NoNo
failedThe file couldn’t be read.NoNo

A file that isn’t ready or processing carries a reason: a fixed code and a fixed message. The message is the one the user sees. It never contains the underlying error text, which stays on the hub.

reason.codestatusreason.message
over_quotasearch_unavailableNot searchable: the library is over its storage quota.
embedding_failedsearch_unavailableNot searchable: the search index could not be built.
no_text_layerno_textNo text could be found in this file.
stored_only_kindstored_onlyThis kind of file is stored but not read.
type_mismatchstored_onlyThe file’s content doesn’t match its type.
corruptfailedThe file could not be read; it may be damaged.
encryptedfailedThe file is password-protected.
parser_limitfailedThe file is too large to read.
timeoutfailedReading the file took too long.
source_missingfailedThe file’s stored content is missing.
retry_exhaustedfailedProcessing kept failing. Try Re-index.

A file that was mostly read is still ready (or search_unavailable), and lists what it skipped in warnings. Each warning is a fixed code, a location and a retryable flag:

codelocationWhat happened
page_unreadablepage NA PDF page couldn’t be read, or took too long (30 seconds by default). The stored text holds the line [page N could not be read] in its place.
sheet_unreadablesheet <name>An Excel worksheet couldn’t be read. The stored text holds [sheet "<name>" could not be read] in its place.
invalid_encodingline LThe text had bytes that aren’t valid UTF-8. They became replacement characters (U+FFFD), and L is the first line affected. A file gets at most one of these.
warnings_truncated<n> moreThe file had more than 100 warnings. The 100th entry counts the rest.
text_truncatednoneOnly on files indexed before the upgrade: the earlier version stopped reading a file at a size cap, so the stored text is incomplete. Re-index the file to read all of it. New indexing never writes this code.

A warning marked retryable (a page that timed out) is tried once more at the end of the run. If the page reads then, its text is added to the end of the stored text after the line [page N, recovered], and the warning goes. If it fails again, the warning stays with retryable: false.

A file is failed only when nothing could be read at all: every page, or every sheet. Plain text is never split into sections, so it is either read whole or not at all.

Treat the code lists as open: show a code you don’t recognise as it is. Files whose text was cut short by the length limit of earlier releases can carry a text_truncated warning (location null) until they are indexed again.

While a file is processing, progress is { step, done, total }; otherwise it is null.

stepdone / total
indexingBytes read for text formats, pages for a PDF, worksheets for an Excel workbook, 0 then 1 for Word
retryingRetryable sections tried again, out of how many there are

Progress is written at most once every two seconds per file, and done never goes down within a step.

The same fields are on every file the hub returns:

  • the user’s library, GET /api/files and GET /api/files/:id;
  • a session’s files, GET /api/sessions/:id/files;
  • the files:status event;
  • the files of a knowledge base, in the admin API.

An abridged file from GET /api/files:

{
"id": "9b2f6c1e-4d0a-4f7e-9a51-2c8d6e0b7f13",
"filename": "handbook.pdf",
"mimeType": "application/pdf",
"sizeBytes": 2483021,
"origin": "UPLOAD",
"status": "ready",
"reason": null,
"warnings": [{ "code": "page_unreadable", "location": "page 14", "retryable": false }],
"progress": null,
"canRead": true,
"canSearch": true,
"isIndexed": true,
"indexBytes": 412880,
"derivedBytes": 0,
"createdAt": "2026-11-06T09:12:44.000Z",
"expiresAt": null
}

Each time the hub writes a file’s status or progress, it pushes files:status with the file’s status fields. The widget applies it in place, and your code can listen to it too:

diosc('on', 'files:status', (file) => {
// { fileId, filename, mimeType, status, reason, warnings, progress, canRead, canSearch, isIndexed }
if (file.status === 'failed') showProblem(file.fileId, file.reason?.message);
});
  • Who gets it. For a signed-in user, every widget that user has open on that Assistant, whichever session uploaded the file, and also for files uploaded outside any session. For an anonymous visitor, that visitor’s session only. Knowledge-base files push nothing.
  • What’s in it. The status fields, so you never need to refetch to learn them. It never carries a storage key, error text or any of the file’s text. isIndexed is the deprecated alias, as above.
  • Pushes aren’t replayed. A widget that was disconnected catches up by reading the list.

files:updated still fires, with { fileId, filename }, to the session that uploaded a file once that file is indexed. It says only that something changed. Prefer files:status.

A user can re-index one of their own files when it is failed or search_unavailable: after freeing storage for an over_quota file, say, or once the embedding provider is back. The widget’s Files dialog shows a Re-index button on those rows.

A ready spreadsheet or CSV file with no workbook outline can be re-indexed too, to build its outline: a file indexed before outlines existed, or a CSV whose outline a redaction removed. This applies to .xlsx, CSV and TSV files that have text stored, as the file was classified. An .ods file, or a spreadsheet with no text stored, can never have an outline, so its re-index is refused. The widget has no button for this case yet; call the API, or ask an administrator for a bulk Re-index.

Over the API:

POST /api/files/:id/reindex

The request has no body and is authenticated like the other /api/files routes: the widget’s REST token, or an anonymous visitor’s files token. The Role needs the files feature.

ResponseWhen
200 with the file, now processingThe Re-index is accepted and an indexing job is queued.
404The file isn’t in the caller’s library, or has expired.
409 with code: "not_reindexable"The file isn’t failed, search_unavailable, or a ready spreadsheet or CSV without an outline: for example a ready file that isn’t a table, a spreadsheet that already has its outline, an .ods file, or a file that is processing, no_text or stored_only. Another Re-index, or an overwrite, may have got there first.
429 with a Retry-After headerThe caller spent the Re-index limit: by default 5 per minute, counted per caller like the other request limits (REINDEX_LIMIT_PER_MINUTE).

The 409 body is:

{
"code": "not_reindexable",
"message": "Only a failed file, one whose search is unavailable, or a ready spreadsheet or CSV with no workbook outline can be re-indexed"
}

The Retry-After header is readable from a cross-origin page, so the widget can say how long to wait: “Too many re-index requests. Try again in 42 seconds.”

Re-index is not a new upload, so the upload quota check doesn’t run. The indexer checks only growth: re-indexing a file that hasn’t changed never trips the quota. A file that now produces a larger index can still stop at the quota.

When a library has a storage quota, the indexer checks it before each batch of chunks is embedded. If the batch would take the library past the quota, embedding stops and the file becomes search_unavailable with reason over_quota. Its full text is still stored, so the assistant can read the file, but file_search can’t return passages from it. To make it searchable, free space, then Re-index it.

Knowledge-base files never count against a user’s quota, and the background re-indexing an administrator starts (below) never stops at one.

The model reaches a file’s content only through tools it calls. Nothing from a file is pasted into the prompt: a user message lists its attached files, and the model then reads, searches or queries them.

ToolWhat it doesAvailable when
read_fileReads one of the user’s files, page by page: its extracted text, or its original bytesThe licence includes file handling and the user’s Role has the files feature
file_searchFinds passages across the user’s files, or inside one fileSame as read_file
describe_workbook, load_table, query_tableSums, filters and other table maths on the user’s own spreadsheets and CSV files; see SpreadsheetsSame as read_file
read_kb_fileReads a knowledge-base file, page by pageThe licence allows knowledge bases
kb_searchFinds passages in the knowledge bases linked to the Assistant and RoleSame as read_kb_file

The reading and searching tools are read-only and never ask for approval. The licence and the Role are checked again on every turn, so a change applies from the next message.

Every piece of file text that read_file, read_kb_file, file_search, kb_search and the table tools hand the model is framed as data, not instructions; see File content the model reads.

A user message with attachments carries a <files> block that lists them for the model:

<files>
<file id="9b2f6c1e-…" name="handbook.pdf" mimeType="application/pdf" sizeBytes="2483021" kind="document" status="ready" pages="42" approxTokens="31800"/>
</files>
AttributeWhat it holds
id, name, mimeType, sizeBytesThe file’s id, name, type and size
kindThe file’s kind, other when it has none
statusThe file’s status when the message was sent
reasonThe reason code, only when the status isn’t ready
pagesThe page count, when known
approxTokensThe size of the extracted text in tokens, rounded up to a multiple of 100
hintThe hint of the admin file type rule that matched the file, cut to 200 characters

The list is a snapshot: the model is told that the status may have changed since, and that read_file reports the current one. pages and approxTokens let it decide how much of a file to read. Every value is escaped, so a file name or hint can’t close the block or add a row.

read_file with the default text view returns one page of the file’s stored text:

{
"fileId": "9b2f6c1e-4d0a-4f7e-9a51-2c8d6e0b7f13",
"name": "handbook.pdf",
"kind": "document",
"view": "text",
"text": "<file-data boundary=\"…\" origin=\"read_file\" file=\"handbook.pdf\">\n[page 1]\nEmployee handbook…\n</file-data boundary=\"…\">",
"location": { "from": { "page": 1 }, "to": { "page": 3 } },
"nextCursor": "eyJmaWxlSWQiOi…",
"progress": { "readBytes": 30412, "totalBytes": 412880, "done": false }
}
  • Markers. The text starts with a marker and carries one wherever a new section starts: [page N] for a PDF, [sheet "Name" rows A-B] for a spreadsheet, [line N] for text and code. They come from the file’s search index. A file without one (search_unavailable) gets a line marker at the start of each page. In read_file text a spreadsheet is marked by sheet and row range. Search hits also carry the chunk’s A1 range and column headers (see From a search hit into the file), and the table tools work by A1 area.
  • Paging. While nextCursor isn’t null, the model calls read_file again with the same file, the same view and that cursor. progress says how much has been read; done is true on the last page. The full text can be read, however long it is. A page never splits a character.
  • Starting somewhere. at starts the read at { "page": N }, { "sheet": "Name", "row": N } (the row is optional), { "line": N } or { "offset": N }. A location that doesn’t exist is refused with location_not_found and a message that says what does exist, such as the page range or the sheet names. A cursor, when given, wins over at.

A page, the whole JSON result included, stays within a page budget the hub works out each turn:

  • a quarter of the smallest context window among the models the session can use (the Assistant’s model, its routing candidates and its fallbacks), counted at 4 characters per token;
  • never more than the tool-result limit, DIOSC_MAX_TOOL_RESULT_CHARS (default 65,536 characters), and never less than 4,096 characters.

A model whose context window isn’t known counts as 32,768 tokens. With a 128K-token model alone, a page is 65,536 characters; if one fallback has a 32K window, every page is 32,768 characters, so a page still fits whichever model answers. Nothing the model sends changes the budget.

A cursor is opaque. It is tied to the file, the view and the version of the stored object it was issued for, and it grants nothing: the file’s owner is checked on every call.

statusWhen
cursor_staleThe file was re-indexed or redacted since the cursor was issued.
cursor_invalidThe cursor isn’t one the hub issued for this file and view.
changedThe file changed while the page was being read. No page is returned.

Each refusal tells the model to start again without a cursor. Text already read is never mixed with text from the new version.

view: "raw" returns the file’s original bytes instead of its extracted text. It works for every file, whatever its kind or status, including files the text view can’t read. It’s for questions the extracted text can’t answer, such as why a strict server rejects a PDF: the version, encryption, scripts or forms, embedded fonts, incremental updates, a broken cross-reference table.

Printable ASCII, line feeds, carriage returns and tabs come through as they are, a backslash as \\, and every other byte as \xHH. Each result carries totalBytes, the byte range it covers, kind, mimeType, nextCursor and progress, within the same page budget. The raw view always starts at the first byte and moves only by cursor; an at offset is refused.

After a redaction deleted a binary file’s original bytes, the raw view answers original_removed, and reads nothing else in its place.

Every file_search and kb_search hit names its file and where it sits:

{
"fileId": "9b2f6c1e-4d0a-4f7e-9a51-2c8d6e0b7f13",
"name": "handbook.pdf",
"location": { "kind": "page", "page": 14 },
"startOffset": 18842,
"score": 0.71,
"content": "<file-data boundary=\"…\" origin=\"file_search\" file=\"handbook.pdf\">\n…\n</file-data boundary=\"…\">"
}

location is a page, a sheet and row range ({ "kind": "sheet", "sheet", "rowFrom", "rowTo", "range" }), a line range ({ "kind": "lines", "lineFrom", "lineTo" }) or null. For a spreadsheet hit, range is the A1 area the chunk covers, such as "C2:E40"; it is left out when it isn’t a plain A1 range. To read around a hit, the model calls read_file (or read_kb_file for a kb_search hit) with at: { "offset": startOffset }. Hits no longer carry storage keys or raw chunk metadata.

Column headers are file text, so they never go in location. A hit from a spreadsheet or CSV file whose header row is known starts its framed content with a Columns: line instead:

Columns: Region, Quarter, Revenue
East,Q1,1200
…

The line holds at most 64 headers, each cut to 200 characters. The same headers go into the text that is embedded for each chunk, so a question that names a column, such as “Revenue for East”, finds the right rows.

file_search also names the files it couldn’t search, with their status and reason:

  • with scope: "file", in file: { fileId, name, kind, status, reason, canRead };
  • with scope: "user", in notSearchable (the 10 newest) and notSearchableTotal (how many there are).

read_file never returns empty text for a file it can’t read. It answers with the file’s status and a fixed message: the reason when the file has one, or a fixed text for the status, such as “This file is still being processed and has no readable text yet. Try again shortly.” for a file being indexed for the first time.

The text view reads a file when canRead is true: ready and search_unavailable files, and a processing file that already has text from an earlier run. For a no_text or stored_only file, and for a readable file whose stored text is missing, the message adds that the raw view can still read the original bytes. A no_text or stored_only answer also carries the hint of the file type rule that matched the file, when there is one.

A file that isn’t in the caller’s library gets one answer, whatever the cause: another user’s file, another session’s, an expired file, a file DioscHub derived for its own use (other than a workbook picture of the caller’s own workbook), or an id that doesn’t exist. It is not_found, “File not found. If this id came from kb_search, use read_kb_file instead.” An anonymous visitor reads only their own session’s files, and only when the session’s Role had files on when the session started.

read_file on an image doesn’t return its bytes. When the image can be shown, the hub adds it to the conversation right after the tool results, as one message that carries the image the way a user-attached image is carried. That message is framed as file data from read_file and marked as the hub’s own: it is not a user turn, and the widget’s history, the admin conversation transcript, session search and compaction leave it out. The read_file result names the frame’s boundary, so the model can tell the real image from text that only looks like one.

An image is shown only when every model the session can use (the Assistant’s model, its routing candidates and its fallbacks) has the vision feature and comes from a provider whose message order DioscHub has verified for it: OpenAI, Azure OpenAI, OpenAI-compatible servers, Anthropic and Ollama. A chain that includes AWS Bedrock, Google, Mistral, DeepSeek, Groq or xAI doesn’t get images from read_file. Images a user attaches to a message are not affected.

statusWhen
image_attachedThe image is shown.
vision_unavailableA model the session can use can’t view images, or its provider isn’t one of those above.
image_unsupportedThe format isn’t PNG, JPEG, WebP or GIF.
image_too_largeThe stored image is over the vision limit, VISION_MAX_INPUT_BYTES (default 32 MiB).
image_limitTwo images have already been shown since the user’s last message.
image_repeatThe same image was already shown since the user’s last message.
image_unavailableThe image couldn’t be loaded.

A shown image is scaled down to at most 1,568 pixels on its longer side, as attached images are; one that is still over 4.5 MiB, or is over the pixel limit (VISION_MAX_INPUT_PIXELS), is image_unavailable. Each image counts as 1,600 tokens in the usage quota and compaction estimates. The turn always goes on: an image that can’t be shown leaves the model a fixed message saying why. If an MCP tool is bound under the name read_file as well, images from read_file are off.

read_kb_file reads knowledge-base files the way read_file reads a user’s: the same views, pages, markers, cursors, at locations, page budget and refusals. It reads a file only when kb_search could search it: its knowledge base is active and linked to the Assistant or to the user’s Role, and, when the bind carried claims, the link’s permissions allow it. That is resolved again on every call. Any other id, a user’s chat file included, gets the same not_found. An image in a knowledge base is not shown; the raw view can still read it.

  • The widget’s history. When a conversation is reopened, each tool result in it, a read_file page included, is cut to 4,000 characters, followed by … [N more characters]. An image read_file showed never appears as a message.
  • The conversation transcript in the admin Conversations explorer shows up to 4,000 characters of each tool result too, read_file pages included.
  • Compaction. When older turns are summarised, the summariser reads at most 4,000 characters of each tool result, so ten full pages can’t overflow the background model.

What the assistant read stays in the conversation after the file changes or is deleted; see Read file content stays in the conversation.

read_file and file_search read a spreadsheet as text. For numbers (a sum, a count, a filter, a group by) the assistant uses three table tools instead, in this order:

  1. describe_workbook shows how the file is laid out, without reading its data.
  2. load_table loads one table from it, so it can be queried.
  3. query_table runs one read-only SQL SELECT over the loaded tables and returns rows.

What each format supports:

Formatread_file, file_searchTable toolsPictures
Excel .xlsxYesYesYes
CSV, TSVYesYes, the whole file is one tableNo
OpenDocument .odsYesNoNo
Excel 97–2003 .xlsStored, not read: only read_file’s raw viewNoNo

Which row applies is decided from how the file was classified when it was stored, never from its name alone. The table tools take the id of one of the user’s own files, such as one from the message’s file list. They come with the Files tools: the licence must include file handling and the user’s Role needs the files feature.

When a spreadsheet or CSV in a user’s library is indexed, the hub also stores an outline of it. describe_workbook reads only that outline; it never opens the file itself.

For an Excel workbook, the outline lists every sheet in workbook order, with:

  • its visibility (visible, hidden or veryHidden) and used range;
  • merged cells, and hidden rows and columns;
  • the table regions found on the sheet (at most 20), each with its A1 range, a confidence (high, medium or low), its header rows and its column names;
  • pictures, with the cell each one sits on (see Pictures in workbooks);
  • charts, with their title, type and the ranges they plot (charts are listed, not drawn);
  • a preview of the first 10 non-empty rows and 10 columns, each cell labelled with its A1 reference and cut to 80 characters. A cell in a hidden row, column or sheet is labelled as hidden.

For a CSV or TSV file, the outline holds its columns, its record count (the header line included) and a preview.

The answer stays within 24,000 characters. When a workbook is larger, the first sheets are shown in full and the later ones by name and visibility only.

For an Excel workbook, the model gives the sheet name exactly as describe_workbook listed it, the table’s A1 area (for example B2:F40) and, when the header isn’t exactly one row, headerRows (0 to 10, default 1). For a CSV or TSV file it gives only the file id: the whole file is loaded, with its first record as the header, and a sheet, range or headerRows it sends anyway is ignored, with a note saying so.

The answer describes the table it loaded:

{
"name": "s1_a1_d20",
"sheet": "<file-data …>Sales</file-data …>",
"range": "A1:D20",
"headerRows": 1,
"rowCount": 19,
"columns": [
{
"name": "revenue",
"header": "<file-data …>Revenue</file-data …>",
"type": "DOUBLE",
"column": "D",
"profile": { "nulls": 0, "min": "<file-data …>120.5</file-data …>", "max": "<file-data …>980</file-data …>", "distinct": 19 }
}
],
"samples": [[{ "ref": "A2", "value": "<file-data …>East</file-data …>" }]],
"notes": ["Distinct counts in the profiles are approximate."]
}
  • Names. The table is s<sheet number>_<top-left>_<bottom-right>, such as s1_a1_d20; a CSV’s table is csv. Columns are named from their headers, in lower case with _ for other characters; an empty header becomes col_<letter>.
  • Types. Each column is BOOLEAN, DATE, TIMESTAMP, BIGINT, DOUBLE or VARCHAR, decided from its values. A column that mixes numbers and text, such as one with a stray N/A, is VARCHAR. In a CSV, codes with a leading zero, such as ZIP codes, stay text.
  • Rows. Every table has _row, the sheet row number (for a CSV, the record number, the header being 1). Hidden rows are loaded and marked in a _row_hidden column, hidden columns are marked hidden, and a hidden sheet loads with a note giving its state.
  • Samples. Up to 5 rows of up to 30 columns, each value cut to 80 characters and labelled with its A1 reference.
  • Size. The answer is at most 16,000 characters. On a wide table, the columns past what fits are listed by name and type only, and a note says so.

A loaded table is kept as a derived file of the spreadsheet, so a later turn or session can query it without loading it again. A spreadsheet keeps at most 8 loaded tables; loading a ninth drops the one used least recently. The kept tables count toward the user’s storage quota. When the quota is full, the table still answers but isn’t kept: it stays usable on that hub instance for up to an hour, and a note tells the model so. When the file changes (it is overwritten or redacted), its tables are dropped and built again on the next load.

query_table takes the file id and one SQL SELECT (a WITH … SELECT is fine) over the tables load_table returned for that file. Anything else is refused. It answers with the rows:

{
"rowCount": 3,
"truncated": false,
"result": "<file-data … origin=\"query_table\" …>{\"columns\":[\"region\",\"total\"],\"rows\":[…]}</file-data …>"
}
  • At most 1,000 rows and 16,000 characters. Values are cut to 200 characters. When rows are cut, truncated is true and a note tells the model to narrow the query.
  • A query runs for at most 10 seconds.
  • On an error the model gets one fixed sentence for the kind of error (syntax, an unknown column, a value that can’t be converted, the time limit) followed by the loaded tables and their columns, so it can fix the SQL. The database’s own error text never reaches the model.
  • A binary column is refused with “Binary columns can’t be shown; select length(x) or cast to VARCHAR.”

Queries run in a sealed, read-only database that can read only the tables loaded from this file. It has no file or network access and its settings are locked. Table functions (other than range and generate_series), schema-qualified names, file paths or URLs in FROM, and functions that describe the database itself are all refused with a fixed Rejected: … sentence. The table build is locked down the same way: no extensions are installed or loaded.

When an Excel workbook in a user’s library is indexed, its pictures are extracted, so the assistant can look at a logo, a scanned form or a chart pasted as an image. describe_workbook lists each picture with its cell and name, and either a fileId or the reason it wasn’t stored. read_file with that fileId shows the picture to the model in the same way as any image, under the same vision rules and limits. The result also names the workbook, the cell the picture sits on, and the sheet’s state when the sheet is hidden.

  • At most 20 pictures per workbook are extracted, in workbook order. Pictures on hidden sheets are included and labelled.
  • PNG, JPEG, GIF and WebP pictures are stored, scaled down to at most 1,568 pixels on the longer side. A GIF keeps its first frame.
  • A picture isn’t stored when its format is another one (EMF, WMF, SVG and so on), when it is too large (over 16 MiB in the workbook, over the pixel limit, over 4.5 MiB once scaled, or past 32 MiB of scaled pictures for the workbook), when it can’t be read, when it is a link to a file outside the workbook, when it is past the twentieth, or when the user’s storage quota is full. describe_workbook gives the reason in words. A picture that isn’t stored never stops the workbook from being indexed.
  • Pictures count toward the user’s storage quota, under Derived in the Files dialog.
  • Pictures are internal: they are not in the file list, and they can’t be downloaded, renamed, deleted or re-indexed on their own. They are deleted with the workbook, and when it is overwritten, redacted or erased, or its anonymous session ends. A re-index of the workbook extracts them again.
  • read_file shows a picture only to the owner of its workbook, and only while the workbook it came from is still the same file. After the workbook is deleted, overwritten or redacted, the picture’s id is not_found.

Every refusal is one fixed sentence. None of them repeats the model’s input, file text or an error.

WhenThe model is told
The id isn’t one of the user’s own files: a knowledge-base file, another user’s or session’s file, a derived file, or an id that doesn’t exist”No file with this id is in your files.” The same words for every cause, so the answer reveals nothing about other files.
The file isn’t a spreadsheet or CSV, an .xls included”describe_workbook works on spreadsheets and CSV files only.” (load_table and query_table say the same with their own name)
The file is still being indexed”This file is still being indexed; try again shortly.”
Indexing failed”This file could not be read, so it has no outline.” (load_table: “…so it cannot be loaded as a table.”)
An .ods file, or a spreadsheet with no text stored”This file has no workbook outline, because of its type or because no text is stored for it; use read_file or file_search to read it.” load_table on an .ods file: “load_table works on .xlsx workbooks and CSV/TSV files only.”
An .xlsx or CSV file with no outline yet: indexed before outlines existed, or changed since (for example redacted)“This file has no workbook outline yet (…); re-indexing the file builds it.” See Re-index. A CSV can still be loaded without one.
The file’s stored content was removed, as a redaction does for an Excel file”The source of this file has been removed.” Its loaded tables are deleted too.
The file is over 256 MB”The file is too large to load as a table (limit 256 MB).”
The area is over 1,000,000 rows, 512 columns or 20,000,000 cells, or a CSV has more than 1,000,000 records”The range is over the table cap (1,000,000 rows, 512 columns).”
The sheet name, A1 area or headerRows is wrong, or the area is outside the sheet’s used rangeA sentence that says which, such as “The range is outside the sheet’s used range A1:F200.”
query_table before any load_table”No table of this file is loaded; load it with load_table first.”
The file changed while its table was being built”The file changed while the table was being built; call load_table again.”

The table tools work on the user’s own files only. For a spreadsheet in a knowledge base, the assistant uses kb_search and read_kb_file: knowledge-base files have no outline, no table tools and no extracted pictures in this release.

  • No length cap. The full text of a file is stored and indexed, however long it is.
  • Word documents are read whole, so their size is bounded by FILE_INDEXING_DOCX_MAX_BYTES (default 268,435,456 bytes, 256 MiB). A larger .docx is failed with reason parser_limit, without being read.
  • PDF pages that take longer than FILE_INDEXING_PDF_PAGE_TIMEOUT_MS (default 30,000 ms) are skipped with a retryable page_unreadable warning.
  • Concurrent indexing per library is INDEXING_PER_OWNER_CONCURRENCY (default 2).
  • Every format has its own input size limit, and zip-based formats, mail and legacy Office files have more; see Parser limits.

Set these on every hub instance. A value that is empty, not a number, or not greater than zero falls back to the default. The two file limits are read when a file is indexed, and the concurrency at startup. Earlier releases’ EXTRACT_MAX_CHARS and FILE_INDEXING_MAX_BYTES are gone; setting them has no effect.

Administrators see how files are doing across the whole deployment on System → Files in the admin portal. It shows counts only, never one user’s files: chat files and knowledge-base files, each by status, by reason, how many have warnings, and by the embedding model of their current index.

The same tab starts a bulk Re-index of the files that match a filter: scope (chat, knowledge base or both), one or more states (any except processing), a reason, files with warnings only, or an embedding model, and the kind, extension or MIME type (see Re-classify stored files). Use it to retry the files a provider outage left search_unavailable, or to move every file onto a new embedding model. To give spreadsheets indexed before this release their outline and pictures, re-index the chat files of kind table.

The model name none means two slightly different things. In the counts, it covers files with no index and files whose index was built before DioscHub recorded the model. As a filter, it matches only files with no index. To reach files indexed before the model was recorded, filter by state instead.

  1. Choose the filter and select Estimate. The portal shows how many files match, how much text they hold and about how many embedding tokens re-indexing them will use.
  2. Confirm. The job starts in the background.

A bulk Re-index runs under the same per-library limit as uploads, and users’ uploads are indexed ahead of it. It never stops at a user’s storage quota, but the bytes it stores still show in that user’s usage afterwards. A file whose re-index fails keeps its previous status and index. Only one bulk Re-index can be queued or running at a time, and there is no cancel in the portal.

Viewing the counts and the estimate needs settings:read; starting a bulk Re-index needs settings:manage, and each start is recorded in the audit trail. Over the admin API:

EndpointReturns
GET /api/admin/file-indexing/status-countschat and kb: total, byStatus, byReason, withWarnings, byEmbeddingModel
GET /api/admin/file-indexing/reindex-estimate?scope=&status=&reason=&hasWarnings=&embeddingModel={ fileCount, textBytes, approxTokens, filesWithoutText, sourceBytes }
POST /api/admin/file-indexing/reindex with { filter, confirm: true, expectedFileCount }201 with { jobId, fileCount }

The start is refused with 400 confirm_required without confirm: true, 400 invalid_filter for a filter it doesn’t accept, 409 estimate_changed (with the new estimate) when more files match than you confirmed, and 409 bulk_running while another bulk Re-index is queued or running.

  • If a data erasure or redaction gives up waiting for a file’s indexing to stop, it fails with index_job_still_running, and that file can be left in processing with nothing to finish it. Run the erasure or redaction again.
  • After a redaction removes a binary file’s original bytes, downloading it (GET /api/files/:id/content) fails with 500 instead of a clear error. The assistant still reads the redacted text.
  • read_file marks only pages, sheet row ranges and lines. A PowerPoint, OpenDocument presentation or EPUB file gets line markers, not slides or chapters, and at: { "page": N } on one is refused; read it by line or from a hit’s startOffset. Search hits from these files have location: null, and still carry startOffset.
  • A sheet whose name contains " or a line break gets no sheet marker, and can’t be reached with at: { "sheet": … }.

Related: Frontend integration · File storage limits, quota & retention · Knowledge bases · Read and write files from a tool