File storage limits, quota & retention
Chat files are the files users upload in the chat, the files tools generate for them, and the internal files DioscHub derives from those (cached tables, extracted pictures). Three controls govern them:
- an upload size limit — the largest single file any upload path accepts;
- a storage quota — how much one user may hold, in bytes and in files;
- retention — how long a file is kept before DioscHub deletes it.
All three are set as a system default on System → Rate Limits → File uploads, and each can be overridden per Role in the role dialog’s Budget section (see Assistants & Roles). An empty role field inherits the system value. The role that applies is the one DioscHub resolved for the user when the session was bound — never a value the browser or a tool sends.
None of these controls is gated by tier: where users can upload files at all, the limit, the quota and retention are available. Changes to the settings and to a Role are recorded in the audit trail.
What a file is — its kind, whether it is read, and the limits on reading it — is separate from these
controls; see File types, kinds & parser limits, which also has the table tool
limits. Every upload path below — the
widget, MCP put_file, and knowledge-base upload and sync — classifies the file once it has the bytes and
refuses an executable with 415 FILE_TYPE_REFUSED; any other type is stored.
Upload size limit
Section titled “Upload size limit”| Setting | Default | Range |
|---|---|---|
| Upload limit (MB) (system) | 100 MB (104,857,600 bytes) | 1 MB to 1024 MB |
| Upload limit (MB) (role) | empty — inherits the system limit | 1 MB to 1024 MB |
The ceiling, 1024 MB (1 GiB, 1,073,741,824 bytes), is fixed in code. The admin API rejects a value
outside the range with 400. A role override applies whether it is higher or lower than the system
limit. The limit is read on every request, so a change applies to the next upload without a restart; an
upload already in progress keeps the limit it started with.
The same limit is enforced on every path that writes a file:
| Path | Limit used |
|---|---|
Upload in the widget (POST /api/files/upload) — file picker, drag and drop, paste, and files delivered by client-side tools | Role, else system |
MCP put_file (POST /api/mcp/files/upload) | The session’s Role, else system |
generate_file — the model tool and POST /api/files/generate | Role, else system. The rendered file is checked before anything is written. |
bundle_files | Role, else system. The summed size of the inputs is checked before any input is read, and the archive written is capped at the limit too. |
Knowledge-base upload (POST /api/admin/knowledge/:id/files/upload) and sync (POST /api/admin/knowledge/:id/sync) | System limit only — no role override |
Uploads stream straight to storage, so the hub stops reading a body as soon as it passes the limit. A file
exactly at the limit is accepted. A refused upload leaves nothing behind — no stored bytes and no file
record. The refusal is HTTP 413:
{ "statusCode": 413, "code": "FILE_TOO_LARGE", "limitBytes": 104857600, "message": "File too large"}The generate_file and bundle_files tools return an error result carrying the same code and
limitBytes, so the model can tell the user why.
The widget reads the limit from the hub (GET /api/files/policy, below) and refuses an oversize file
before sending any bytes, with a message such as “report.pdf is larger than the 100 MB upload limit”.
There is no client-side limit to configure; the hub’s limit is the only one.
Storage quota
Section titled “Storage quota”| Setting | Default | Range |
|---|---|---|
| Storage quota (MB) (system) | empty — no quota | 1 MB to 1,048,576 MB (1 TiB) |
| File count limit (system) | empty — no limit | 1 to 1,000,000 files |
| Storage quota (MB), File count limit (role) | empty — inherits the system value | same ranges |
Each field inherits separately, so a Role can set a byte quota and keep the system file count. The quota is per user, per Assistant: it covers that user’s file library on that assistant.
What counts
Section titled “What counts”A user’s storage used is the sum, over every file in their library, of:
- the file itself — uploaded files, generated files (
generate_file,bundle_files, MCPput_file), and the derived files DioscHub keeps for them; - the extracted text of the file, which is kept in the file store next to the file;
- the search index built from it (chunk text, embeddings and chunk metadata).
The file count counts the user’s own files; derived files are not counted. Knowledge-base files never count toward a user’s quota. Expired files (see Retention) stop counting as soon as they expire, before the sweep removes them.
Extracted text and index bytes are recorded while the file is indexed, after the upload was accepted.
Indexing checks the quota before each batch of the search index is built. If the index would take the user
past the quota, indexing stops there: the file is kept, its text is stored, and it reports the status
search_unavailable with the reason over_quota. The assistant can still read it, but file_search can’t
return passages from it until the user frees space and re-indexes it. See Chat files &
indexing.
How it is enforced
Section titled “How it is enforced”Every write that creates or replaces one of the user’s files checks the quota: widget upload, MCP
put_file, generate_file and bundle_files. The check runs in the same Postgres transaction that
records the file, under a per-user lock, so two uploads racing each other — in one instance or across a
cluster — can’t both pass. Uploads by different users never wait on each other.
- A write is refused when the bytes it adds would take storage used past the byte quota, or when a new file would take the count past the file count limit.
- Replacing a file counts only the growth. A replacement that does not grow the library is always allowed and adds no file.
- Derived files are counted. Cached tables and workbook pictures are checked against the byte quota as
they are stored: over it, a table still answers the assistant but isn’t kept (it stays usable on that
hub instance for up to an hour), and a picture isn’t stored (
describe_workbooksays the storage quota is full). Other derived files are never refused; the next upload is refused instead. - An upload that would take the library past the byte quota first deletes that library’s cached tables, least recently used first, until the upload fits. A deleted table is built again the next time the assistant loads it. If deleting all of them wouldn’t make room, none is deleted and the upload is refused. Pictures and the user’s own files are never deleted to make room.
- The hub checks the quota once it has received the file. The widget checks first from the cached policy, so in practice an over-quota upload from the widget is refused before any bytes are sent.
- Lowering a quota below what users already hold deletes nothing. It only blocks new uploads until they free space. Apart from cached tables (above), DioscHub never evicts files to make room.
A refused write leaves no stored bytes and no file record. The refusal is HTTP 413:
{ "statusCode": 413, "code": "QUOTA_EXCEEDED", "message": "Storage quota exceeded", "usageBytes": 5767168, "quotaBytes": 6291456, "usageFiles": 4, "quotaFiles": 20, "requiredBytes": 2097152}quotaBytes or quotaFiles is null when that dimension has no quota. requiredBytes is what the
refused write needed: the file’s size, or the growth of a replacement. The generate_file and
bundle_files tools return the same fields in their error result.
What the user sees
Section titled “What the user sees”When an upload is refused for the quota, the widget shows a toast — “Storage full: 5.5 MB of 6 MB used”, or “Storage full: 20 of 20 files used” when only the file count is full — and opens the Files dialog sorted largest first. The dialog’s header shows storage used against the quota, and each file shows its breakdown:
| Column | What it is |
|---|---|
| File size | The file itself |
| Search index | The search index, extracted text included |
| Derived | The derived files kept for it |
| Total | What the file costs against the quota; sorting by size uses this |
Opened normally, the dialog shows the same header and columns, sorted newest first.
What lowers usage
Section titled “What lowers usage”- Deleting a file from the Files dialog (or
DELETE /api/files/:id) removes the file, its extracted text, its search index and its derived files. - Data erasure of a file, a session or a user removes the erased files and everything counted for them.
- Redaction of a file sets its size to the rewritten file (zero when the file’s bytes are removed), and its text and index bytes are recomputed from the redacted text and the new index.
- Retention: an expired file stops counting at once and is deleted by the next sweep.
Retention
Section titled “Retention”| Setting | Default | Range |
|---|---|---|
| File retention (days) (system) | empty — off, files are kept until deleted | 1 to 3650 days |
| File retention (days) (role) | empty — inherits the system value | 0 to 3650 days; 0 means this role’s files never expire |
Retention is off by default. When it is on, each file is stamped with an expiry when it is written — the write time plus the retention period of the user’s Role, else the system’s.
- It applies to files written after you set it. Turning retention on, changing the period or turning it off does not re-stamp files that already exist, and an expiry already stamped is honoured.
- Replacing a file re-stamps it from the current setting. Renaming does not change its expiry.
- Derived files have no expiry of their own; they are deleted with the file they came from.
- A file whose expiry has passed is expired. From that moment it is hidden everywhere, even before
the sweep deletes it: the Files list (
GET /api/files), the session’s file list, attachments on new messages and the file list the model is given,file_search,read_file(not_found),bundle_files(reported missing), MCPget_file(404), and every by-id route — content, download, rename and delete answer404as for a file that does not exist. It also stops counting toward the quota.
The sweep
Section titled “The sweep”A sweep deletes expired files through the normal delete path: the stored bytes, the extracted text, the
search index, the file record and its derived files. It runs hourly by default; set CHAT_FILE_RETENTION_SWEEP_CRON (a
cron expression) to change the schedule. In a cluster it runs on one instance per run, under a
distributed lock. Each run handles at most 2,000 expired files and leaves the rest for the next run. A
file that was replaced or deleted after the sweep picked it is skipped.
Each run logs one summary line, Chat-file retention sweep finished, with the counts expired (files
examined), deleted, failed, and endedSessions (files of ended
anonymous sessions deleted in the same run). A file
that fails is logged and retried on the next run.
What the user sees
Section titled “What the user sees”In the Files dialog, a file with an expiry shows “Expires in N days”, N rounded up to whole days (minimum 1; “Expires in 1 day”). Files that do not expire show nothing.
Anonymous visitors’ files follow a separate, fixed rule — at most 24 hours. See Anonymous sessions.
The retention window removes unused files too
Section titled “The retention window removes unused files too”Separate from file retention, the conversation retention window on the Data Privacy page, when it is on, deletes a user’s file once its last use (read, attached or downloaded; else its upload) is older than the window, whether or not the file has an expiry. When both are on, whichever comes first removes the file. The Files dialog’s “Expires in N days” shows file retention only.
The upload policy endpoint
Section titled “The upload policy endpoint”The widget reads the rules that apply to the current user from one endpoint, authenticated like its other file calls (the widget’s REST token, or an anonymous visitor’s session-scoped files token):
GET /api/files/policy
{ "limitBytes": 104857600, "usage": { "bytes": 3145728, "files": 4 }, "quota": { "bytes": 5368709120, "files": null }, "acceptedTypes": [".7z", ".avi", ".avif", ".bmp", ".bz2", ".c", ".cpp", ".css", ".csv", ".zip", "application/x-hl7"]}(The acceptedTypes list is shortened here; the real one names every built-in extension.)
| Field | Meaning |
|---|---|
limitBytes | The upload limit for this user’s Role (role override, else system). |
usage.bytes, usage.files | Storage used and file count, counted as in What counts. |
quota.bytes, quota.files | The quota for this user’s Role; null = no quota on that dimension. |
acceptedTypes | The widget picker’s list: a . extension for every built-in type and every extension rule (sorted), then the MIME type of every MIME rule (sorted). The same for every caller. A hint for the picker, not a gate; see File types. |
It answers only for the caller — the user and Assistant of the token, and the Role resolved at bind; no
parameter can name another user. It sits behind the same checks as the other /api/files routes: no
valid token is 401, and a Role without the files feature is 403. For an anonymous visitor with file
upload enabled, it reports the session’s usage, and quota holds the per-session caps.
The widget caches the policy for up to a minute and drops the cache after an upload, a delete or a
re-bind. The hub stays authoritative: if the limit or quota changed in the meantime, the hub’s 413
refusal wins and the widget adopts the values it carries.
Large uploads behind a reverse proxy
Section titled “Large uploads behind a reverse proxy”Uploads stream to storage, so the hub’s memory does not grow with file size. Whatever sits in front of the hub still has to let a large body through, quickly enough.
What the image ships
Section titled “What the image ships”The web server inside the DioscHub image (nginx) has a dedicated location for the four upload routes —
/api/files/upload, /api/mcp/files/upload, /api/admin/knowledge/<id>/files/upload and
/api/admin/knowledge/<id>/sync:
location ~ ^/api/(files/upload|mcp/files/upload|admin/knowledge/[^/]+/files/upload|admin/knowledge/[^/]+/sync)$ { proxy_request_buffering off; # stream the body to the hub, don't spool it first proxy_read_timeout 1800s; proxy_send_timeout 1800s; client_max_body_size 1100m; # the 1 GB ceiling plus multipart headroom # proxy_pass and headers as for /api/}Every other /api/ route keeps a 100m body limit and 300s timeouts. That includes
POST /api/files/generate, so with the shipped configuration a generate request whose body is over
100 MB is stopped by nginx before the hub sees it.
The hub’s own HTTP request timeout is 30 minutes, so a ceiling-sized upload on a slow link is not cut
off. Override it with HTTP_REQUEST_TIMEOUT_MS (a positive whole number of milliseconds). The
header-phase timeout keeps Node’s default.
Your own proxy, load balancer or CDN
Section titled “Your own proxy, load balancer or CDN”If you put another proxy, load balancer or CDN in front of the image, give the upload routes the same treatment:
- Body size at least your largest upload limit plus some headroom for multipart framing. nginx’s own
default
client_max_body_sizeis only 1 MB. - Request buffering off, so the body streams through instead of being written to the proxy’s disk first.
- Timeouts at least as long as the hub’s request timeout.
A CDN may cap request bodies regardless of your configuration — Cloudflare, for example, caps them by plan
(100 MB on Free and Pro, 200 MB on Business, 500 MB on Enterprise by default). Keep the upload limit at or
below the smallest cap on the path. A body stopped at the edge never reaches the hub, so the user gets the
edge’s error instead of a FILE_TOO_LARGE refusal that names the limit.
Related: File types, kinds & parser limits · System settings · Rate limiting & cost controls · Read and write files from a tool