Artifacts

Agents and workflows often produce files: a generated report, an exported CSV, an image. mayura/artifacts stores such files on the local filesystem. Each file is addressed by its SHA-256 digest, kept apart per tenant, checked again every time it is read, and handed out only as a download under a policy you choose. You get back a small JSON reference, which you save next to your own records.

import { createLocalArtifactStore } from 'mayura/artifacts';

const artifacts = createLocalArtifactStore({
  rootDirectory: '/var/lib/my-app/artifacts',
  maxArtifactBytes: 10 * 1024 * 1024,
});
const scope = { principalId: 'acme', projectId: 'reports' };

const staged = await artifacts.stage({
  scope,
  content: new TextEncoder().encode('order,total\nord-1001,42.00\n'),
  mediaType: 'text/csv',
  classification: 'internal',
  filename: 'orders.csv',
});
const reference = await artifacts.commit(staged);
// Save `reference` (plain JSON) with your own record, for example the workflow run that produced it.

const bytes = await artifacts.read(reference, scope);

#Store options

#Stage, commit, discard

Storing is two steps. stage writes the bytes to a private staging area and computes their digest. commit checks the staged file again and moves it into place atomically, returning the reference. If you decide not to keep a staged file, call discard(staged).

A stage takes:

Committing the same bytes with the same metadata in the same scope returns the same reference. References are frozen JSON objects that bind the scope, digest, size, media type, classification, filename and expiry together; changing any field makes the reference invalid.

#Read and download

read(reference, scope) returns the bytes after checking that the reference is intact, the scope matches, the file exists, and its size and digest still match. A tampered or missing file fails with an integrity error and returns nothing. A reference for another scope fails with PERMISSION_DENIED.

disclose prepares a download for an HTTP response:

const download = await artifacts.disclose(reference, scope, {
  classifications: ['public', 'internal'],
  maxBytes: 5 * 1024 * 1024,
  mediaTypes: ['text/csv', 'application/pdf'],
});
// download.body is the bytes; download.headers has Content-Type, Content-Length,
// Content-Disposition: attachment, and X-Content-Type-Options: nosniff.

A disclosure is always an attachment, never rendered inline. HTML, SVG and XML types are refused even when their classification is allowed, so a stored file cannot run script in your site. mediaTypes narrows the allowed types further.

#Delete, clean up and audit

• delete(reference, scope) removes one committed file.
• reconcileStaging({ olderThan, maxDeletes }) removes staged files older than a cutoff, left behind when a process stopped between stage and commit. Run it at startup or on a timer.
• audit(references, scope, { maxTotalBytes }) checks a list of references and reports each as ok, missing, expired or integrity_failed, without returning content.
• planReconciliation and applyReconciliation delete committed files that your records no longer reference. You pass the complete set of references you keep for the scope, an age cutoff and limits; the plan lists candidates, and applying it rechecks each file and skips any that changed. A plan can be applied once, by the store that made it.

#Back up and restore

backup({ scope, references, authoritativeSetComplete: true, maxTotalBytes }) returns one archive (bytes) for a whole scope, with an integrity digest over its contents. restore(archive, scope, { maxArchiveBytes, maxTotalBytes, maxArtifacts }) installs the files into a store that is empty or already holds an exact subset of the archive, so an interrupted restore can simply be retried. Run audit after a restore, before reopening access.

An archive holds at most 256 files and 64 MiB of content. It is not encrypted: store it somewhere protected, and keep it together with the database backup that holds your references, because restoring files alone does not restore the records that point at them.

#Good to know

• Holding a reference is not permission to read the file. Authenticate the caller, derive the scope from their verified identity, and check they may see the record the artifact belongs to.
• The store is local to one host. It is not a shared network filesystem or an object store, and it does not scan for malware, encrypt files or schedule backups.
• Your database and the artifact folder are not updated in one transaction. Keep references in your records, and use reconcileStaging and the reconciliation plan to clean up after crashes.
• To download from a remote source into any staging sink with a size limit and digest check, see transferArtifact in helpers.

#Related

• Storage
• Durable workflows
• Server and client
• Helpers