REST handles commands.
Uploads, review edits, job control, and publication go through authenticated REST endpoints. The server checks permissions before changing data.
Architecture
How DerridAI is built
DerridAI keeps source files, record revisions, and review decisions separate from the indexes and models that use them. You can rebuild an index or switch models while preserving the research record.
One corpus, two paths
Follow the records into a publication or see how they support a research answer.
Corpus and publication
Source material
Documents and recordings with page, passage, or time locators.
Versioned records
SQLite keeps record revisions, metadata assertions, and review decisions.
Published corpus
An immutable snapshot can be exported as JSONL or a research site.
Search indexes can be rebuilt from the versioned records.
Research pipeline
Search index
Keyword and semantic indexes help find relevant passages.
Evidence packet
Selected passages travel with their source and citation metadata.
Cited answer
A model synthesizes the evidence. Citation binding resolves source locations.
Runtime and interfaces
The browser and the server
The Docker Compose stack starts a Vue 3 frontend served by nginx and a FastAPI backend. The API owns authentication, scholarly state, and background work. Model providers and an external Chroma server can be configured separately.
GraphQL exposes typed reads.
The read-only GraphQL endpoint composes cELF queries over shared backend services. It exposes scholarly objects without adding a separate mutation path.
WebSocket signals changes.
Realtime events report job and corpus activity. Clients use REST or GraphQL to refresh authoritative state after a change or reconnect.
Storage and provenance
How corpus data is stored
Corpus Builder registers sources, extracts or transcribes them, and maps records to pages, text spans, or audio intervals. Metadata enrichment proposes assertions. Review decisions preserve the assertion’s origin and supporting evidence.
Canonical corpus state
Source assets and extracted blocks retain documentary identity. Interactive build records live in per-build SQLite databases. Publication and export produce JSONL projections of the corpus state.
Durable application state
Separate SQLite stores hold authentication data and operational state such as job history, provider profiles, annotations, and provenance. Record identity and evidence links remain distinct from run history.
Rebuildable projections
Chroma stores search indexes and semantic projections. Reviewed metadata precedents can guide later enrichment. Cached responses and research memory serve different purposes and remain separate from authoritative corpus records.
Evidence and generation
From retrieval to citation
Pipeline Studio shows the steps in a run and the settings each step uses. Retrieval ranks passages by relevance. Researchers can then check whether those passages support the interpretation.
Assemble the evidence.
Combine dense and lexical search, diversity selection, deduplication, and optional reranking. A researcher can also select passages directly. The resulting evidence packet carries source identity and citation metadata into generation.
Resolve citations in code.
Where stored identifiers and locators resolve exactly, deterministic code handles citation binding. The model synthesizes from the evidence packet. Claim support can be inspected against the passage and record revision.
Evaluate the run.
Keep retrieval and generation settings visible so runs can be compared. Check attribution and evidence fidelity alongside latency. A structural or provenance check establishes only the condition it tested.
Extension points and limits
What can be configured
Metadata schemas, versioned pipelines, and provider profiles let teams adapt their workflow. The underlying records keep stable identities as processing choices change.
Providers and analysis
Use a local Ollama service or an OpenAI-compatible endpoint. Optional Document Intelligence tools contribute derived linguistic annotations, such as entities and quotation attribution. Those annotations can guide analysis without changing source authority.
Publication and integration
Export a portable corpus snapshot with the browser SDK, or use the typed read API from a custom client. A published snapshot represents a specific corpus state and can be served separately from the main application.
Deployment boundary
Background jobs run in one API process. Job history is durable, while realtime notifications and in-flight execution are process-local. The current architecture does not coordinate active jobs across multiple API processes.