Knowledge, Memory, Embeddings, and RAG
1. Purpose
The Site Application must distinguish knowledge, retrieval, and memory. Treating them as one storage concept creates weak architecture and incorrect security boundaries.
2. Core Concepts
| Concept | Meaning | Typical Persistence |
|---|---|---|
| Knowledge source | Authoritative source material available for retrieval | Document/object store |
| Chunk | Retrievable segment derived from source content | Retrieval index/vector store |
| Embedding | Floating-point vector representation of content | Vector store/index |
| Metadata | Source, object, site, version, classification, permissions | Vector store / metadata store |
| Retrieval | Search operation that finds relevant chunks | Runtime operation |
| Re-ranking | Secondary scoring/reordering of candidates | Runtime operation |
| RAG | Supplying retrieved context to a model before generation | Runtime pattern |
| Conversation state | Messages/events in a current or prior dialogue | Session store |
| Long-term memory | Durable user/task facts or learned state | Memory service/store |
3. Embedding Representation
An embedding is a list/vector of floating-point numbers representing semantic properties of data. It is not simply a binary conversion of the document.
flowchart TD
S(["Source Content"]):::actor --> P["Pre-processing"]:::process
P --> C["Chunking"]:::process
C --> E[["Embedding Model"]]:::model
E --> V["Floating-point Vector"]:::store
V --> I[("Vector Index")]:::store
classDef actor fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1px,color:#0B1220
classDef process fill:#EEF2F7,stroke:#475569,stroke-width:1px,color:#0B1220
classDef model fill:#EDE9FE,stroke:#6D28D9,stroke-width:1px,color:#0B1220
classDef store fill:#CCFBF1,stroke:#0F766E,stroke-width:1px,color:#0B1220
Similarity search compares vectors using a distance or similarity function such as cosine similarity, depending on the chosen engine.
4. Knowledge Ingestion Flow
flowchart TD
D(["Document / Text / Object Data"]):::actor --> E["Content Extractor"]:::process
E --> N["Normalization"]:::process
N --> C["Chunking"]:::process
C --> M[["Embedding Model"]]:::model
M --> V["Embeddings"]:::store
C --> MD["Metadata"]:::store
V --> S[("Vector Store")]:::store
MD --> S
classDef actor fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1px,color:#0B1220
classDef process fill:#EEF2F7,stroke:#475569,stroke-width:1px,color:#0B1220
classDef model fill:#EDE9FE,stroke:#6D28D9,stroke-width:1px,color:#0B1220
classDef store fill:#CCFBF1,stroke:#0F766E,stroke-width:1px,color:#0B1220
Metadata should retain enough information to enforce traceability and authorization.
Recommended metadata:
source_id: document-or-object-id
source_type: document
site_id: site-id
application_id: application-id
business_object_id: optional-business-object-id
version_id: source-version
chunk_id: stable-chunk-id
content_hash: hash
classification: internal
acl_ref: permission-reference
language: en
created_at: timestamp
5. Chunking
Chunk sizes such as 1000-1500 characters with 200-300 characters of overlap can be used as tuning examples, but they are not a platform standard.
Chunking should be configurable by:
- token count;
- semantic section boundaries;
- headings/paragraphs;
- document type;
- overlap;
- maximum context budget;
- model tokenizer;
- retrieval quality tests.
For comparison, OpenAI-managed vector stores currently document a default static chunking strategy of 800 tokens with 400 tokens overlap, and allow configuration within documented limits. That is an implementation default, not a Site-wide standard.
6. Retrieval Flow
flowchart TD
Q(["User / Agent Query"]):::actor --> N["Query Normalization"]:::process
N --> E["Query Embedding"]:::model
E --> A["Metadata / ACL Filter"]:::govern
A --> S["Vector / Hybrid Search"]:::store
S --> C["Candidate Chunks"]:::store
C --> R["Optional Re-ranking"]:::process
R --> T["Top Context"]:::process
T --> L[["LLM / Agent"]]:::model
classDef actor fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1px,color:#0B1220
classDef process fill:#EEF2F7,stroke:#475569,stroke-width:1px,color:#0B1220
classDef model fill:#EDE9FE,stroke:#6D28D9,stroke-width:1px,color:#0B1220
classDef store fill:#CCFBF1,stroke:#0F766E,stroke-width:1px,color:#0B1220
classDef govern fill:#FFE4E6,stroke:#BE123C,stroke-width:1px,color:#0B1220
The platform should support:
- vector similarity search;
- metadata filtering;
- hybrid lexical + semantic search where appropriate;
- re-ranking;
- minimum score thresholds;
- top-k controls;
- source attribution;
- permission-aware retrieval.
7. RAG Does Not Store Documents Inside the LLM
The source material remains in external storage/indexes. The runtime retrieves relevant context and sends selected content to the model for a request.
flowchart TD
K[("Knowledge Store")]:::store --> R["Retrieve selected authorized chunks"]:::govern
R --> P["Prompt Context + User Query"]:::process
P --> M[["Model"]]:::model
M --> O(["Grounded Response"]):::output
classDef process fill:#EEF2F7,stroke:#475569,stroke-width:1px,color:#0B1220
classDef model fill:#EDE9FE,stroke:#6D28D9,stroke-width:1px,color:#0B1220
classDef store fill:#CCFBF1,stroke:#0F766E,stroke-width:1px,color:#0B1220
classDef govern fill:#FFE4E6,stroke:#BE123C,stroke-width:1px,color:#0B1220
classDef output fill:#DCFCE7,stroke:#15803D,stroke-width:1px,color:#0B1220
This distinction is important for updates, deletion, access control, audit, and versioning.
8. Memory vs Knowledge
Knowledge
Knowledge is usually shared, source-backed, versioned, and permission-controlled.
Examples:
- engineering manuals;
- business procedures;
- equipment specifications;
- Site documents;
- standards;
- package data.
Conversation Memory
Conversation memory preserves dialogue state so the assistant or agent can continue coherently.
Examples:
- previous user message;
- current task state;
- previous tool result;
- unresolved question.
Long-Term Memory
Long-term memory stores durable information derived from interactions or operations. It requires stronger governance because it can outlive the source conversation.
Recommended controls:
- explicit memory type;
- retention policy;
- provenance;
- owner/user scope;
- sensitivity classification;
- update/forget mechanism;
- audit trail.
9. Knowledge Source Configuration
A Site capability can bind one or more knowledge sources.
knowledge:
sources:
- type: document-package
ref: equipment-specifications
- type: business-object
ref: asset
retrieval:
strategy: hybrid
top_k: 8
rerank: true
authorization:
enforce_source_acl: true
10. Business Object Context
Knowledge should be attachable to application- and business-object-level context.
flowchart TD
S(["Site"]):::actor --> A[["Application"]]:::process
A --> AK[("Application Knowledge Sources")]:::store
S --> B[("Business Object: Asset")]:::store
B --> D[("Documents")]:::store
B --> SP[("Specifications")]:::store
B --> T[("Telemetry References")]:::store
B --> BK[("Business-object Knowledge Sources")]:::store
classDef actor fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1px,color:#0B1220
classDef process fill:#EEF2F7,stroke:#475569,stroke-width:1px,color:#0B1220
classDef store fill:#CCFBF1,stroke:#0F766E,stroke-width:1px,color:#0B1220
An assistant or agent can then retrieve information within the correct business scope instead of searching a global unfiltered corpus.
11. Multimodal Sources
A file ingestion pipeline may accept text, documents, images, audio, or video, but each modality requires an explicit extraction strategy. For example, video can be processed through transcripts, sampled frames, scene segmentation, or multimodal models. Converting every video frame to text by default is inefficient and should not be the platform rule.
12. Reliability and Security
Minimum controls:
- permission filters before context is exposed to the model;
- version-aware re-indexing;
- deletion propagation;
- source provenance;
- embedding-model version tracking;
- failed-ingestion state;
- retry/idempotency;
- data-classification enforcement;
- encryption and secret isolation;
- retrieval telemetry without leaking sensitive content.