Back to page

Vector Context Embedding Matrices

Raw Markdown & AI/RAG Chunks
Raw Markdown Source 2451 chars
# Vector Context Embedding Matrices

To serve natural language document lookups efficiently, **Asterisk 2** maps document contents to dense vector arrays. The database schema manages multi-dimensional floating point representations allowing search controllers to rank text snippets via mathematical similarity matching.

---

## Vector Store Layout Architecture

The flow block below illustrates how numerical arrays store semantic context weights for rapid processing:

```mermaid
graph LR
    %% Text Source
    subgraph TextSource["Pre-Parsed Chunk Source"]
        ChunkEntity["PageChunk Row<br/>(Plain Text Content String)"]
    end

    %% Math Operations
    subgraph MathematicalTier["Embedding Processing Layer"]
        ModelMatrix["Vector Float Array<br/>(Dimensions: 1536 Floats)"]
    end

    %% Storage Node
    subgraph StorageLedger["Relational Vector Ledger"]
        VectorRow[("ContextEmbeddings Array<br/>(Binary Indexed Bytes)")]
    end

    %% Search Node
    subgraph SearchEvaluation["Cosine Distance Engine"]
        EvalCosine["Rank by Highest Similarity Score"]
    end

    %% Links
    ChunkEntity -->|"Extract Context"| ModelMatrix
    ModelMatrix ==>|"Serialize Vector Array"| VectorRow
    VectorRow ==>|"Cosine Math Matching"| EvalCosine

    %% Theming Customization
    classDef sourceToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef mathToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef storeToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;

    class TextSource,ChunkEntity sourceToken;
    class MathematicalTier,ModelMatrix mathToken;
    class StorageLedger,VectorRow storeToken;
    class SearchEvaluation,EvalCosine mathToken;
```

---

## Model Attribute Configuration

### 1. High-Density Array Mappings
- **`EmbeddingId`** (`long`): Primary automatic auto-increment integer ID.
- **`ChunkId`** (`int`): Foreign key matching the embedding matrix directly to specific raw markdown sub-documents.
- **`VectorBytes`** (`byte[]`): Compact serialized binary columns retaining multi-dimensional floating point weights natively.

### 2. Index Efficiency Rules
To maximize search sorting performance, vector tables completely bypass Entity Framework Core change tracking memory allocations (`AsNoTracking`). Database connections fetch matching binary arrays to execute fast memory calculations instantly.
AI Chunks (RAG) 3 chunks
Chunk #1 Vector Context Embedding Matrices
# Vector Context Embedding Matrices

To serve natural language document lookups efficiently, **Asterisk 2** maps document contents to dense vector arrays. The database schema manages multi-dimensional floating point representations allowing search controllers to rank text snippets via mathematical similarity matching.

---
Chunk #2 Vector Store Layout Architecture
## Vector Store Layout Architecture

The flow block below illustrates how numerical arrays store semantic context weights for rapid processing:

```mermaid
graph LR
    %% Text Source
    subgraph TextSource["Pre-Parsed Chunk Source"]
        ChunkEntity["PageChunk Row<br/>(Plain Text Content String)"]
    end

    %% Math Operations
    subgraph MathematicalTier["Embedding Processing Layer"]
        ModelMatrix["Vector Float Array<br/>(Dimensions: 1536 Floats)"]
    end

    %% Storage Node
    subgraph StorageLedger["Relational Vector Ledger"]
        VectorRow[("ContextEmbeddings Array<br/>(Binary Indexed Bytes)")]
    end

    %% Search Node
    subgraph SearchEvaluation["Cosine Distance Engine"]
        EvalCosine["Rank by Highest Similarity Score"]
    end

    %% Links
    ChunkEntity -->|"Extract Context"| ModelMatrix
    ModelMatrix ==>|"Serialize Vector Array"| VectorRow
    VectorRow ==>|"Cosine Math Matching"| EvalCosine

    %% Theming Customization
    classDef sourceToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef mathToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef storeToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;

    class TextSource,ChunkEntity sourceToken;
    class MathematicalTier,ModelMatrix mathToken;
    class StorageLedger,VectorRow storeToken;
    class SearchEvaluation,EvalCosine mathToken;
```

---
Chunk #3 Model Attribute Configuration
## Model Attribute Configuration

### 1. High-Density Array Mappings
- **`EmbeddingId`** (`long`): Primary automatic auto-increment integer ID.
- **`ChunkId`** (`int`): Foreign key matching the embedding matrix directly to specific raw markdown sub-documents.
- **`VectorBytes`** (`byte[]`): Compact serialized binary columns retaining multi-dimensional floating point weights natively.

### 2. Index Efficiency Rules
To maximize search sorting performance, vector tables completely bypass Entity Framework Core change tracking memory allocations (`AsNoTracking`). Database connections fetch matching binary arrays to execute fast memory calculations instantly.