Document RAG Indexing Structures

The Asterisk 2 AI integration relies on structured relational models to serve Retrieval-Augmented Generation (RAG) prompts. The database schema stores pre-chunked markdown text strings and multi-dimensional vector array weights to speed up semantic natural language matching.

Semantic Indexing Pipeline Map

The diagram below traces how documentation strings break down into embedded contextual knowledge vectors:

graph TD
    %% Document Lifecycle
    subgraph StorageTier["Raw Documentation Nodes"]
        PageNode["Page Record<br/>(Full Unparsed Markdown String)"]
    end

    %% Auto Chunking logic
    subgraph ParserTier["Markdown Auto-Chunker Tier"]
        RechunkWorker["RechunkPage Action<br/>(Parses text by headers)"]
    end

    %% Pre-computed Data Store
    subgraph EmbeddingTier["Pre-Computed Vector Ledger"]
        ChunkRow[("PageChunks Table<br/>(PK: ChunkId, Text: String)")]
        VectorRow[("ContextEmbedding Models<br/>(Floating Point Arrays)")]
    end

    %% Consumer Tier
    subgraph AIClient["Semantic Matching Engine"]
        SearchApi["SearchController.cs<br/>(Cosine Similarity Matching)"]
    end

    %% Path Links
    PageNode -->|"Update Trigger"| RechunkWorker
    RechunkWorker ==>|"Extract Semantic Chunks"| ChunkRow
    ChunkRow ==>|"Generate Array Weight"| VectorRow
    VectorRow ==>|"High-Speed Lookup"| SearchApi

    %% Curated Palette Tokens
    classDef rawToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef parseToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef embedToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef aiToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;

    class StorageTier,PageNode rawToken;
    class ParserTier,RechunkWorker parseToken;
    class EmbeddingTier,ChunkRow,VectorRow embedToken;
    class AIClient,SearchApi aiToken;

Structural Engine Properties

1. Chunk Granularity Criteria

To ensure high accuracy during RAG prompt inclusion generation, markdown content strings break down into discrete schema rows bounded cleanly by structural headers.

2. Multi-Tenant Vector Segregation

Every pre-computed chunk row maps to foreign key references linking the text array directly to parent document configurations. This completely blocks prompt search scripts from retrieving document chunks belonging to unauthorized client accounts.

# Document RAG Indexing Structures

The **Asterisk 2** AI integration relies on structured relational models to serve Retrieval-Augmented Generation (RAG) prompts. The database schema stores pre-chunked markdown text strings and multi-dimensional vector array weights to speed up semantic natural language matching.

---

## Semantic Indexing Pipeline Map

The diagram below traces how documentation strings break down into embedded contextual knowledge vectors:

```mermaid
graph TD
 %% Document Lifecycle
 subgraph StorageTier["Raw Documentation Nodes"]
 PageNode["Page Record (Full Unparsed Markdown String)"]
 end

%% Auto Chunking logic
 subgraph ParserTier["Markdown Auto-Chunker Tier"]
 RechunkWorker["RechunkPage Action (Parses text by headers)"]
 end

%% Pre-computed Data Store
 subgraph EmbeddingTier["Pre-Computed Vector Ledger"]
 ChunkRow[("PageChunks Table (PK: ChunkId, Text: String)")]
 VectorRow[("ContextEmbedding Models (Floating Point Arrays)")]
 end

%% Consumer Tier
 subgraph AIClient["Semantic Matching Engine"]
 SearchApi["SearchController.cs (Cosine Similarity Matching)"]
 end

%% Curated Palette Tokens
    classDef rawToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef parseToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef embedToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef aiToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;

class StorageTier,PageNode rawToken;
    class ParserTier,RechunkWorker parseToken;
    class EmbeddingTier,ChunkRow,VectorRow embedToken;
    class AIClient,SearchApi aiToken;
```

---

## Structural Engine Properties

### 1. Chunk Granularity Criteria
To ensure high accuracy during RAG prompt inclusion generation, markdown content strings break down into discrete schema rows bounded cleanly by structural headers.

### 2. Multi-Tenant Vector Segregation
Every pre-computed chunk row maps to foreign key references linking the text array directly to parent document configurations. This completely blocks prompt search scripts from retrieving document chunks belonging to unauthorized client accounts.