Back to page

Pre-Computed Markdown Chunk Models

Raw Markdown & AI/RAG Chunks
Raw Markdown Source 2817 chars
# Pre-Computed Markdown Chunk Models

To avoid slow parsing scripts during user prompt searches, **Asterisk 2** splits document text into highly focused storage objects called **Page Chunks**. The application controller processes modified markdown strings to keep search databases clean and optimized.

---

## Chunk Generation Lifecycle

The diagram below details the automatic parsing loop maintaining clean RAG snippet databases:

```mermaid
graph TD
    %% Trigger Action
    subgraph ActionTrigger["Document Update Operation"]
        PostUpdate["POST /api/Documentation/page<br/>(Markdown Saved)"]
    end

    %% Chunking logic
    subgraph LogicTier["Automatic Regex Processing"]
        HeaderSplit["Split Document by Regex<br/>(Matches Line Starters)"]
        CleanPass["Scrub Extraneous Carriage Returns"]
    end

    %% Relational Ledger
    subgraph StorageTier["Chunk Ledger Database"]
        PurgeOld["DELETE from PageChunks<br/>where PageId == Target.Id"]
        InsertNew[("INSERT PageChunk Rows<br/>(ChunkIndex, Content Text)")]
    end

    %% Downstream Delivery
    subgraph ConsumerTarget["AI Context Pipeline"]
        DeliverRAG["Return Focused Text Snippet<br/>to User RAG Search"]
    end

    %% Flow Vectors
    PostUpdate --> HeaderSplit
    HeaderSplit --> CleanPass
    CleanPass ==>|"Execute Transaction"| PurgeOld
    PurgeOld ==>|"Atomic Commit"| InsertNew
    InsertNew ==>|"Index Pre-Computed"| DeliverRAG

    %% Custom Curated Colors
    classDef triggerToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef logicToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef storeToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef targetToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;

    class ActionTrigger,PostUpdate triggerToken;
    class LogicTier,HeaderSplit,CleanPass logicToken;
    class StorageTier,PurgeOld,InsertNew storeToken;
    class ConsumerTarget,DeliverRAG targetToken;
```

---

## Model Attribute Blueprint

### 1. `PageChunk` Entity Fields
- **`ChunkId`** (`int`): Primary automatic auto-increment identity integer.
- **`PageId`** (`int`): Mandatory foreign key binding the isolated snippet directly to its original parent document.
- **`ChunkIndex`** (`int`): Sequential numerical integer preserving the real reading layout sequence of the sub-headings.
- **`Content`** (`string`): Indexed text fields retaining unescaped markdown sub-block instructions.

---

## Referential Safeties
If a parent `Page` record triggers deletion inside `ApplicationDbContext`, relational cascading triggers immediately drop all mapped `PageChunk` entries, completely preventing obsolete RAG tokens from skewing prompt generation logic.
AI Chunks (RAG) 4 chunks
Chunk #1 Pre-Computed Markdown Chunk Models
# Pre-Computed Markdown Chunk Models

To avoid slow parsing scripts during user prompt searches, **Asterisk 2** splits document text into highly focused storage objects called **Page Chunks**. The application controller processes modified markdown strings to keep search databases clean and optimized.

---
Chunk #2 Chunk Generation Lifecycle
## Chunk Generation Lifecycle

The diagram below details the automatic parsing loop maintaining clean RAG snippet databases:

```mermaid
graph TD
    %% Trigger Action
    subgraph ActionTrigger["Document Update Operation"]
        PostUpdate["POST /api/Documentation/page<br/>(Markdown Saved)"]
    end

    %% Chunking logic
    subgraph LogicTier["Automatic Regex Processing"]
        HeaderSplit["Split Document by Regex<br/>(Matches Line Starters)"]
        CleanPass["Scrub Extraneous Carriage Returns"]
    end

    %% Relational Ledger
    subgraph StorageTier["Chunk Ledger Database"]
        PurgeOld["DELETE from PageChunks<br/>where PageId == Target.Id"]
        InsertNew[("INSERT PageChunk Rows<br/>(ChunkIndex, Content Text)")]
    end

    %% Downstream Delivery
    subgraph ConsumerTarget["AI Context Pipeline"]
        DeliverRAG["Return Focused Text Snippet<br/>to User RAG Search"]
    end

    %% Flow Vectors
    PostUpdate --> HeaderSplit
    HeaderSplit --> CleanPass
    CleanPass ==>|"Execute Transaction"| PurgeOld
    PurgeOld ==>|"Atomic Commit"| InsertNew
    InsertNew ==>|"Index Pre-Computed"| DeliverRAG

    %% Custom Curated Colors
    classDef triggerToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef logicToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef storeToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
    classDef targetToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;

    class ActionTrigger,PostUpdate triggerToken;
    class LogicTier,HeaderSplit,CleanPass logicToken;
    class StorageTier,PurgeOld,InsertNew storeToken;
    class ConsumerTarget,DeliverRAG targetToken;
```

---
Chunk #3 Model Attribute Blueprint
## Model Attribute Blueprint

### 1. `PageChunk` Entity Fields
- **`ChunkId`** (`int`): Primary automatic auto-increment identity integer.
- **`PageId`** (`int`): Mandatory foreign key binding the isolated snippet directly to its original parent document.
- **`ChunkIndex`** (`int`): Sequential numerical integer preserving the real reading layout sequence of the sub-headings.
- **`Content`** (`string`): Indexed text fields retaining unescaped markdown sub-block instructions.

---
Chunk #4 Referential Safeties
## Referential Safeties
If a parent `Page` record triggers deletion inside `ApplicationDbContext`, relational cascading triggers immediately drop all mapped `PageChunk` entries, completely preventing obsolete RAG tokens from skewing prompt generation logic.