Back to page
Pre-Computed Markdown Chunk Models
Raw Markdown & AI/RAG Chunks
Raw Markdown Source
2817 chars
# Pre-Computed Markdown Chunk Models
To avoid slow parsing scripts during user prompt searches, **Asterisk 2** splits document text into highly focused storage objects called **Page Chunks**. The application controller processes modified markdown strings to keep search databases clean and optimized.
---
## Chunk Generation Lifecycle
The diagram below details the automatic parsing loop maintaining clean RAG snippet databases:
```mermaid
graph TD
%% Trigger Action
subgraph ActionTrigger["Document Update Operation"]
PostUpdate["POST /api/Documentation/page<br/>(Markdown Saved)"]
end
%% Chunking logic
subgraph LogicTier["Automatic Regex Processing"]
HeaderSplit["Split Document by Regex<br/>(Matches Line Starters)"]
CleanPass["Scrub Extraneous Carriage Returns"]
end
%% Relational Ledger
subgraph StorageTier["Chunk Ledger Database"]
PurgeOld["DELETE from PageChunks<br/>where PageId == Target.Id"]
InsertNew[("INSERT PageChunk Rows<br/>(ChunkIndex, Content Text)")]
end
%% Downstream Delivery
subgraph ConsumerTarget["AI Context Pipeline"]
DeliverRAG["Return Focused Text Snippet<br/>to User RAG Search"]
end
%% Flow Vectors
PostUpdate --> HeaderSplit
HeaderSplit --> CleanPass
CleanPass ==>|"Execute Transaction"| PurgeOld
PurgeOld ==>|"Atomic Commit"| InsertNew
InsertNew ==>|"Index Pre-Computed"| DeliverRAG
%% Custom Curated Colors
classDef triggerToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef logicToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef storeToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef targetToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;
class ActionTrigger,PostUpdate triggerToken;
class LogicTier,HeaderSplit,CleanPass logicToken;
class StorageTier,PurgeOld,InsertNew storeToken;
class ConsumerTarget,DeliverRAG targetToken;
```
---
## Model Attribute Blueprint
### 1. `PageChunk` Entity Fields
- **`ChunkId`** (`int`): Primary automatic auto-increment identity integer.
- **`PageId`** (`int`): Mandatory foreign key binding the isolated snippet directly to its original parent document.
- **`ChunkIndex`** (`int`): Sequential numerical integer preserving the real reading layout sequence of the sub-headings.
- **`Content`** (`string`): Indexed text fields retaining unescaped markdown sub-block instructions.
---
## Referential Safeties
If a parent `Page` record triggers deletion inside `ApplicationDbContext`, relational cascading triggers immediately drop all mapped `PageChunk` entries, completely preventing obsolete RAG tokens from skewing prompt generation logic.
AI Chunks (RAG)
4 chunks
Chunk #1
Pre-Computed Markdown Chunk Models
# Pre-Computed Markdown Chunk Models To avoid slow parsing scripts during user prompt searches, **Asterisk 2** splits document text into highly focused storage objects called **Page Chunks**. The application controller processes modified markdown strings to keep search databases clean and optimized. ---
Chunk #2
Chunk Generation Lifecycle
## Chunk Generation Lifecycle
The diagram below details the automatic parsing loop maintaining clean RAG snippet databases:
```mermaid
graph TD
%% Trigger Action
subgraph ActionTrigger["Document Update Operation"]
PostUpdate["POST /api/Documentation/page<br/>(Markdown Saved)"]
end
%% Chunking logic
subgraph LogicTier["Automatic Regex Processing"]
HeaderSplit["Split Document by Regex<br/>(Matches Line Starters)"]
CleanPass["Scrub Extraneous Carriage Returns"]
end
%% Relational Ledger
subgraph StorageTier["Chunk Ledger Database"]
PurgeOld["DELETE from PageChunks<br/>where PageId == Target.Id"]
InsertNew[("INSERT PageChunk Rows<br/>(ChunkIndex, Content Text)")]
end
%% Downstream Delivery
subgraph ConsumerTarget["AI Context Pipeline"]
DeliverRAG["Return Focused Text Snippet<br/>to User RAG Search"]
end
%% Flow Vectors
PostUpdate --> HeaderSplit
HeaderSplit --> CleanPass
CleanPass ==>|"Execute Transaction"| PurgeOld
PurgeOld ==>|"Atomic Commit"| InsertNew
InsertNew ==>|"Index Pre-Computed"| DeliverRAG
%% Custom Curated Colors
classDef triggerToken fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef logicToken fill:#1e293b,stroke:#a855f7,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef storeToken fill:#312e81,stroke:#ec4899,stroke-width:2px,color:#fff,rx:6px,ry:6px;
classDef targetToken fill:#065f46,stroke:#10b981,stroke-width:2px,color:#fff,rx:6px,ry:6px;
class ActionTrigger,PostUpdate triggerToken;
class LogicTier,HeaderSplit,CleanPass logicToken;
class StorageTier,PurgeOld,InsertNew storeToken;
class ConsumerTarget,DeliverRAG targetToken;
```
---
Chunk #3
Model Attribute Blueprint
## Model Attribute Blueprint ### 1. `PageChunk` Entity Fields - **`ChunkId`** (`int`): Primary automatic auto-increment identity integer. - **`PageId`** (`int`): Mandatory foreign key binding the isolated snippet directly to its original parent document. - **`ChunkIndex`** (`int`): Sequential numerical integer preserving the real reading layout sequence of the sub-headings. - **`Content`** (`string`): Indexed text fields retaining unescaped markdown sub-block instructions. ---
Chunk #4
Referential Safeties
## Referential Safeties If a parent `Page` record triggers deletion inside `ApplicationDbContext`, relational cascading triggers immediately drop all mapped `PageChunk` entries, completely preventing obsolete RAG tokens from skewing prompt generation logic.