// SERVICE SPECIFICATION - 10 DAYS

AI Second Brain

Build a private, Git-backed team knowledge vault in plain Markdown that captures web clippings, PDFs, and meeting notes, organizes them with local models, and answers complex questions with verifiable citations.

Table of Contents

1. Solving the Institutional Memory Problem
2. Plain-Text Markdown & Git Vault Architecture
3. Local-First AI Organization & Privacy Boundaries
4. Cited Vector Retrieval & Weekly Digest
5. Enterprise RBAC, Audit & Knowledge Governance
6. Failure-Mode Recovery & Index Reconciliation
7. Tutorial 1: Production Python Vault Indexer & Cited Query
8. Tutorial 2: Production Frontmatter & Tagging Prompt
9. Tutorial 3: Step-by-Step 5-Day Implementation Runbook
10. Tutorial 4: Production Python Git-to-Vector Reconciler
11. Tutorial 5: Production Knowledge Conflict & Deprecation Prompt
12. Tutorial 6: Step-by-Step Governance & Backup Audit
13. Live References

Solving the Institutional Memory Problem

Every growing organization leaks knowledge. Architectural decisions live in buried Slack threads, client onboarding playbooks sit inside personal Google Docs, and post-mortem lessons vanish when engineers rotate teams. Traditional wikis fail because they demand manual curation: busy teams rarely have time to tag, link, and summarize every document they produce.

An AI Second Brain solves this by decoupling capture from organization. Team members drop raw web clippings, PDFs, email exports, and quick voice-to-text notes into a single inbox folder. Automated Python pipelines normalize those inputs into clean Markdown, deduplicate them via SHA-256 content hashes, and invoke a local language model to propose titles, summaries, and canonical tags - always requiring human approval before committing to the shared vault.

Plain-Text Markdown & Git Vault Architecture

Proprietary SaaS knowledge bases trap your company's intellectual property behind proprietary databases and unpredictable subscription tiers. By standardizing on Obsidian Markdown vaults backed by Git, your team gains full cryptographic history, branch-based collaboration, and instant portability.

We structure the vault using a disciplined PARA hierarchy (`00-inbox`, `10-projects`, `20-areas`, `30-resources`, `90-archive`) paired with strict YAML frontmatter schemas (`title`, `date`, `tags`, `source`, `status`). A curated taxonomy of at most twenty canonical tags prevents tag sprawl and ensures deterministic filtering.

Local-First AI Organization & Privacy Boundaries

Sending proprietary contracts, HR notes, or unreleased product roadmaps to third-party cloud APIs introduces unnecessary compliance risk. Using Ollama local model serving, the organization pipeline runs entirely on your own hardware. The local model reads incoming inbox notes, generates a concise three-sentence summary, and proposes tags strictly from your approved taxonomy.

Crucially, AI suggestions are written into a `proposed_` frontmatter block rather than overwriting canonical metadata. A team member reviews and promotes approved notes during a ten-minute daily triage, guaranteeing zero silent hallucinations in your knowledge base.

Cited Vector Retrieval & Weekly Digest

To make the vault conversational, we split Markdown files by heading boundaries (capped at roughly 800 words per chunk) and generate dense vector embeddings stored in Chroma local vector database. When a team member asks a question, the retriever fetches the top four semantic matches and instructs the model to answer exclusively from those chunks while citing the exact source filenames.

Every Sunday evening, a scheduled cron job compiles a Weekly Review Digest: listing newly captured notes, stale drafts awaiting review, and suggested wikilinks between related projects.

Enterprise RBAC, Audit & Knowledge Governance

When an organization consolidates engineering runbooks, executive strategy memos, and client post-mortems into a unified AI Second Brain, access control cannot remain an afterthought. A flat vector index where every employee can query every document creates immediate confidentiality leaks - for example, surfacing unreleased M&A notes or HR compensation bands to unauthorized teams. Enterprise deployments therefore enforce document-level Role-Based Access Control (RBAC) at both the Git repository boundary and the vector retrieval filter layer.

Each Markdown note carries immutable YAML frontmatter declaring its classification (public, internal, confidential, restricted) and allowed_groups. During ingestion, these metadata attributes are indexed alongside the embedding vectors inside Chroma metadata filters or PostgreSQL pgvector row-level security policies. Before any similarity search executes, the query engine injects the authenticated user's group claims as a mandatory pre-filter, guaranteeing that restricted chunks never enter the LLM context window.

Every retrieval and synthesis event is written to an append-only audit log recording the user identity, query hash, retrieved document URIs, and timestamp. Combined with Git commit history, compliance officers gain complete lineage from any AI-generated answer back to the exact human author and pull request that approved the underlying knowledge.

Failure-Mode Recovery & Index Reconciliation

A common failure mode in RAG-backed knowledge bases is silent index drift: files are renamed, deleted, or merged in Git, yet their orphaned embeddings persist in the vector database. When employees query the Second Brain, the retriever surfaces deprecated runbooks or broken file paths, eroding trust in the entire system.

To eliminate drift, the vault indexer operates as a deterministic reconciler rather than an append-only script. On every Git push or scheduled nightly sweep, the reconciler computes a SHA-256 content hash for each Markdown file and compares it against the stored content_hash metadata in the vector store. Unchanged files are skipped in milliseconds, modified files atomically replace their prior chunks, and deleted files trigger immediate vector tombstone purges.

If a local embedding model such as Ollama nomic-embed-text is temporarily unavailable during a bulk re-index, the reconciler checkpoints its progress in a local SQLite state table and resumes from the last committed hash without duplicating vectors.

Tutorial 1: Production Python Vault Indexer & Cited Query

This Python script indexes Markdown files into Chroma using local Ollama embeddings and answers questions with explicit filename citations.

import pathlib
import chromadb
import ollama

client = chromadb.PersistentClient(path="vault/index")
collection = client.get_or_create_collection("team_vault")


def embed_text(text: str) -> list[float]:
    res = ollama.embeddings(model="nomic-embed-text", prompt=text)
    return res["embedding"]


def index_vault(vault_dir: str = "vault") -> None:
    for md_file in pathlib.Path(vault_dir).rglob("*.md"):
        if "index" in md_file.parts:
            continue
        content = md_file.read_text(encoding="utf-8")[:4000]
        collection.upsert(
            ids=[str(md_file)],
            embeddings=[embed_text(content)],
            documents=[content],
            metadatas=[{"source": str(md_file)}],
        )


def ask_vault(question: str) -> str:
    hits = collection.query(query_embeddings=[embed_text(question)], n_results=4)
    context_blocks = []
    for doc, meta in zip(hits["documents"][0], hits["metadatas"][0]):
        context_blocks.append(f"[SOURCE: {meta['source']}]\n{doc}")
    context = "\n---\n".join(context_blocks)
    prompt = (
        "Answer ONLY using the provided sources. Cite every claim with [SOURCE: path].\n\n"
        f"{context}\n\nQuestion: {question}"
    )
    resp = ollama.chat(model="qwen3:8b", messages=[{"role": "user", "content": prompt}])
    return resp["message"]["content"]

Tutorial 2: Production Frontmatter & Tagging Prompt

Use this prompt with your local Ollama model to classify incoming inbox notes against your approved tag taxonomy without inventing arbitrary labels.

SYSTEM ROLE:
You are a Knowledge Base Librarian organizing Markdown notes.

APPROVED TAGS (SELECT UP TO 3 ONLY FROM THIS LIST):
architecture, security, onboarding, sales, product, devops, compliance, research

NOTE CONTENT:
{{raw_markdown_note}}

INSTRUCTIONS:
1. Propose a concise, descriptive title (max 8 words).
2. Write a 2-sentence executive summary.
3. Select 1 to 3 tags strictly from APPROVED TAGS.

OUTPUT FORMAT (STRICT JSON):
{
  "proposed_title": "...",
  "proposed_summary": "...",
  "proposed_tags": ["...", "..."]
}

Tutorial 3: Step-by-Step 5-Day Implementation Runbook

1. Day 1 - Vault & Git Initialization: Create the PARA directory structure in Obsidian, define YAML frontmatter templates, and initialize a private Git repository with `.gitignore` excluding the vector index.
2. Day 2 - Capture & Deduplication: Configure web clippers and email-to-Markdown scripts routing into `00-inbox/`, guarded by SHA-256 content-hash deduplication.
3. Day 3 - Local Model Triage: Deploy Ollama with a local chat model to populate `proposed_` frontmatter fields, enforcing your 20-tag ceiling.
4. Day 4 - Vector Indexing & Cited Q&A: Chunk notes by Markdown heading, generate local embeddings into Chroma, and validate 10 real team queries for citation accuracy.
5. Day 5 - Weekly Digest & Handover: Schedule nightly incremental re-indexing and Sunday digest generation via cron, then verify a full Git restore test.

Tutorial 4: Production Python Git-to-Vector Reconciler

This reconciler computes SHA-256 hashes across your Markdown vault, skips unchanged notes, atomically updates modified notes, and purges deleted file vectors to prevent index drift.

import hashlib
from pathlib import Path
from typing import Dict, Set

def compute_vault_hashes(vault_dir: Path) -> Dict[str, str]:
    hashes: Dict[str, str] = {}
    for md_file in vault_dir.rglob("*.md"):
        rel = str(md_file.relative_to(vault_dir))
        digest = hashlib.sha256(md_file.read_bytes()).hexdigest()
        hashes[rel] = digest
    return hashes

def reconcile_index(vault_dir: Path, indexed_hashes: Dict[str, str]) -> Dict[str, Set[str]]:
    current = compute_vault_hashes(vault_dir)
    current_keys = set(current.keys())
    indexed_keys = set(indexed_hashes.keys())

    to_delete = indexed_keys - current_keys
    to_upsert = {
        path for path in current_keys
        if path not in indexed_hashes or indexed_hashes[path] != current[path]
    }
    unchanged = current_keys - to_upsert
    return {"upsert": to_upsert, "delete": to_delete, "skip": unchanged}

Tutorial 5: Production Knowledge Conflict & Deprecation Prompt

Run this prompt during weekly vault maintenance to detect contradictory SOPs, flag stale runbooks, and propose canonical merge targets.

SYSTEM_PROMPT = """
You are an enterprise knowledge-base auditor. Compare the two Markdown notes provided in <note_a> and <note_b>.
Identify any factual contradictions, superseded commands, or conflicting policy thresholds.
Cite exact YAML frontmatter dates to determine which note is newer.

Return ONLY valid JSON:
{
  "conflict_detected": boolean,
  "canonical_uri": "path/to/newer_note.md",
  "deprecated_uri": "path/to/older_note.md",
  "contradiction_summary": "1-sentence explanation",
  "recommended_action": "archive_older" | "merge_sections" | "no_action"
}
"""

Tutorial 6: Step-by-Step Governance & Backup Audit

1. Enforce Frontmatter Linting in CI: Add a GitHub Actions pre-merge check that rejects any Markdown note missing classification, owner, or allowed_groups.
2. Verify RBAC Pre-Filter Isolation: Execute automated integration tests with a restricted-group token to confirm zero confidential chunks appear in vector query results.
3. Schedule Nightly Hash Reconciliation: Run vault_reconciler.py via cron or CI to purge deleted notes and re-embed modified files.
4. Snapshot Encrypted Git Mirrors: Replicate the bare Git repository to an immutable, encrypted object-storage bucket with 90-day point-in-time recovery.
5. Review Staleness Digest: Distribute a weekly automated report listing notes untouched for 180+ days to their designated owners for archive or renewal.

Live References

• Obsidian Official Help & Markdown Vault Documentation
• Ollama Local Model & Embedding Library
• Chroma Vector Database Documentation
• Anthropic Claude API Documentation
• pgvector: Open-Source Vector Similarity Search for PostgreSQL
• PostgreSQL Row-Level Security (RLS) Policies Documentation
• Git Official Reference & Bare Repository Mirroring
• Pydantic Data Validation & Frontmatter Schema Enforcement
• SQLite Documentation for Local State Checkpointing
• OpenAI Embeddings & Semantic Retrieval Guide

Own Your Team's Institutional Memory

Commission a 10-day AI Second Brain engagement: 5 days of live engineering training plus 5 days of dedicated support.