Alkeyword — Pipeline Spec v0.1 (draft)

Input: a bare domain name. Nothing else. Output: a ranked queue of measured visibility gaps, and content that closes them — with every factual sentence traceable to something the site already said.

Governing rule: transmute, never fabricate. If the site does not contain the raw material to ground a page, the system refuses to generate it and says why.

0. Pipeline overview

  domain
    │
    ▼
┌───────────┐   pages, claims, contradictions
│ 1 CRAWL   │──────────────────────────────────┐
└───────────┘                                  │
    │ entity + category inferred               │
    ▼                                          ▼
┌───────────┐   answers, citations,      ┌──────────────┐
│ 2 PROBE   │──  mention rates  ────────▶│   3 DIFF     │
└───────────┘                            │  gap queue   │
                                         └──────────────┘
                                                │ gaps with sufficient material
                                                ▼
                                         ┌──────────────┐
                                         │ 4 GENERATE   │ provenance-validated
                                         └──────────────┘
                                                │
                                                ▼  T+30d
                                         ┌──────────────┐
                                         │ 5 RE-PROBE   │──▶ calibration store
                                         └──────────────┘

Stages 1–3 are the free tier (cheap, produces the shareable artifact). Stages 4–5 are paid (expensive, produces the deliverable).

1. CRAWL

1.1 Discovery

1. Fetch robots.txt; honor it. Record any AI-crawler blocks (GPTBot, ClaudeBot,

PerplexityBot, OAI-SearchBot, Google-Extended) — a site blocking these has an answer before any analysis runs, and it's a finding worth reporting on its own.

2. Fetch sitemap.xml / sitemap index. If absent, BFS from / following same-origin links.

3. Cap at MAX_PAGES (default 200; tiered). Prioritize: homepage, nav-linked, /pricing,

/about, /faq, /docs, /blog index, then depth order.

1.2 Fetch strategy (cost control)

Static GET first. Escalate to headless render only when the static HTML yields < 100 words of extracted body text while containing <script> tags — i.e. a real SPA. Record which path was used. Headless is ~50–100× the cost of a static fetch; never make it the default.

1.3 Per-page extraction

Page {
  url, canonical, status, lang, fetched_at, render_mode: static|headless
  title, meta_description
  outline: [{level, text}]          // h1–h4, in document order
  body_text                          // readability-style main-content extraction
  schema_blocks: [JSON-LD | microdata]
  author, published_at, modified_at  // from schema, meta, or visible byline
  internal_links_out: [url]
  external_links_out: [url]
  images: [{src, alt}]
  word_count
}

1.4 Claim extraction — the core

A claim is one atomic assertion the site makes, anchored to where it says it.

Claim {
  id
  type: entity | attribute | quantity | process | definition
      | comparison | question | quote | credential
  subject                 // resolved entity ref where possible
  predicate
  object
  text_span               // verbatim source text
  source_url
  source_anchor           // heading path or CSS selector
  confidence: 0..1        // extraction confidence, not truth
  first_seen, last_seen
}

Type notes — these are not decoration, each drives different downstream behavior:

1.5 Graph edges

supports · contradicts · elaborates · restates (claim↔claim) about (claim→entity) · links_to (page→page) · mentions (page→entity)

1.6 Entity + category inference

Resolve the primary entity (name, aliases, one-line description, category) from Organization/ Product schema where present, else from homepage title/h1/meta plus repeated noun phrases. Infer category as a short label plus 3–8 sibling/competitor candidates discovered from the site's own comparison claims and outbound links. Competitors named by the site itself are far more reliable than guessed ones — prefer them.

1.7 Immediate free findings (no probing required)

Emit before any engine call:

• Self-contradictions — two claims that conflict (pricing page says $49, blog says $39).

• Orphan claims — high-value claim on a page with ≤1 internal inbound link.

• Unsourced quantities — a number with no citation or supporting page.

• AI-crawler blocks in robots.txt.

These cost nothing beyond the crawl and buy trust on day one.

2. PROBE

2.1 Question set generation

Three generators, all derivable from stage 1 alone:

Brand questions (from the entity) — cheapest to win, highest emotional impact: What is <brand>? · Is <brand> legit? · How much does <brand> cost? · <brand> vs <competitor> · <brand> reviews · Who is <brand> for? · Does <brand> do <feature>?

Category questions (from the inferred category) — where you're absent and competitors get cited: best <category> for <ICP> · how do I <job-to-be-done> · what is <term> · <category> alternatives · <competitor> alternatives

Claim-derived questions — invert each high-value claim into the question it answers. A quantity claim "99.9% uptime" → "what uptime does \<brand\> guarantee?". A definition claim → "what is \<term\>?". This is the generator that makes the question set specific to this site rather than generic to the vertical.

Cap and rank by expected value. Brand questions first — they're winnable and the answers are either already right or embarrassingly wrong.

2.2 Engines

Cross-engine overlap is only ~11%, so do not average engines into one "visibility score." Report per-engine. A single blended number is exactly the opacity marketers are already complaining about.

2.3 Non-determinism — handle this honestly

The same prompt returns different answers across runs. This is the single largest credibility problem in the GEO category; agencies already treat these tools as a benchmark, not a source of truth.

Design response: run each (question, engine) N=3 times and report a rate, never a binary. "Cited in 2 of 3 runs" is the unit. Never state a single-run result as fact. Record variance and surface it. Competitors report single samples as truth — being the tool that shows its error bars is a real, cheap differentiator.

2.4 Per-probe record

Probe {
  question_id, engine, run_index, run_at
  answer_text
  cited_urls: [url]
  brand_mentioned: bool
  brand_position: int|null          // ordinal within the answer
  own_domain_cited: bool
  competitors_mentioned: [entity]
  competitor_urls_cited: [url]
  sentiment: pos|neu|neg
  asserted_facts: [Claim]           // extracted from the answer, same schema as §1.4
}

asserted_facts is what makes stage 3 possible — you need the model's claims in the same shape as the site's claims in order to diff them.

2.5 Cost control

• Cache by (question, engine, week).

• Share category-level probes across customers in the same vertical. Ten customers asking

"best CRM for nonprofits" is one probe, not ten. This turns the dominant recurring cost into a fixed amortizable one and is a structural advantage over per-customer-polling incumbents.

• Brand questions are per-customer and cannot be shared. Budget accordingly.

3. DIFF — the gap library

Each gap type = a detection rule + a prescribed action + a score. This library is the moat; it compounds with every re-probe cycle and can't be copied from a screenshot.

3.1 Scoring

score = reach × winnability × material_sufficiency × (1 / effort)

• reach — how many probe questions the gap touches, weighted by brand vs category

• winnability — inverse of how entrenched the incumbent citation is (a Wikipedia citation is

near-unwinnable; a 2023 competitor blog post is not)

• material_sufficiency — the gate, see §3.2

• effort — page count / word count / whether a human must supply a fact

3.2 The refusal rule

material_sufficiency is computed from the count and type-mix of claims available to ground the target page. Below threshold, the gap is reported but generation is refused, with the reason stated: "You have 3 claims about X. A comparison page needs at least 8, including 2 quantities. Here's what's missing."

This is the rule that separates this product from the autoblog segment. A 3-page brochure site gets a gap report and an honest "there's nothing here to transmute yet." Enforce it — the temptation to relax it under revenue pressure is the failure mode that turns this into scaled content abuse.

4. BRIEF — the primary deliverable

The product is the research, not the prose. Alkeyword produces a work order per opportunity; the human writes. Generation (§4b) is an opt-in extra, not the default.

4.0 Why the brief is the product, not a fallback

• **AI authorship is not itself the differentiator, and the once-cited "over 90% of cited pages

contain AI text" figure is withdrawn** — it carried no source here, and the best public measurement points the other way: Graphite (June 2025) classified 18% of articles cited by ChatGPT and Perplexity as AI-generated. The argument does not need it. The differentiator is original research and expertise added on top.

• **One study (NP Digital, 744 articles across 68 sites) measured human-written content at

~5.44× the traffic** of AI-generated, holding it over months rather than fluctuating.

• Editorial oversight is exactly what Google's scaled-content-abuse policy tests for. A tool

that hands a human a brief cannot produce the penalized pattern; a tool that publishes at volume can.

• The brief is where the human's judgment enters. That is the part a tool cannot supply and

should not pretend to.

4.1 The anti-dashboard rule

The known failure mode of this whole category: "brands that buy trackers without a content engine to act on the data watch flat dashboards for six months and churn." A research tool is at risk of being exactly that.

The discriminator is whether the reader must go do research after reading it. A dashboard reports a score. A brief must be specific enough that writing from it is assembly, not research — the raw material travels with the assignment.

Therefore every brief carries the actual claims, verbatim, with source URLs. If a brief would send the user off to gather their own facts, it is not finished.

4.2 Brief structure (one per ranked opportunity)

Brief {
  rank, score, gap_id                      // why this one, and why here
  questions: [                             // verbatim, as people really type them
    {text, is_question, source: autocomplete|probe|claim-derived, engine_state}
  ]
  keyword_cluster: [...]                   // the classic-SEO surface, same topic
  current_state {                          // measured, not assumed
    own_domain_cited: "0/3", competitors_cited: [{domain, rate}], engine
  }
  raw_material: [                          // ← the part that makes it a brief
    {claim_id, type, text_span, source_url, source_anchor}
  ]
  missing: [...]                           // "needs 2 more quantities, here's the shape"
  recommended_form: comparison|glossary|faq|statistics|how-it-works|entity
  outline: [...]                           // section headings, each mapped to claim_ids
  reprobe_at
}

raw_material is non-negotiable. A brief without the site's own claims attached is a keyword list with extra steps.

4.3 The refusal rule becomes a shopping list

In generation mode, refusal means "I won't write this." In brief mode it becomes strictly more useful and less paternalistic:

Not yet. A comparison page needs ≥8 shared attributes and ≥2 quantities. You have 3 claims about X, all on /features. Gather: a price or plan figure, one third-party benchmark or citation, and 4 more attribute rows. Then this becomes a ready brief.

Same threshold, same discipline — but the output is an assignment rather than a door closing.

4.4 Query expansion (free)

Real queries, no paid keyword API: Google autocomplete and DuckDuckGo suggest, seeded from the claim graph rather than a generic niche guess — definition subjects (the site's own vocabulary), competitors the site itself names, and recurring heading noun-phrases. Modifier sweep across question words (how/what/why/which/is/does), commercial intent (best/vs/alternative to), and suffixes (vs/alternatives/pricing/review/for).

These return queries people actually typed, not modeled volume — which is the better input for generative engines anyway, where the unit of demand is a question, not a keyword with a number beside it. Search Console (free, OAuth) is the natural second source: the user's own ground truth.

Paid volume data (DataForSEO ≈ $0.0006/req, $50 minimum) stays optional. It buys a number to sort by, not the opportunity itself.

4.5 Material verdict — the column nobody else has

Every mined query is cross-referenced against the claim graph:

A keyword list without this column is undifferentiated. This column is the product.

4b. GENERATE (opt-in)

Retained as an explicit per-brief action — "draft this one" — never a default and never a bulk mode. The brief already contains everything a draft needs, so the marginal cost is small; but the product identity is research, and the volume cap and provenance validator below still apply in full.

4b.1 The provenance constraint

Every sentence in the output is either:

• (a) traceable to ≥1 source claim, or

• (b) connective tissue containing no factual assertion.

Enforce mechanically, not by prompting. The generator emits sentences tagged with the claim IDs they draw on; a validator then checks each factual sentence (i) carries ≥1 claim ID and (ii) is actually entailed by those claims' text_spans. Sentences failing either check are dropped or flagged for human fill-in — never shipped silently.

This is the trust story and the demo: hover any sentence, see the source page and the exact text it came from.

4.2 Form templates

Each form declares its material requirements, which feed material_sufficiency:

Every form emits appropriate JSON-LD (FAQPage, DefinedTerm, HowTo, Organization, Article with author) — schema is the page-level counterpart to entity resolution.

4.3 Output artifact (per piece)

Piece {
  gap_id, form, target_questions: [question_id]
  markdown, html_with_schema
  provenance: [{sentence_idx, claim_ids, source_urls, text_spans}]
  unresolved: [{sentence_idx, reason}]   // needs a human fact
  reprobe_at                              // T+30d
}

4.4 Volume cap

Hard cap per cycle (default 6 pieces/month). This is a product constraint, not a pricing tier to be upsold away. Removing the cap produces exactly the shape Google's scaled-content-abuse policy describes (March 2024 spam update: many pages made primarily to manipulate rankings, "no matter how it's created"). Sell "6 pieces that close 6 measured gaps," never "30 articles a month."

Human approval before publish is the default. Fully-automatic publishing is where both brand risk and Google risk live.

5. RE-PROBE + calibration

At T+30d, re-run the exact questions the piece targeted, same N=3 protocol. Report movement as a rate delta (cited 0/3 → 2/3), not a score change.

Persist every (gap_type, form, vertical, outcome) tuple. Over cycles this answers the question nobody in the category can currently answer: which gap types and which page forms actually move citations, and in which verticals. That's the compounding asset — the gap library's scores stop being guesses and become measured priors.

It also produces the before/after proof loop whose absence is driving churn everywhere else ("brands watch flat dashboards for six months and churn").

6. The deliverable — "Transmutation Report"

1. The delta — your entity as the models understand it, beside your entity as your site states

it. Wrong facts (G2) at the top. This page is the product's argument.

2. The queue — ranked gaps with scores, each with its prescribed action and material verdict

(mintable now / needs N more facts / refused).

3. The pieces — generated content with hoverable provenance.

4. The proof — after the first re-probe cycle, rate deltas per targeted question.

7. Build notes

• Crawl + probe fan-out is a supervision-tree problem — many concurrent, rate-limited,

independently-failing I/O workers with backoff. Elixir/OTP is the right fit and is already in this portfolio.

• The claim graph is Graphonomous's shape — nodes, typed edges, edge extraction, evidence-path

tracing, and belief revision when a re-crawl contradicts a prior crawl (§1.7 contradictions, §5 re-probe). Evaluate it as the engine before building graph infrastructure from scratch.

• Publishing is deliberately out of scope for v1. Domain-only input means no credentials.

Detect the CMS during the crawl and offer the matching export (markdown / HTML+schema / WordPress draft file). Real integrations are v2 and are where incumbents are genuinely ahead.

8. Failure modes to watch