Version. 0.1
Date. 2026-05-04
Author. Marvin Percival (DUNIN7), prepared via Claude.
Target. /Users/dunin7/loomworks (substrate) on DUNIN7-M4 (MacMini).
Priority. Standard.
Confidential. Internal.
Supersedes. Nothing for Phase 38 specifically.
Companion to. Phase 38 scoping note v0.1, engine implementation strategy v0.2 §8, Discovery record — Shape as specification v0.4 (Loompa grammar), Loompa skill (SKILL.md).
Status. First draft. Consumes settled scoping decisions from scoping note v0.1 without relitigating them.
Phase 38 makes the engine grammar-aware without making the engine grammar-specific.
Today, the grammar a shape type produces is implicit — buried in shaping instructions text. The engine cannot answer "what grammar does this shape type declare?" or "does this specialist accept shapes of this grammar?" or "is this shape structurally complete?" without re-reading the shaping instructions.
Phase 38 adds four capabilities:
1 — SpecificationGrammar model and grammar registry. A SpecificationGrammar Pydantic model declaring grammar name, structural elements, completeness criteria, and testability criteria. A registry module with six well-known grammars registered at startup.
2 — DeclaredShapeType extension. specification_grammar (JSONB) on DeclaredShapeType. New shape type declarations require a grammar. Existing shape types backfilled.
3 — Specialist grammar matching. accepts_grammars: list[str] on RenderSpecialist. Advisory when empty (backward-compatible). GrammarMismatchError when the specialist declares grammars and the shape's grammar is not in the list.
4 — Completeness evaluation at confirmation. Before a shape is confirmed, the engine evaluates the grammar's completeness criteria against the shape content. Failures surface as a report. The Operator can override with required rationale.
Substrate-only phase. One checkpoint.
Phase 38 is grounded in three layers, consulted in order.
Layer 1 — Methodology document v0.20. Shape as complete specification: the Loompa grammar (thirteen universal specification elements) establishes that all specifications share structural elements regardless of domain. The grammar was derived empirically from 78 artifact types (Discovery v0.4). Extensibility-first: the grammar registry specifies the boundary new grammars attach at; it does not enumerate a closed set. Recognition is load-bearing: grammar declaration makes the engine's structural awareness visible and auditable.
Layer 2 — Reference Design v0.1.2. Shape types already carry shaping_instructions_ref as implicit grammar. Phase 38 makes the grammar explicit. Parallel to the way Phase 17 made assertion identity explicit (display numbers) — something that existed implicitly becomes a first-class field the engine can reason about.
Layer 3 — Specification (Playground Spec v0.10 and prior phase CRs). Phase 9 (shaping): shapes are produced by reading shaping instructions; the grammar is the structural expectation those instructions encode. Phase 10 (render dispatch): dispatch gains grammar-matching awareness. The Loompa skill (SKILL.md) implements the thirteen elements; the engine declares the grammar at the metadata level.
CC confirms before Step 1:
uv run pytest -v reports the expected passed count (post-Phase 36). If the count differs significantly, stop and reconcile.alembic heads shows a single head. Current migration is 0054 (or whatever Phase 36 left).DeclaredShapeType model location identified. Confirm fields include name, shaping_instructions_ref. Confirm no existing specification_grammar field.RenderSpecialist base class location identified. Confirm no existing accepts_grammars attribute.StubRenderSpecialist location identified. Confirm no existing accepts_grammars attribute.pending_confirmation to confirmed).DeclaredShapeTypeResponse schema location identified. Confirm no existing specification_grammar field.DeclaredShapeType instances counted. Expected: 5 in the Loomworks engagement (Phase execution context, Project state, Methodology narrative, Discovery trajectory, Education). Plus any in the Goosey engagement (ad-hoc shape types).current_memory_objects materialized view location identified (for view refresh).
Archive this CR to docs/phase-crs/phase-38-cr-specification-grammar-declaration-v0_1.md at Step 0 before Step 1 begins, per standing archival discipline.
The engine-level SpecificationGrammar model declares: grammar name, structural elements (a list of named required/optional sections), completeness criteria, and testability criteria. The Loompa skill (SKILL.md) implements the thirteen universal specification elements — the actual grammar work. Phase 38 does not embed the skill into the engine. The engine is grammar-indifferent (strategy doc §8.1): it knows which grammar applies and can check structural metadata; it does not perform the grammar's specification work.
The engine defines the thirteen element names as a well-known set in the grammar module:
UNIVERSAL_SPECIFICATION_ELEMENTS = [
"subject_definition",
"component_inventory",
"component_arrangement",
"logical_relationships",
"properties_and_attributes",
"constraints_and_rules",
"dependencies",
"provenance_links",
"acceptance_criteria",
"consumer_definition",
"medium_and_format",
"composition_grammar",
"consumption_mode",
]
Grammar registrations reference these names in their structural_elements list. Individual grammars declare which elements are required and which are optional for their domain. All thirteen are required by default — a grammar must explicitly mark an element as optional.
accepts_grammars: list[str] = [] on RenderSpecialist. Empty means accepts all — no behavioral change for existing specialists. When a specialist declares specific grammars and the shape's grammar is not in the list, GrammarMismatchError is raised at dispatch time. A warning is logged when a specialist with empty accepts_grammars receives a shape with a declared grammar. This follows "ship incrementally" — advisory first, strict later.
Completeness evaluation produces a CompletenessReport. If all criteria pass, confirmation proceeds. If any fail, the report is surfaced and the Operator confirms with required rationale (completeness_override_rationale: str | None on the confirmation endpoint). The override rationale is recorded in the confirmation event payload. No separate exception endpoint — the confirmation endpoint handles both cases.
The five Loomworks DeclaredShapeTypes map to known grammars:
| Shape type | Grammar | |---|---| | Phase execution context | presentational-narrative | | Project state | presentational-declarative | | Methodology narrative | presentational-narrative | | Discovery trajectory | presentational-narrative | | Education | presentational-narrative |
Ad-hoc shape types (Goosey engagement) are backfilled with specification_grammar = None. They were created without grammar awareness and remain grammar-optional until the Operator declares one.
The Operator reviews the backfill assignments at Checkpoint A.
Six grammars register at startup, per the engine strategy doc §8.4 and §13.2:
Each grammar carries fully populated structural_elements, completeness_criteria, and testability_criteria. No placeholder registrations.
from pydantic import BaseModel
class StructuralElement(BaseModel):
"""A named section or field in the specification structure."""
name: str # e.g., "subject_definition", "component_inventory"
required: bool = True # Whether this element must be present
repeats: bool = False # Whether multiple instances are allowed
description: str | None = None # Human-readable description of this element
class CompletenessCriterion(BaseModel):
"""A check that must pass for the shape to be confirmable."""
name: str # e.g., "all_components_named"
description: str # What this criterion checks
evaluator: str | None = None # Optional: a reference to evaluation logic
class TestabilityCriterion(BaseModel):
"""How to evaluate a produced render against the specification."""
name: str # e.g., "every_clause_has_jurisdiction_marker"
description: str # What this criterion tests
evaluator: str | None = None
class SpecificationGrammar(BaseModel):
"""Explicit declaration of what a shape type's content looks like."""
grammar_name: str # e.g., "req-table", "narrative-storybook-pages"
structural_elements: list[StructuralElement]
completeness_criteria: list[CompletenessCriterion]
testability_criteria: list[TestabilityCriterion]
No primary_artifact_type field. The grammar name is the matching key. Artifact type classification is skill-level knowledge (per scoping note N1, Position A adopted).
# src/loomworks/grammars/registry.py
GRAMMAR_REGISTRY: dict[str, SpecificationGrammar] = {}
def register_grammar(grammar: SpecificationGrammar) -> None:
"""Register a grammar. Raises ValueError on duplicate name."""
if grammar.grammar_name in GRAMMAR_REGISTRY:
raise ValueError(f"Grammar '{grammar.grammar_name}' already registered")
GRAMMAR_REGISTRY[grammar.grammar_name] = grammar
def get_grammar(name: str) -> SpecificationGrammar | None:
"""Look up a grammar by name. Returns None if not registered."""
return GRAMMAR_REGISTRY.get(name)
def list_grammars() -> list[str]:
"""Return all registered grammar names."""
return list(GRAMMAR_REGISTRY.keys())
A separate module src/loomworks/grammars/defaults.py defines the six default grammars and registers them. This module is imported at application startup (e.g., in the lifespan or app factory). CC populates the structural elements, completeness criteria, and testability criteria for each grammar during Step 2, drawing from the Loompa skill's thirteen elements and the domain knowledge in the Discovery v0.4 and examples.md.
The UNIVERSAL_SPECIFICATION_ELEMENTS list also lives in src/loomworks/grammars/__init__.py as a well-known constant.
New field on DeclaredShapeType:
specification_grammar: SpecificationGrammar | None = None
At declaration time (new shape types): the grammar is required. Shape type creation without a grammar → 422. The grammar must reference a registered grammar name or provide a complete inline SpecificationGrammar.
For existing shape types: specification_grammar is nullable. The migration backfills the five Loomworks shape types with their assigned grammars (D5). Ad-hoc shape types remain null.
Projector extension: payload.get('specification_grammar', None). Pre-Phase-38 events default cleanly.
class RenderSpecialist:
# Existing: produce_render(...), supports_incremental_rerender, rerender(...)
# New in Phase 38:
accepts_grammars: list[str] = [] # Class-level declaration
In the render dispatch path (after the Phase 36 orchestration branch):
# Grammar matching (advisory when accepts_grammars is empty)
shape_grammar_name = shape.declared_shape_type.specification_grammar.grammar_name \
if shape.declared_shape_type and shape.declared_shape_type.specification_grammar \
else None
if specialist.accepts_grammars:
if shape_grammar_name and shape_grammar_name not in specialist.accepts_grammars:
raise GrammarMismatchError(
f"Specialist does not accept grammar '{shape_grammar_name}'. "
f"Accepts: {specialist.accepts_grammars}"
)
elif shape_grammar_name:
logger.warning(
"Specialist %s has no grammar declarations; accepting shape with grammar '%s'",
specialist.__class__.__name__, shape_grammar_name
)
GrammarMismatchError is a new exception. The dispatch wrapper catches it and returns an appropriate error (likely 422 or a specific dispatch-failure status).
StubRenderSpecialist gains a configurable accepts_grammars list (default empty), settable at construction: StubRenderSpecialist(accepts_grammars=["req-table", "narrative-storybook-pages"]).
class CompletenessReport(BaseModel):
grammar_name: str
passed: bool
results: list[CriterionResult]
class CriterionResult(BaseModel):
criterion_name: str
passed: bool
detail: str | None = None
async def evaluate_completeness(
shape_content: dict,
grammar: SpecificationGrammar,
) -> CompletenessReport:
"""Evaluate shape content against the grammar's completeness criteria.
Returns a report. Does not raise on failure — the caller decides
whether to proceed or block.
"""
results = []
for criterion in grammar.completeness_criteria:
# Structural check: does the content have the expected top-level keys?
# For criteria with evaluators: run the evaluator (future work).
# For criteria without evaluators: check key presence.
passed = _check_criterion(shape_content, criterion)
results.append(CriterionResult(
criterion_name=criterion.name,
passed=passed,
detail=None if passed else f"Criterion '{criterion.name}' not satisfied"
))
return CompletenessReport(
grammar_name=grammar.grammar_name,
passed=all(r.passed for r in results),
results=results,
)
The shape confirmation endpoint gains:
DeclaredShapeType has a specification_grammar, call evaluate_completeness.completeness_report field. The confirmation does not proceed unless the request includes completeness_override_rationale: str.Confirmation endpoint request schema extension:
class ConfirmShapeRequest(BaseModel):
# Existing fields...
completeness_override_rationale: str | None = None
Confirmation response schema extension:
class ConfirmShapeResponse(BaseModel):
# Existing fields...
completeness_report: CompletenessReport | None = None
Migration number is the next available at execution time (expected 0055 after Phase 36's 0054).
Add to the DeclaredShapeType storage table:
specification_grammar — JSONB, nullable.
Within the migration, update the five Loomworks DeclaredShapeTypes with their grammar assignments per D5. The grammar content is the full SpecificationGrammar JSON for each assigned grammar.
Ad-hoc shape types (Goosey) remain null.
Refresh current_memory_objects materialized view to include the new column.
The projector reads: payload.get('specification_grammar', None). Pre-Phase-38 events default cleanly.
DeclaredShapeTypeResponse gains:
specification_grammar: dict | None = None # The grammar as a dict for API consumers
New endpoint:
GET /grammars
Returns all registered grammar names with their structural element counts. Lightweight — no authentication required (grammar declarations are not engagement-specific).
class GrammarListResponse(BaseModel):
grammars: list[GrammarSummary]
class GrammarSummary(BaseModel):
grammar_name: str
structural_element_count: int
completeness_criteria_count: int
testability_criteria_count: int
test_register_grammar — Register a grammar. Verify get_grammar returns it.
test_register_duplicate_raises — Register the same grammar name twice. Verify ValueError.
test_list_grammars — Register multiple grammars. Verify list_grammars returns all names.
test_default_grammars_registered — Verify all six default grammars are registered at startup.
test_universal_elements_constant — Verify UNIVERSAL_SPECIFICATION_ELEMENTS contains 13 elements.
test_new_shape_type_requires_grammar — Create a new DeclaredShapeType without specification_grammar. Verify 422 or validation error.
test_new_shape_type_with_grammar — Create a new DeclaredShapeType with a valid grammar. Verify it is stored and retrievable.
test_backfill_assigns_correct_grammars — Verify the migration backfill assigned the correct grammars to the five Loomworks DeclaredShapeTypes.
test_adhoc_shape_type_no_grammar — Verify ad-hoc shape types (Goosey) have null specification_grammar after migration.
test_grammar_persists_through_projector — Create a shape type with grammar. Verify the projector reads it back correctly from payload.
test_specialist_matching_accepted — Specialist declares accepts_grammars=["req-table"]. Dispatch a shape with req-table grammar. Verify dispatch proceeds.
test_specialist_matching_rejected — Specialist declares accepts_grammars=["req-table"]. Dispatch a shape with narrative-storybook-pages grammar. Verify GrammarMismatchError.
test_specialist_empty_accepts_all — Specialist has empty accepts_grammars. Dispatch a shape with any grammar. Verify dispatch proceeds (with warning logged).
test_specialist_matching_no_grammar_on_shape — Specialist declares grammars. Dispatch a shape with no grammar (ad-hoc). Verify dispatch proceeds (no grammar to mismatch against).
test_stub_specialist_configurable_grammars — StubRenderSpecialist(accepts_grammars=["req-table"]). Verify the flag is set.
test_completeness_all_pass — Shape content satisfies all completeness criteria. Verify report passes.
test_completeness_failure — Shape content missing a required element. Verify report fails with detail.
test_confirmation_proceeds_when_complete — Confirm a shape with passing completeness. Verify confirmation succeeds.
test_confirmation_blocked_when_incomplete — Confirm a shape with failing completeness and no override. Verify confirmation does not proceed. Verify response includes completeness_report.
test_confirmation_override_with_rationale — Confirm a shape with failing completeness and override rationale. Verify confirmation proceeds. Verify rationale recorded in event payload.
test_confirmation_no_grammar_no_evaluation — Confirm a shape with no grammar (ad-hoc). Verify no completeness evaluation. Verify confirmation proceeds.
test_grammar_list_endpoint — GET /grammars returns all six grammars with correct counts.
Auto-mode posture: Steps 1–7 auto-mode-proceed. Checkpoint A halts until Operator confirms grammar assignments. No frontend work in this phase.
Archive this CR to docs/phase-crs/phase-38-cr-specification-grammar-declaration-v0_1.md. Run pre-flight checks (Section 3.2). Confirm baseline.
Verification: post-Phase-36 test count. Alembic single head. CR archived. All pre-flight items confirmed.
Commit: Phase 38 step 0: CR archival and pre-flight.
Create src/loomworks/grammars/__init__.py with UNIVERSAL_SPECIFICATION_ELEMENTS. Create src/loomworks/grammars/models.py with SpecificationGrammar, StructuralElement, CompletenessCriterion, TestabilityCriterion. Create src/loomworks/grammars/registry.py with register_grammar, get_grammar, list_grammars.
Write: test_register_grammar, test_register_duplicate_raises, test_list_grammars, test_universal_elements_constant.
Verification: uv run pytest -v green.
Commit: Phase 38 step 1: specification grammar model and registry.
Create src/loomworks/grammars/defaults.py. Define all six grammars with fully populated structural elements, completeness criteria, and testability criteria. Register them. Import at application startup.
Write: test_default_grammars_registered.
Verification: uv run pytest -v green.
Commit: Phase 38 step 2: six default grammar registrations.
Create migration (next available number). Add specification_grammar (JSONB, nullable) to the shape-type storage table. Backfill the five Loomworks DeclaredShapeTypes with their assigned grammars. Refresh materialized views. Update projector.
Write: test_backfill_assigns_correct_grammars, test_adhoc_shape_type_no_grammar, test_grammar_persists_through_projector.
Verification: uv run pytest -v green. Migration applies and reverses cleanly.
Commit: Phase 38 step 3: migration and backfill.
Modify shape type creation to require specification_grammar on new declarations. Existing null-grammar shape types remain valid.
Write: test_new_shape_type_requires_grammar, test_new_shape_type_with_grammar.
Verification: uv run pytest -v green.
Commit: Phase 38 step 4: grammar required on new shape type declarations.
Add accepts_grammars: list[str] = [] to RenderSpecialist. Add grammar-matching check to dispatch path. Define GrammarMismatchError. Extend StubRenderSpecialist with configurable accepts_grammars.
Write: test_specialist_matching_accepted, test_specialist_matching_rejected, test_specialist_empty_accepts_all, test_specialist_matching_no_grammar_on_shape, test_stub_specialist_configurable_grammars.
Verification: uv run pytest -v green.
Commit: Phase 38 step 5: specialist grammar matching.
Implement evaluate_completeness function. Modify shape confirmation endpoint: evaluate if grammar exists, block if incomplete without override rationale, record override in event payload. Extend request and response schemas.
Write: test_completeness_all_pass, test_completeness_failure, test_confirmation_proceeds_when_complete, test_confirmation_blocked_when_incomplete, test_confirmation_override_with_rationale, test_confirmation_no_grammar_no_evaluation.
Verification: uv run pytest -v green.
Commit: Phase 38 step 6: completeness evaluation at confirmation.
Add specification_grammar to DeclaredShapeTypeResponse. Implement GET /grammars endpoint.
Write: test_grammar_list_endpoint.
Verification: uv run pytest -v green.
Commit: Phase 38 step 7: response schema and grammar listing endpoint.
Operator confirms. Tag substrate repo.
All Phase 38 work complete. Grammar registry operational. Six grammars registered. Five Loomworks shape types backfilled. Specialist matching advisory. Completeness evaluation at confirmation with override support. All tests green.
Operator reviews the backfill grammar assignments for the five Loomworks DeclaredShapeTypes and confirms they are correct.
Tag: phase-38-specification-grammar-declaration on DUNIN7/loomworks.
Implementation notes: docs/phase-impl-notes/phase-38-implementation-notes-v0_1.md absorbs execution-time surprises and findings.
SpecificationGrammar, StructuralElement, CompletenessCriterion, TestabilityCriterion models exist and are importable.UNIVERSAL_SPECIFICATION_ELEMENTS constant contains 13 elements.register_grammar, get_grammar, list_grammars.DeclaredShapeType has specification_grammar field (JSONB, nullable).specification_grammar.GrammarMismatchError.accepts_grammars → dispatch proceeds with warning.GET /grammars returns all six grammars.DeclaredShapeTypeResponse includes specification_grammar.specification_grammar from payload with clean default.phase-38-specification-grammar-declaration on substrate repo.src/loomworks/grammars/ with models, registry, defaults.GET /grammars.GrammarMismatchError.evaluator field on criteria is declared but not executed in Phase 38. Evaluation logic is key-presence checking only. Custom evaluator execution is future work.
Read the Change Request document at the path I supply below. This is
CR-2026-050 v0.1, the Phase 38 Change Request. You are the executing
agent named in the CR.
CR path: ~/Downloads/phase-38-cr-specification-grammar-declaration-v0_1.md
Phase 38 adds specification grammar declaration to the engine:
SpecificationGrammar model with structural elements, completeness
criteria, and testability criteria. Grammar registry with six
default grammars (req-table, narrative-storybook-pages,
legal-document, procedural-3d, application-specification,
scene-specification). DeclaredShapeType.specification_grammar
field with migration and backfill. Specialist accepts_grammars
matching (advisory when empty). Completeness evaluation at shape
confirmation with override rationale. GET /grammars endpoint.
Substrate-only phase. Seven steps, one checkpoint.
Code baseline: at HEAD after Phase 36. Post-Phase-36 test count.
Alembic at 0054 (or whatever Phase 36 left).
Run pre-flight (Step 0) per Section 3.2. The Step 0 checklist
includes: DeclaredShapeType model location and field absence;
RenderSpecialist base class location; shape confirmation flow
location; existing DeclaredShapeType count; shape-type storage
table location.
Per Section 3.3: archive this CR to
docs/phase-crs/phase-38-cr-specification-grammar-declaration-v0_1.md
at Step 0 before Step 1 begins.
Per Section 13, seven steps with one checkpoint. Auto-mode posture:
Steps 1–7 auto-mode-proceed; Checkpoint A halts for Operator to
review grammar backfill assignments and tag.
The thirteen universal specification elements from the Loompa
grammar (Discovery v0.4) are defined as engine-level constants.
The six default grammars reference these elements with
domain-specific required/optional markings.
Pre-flight surprises (Section 3.2 ground-truth divergence) stop
execution at Step 0 and drive a CR v0.2 revision; do not proceed
through divergence.
Implementation notes at Checkpoint A:
docs/phase-impl-notes/phase-38-implementation-notes-v0_1.md
absorbs execution-time surprises and findings.
DUNIN7 — Done In Seven LLC — Miami, Florida Phase 38: Specification Grammar Declaration — CR v0.1 — 2026-05-04