Version. 0.1
Date. 2026-05-09
Status. Investigation. Thinking material — pattern crystallized, not yet specified for build.
Author. Claude.ai (investigation layer). Operator: Marvin Percival.
Provenance. Conversation initiated by the Operator's question: Would it be possible to have the Companion present and listening on my mobile (iPhone or Android) similar to how Siri is always present on my iPhone? I would like to be able to say "Hey Companion, I parked on level 10." The conversation surfaced two engagements rather than one: a substrate intent-class engagement (quick-capture) and a capture-device engagement (mobile presence). This document is the second of the pair, treating the capture-device side. The companion document is loomworks-quick-capture-engagement-investigation-v0_2.md, which treats the substrate side and is referenced extensively here.
Informed by. Quick-capture investigation v0.2 (the substrate it captures into; the routing layer; the held-with-supersession lifecycle; the four-template voice register family; the name-agnostic router). Phase 41 (companion_name column; PATCH /me/companion-name; per-person Companion identity). Phase 46 (Operator Layer frontend; existing web/desktop surface). Operator Layer discovery v0.4. Companion as Agent Investigation v0.1 (capability tiers; attribution model). Knowledge Elevation Pathway Investigation v0.1 (commit lifecycle as universal write path). Loomworks Deployment Strategy v0.2 (jurisdiction as deployment parameter — relevant for mobile-app distribution).
The Operator's original question carries three distinct claims:
This document treats claims 1 and 2.
The literal interpretation of "always listening" — Siri-equivalent system-level wake-word recognition while the phone is locked, screen off, low-power — turns out to be unattainable as a third party on iOS and effectively unattainable on Android. But the spirit of the Operator's question is achievable through several composable surfaces, each with its own constraints and properties. The mobile-presence question is therefore not "how do we achieve Siri-equivalence" but rather "what is the right composition of surfaces such that the Operator experiences the Companion as present and reachable, with as little friction as the platforms permit?"
The framing question this document treats: what is the architecture of the Operator's mobile experience of the Companion — capture, identity, presence, posture, friction — and how does it compose with the existing Loomworks substrate (especially quick-capture) and the existing Operator Layer (web/desktop)?
The architectural form of the answer:
Mobile presence is multi-surface, not single-surface. The Operator's experience of the Companion on mobile is the composition of several distinct entry points, each appropriate for a different posture (sitting in a chair vs. driving vs. walking; phone in hand vs. phone in pocket vs. wrist only). No single surface delivers Siri-class always-on for a third-party app; the composition delivers something close.
The composition is layered:
companion_name. Bounded by foreground posture.Mobile presence is an Operator Layer extension, not a separate identity. The mobile app shares person identity, Companion identity, engagement set, Memory state with the existing web/desktop Operator Layer. A single Operator with a single Companion across all surfaces. Mobile is a surface, not a fork.
Posture-appropriate friction trade-offs are the design language. Different surfaces accept different friction in exchange for different capability. App Intents pays a one-tap onboarding ceremony to get OS-class always-on. Foreground listener pays foreground-only constraint to get Operator-named wake word. Watch complication pays "no wake word; tap required" to get wrist-only access. Each is the right trade-off for its posture; the design isn't optimizing all surfaces toward one thing.
Companion-name authority drives every surface. Same source (companion_name on person); each surface reads from there with surface-appropriate integration. The unifying principle from quick-capture v0.2 §4.3.
The mobile app form factor. The recommended primary form is a native mobile app (iOS Swift/SwiftUI, Android Kotlin/Jetpack Compose). PWA is the alternative considered (and set aside for the alpha). Native is required for App Intents, foreground wake-word detection, and the tap-to-speak surfaces. PWA can deliver the conversational surface and non-voice quick-capture, but most of the value sits in the native-only capabilities.
What does not land in this document: an App Intents schema specification; a wake-word library decision; a build/distribution pipeline; an iOS-vs-Android phasing; the substrate side of quick-capture (treated in the quick-capture investigation).
Initial framing. The Operator's question implies Siri-equivalent behavior: phone-locked, screen-off, low-power, constantly listening for "Hey Companion."
Why the literal interpretation fails — iOS. Apple reserves the always-on, low-power, system-level wake-word listener for Siri. SiriKit doesn't expose wake-word registration to third parties. There is no "Hey [your app name]" hardware-or-system-level wake on iOS. Background mic access exists in narrow categories (CallKit; some VoIP exemptions) but not "phone in pocket, screen off, listening for a wake word." Apple actively kills background mic access for battery and privacy.
Why the literal interpretation fails — Android. Android historically allowed VoiceInteractionService for OEM assistants (the path Google Assistant uses on Pixel) but not arbitrary third-party apps. Foreground services can hold the mic with a persistent notification (the "your phone is recording" indicator), but this is foreground-service posture, not always-on-when-phone-locked posture. Even within foreground services, battery cost is real and OEMs aggressively kill long-running foreground services in low-memory situations.
What landed. Siri-equivalent literal "Hey Companion" is closed for third parties on both platforms. The right framing is not "how do we achieve Siri-equivalence" but "what is the right composition of surfaces." The composition turns out to be richer than a single wake-word would have been — different surfaces serve different postures, and that's a feature.
The pattern. The OS owns the wake word; the OS recognizes that the app exists; the user invokes the app's specific intent through the OS. For iOS:
"Hey Siri, Freda, I parked on level 10."
↓
Siri wake-word recognition (system-level, always-on, allowed)
↓
Siri intent matching: "Freda" → matches Loomworks Siri Shortcut named "Freda"
↓
Loomworks app's quick-capture intent invoked with parameter "I parked on level 10"
↓
App backgrounds; intent receiver writes to /quick-capture; voice ack via SiriKit response
The Operator pays a small prefix tax — "Hey Siri, Freda, …" instead of "Hey Freda, …" — and gets Siri-class always-on for free. App is not even running when the wake word fires; Siri starts it.
Three options for "Freda" as the invocation. Treated in detail in quick-capture v0.2 §4.3:
Apple's INShortcut API — the suggestion mechanism. The app, at first run after the Operator names the Companion, calls INVoiceShortcutCenter.shared.setShortcutSuggestions(...) with a suggested shortcut whose phrase is the configured companion_name. iOS shows this in the Shortcuts app and (depending on iOS version) suggests it on Siri lookups. The Operator confirms once; from then on, "Hey Siri, Freda" works.
Android equivalent. App Actions (built on Google Assistant) — the app declares intents in actions.xml and shortcut invocation phrases. The shortcut name can be Operator-configurable through similar suggestion mechanisms. Less mature ecosystem than iOS's; more variation across OEMs (the path differs on Pixel vs. Samsung vs. others).
What landed. App Intents / App Actions is the OS-mediated path; Siri Shortcut suggestion at first run is the cleanest fit for Operator-authoritative naming; the Operator pays one-time onboarding friction (one tap) for ongoing always-on access. iOS-first because the ecosystem is more mature; Android in close second.
When the Loomworks mobile app is open and mic permission is granted, the app runs its own wake-word detector. No platform constraint applies at this surface; the wake word literally is the Operator's chosen companion_name.
Wake-word detector options:
SpeechRecognizer can be used with continuous listening but is power-hungry and not designed for wake-word.Recommendation: Picovoice Porcupine for alpha. Best support for custom wake words; predictable behavior; reasonable license posture.
Constraints:
The composition with App Intents. The Operator gets both: when the app is open, foreground listener responds to "Hey Freda" directly; when the app is closed, "Hey Siri, Freda" routes through Siri. Different invocations for different postures. The Operator learns both naturally — open the app for sustained capture sessions; "Hey Siri, Freda, …" for one-off captures from the home screen.
Voice surfaces aren't always the right answer. Three tap-to-speak surfaces complete the composition:
Lock Screen widget (iOS). A widget on the Lock Screen — tap to record-and-send. Surfaces "Note to Freda" or comparable. iOS 16+ widget capabilities allow this. Operator unlocks once; tap once; speaks; releases; capture sent. Useful in the "phone in hand, screen on, but I don't want to fuss with Siri" posture.
Home-screen widget (Android). Equivalent; richer surface area than iOS widgets historically. Same shape: tap; record; release; sent.
Watch complication (Apple Watch / Wear OS). Tap on the watch face; record on watch (or pass through to phone if watch's mic is unavailable); send. Useful in the "phone in pocket, wrist accessible" posture — the highest-value form factor for quick capture, because the friction of getting the phone out is the highest friction of all the surfaces, and the watch eliminates it.
Common properties:
companion_name. "Note to Freda" (not "Note to Companion"). Operator-authoritative naming surfaces visually.The watch complication is the most interesting form factor for this investigation. Quick-capture's highest-frequency case is the in-the-moment fact-logging — "I parked on level 10" exactly as the Operator opened the question with. Many of those moments are when the phone is not in hand. The watch surface delivers highest-value quick-capture access at lowest-friction posture.
The mobile app must also surface the conversational Companion (Phase 42). Not just quick-capture. The Operator wants to ask the Companion questions, work through ideas, review held assertions for batch commit, explore Memory across engagements — all of which are the conversational pipeline, not quick-capture.
The conversational mobile surface is a chat UI mirroring the existing Operator Layer's chat. Same converse pipeline; same SSE stream from Phase 44; same approval cards from Phase 45. The mobile chat is a thin client — most of the rendering and state is server-side.
Things this surface needs:
/me/credits endpoint, the same SSE stream, the same /converse endpoint)./me/dashboard/* endpoints from Phase 39.Composition with quick-capture. The Operator dictates "I parked on level 10" via App Intents (no app open); the assertion lands held. Later, the Operator opens the mobile chat; sees the held assertion in their daily-log engagement's queue; commits it (or batch-commits the day's queue). The mobile chat is also where the Operator answers any pending disambiguation questions from write-with-disambiguation captures that happened earlier.
Voice isn't always available (meeting; quiet environment; speech-to-text errors with technical terms). Non-voice quick-capture surfaces:
/quick-capture. Implementation through keyboard extension is non-trivial (keyboard extensions are sandboxed); the cleaner alternative is a system share-sheet target.The composition with voice surfaces means there is no single "quick-capture path" — there's a set of capture devices, each surface-appropriate, all routing to the same backend.
The mobile app is not a fork. The Operator's Companion identity, engagement set, Memory, conversation history, held queue, credit balance — all of these live in the substrate, accessed by the mobile app through the same APIs the web/desktop Operator Layer uses.
What the mobile app shares with web/desktop:
/auth/* endpoints).companion_name; PATCH /me/companion-name).GET /me/credits).What the mobile app adds beyond web/desktop:
What the mobile app cannot do better than web/desktop:
The right framing. Mobile is the capture and quick-review surface. Desktop is the production surface. Watch is the capture-only surface. Each surface optimizes for its strongest posture; the federation is what makes them feel like one Companion across all of them.
Mobile means offline. The Operator will quick-capture in dead zones — parking garages (the very example the Operator opened with!), elevators, basement floors, airplane mode. The capture surface must handle offline.
The model: quick-capture utterances queue locally when offline; replay to /quick-capture when connectivity returns. The mobile app maintains a local queue; on each app launch / foreground / connectivity-change, the queue drains.
Properties:
This is one of the most under-discussed properties of mobile-presence — the OS-mediated path (App Intents) is offline-uncertain (Siri may or may not let an offline app run an intent depending on iOS version and intent type), but the foreground listener + tap-to-speak surfaces handle offline cleanly because the app is in the foreground when capture happens.
Mobile presence has a non-trivial onboarding that the existing Operator Layer doesn't have. First-run flow:
companion_name. App reads from substrate; defaults to "Companion" if unset.INVoiceShortcutCenter with the configured name. Surfaces the Shortcuts app dialog — Operator confirms, name pre-filled.Key property: every step is skippable. The Operator who only wants the conversational chat surface skips the mic permission, the shortcut suggestion, the wake-word setup, the widget, the complication. They still get a working mobile chat. Mobile presence is layered, and the Operator chooses how many layers to enable.
This connects to the methodology principle "only show what is available" — surfaces the Operator hasn't enabled don't appear. No grayed-out widget; no disabled wake-word. If the Operator never granted mic permission, there's no "say Hey Freda" tip showing up in the UI.
[OS-mediated path — App Intents / App Actions]
"Hey Siri, Freda, …" or "Hey Google, Freda, …"
↓ (Siri / Assistant routes; app may be backgrounded)
↓
[Loomworks mobile app — intent receiver]
↓ (writes to /quick-capture; reads companion_name; renders ack via SiriKit response)
↓
[Substrate — quick-capture surface from quick-capture v0.2]
[Foreground listener — wake-word detector inside the app]
"Hey Freda, …" (when app is open)
↓ (Picovoice Porcupine or equivalent; listens for companion_name)
↓
[Loomworks mobile app — capture handler]
↓ (writes to /quick-capture; voice ack from local templates)
↓
[Substrate — quick-capture surface]
[Tap-to-speak surfaces — Lock Screen widget / home widget / watch complication]
Tap → record → release
↓ (visible label binds companion_name)
↓
[Loomworks mobile app — capture handler]
↓
[Substrate — quick-capture surface]
[Non-voice surfaces — share sheet / typed quick-capture / keyboard shortcut]
Type / share text
↓
[Substrate — quick-capture surface]
[Conversational surface — mobile chat UI]
Standard chat
↓ (uses /converse, SSE, /me/dashboard/*, Phase 45 approval cards)
↓
[Substrate — converse pipeline]
[Offline queue — local persistence when network unavailable]
Captures land in local queue with local timestamps
↓ (drains on connectivity return)
↓
[Substrate — quick-capture surface, idempotency-aware]
The substrate work is small — the mobile app talks to existing endpoints (after quick-capture's /quick-capture lands). The mobile-side work is where the volume sits: native iOS app + native Android app + watch app + widgets + offline queue + onboarding.
The build-volume estimate is meaningful. Mobile is not a small build. It is plausibly the largest single capture-device surface Loomworks ships, comparable in effort to the Operator Layer's web frontend. Phasing matters.
Phase 41 — Companion identity. Mobile reads companion_name; surfaces it on every UI element; passes the read into Siri Shortcut suggestion, Picovoice Porcupine configuration, watch complication label. PATCH /me/companion-name is reachable from mobile (chat surface; settings).
Phase 42 — converse pipeline. Mobile chat hits /converse; same intent classification, same persona system prompt, same responder. Mobile is a thin client.
Phase 43 — held + commit lifecycle. Mobile chat is one of the surfaces where held assertions surface for commit. Mobile quick-capture (offline-queued or not) lands held; the mobile chat (or web/desktop) is where the Operator commits.
Phase 44 — SSE + proactive behavior. Mobile maintains the SSE connection while foregrounded; reconnects on foreground. Proactive Companion turns surface as notifications when backgrounded (push notifications carrying the converse turn; tapping opens the chat surface to that engagement).
Phase 45 — approval cards and delegation contract. Mobile renders approval cards in the chat surface (the same <ProposalApprovalCard> / <GrantDecisionApprovalCard> family from Phases 49/50, rendered natively or via WebView). Approve/Decline tap dispatches per Phase 45.
Phase 46 — Operator Layer frontend. Mobile app is an Operator Layer surface. Federation: same auth, same person, same Companion. The web Operator Layer is "Operator Layer (web)"; mobile is "Operator Layer (mobile)." Same family, different form factor.
Phase 48 — production passkey sign-in. Mobile uses passkey-based auth. iOS Passkeys / Android Credential Manager. Cross-device key management is the standard platform behavior.
Quick-capture investigation v0.2. The substrate side. Every mobile capture surface (App Intents, foreground listener, widgets, watch complication, share sheet) routes to the substrate's /quick-capture endpoint. The router handles destination resolution, conflict detection, supersession; mobile is one of many capture devices feeding the same substrate.
Closed-loop investigation v0.1 — observation distributed, attribution centralized. Mobile is one observer (or rather, mobile contains multiple observers — App Intents, foreground listener, widgets, etc.). All route through the Companion to author held assertions. The principle from the closed-loop investigation generalizes cleanly.
Knowledge elevation pathway investigation v0.1 — held + commit as universal write path. Mobile quick-capture is a high-frequency exercise of this pattern; mobile chat is where commit-or-retract decisions land for Operator review.
Loomworks deployment strategy v0.2 — jurisdiction as deployment parameter. Mobile distribution interacts with jurisdiction in ways the web doesn't. App Store / Play Store regional approval; data residency for the mobile-substrate connection (the mobile app must connect to the right Loomworks instance per Operator jurisdiction). The Operator's account binds to a specific Loomworks deployment; the mobile app discovers this at sign-in and connects accordingly. No Loomworks-as-monolith assumption.
The question. Both platforms are needed eventually. Which goes first, and by how much?
Considerations.
This investigation does not settle. Scoping decides based on Operator distribution at the time and engineering capacity.
The question. Native iOS + native Android, vs. a PWA, vs. a cross-platform framework (React Native, Flutter).
The case for native (recommended). App Intents / App Actions, foreground wake-word, watch complication, widgets, share sheet — most of these require native. PWA can deliver the conversational surface and basic quick-capture but loses the high-value capabilities.
The case for PWA. Faster to ship; Operator-Layer code reuse is high; no app-store gatekeeping. Loses most of mobile-presence's value. Acceptable as a stopgap for the conversational surface only; not as the primary mobile presence.
The case for cross-platform (React Native / Flutter / Kotlin Multiplatform). Two-ish platforms with one codebase. Trade-off: platform-native feel suffers; some platform integrations (especially the deeper App Intents / App Actions integrations and watch complications) require native bridges anyway.
This investigation does not settle. Scoping decides. Recommendation leaning: native for alpha because the high-value surfaces require it, with the door open to revisit if engineering capacity becomes binding.
The question. Picovoice Porcupine vs. Snowboy vs. platform-native vs. custom on-device model.
Investigation lean: Picovoice Porcupine. Best support for custom wake words; predictable behavior; reasonable license posture.
This investigation does not settle. Cost analysis (Picovoice license vs. engineering effort for alternatives) is a scoping concern.
The question. App Store / Play Store distribution vs. TestFlight / internal track for alpha. Code signing. CI/CD for two native platforms.
This investigation does not address. Standard mobile-app shipping concerns; well-trodden ground; no Loomworks-specific architecture questions.
The question. Is the watch app its own substantial deliverable, or a thin extension of the iPhone/Android app?
The case for thin extension (recommended for alpha). Watch surfaces only the tap-to-speak quick-capture and the visible Companion presence (complication). All capture goes through the phone's connection. Thin = tractable.
The case for substantial. Long-term, the watch is a high-value form factor (per §3.4). A direct watch-to-substrate path (no phone tether) widens the always-accessible window.
This investigation does not settle. Scoping decides. Recommendation: thin extension for alpha; revisit for a later phase if watch usage justifies the investment.
The question. Phase 44 SSE drives proactive Companion behavior on web. Mobile equivalent uses APNs (iOS) / FCM (Android) push notifications when the app is backgrounded. Tapping the notification opens the chat to the relevant engagement.
Sub-questions. Privacy posture (push notifications carry text — sensitive Companion turns might warrant content-stripping); rate limits (how often does proactive Companion behavior fire); offline interaction (push-while-offline → notification reappears on next foreground? Or no?).
This investigation surfaces but does not settle.
The question. Operator opens the mobile chat mid-conversation that was happening on desktop. Does the mobile chat see the conversation history? Pick up where desktop left off?
Architectural answer (sketched). Yes — same /converse endpoint, same conversation history, same SSE stream. The Operator can switch surfaces mid-thought.
Sub-question this raises. SSE connection from two surfaces to the same person — broadcast vs. pinned to one device? Probably broadcast to all of the person's connected devices; each device renders the same SSE event. This investigation does not settle the design.
loomworks-quick-capture-engagement-investigation-v0_2.md).This investigation produces architectural framing and a list of decisions for next stages. The output is thinking material, not building material.
Mobile presence is not implementation-ready. The §6 architectural questions and §8 open questions need scoping, and the substrate-side quick-capture work is the prerequisite — there is no mobile capture surface to ship until /quick-capture exists.
Three upstream considerations bear on implementation timing:
The earliest reasonable timing: mobile-presence scoping in parallel with quick-capture build (Phase 53+ scoping); mobile-presence build Phase 54+ or later. Watch complication likely a separate phase after the iPhone / Android base apps.
/quick-capture endpoint contract) and answers the §6 / §8 questions. The two scoping notes can be co-drafted or sequenced; co-drafting is cleaner because the substrate decisions and the mobile decisions interact at the contract level.loomworks-queued-directions-and-deferred-work-v0_3.md (next version) for the mobile-presence + quick-capture pair. Cross-references to:v0.1 (2026-05-09). Initial investigation. Nine rounds of working-through (§3.1–§3.9). Layered composition framing — App Intents + foreground listener + tap-to-speak + non-voice + conversational + offline + onboarding. Companion-name authority unified across surfaces. Federation with the existing Operator Layer named. Five architectural questions surfaced as not-settled (§6). Fifteen open questions for next stages (§8). Implementation-readiness Phase 53+ scoping; Phase 54+ build at earliest, with engineering capacity as the larger pacing constraint.
The investigation pairs with loomworks-quick-capture-engagement-investigation-v0_2.md. The two together describe the substrate side and the capture-device side of the Operator's mobile experience of the Companion.
DUNIN7 — Done In Seven LLC — Miami, Florida Loomworks Mobile Presence Investigation — v0.1 — 2026-05-09