Northstar operational review
A first release that makes exception review faster without weakening traceability, access control or human accountability
Swathi Perumalla
Draft presentation, November 2026
Built on Kamiwaza's on-premises platform architecture
How to read this deck
E01-E14 cite the supplied Evidence Pack. Assumption marks anything not in the case. Kamiwaza claims carry their verification status. The case and Northstar are fictional.
Evidence hunting slows analysts; untraceable claims slow reviewers
Context
Northstar Financial, a fictional regional financial-services company, runs Kamiwaza on premises because operational and customer data cannot leave its environment.
In a recurring operational review, analysts investigate service exceptions, assemble evidence from several internal systems, recommend a disposition and send the case to an authorized reviewer, who checks it, returns it or records the final disposition.
Today analysts move among systems, build the packet by hand and copy the outcome back into the case system. Reviewers repeat work when a claim lacks a clear source.
Goal Shorten the time from exception creation to an authorized, auditable disposition.
median analyst time per exception case; 4.2 h of it locating and reconciling source records
E01of sampled pilot summaries had a claim reviewers could not trace to a displayed source
E03of eligible weekly cases used the draft-summary pilot more than once
E02of pilot outputs were manually reformatted before review
E10Root cause in users' words: the draft does not show why a source was selected or what was omitted, so analysts reopen source systems E04.
One primary user, one approver, one guardian of the record
Operations analyst
- Investigates the exception, finds and reconciles records, separates reliable evidence from open uncertainty, recommends a disposition (Scenario)
- Spends 4.2 of 11.5 hours locating and reconciling records E01
- Values a starting draft, but reopens source systems when it hides why sources were chosen or omitted E04
- Reformats 27% of outputs and copies the packet into the case system E10
Authorized reviewer
- Checks the evidence, returns the case for more work or records the final disposition (Scenario)
- Repeats parts of the investigation when a claim lacks a clear source (Scenario, E03)
- Notices an inline explanation and a required confirmation step, misses a small badge E13 five participants
Compliance lead
- Every material claim links to an authorized source E06
- The record keeps approver, source set, model and prompt version, edits and final disposition E06
- No user sees or retrieves a source they could not access; exports reveal no restricted content E07
- Has not approved automated dispositions E06
Stakeholders who shape scope
Fourteen signals, and what each one means for the first release
| ID | Source | Observation | Limitation | Implication for release 1 |
|---|---|---|---|---|
| E01 | Analyst interviews | Median 11.5 h per case; 4.2 h locating and reconciling records | 6 interviews, one team | Attack evidence gathering first |
| E02 | Pilot telemetry | 18% of eligible weekly cases used the pilot more than once | Does not explain abandonment | Adoption is the risk; learn why users leave |
| E03 | Pilot quality review | 84% materially accurate; 14% had an untraceable claim | 50 sampled cases | Every claim needs a visible source |
| E04 | Frontline interview | Drafts hide why a source was selected or what was omitted | 4 of 6 interviews | Show selection and omission reasons |
| E05 | Operations leader | Standard packet; 40% faster median cycle this quarter; narrower release OK | Commitment discussed with execs | Narrow scope, consistent use |
| E06 | Compliance lead | Claims link to authorized sources; keep approver, sources, model and prompt version, edits, disposition | No automated dispositions | Versioned audit record; human decides |
| E07 | Compliance lead | No access to sources a user cannot see; exports reveal nothing restricted | Existing access groups only | Inherit source permissions; safe export |
| E08 | CTO | Six-week demo: 5 sources, autonomous research, auto disposition, dashboard | No engineering estimate | Reframe the demo around what can ship |
| E09 | Engineering discovery | Case system and document repository connectors exist; 3 others need new connector and security review | 2 engineers for 4 weeks, then 1 moves | Two sources in release 1 |
| E10 | Support observations | 27% of outputs reformatted; packet copied into case system | 63 support-tagged cases | Standard packet format and write-back |
| E11 | Incident report | Revoked document stayed in 3 generated drafts until refresh | No exposure outside authorized users | Revocation invalidates dependent claims |
| E12 | Sales note | Renewal needs measurable adoption and a credible coverage path | Decision in ten weeks | Instrument adoption from day one |
| E13 | Usability session | Users missed a small low-confidence badge; noticed inline explanation and confirmation | 5 participants | Inline uncertainty plus confirm step |
| E14 | Security architecture | On-prem endpoints, role groups, append-only audit, source permission checks; no field-level redaction | Current as of case date | Exclude, do not redact, on export |
Seven stories define the minimum first release
| # | As a | I want | so that | Evidence |
|---|---|---|---|---|
| US1 | Analyst | the records for my case gathered from the case system and document repository into one evidence set | I stop spending hours locating and reconciling records | E01, E09 |
| US2 | Analyst | to see why each source was selected and what was left out | I trust the draft without reopening every source system | E04, E03 |
| US3 | Reviewer | to open the authorized source behind every material claim and return the case with comments | I verify instead of re-investigating | E06, Scenario |
| US4 | Compliance lead | the assistant to retrieve only what the requesting user can access, and exports to carry no restricted content | source permissions are never weakened | E07, E14 |
| US5 | Analyst | low confidence and revoked sources flagged inline, with a confirmation I must make | I never submit a claim I have not checked | E13, E11 |
| US6 | Compliance lead | approver, source set, model and prompt version, edits and disposition kept in an append-only record | every decision is auditable | E06, E14 |
| US7 | Analyst | a standard packet written back to the case system | I stop reformatting and copying by hand | E10, E05 |
A governed review app on Kamiwaza's Inference Mesh: data stays put
Results flow up as permitted results only. Source data is processed in place and never pooled into a central copy. Dashed = later.
How it fits
Inference goes to the data. Inference Mesh routes AI to each location and processes it locally; no data leaves Northstar's control (Inference Mesh page; Scenario).
No central copy. No data lake or ETL step; the mesh merges results, not records (Inference Mesh page).
Permissions at query time. ReBAC applies the user's access before results return (Kamiwaza, verified page read; E07E14).
Path to coverage. Dashed sources join after connector and security review E09E12.
Five steps to a disposition, with the AI's work kept on the record
Case opens
Inference runs at the case system and document repository; only results the analyst may see come back.
Sources the analyst cannot access are not retrieved or shown.
AI drafts
Draft links every material claim to its source, with why it was chosen and what was omitted.
Draft v1 saved with source set, model and prompt version.
Flags raised
Low-confidence claims get an inline explanation; a revoked source marks its claims stale.
Submit is blocked until the analyst confirms or refreshes.
Analyst edits
Analyst corrects the draft and submits a standard packet to the reviewer.
Each edit is a new version; the AI original is kept beside it.
Reviewer decides
Reviewer opens each claim's source, returns the case or records the final disposition.
Only a named human disposes; record written back and sealed. Never 100% automatic.
Every link records who, when, source set, model and prompt version E06, in append-only storage E14. Version numbers are illustrative.
AI assumptions, decisions, prompts and commands are never overwritten
Logged E06E14
Every AI assumption, prompt, command, retrieved source set and output is written to the append-only audit store as it happens.
Versioned E06
Prompt templates, model versions, drafts and edits each get a version ID. Nothing is updated in place; a change is a new version.
Human-approved E06design proposal
A new prompt template, a changed AI assumption or a disposition takes effect only after a named human approves it.
Original kept E06E03
The AI's original output stays beside every human edit, so the reviewer sees the diff and the reasons.
Audit trail E06E07
One record per case: who acted, when, on which sources, with which model and prompt version and permission rule.
One claim, four versions (illustrative)
Recommends disposition A; cites two sources; model and prompt version recorded
Changes the recommendation; reason required; diff against v1 stored
Asks for one more source; comment linked to the claim
A named human reviewer records the disposition; record sealed and written back
Never: in-place edits, auto-applied prompt changes, deleted history. Dispositions are never automated (RULE).
Eight principles the first release will not trade away
Inherit permissions
The assistant sees only what the requesting user can see, checked at query time.
Trace every claim
Each material claim links to an authorized source; untraceable claims are flagged, not hidden.
Show uncertainty where eyes are
Inline explanation and a required confirmation, not a small badge.
Humans decide
AI drafts and recommends; a named human reviewer records every disposition. Never 100% automatic, even when assisted.
Fresh or flagged
A revoked source invalidates the claims built on it until the set is refreshed.
Data stays in place
Inference runs where each source lives, on premises; no central copy of case data.
Exclude rather than redact
With no field-level redaction, exports drop restricted documents whole.
Check before answering
Kamiwaza notes after-the-fact logging does not prevent overexposure at answer time, so filtering happens before generation.
What the first release has to fit inside
Technical
- On premises only; data cannot leave Northstar Scenario
- Reliable connectors only for case system and document repository E09
- Ticketing, call recording and policy each need a new connector and security review E09
- No field-level redaction across sources E14
- No broader integrations, model training or new identity infrastructure in the window Scenario
Compliance
- Every material claim links to an authorized source E06
- Record keeps approver, source set, model and prompt version, edits, disposition E06
- Automated dispositions not approved E06
- Use existing access groups; no new identity system E07
- Exports reveal no restricted content E07
Capacity
- Two engineers and one product designer for the first four weeks Scenario
- One engineer moves to another commitment after four weeks E09
- No engineering estimate exists for the CTO's demo E08
- Designer availability after week 4: unknown Assumption gap
Timing
- Credible first release in six weeks Scenario
- Six-week executive demo E08
- 40% shorter median cycle time this quarter E05
- Renewal decision in ten weeks E12
Show a governed slice that ships, plus a costed path to the rest
| Pressure | We ship | Why | Evidence |
|---|---|---|---|
| All five sources in the demo (E08) | Two sources with existing connectors; a sequenced plan for the other three | Only two connectors are reliable; the sponsor does not need all five at launch | E09, E12 |
| Autonomous research and automatic disposition (E08) (conflicts with RULE; not planned) | AI-drafted evidence and recommendation; a human disposes | Compliance has not approved automated dispositions | E06 |
| Configurable executive dashboard (E08) | A fixed adoption and cycle-time view | Renewal hinges on measurable adoption, not configurability | E12, E05 |
| Speed at every step (E05) | A required confirmation only on low-confidence or stale claims | Users miss badges; friction is limited to where risk is | E13, E11 |
| Field-level redaction (E07) | Exclude restricted documents from exports | No redaction exists; building it does not fit four weeks | E14, E09 |
Build in four weeks, prove use by week six, bring proof to renewal
Lead with adoption and trust, judge on cycle time
Leading
| Measure | Baseline | Target (assumed) |
|---|---|---|
| Eligible cases using the tool repeatedly, weekly | 18% in pilot (E02) | Majority of pilot team's cases |
| Hours locating and reconciling records | 4.2 h per case (E01) | Halved |
| Material claims with a linked authorized source | 14% had an untraceable claim (E03) | 100% linked |
| Packets reformatted by hand | 27% (E10) | Under 5% |
Outcome and guardrails
| Measure | Baseline | Target (assumed) |
|---|---|---|
| Median cycle time, exception to disposition | Not in case; measure in week 1 | 40% shorter (E05 goal) |
| Cases returned by reviewer | Not in case | Falling trend |
| Permission or revoked-source incidents | 3 revoked-doc cases (E11) | Zero |
| Records with all E06 fields | Not in case | 100% |
What could make this wrong, and how I would find out early
Risks
- Adoption stays low: the pilot data does not say why users left E02
- Thin evidence: six interviews in one team, five usability participants, fifty sampled cases E01E13E03
- Revocations reach drafts late; sync speed is not documented E11Kamiwaza unverified
- CTO expects a five-source autonomous demo E08
- Capacity halves after week 4 E09
Next validation
- Interview pilot users who stopped using it E02
- Test the packet and reviewer view with reviewers on real cases E03E13
- Measure end-to-end cycle time in week 1 E05
- Run a permission and revocation test suite before the pilot E07E11
- Change course if reviewers still re-investigate linked claims E03
Open questions
- Who owns the release decision: operations leader or CTO? E05E08
- Can the case-system connector write back? E09
- Who stays on the team after week 4? E09
- What confidence signal do the on-prem models give? E13E14
- Will compliance accept AI-drafted recommendations? E06
AI use: what I accepted, what I corrected, how I verified
Candidate Instructions, AI use: “one suggestion you accepted and why; one suggestion you rejected or corrected and why”. Product Brief Template, AI working notes: “Summarize one AI suggestion you accepted, one you rejected or corrected, and how you verified the work.”