> For the complete documentation index, see [llms.txt](https://t420.gitbook.io/t420/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://t420.gitbook.io/t420/shadow-training-layer.md).

# shadow-training-layer

## FleetOS Shadow Training Layer

### Objective

Design a **shadow training layer** inside FleetOS that learns how real fleet-service coordination happens without taking unsafe autonomous action too early.

This layer should:

* observe real coordination workflows
* structure the decision points inside those workflows
* capture human choices and outcomes
* generate training-quality data for future autonomous coordination agents
* initially operate in **shadow mode** (suggest, score, compare), not full control mode

The goal is not just to log activity. The goal is to create a system that can learn:

* what information was needed
* what action should have happened next
* what humans actually did
* whether it worked
* how long it took
* where the workflow got stuck

***

### Why This Matters

FleetOS is not only a workflow layer. If designed correctly, it also becomes a **training environment for autonomous fleet-service coordination agents**.

Every real job flowing through FleetOS can teach the system:

* what a good intake looks like
* when a quote is incomplete
* when approval is likely to fail
* which supplier should be routed first
* which part is likely required
* which technician / bay assignment is feasible
* what follow-up should happen next
* when escalation is required

But this only works if the workflow is instrumented deliberately.

***

### Design Principles

#### 1. Shadow first

The agent should first observe and suggest. It should not silently take over real operations early.

#### 2. Decision-centric tracking

Do not just log events. Log **decision points**, candidate actions, chosen actions, and outcomes.

#### 3. Outcome-linked learning

A suggestion without an outcome is weak training data. Every major recommendation should link to what happened next.

#### 4. Human override is signal

Human disagreement is useful training data, not noise.

#### 5. Structured before unstructured

Capture structured fields wherever possible. Keep raw messages too, but do not rely on them alone.

***

## Part 1 — Layered Architecture

### Layer A — Workflow Event Layer

This is the raw operational timeline.

Examples:

* work order created
* quote submitted
* quote approved
* technician assigned
* part added
* part status updated
* work started
* work completed
* job cancelled

This layer answers:

* what happened
* when it happened
* who did it

### Layer B — Decision Point Layer

This is the key layer. It records moments where a choice existed.

Examples:

* intake missing required info
* which supplier to route to
* whether quote is acceptable
* whether approval is needed
* whether job should be escalated
* which technician should be assigned
* whether a part is likely required
* whether delay risk is high
* what follow-up should be sent

This layer answers:

* what decision existed
* what options were available
* what the system would recommend
* what the human chose

### Layer C — Outcome Layer

This links choices to results.

Examples:

* quote approved or rejected
* completion within deadline or not
* budget variance
* downtime duration
* supplier response time
* rework required or not
* customer escalation or not

This layer answers:

* did the decision work
* was the job faster / cheaper / cleaner
* what was the downstream consequence

### Layer D — Training Example Layer

This converts prior layers into model-ready examples.

Examples:

* input state
* candidate next actions
* preferred action
* actual action taken
* reward / outcome score

This layer answers:

* what should the agent learn from this case

***

## Part 2 — Core Objects

### 1. CoordinationSession

Represents the full life of a service coordination case.

Suggested fields:

* `coordination_session_id`
* `work_order_id`
* `fleet_company_id`
* `supplier_company_id`
* `opened_at`
* `closed_at`
* `final_outcome`
* `success_score`

### 2. WorkflowEvent

Raw event.

Suggested fields:

* `event_id`
* `coordination_session_id`
* `work_order_id`
* `event_type`
* `actor_type` — `fleet | supplier | system | agent`
* `actor_id`
* `actor_name`
* `channel` — `api | ui | whatsapp | email | webform | agent`
* `payload`
* `created_at`

### 3. DecisionPoint

The most important object.

Suggested fields:

* `decision_point_id`
* `coordination_session_id`
* `work_order_id`
* `decision_type`
* `state_snapshot_id`
* `trigger_event_id`
* `opened_at`
* `resolved_at`
* `resolved_by_actor_type`
* `resolved_by_actor_id`

#### Example decision types

* `missing_intake_info`
* `supplier_selection`
* `quote_evaluation`
* `approval_required`
* `technician_assignment`
* `bay_assignment`
* `part_requirement_prediction`
* `follow_up_needed`
* `delay_escalation`
* `vehicle_delivery_request`

### 4. CandidateAction

Represents options available at a decision point.

Suggested fields:

* `candidate_action_id`
* `decision_point_id`
* `action_type`
* `action_payload`
* `proposed_by` — `human | rule_engine | model`
* `confidence_score`
* `rank`

### 5. SelectedAction

Represents what was actually chosen.

Suggested fields:

* `selected_action_id`
* `decision_point_id`
* `candidate_action_id` (nullable if human chose something outside candidate set)
* `selected_action_type`
* `selected_action_payload`
* `selected_by_actor_type`
* `selected_by_actor_id`
* `selected_at`
* `selection_reason`

### 6. OutcomeRecord

Represents the downstream result linked to one or more decisions.

Suggested fields:

* `outcome_record_id`
* `coordination_session_id`
* `work_order_id`
* `linked_decision_point_ids`
* `outcome_type`
* `outcome_value`
* `measured_at`

#### Example outcome types

* `quote_approved`
* `quote_rejected`
* `job_completed`
* `deadline_met`
* `budget_variance`
* `response_time_minutes`
* `completion_time_minutes`
* `reassignment_required`
* `escalation_occurred`
* `customer_complaint`

### 7. StateSnapshot

Frozen structured view of the case at a decision moment.

Suggested fields:

* `state_snapshot_id`
* `coordination_session_id`
* `work_order_id`
* `snapshot_json`
* `created_at`

This should capture things like:

* priority
* vehicle
* location
* category
* quote amount
* budget auth
* current technician
* bay availability
* part status
* elapsed time
* channels used
* missing fields

### 8. TrainingExample

Model-ready representation.

Suggested fields:

* `training_example_id`
* `decision_point_id`
* `input_snapshot_id`
* `target_action_type`
* `target_action_payload`
* `human_action_type`
* `human_action_payload`
* `model_action_type`
* `model_action_payload`
* `agreement_score`
* `outcome_score`
* `created_at`

***

## Part 3 — Shadow Mode Operation

### Mode 1 — Observe only

The system does not suggest anything visible to users. It only logs:

* decision point
* state snapshot
* what human chose
* outcome

Use this first if data quality is poor.

### Mode 2 — Silent recommendation

The model generates a recommendation, but users do not see it. FleetOS stores:

* what model would have done
* what human actually did
* whether model would have been correct

This is ideal for early offline evaluation.

### Mode 3 — Suggested action

Users see recommendations, but must confirm.

Examples:

* suggest likely required part
* suggest best supplier to route
* suggest follow-up message
* suggest technician / bay assignment

### Mode 4 — Guardrailed autonomy

Only after enough evidence. Allow autonomous action on narrow low-risk tasks.

Examples:

* send follow-up reminder
* request missing photos
* normalize quote format
* update internal status after explicit supplier signal

Not for early autonomy:

* approving spend
* choosing a supplier without rules/human review
* closing work orders without evidence

***

## Part 4 — Decision Types To Instrument First

### 1. Intake completeness

Question: Is there enough information to route the job or quote it?

Track:

* missing vehicle info
* missing location detail
* missing symptom description
* missing urgency / deadline
* missing approval authority

### 2. Supplier selection

Question: Which supplier should receive the request first?

Track:

* category fit
* geography
* historical response time
* completion quality
* pricing behavior
* open workload if known

### 3. Quote evaluation

Question: Is the quote likely acceptable / risky / above budget?

Track:

* quote amount vs budget
* quote amount vs historical range
* parts included or not
* completeness of quote
* attachment evidence

### 4. Assignment feasibility

Question: Which technician / bay / slot is feasible?

Track:

* technician workload
* bay occupancy
* required part status
* vehicle ETA
* deadline pressure

### 5. Follow-up / escalation

Question: Should the system follow up, wait, or escalate?

Track:

* elapsed time since last action
* priority
* pending dependency
* human SLA expectations

These are the best early training targets.

***

## Part 5 — Suggested Data Captures By Workflow Stage

### Intake stage

Capture:

* structured intake fields
* raw message text
* attachments
* identified missing fields
* model-inferred category
* model-inferred urgency

### Supplier response stage

Capture:

* quote amount
* quote completeness
* response latency
* questions asked back to fleet
* model-predicted acceptance probability

### Approval stage

Capture:

* approval authority present or missing
* approval turnaround time
* approval outcome
* reasons for rejection

### Scheduling stage

Capture:

* technician candidates
* bay candidates
* part readiness
* conflicts detected
* human override of system suggestion

### Completion stage

Capture:

* final cost
* variance vs quote
* evidence present or missing
* completion time
* deadline met or missed

***

## Part 6 — Scoring Framework

### Purpose

Turn raw coordination outcomes into learning signals.

### Suggested scores

#### 1. Coordination success score

Composite of:

* completed
* within deadline
* within budget tolerance
* no unnecessary reassignment
* no unresolved escalation

#### 2. Response quality score

Composite of:

* fast first response
* complete quote
* minimal clarification loops

#### 3. Scheduling quality score

Composite of:

* no resource conflict
* no avoidable delays
* technician / bay utilization fit

#### 4. Recommendation agreement score

How often human chose the same action as the model.

#### 5. Recommendation usefulness score

Whether model recommendation improved downstream outcomes even when humans did not initially agree.

***

## Part 7 — Privacy and Safety Guardrails

### Requirements

* do not train directly on uncontrolled raw personal data without filtering
* separate operational logs from model-training extracts
* support redaction for phone/email/message bodies where needed
* keep a clear distinction between audit storage and model training storage
* version training datasets

### Human oversight

The shadow layer should support:

* reviewable recommendations
* traceable reasoning artifacts where possible
* rollback of automation scope
* per-action autonomy controls

***

## Part 8 — Suggested Initial Tables

Create a first version with these tables:

* `coordination_sessions`
* `workflow_events`
* `decision_points`
* `candidate_actions`
* `selected_actions`
* `outcome_records`
* `state_snapshots`
* `training_examples`

Keep v1 narrow and useful. Do not overbuild a massive ML platform on day one.

***

## Part 9 — Recommended Build Sequence

### Phase 1 — Instrumentation

Build:

* workflow event logging
* decision point creation
* state snapshot capture
* selected action capture
* outcome capture

No autonomy yet.

### Phase 2 — Silent shadow model

Build:

* internal recommendation engine
* compare model suggestion vs human choice
* offline evaluation dashboards

### Phase 3 — Human-visible suggestions

Build:

* suggest next best action
* suggest supplier routing
* suggest part requirements
* suggest follow-up actions

### Phase 4 — Guardrailed low-risk autonomy

Build only after evidence:

* automated reminders
* missing-info requests
* quote normalization
* low-risk coordination nudges

***

## Part 10 — Definition of Success

The shadow training layer is successful when:

* FleetOS captures decision-quality data, not just raw logs
* humans and model recommendations can be compared cleanly
* outcomes are linked back to decisions
* the system can evaluate which coordination actions actually work
* FleetOS becomes a training ground for autonomous coordination agents
* autonomy can be introduced safely in narrow areas over time


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://t420.gitbook.io/t420/shadow-training-layer.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.