13. Fleet as Instrumented Experiment
Thursday morning: five units on the bench, and no two run logs structured the same way.
By Friday the team had a story about each unit and no way to compare them — which build carried which change, which result came from which configuration, none of it lined up.
That is a missed experiment, not a successful fleet. The distinction matters because the fleet's job is not to confirm a prototype — it is to surface variation across units so the risk register moves on evidence, not on which unit the program lead happened to visit last. Getting there requires structured per-unit records, a build matrix that controls variables, and a weekly cadence that forces comparison. Build those three and the fleet becomes an experiment; skip them and it stays a pile of units with good stories attached.
Fleet mindset: learning asset, not milestone artifact
A fleet build is valuable when each unit produces usable evidence.
Treating prototypes as proof-by-existence wastes the most expensive learning window in the program.
Fleet intent should be explicit before build:
- Which decisions this fleet is meant to inform,
- Which requirements/risks each unit is probing,
- Which measurements and conditions must be captured.
A unit's evidence is gate-complete when the record answers all three from the record alone — without asking the engineer who ran the test.
Minimum per-unit record
Six fields per unit. If the record cannot survive the question "what was different about this unit," it is not a unit record — it is a bench note.
For each serial unit, capture:
- configuration (revision/state),
- manufacturing context (process/material lots),
- test conditions,
- results against requirement IDs,
- anomalies and disposition,
- owner/date for record completeness.
If there is no per-unit record, you cannot defend the cross-unit comparison at gate — and the anomaly that matters will be in the unit whose log is thinnest.
Build matrix, not pile of units
Plan fleet variation deliberately:
- what to hold constant,
- what to vary,
- which interactions are intentionally sampled,
- which units are designated for destructive/edge tests.
If all units are "nominal," fleet learning is shallow and late surprises remain likely. A common miss: a five-unit fleet built to identical nominal conditions produces five confirmations of one operating point rather than a sample of the variation the program will face at scale.
Connect fleet data to program controls
Fleet evidence earns its keep only when it moves the same program-control records every result should — requirement confidence, the risk register, the one-page status, and the next decision in the queue. When the chain runs, that propagation is visible: Unit 03 measured a leak rate well above Unit 01 at hot soak, the seal leak requirement confidence moved to red, and the next gate call shifted from go to hold pending seal redesign evidence.
If fleet data stays in test notes, the program keeps running on stale assumptions.
At a battery-adjacent program, five prototype units were built and instrumented over four weeks. Unit 03 measured a leak rate more than twice Unit 01's baseline at hot soak — not a sensor artifact, the same delta confirmed across multiple test runs. The discrepancy sat in the test engineer's notes for eleven days before it reached the program's risk register, because there was no per-unit record requirement that forced the delta into the risk log. The test engineer had mentioned it verbally on day three. It was in nobody's written record. The thermal requirement moved to red only after the team had already placed tooling orders on the assumption that Unit 01 was representative.
The program added a per-unit record requirement and a weekly fleet review cadence to its standing controls. The next fleet campaign surfaced three requirement deltas in the first review week rather than the fifth. That gap had cost tooling decisions made on unrepresentative data — decisions that required a correction order once the real distribution of results was visible.
Failure patterns in fleet execution
Common misses and what catches each:
- unit history lost after rework — per-unit record requires a rework entry before the unit re-enters evidence
- instrumentation setup drift between units — per-unit test-conditions field, visible at the evidence completeness check
- anomalies documented but not tied to decision owners — step 3 of the weekly review (unresolved anomalies and owners)
- test logs decoupled from build configuration — step 1 of the weekly review (unit-by-unit evidence completeness check)
Without the pairing, these failures make apparently rich data unusable for high-stakes decisions.
Practical weekly fleet loop
During fleet campaigns, run a weekly review:
- Unit-by-unit evidence completeness check,
- requirement/risk deltas from new data,
- unresolved anomalies and owners,
- decision queue updates for next gate.
Short, strict, repeatable. Each cycle ends with a documented delta note: which requirement or risk rows moved and what evidence moved them.
Delta discipline scales to fleet
Delta and traceability discipline at component and subsystem level scales directly into fleet work — the same logic, applied across physical units in real variation.
A fleet built this way surfaces process variation before tooling orders commit and supplier qualifications lock it in.
Field test: If your weekly fleet review this week can name one risk-register row that moved because of a specific unit's measured delta, the campaign is doing its job. If it cannot, you are collecting prototypes, not evidence.
Fleet deltas change the program's risk register only if gate-closure criteria require evidence from the failure-mode log — programs that close gates on action-item status rather than test evidence are done on paper long before they are done in the hardware.