Verify: A Worked Decision #
Worked recommendation: Observe the complete journey in one service before expanding recruitment. Keep a usable route available while identifying where applicants fail to reach the task’s end.
Complete the student exercise first. This is an instructor’s proposed response to evidence available after 5 March 2019. It does not report a government trial or use later events to prove the recommendation correct.
Repair the briefing #
Supported replacement: “The reported 48% concerns Verify sign-up in a single attempt under the source’s counting rules. We need separate evidence of completion across the intended service journey.” The NAO’s report supplies the limits behind that correction. BS-0081
The original briefing makes an additional unsupported move: it assumes improving an intermediate event will improve the final task. Even if both measures later rose, a before-and-after pattern would still need a suitable comparison before attributing the change to a particular intervention.
Historical outcome boundary: this case begins after an audit of disappointing program performance. The audit is the starting evidence. The investigation and routes below are teaching proposals, so there is no historical result to reveal for them.
A completed decision record #
| Field | Instructor response |
|---|---|
| Useful outcome | An eligible applicant submits the selected online application and receives confirmation of acceptance for processing, with required identity assurance maintained. |
| Target action | During an application episode, the applicant supplies the required information, completes the permitted identity route and submits the request. |
| Population | People beginning their first application to the selected service; record whether they already hold a usable identity account. |
| Immediate commitment | Investigate the full journey and record the support it requires before increasing recruitment into that route. |
| Strongest alternative | Start a bounded assisted-route pilot alongside observation if current access problems justify earlier support investment. |
| Key uncertainty | Which stage prevents useful completion, for which applicants, under which conditions? |
| Limit on this decision | No claim that a route has achieved BMF, that extra support will work, or that the program has one dominant failure cause. |
The selected behavior contains several necessary actions. The purpose of the chain is to expose them, not to assign all responsibility to the applicant.
Define the chain and allocate the work #
The following is an original model for the exercise’s application service. It is not a reproduction of a government workflow diagram.
| Stage | Work required | Observation and possible failure |
|---|---|---|
| Enter the service | Applicant identifies the relevant service and starts the application journey. | Record entry before identity-provider selection. Wrong-service visits and unresolved eligibility need explicit handling. |
| Select a permitted identity route | Applicant and service establish whether a usable account or another route is available. | Record the route offered, chosen or declined. Avoid assuming every entrant needs a new account. |
| Supply necessary evidence | Applicant gathers and provides the required information; support may assist with permitted steps. | Distinguish missing information, lack of access, inability to complete the step and deliberate refusal. |
| Complete identity checking | Provider performs the required checks. | Record successful and unsuccessful attempts with reasons where known. |
| Reach the application | Provider and service pass the result; the service matches it to the relevant records. | Observe a successful handoff rather than assuming that verification guarantees access. |
| Submit and confirm | Applicant finishes the application; the service confirms acceptance for processing. | Count this as the exercise’s completed task. A later service decision is a separate outcome. |
| Use support or another route | Applicant and staff resolve a problem or continue through an available alternative. | Record completion, time and staff work across routes; do not classify every switch as failure. |
This model tests the whole behavior chain. It also makes an option requiring less applicant work visible: a permitted existing step may be reused, or work may move to the provider. Whether that option is technically and operationally available must be established.
Specify the observations before collecting them #
The following plan is an instructional example, not a study conducted by Verify. Its timing is a design assumption to adjust to the real service.
Start with a process study. During a two-week recruitment period, invite consenting eligible entrants across the service’s normal arrival periods. Include relevant differences in device access, available documentation, prior accounts and need for assistance. Follow each included application for seven days after entry. Record who was invited, who declined and why inclusion was limited where that information is available. This selected observation group cannot establish a population-wide rate.
Link the episode. Use a study reference to connect the same application across its stages and repeat attempts. Do not collect copies of identity documents merely to count progress. Keep application episodes distinct from people and individual attempts.
Record the operating conditions. Keep the service’s required identity checks and existing assistance available. Record extra research help separately. Staff effort is part of evaluating an assisted service; supported completion does not establish unaided completion.
| Measure | Definition for this proposed study |
|---|---|
| Confirmed task completion | Included application episodes with an acceptance confirmation within seven days, divided by all included eligible application episodes. |
| Unresolved outcome share | Included episodes whose final state cannot be established, divided by the same denominator. Report alongside completion. |
| Stage progress | Episodes reaching each named checkpoint, using both the full cohort and the previous checkpoint as explicitly different denominators. |
| Repeated attempts | Additional attempts within an existing application episode; report separately from new people or applications. |
| Supported completion | Confirmed completions with recorded assistance, with the type and amount of assistance visible. |
| Alternate-route completion | Included episodes completed through another permitted route within the window. |
| Coverage | Counts invited and included, the contexts represented, and known differences between participants and the intended service population. |
No result values are supplied. An unknown final state is not a confirmed failure. The observed completion fraction is incomplete when outcomes remain unresolved; present that limitation alongside the number. Use the measurement lab to practice the counting rules with fictional records.
Decide what the observations would permit #
These rules guide the next investigation. They do not make a small observation study a causal comparison.
| Possible finding | Defensible next decision |
|---|---|
| Events cannot be connected across the journey. | Repair observation first. Do not select a winning route from incomparable denominators. |
| A recurring obstacle appears before identity checking. | Test a targeted preparation or support change. Treat the proposed cause as a hypothesis and observe the later task as well. |
| Identity checking succeeds but applicants cannot reach the application. | Prioritize investigation of the service handoff and record matching before increasing recruitment. |
| An existing route completes the task with acceptable assurance and effort. | Retain it as a serious alternative. More new registrations have no independent value under this objective. |
| An assisted route appears usable. | Test whether its staffing, availability and cost can be sustained; do not infer unsupported self-service performance. |
| Too few relevant episodes are observed, or outcomes remain uncertain. | Record an inconclusive result and the specific evidence still needed. |
Before expanding a proposed change, the service owner must set an acceptable completion requirement, observation window, assurance condition and support-cost limit. A comparative trial would also need an appropriate allocation method, adequate sample and uncertainty analysis. The audit does not supply those local decision requirements. A completed recommendation can therefore authorize the next investigation while explicitly withholding a broader rollout decision.
A defensible alternative #
Another response could prioritize an assisted-route pilot immediately, while collecting the same end-to-end observations. This may be reasonable if the service owner gives greater weight to providing current applicants with a workable route than to delaying support costs. That priority is an explicit decision assumption; the packet supplies no local cost-benefit calculation.
The alternative must keep the required assurance, record staff work and preserve observation of the final task. It cannot claim that additional help is automatically effective. A comparison with current practice would still be necessary to estimate the change in completion attributable to the pilot.
Assess the reasoning #
A sound response distinguishes the applicant’s goal from the registration metric, names all necessary actors, retains unknowns and defines the next decision. It can favor investigation or a bounded pilot if it explains the different commitments.
Revise a response that treats the projection as a control group, labels every departure a motivation problem, excludes early failures to improve a reported percentage, or counts an account as a completed public service. These errors concern what the evidence permits, independent of which route you prefer.
Sources and chronology #
The NAO investigation and its findings release were published on 5 March 2019. The detailed measurement boundary is in the full report, printed page 16, paragraphs 1.16 and 1.17. They describe the same audit, not independent replications. Sources checked 5 September 2026.
The action chain, study plan, decision rules and alternative response above are instructor analyses. The audit does not show government use of DRIVE, BMF or BFA, and this reconstruction does not validate those methods.
Original teaching material by Jason Hreha, version 1.0, licensed CC BY 4.0. External sources retain their own terms. See teaching notes and reuse guidance.