DUBSAR for Head of AI / AI Lead
Role · Head of AI / AI Lead
Heads of AI: what needs reassessing when a mission changes?
A change of model, instructions or tool can produce text that looks just as convincing as the previous version. It can also change the data sent, its recipient or the actions an agent can request.
Before proceeding with the change, the integrator and the client’s Head of AI need to establish what remains valid, what needs retesting and what requires fresh authorisation. A change record prepares that decision without rebuilding the entire mission record from scratch.
In brief
The question and the useful result.
- Question
- Which decisions remain valid after a change to a mission’s model, instructions, tools, data or permissions?
- Useful result
- A change record separating retained findings, targeted tests and authorisations needed before proceeding within a reassessed scope.
DUBSAR for Head of AI / AI Lead
Describe the difference between two identifiable versions
Identify the mission, its reference version and the proposed change. Keep the references actually available: model or service, instructions, workflow configuration, accessible tools, data scope and granted permissions. Mark any missing reference explicitly.
The purpose is to compare operating conditions as well as model responses. Broader instructions may change the data requested; a different tool may expose another operation; a new service may receive information previously sent to another recipient.
For each difference, identify the earlier decisions affected and the assumptions behind them. A decision may still apply if its scope and conditions remain established. Keeping the same label in a configuration is not enough to demonstrate that.
DUBSAR for Head of AI / AI Lead
Choose checks according to the effect of the change
The review should produce three useful categories:
- Keep: unchanged elements and the reasons why earlier checks remain relevant.
- Retest: affected behaviour, test conditions, the expected result and the observation needed to reach a conclusion.
- Seek fresh authorisation: data, recipients or actions outside the scope already granted, with the person empowered to decide.
Match the checks to the change. Assess response quality when instructions change; examine the information actually sent when a connection changes; check accessible operations and the meaning of responses when a tool changes. Each finding remains tied to the conditions of its test.
An acknowledgement from the new tool may establish that it received a request without confirming the intended business effect. The review needs to distinguish the request, authorisation, attempt, acknowledgement and confirmed effect. A change that makes the response less informative may limit the acceptable scope, even if the generated text looks satisfactory.
DUBSAR for Head of AI / AI Lead
Example: the same summary, a more powerful tool
In this illustrative scenario, an agent prepares a summary from selected fields in a customer record. The integrator is considering replacing the lookup tool. The new tool returns similar information but can receive the full record and also supports an update operation. No available DUBSAR CRM integration is assumed.
| Finding to examine | What it does not establish by itself | Follow-up to record |
|---|---|---|
| The summary appears comparable to the previous version’s output. | That the same data was sent to the same recipient. | Compare actual inputs and identify the receiving service. |
| An update operation is accessible through the new tool. | That the mission has permission to use it. | Clarify the permitted scope and check how the connection implements it. |
| A test returns an acknowledgement. | That the expected effect completed, or that it was authorised. | Establish what the response means and how the outcome needs to be checked. |
The decision might retain lookup within a verified scope, request a connection change or defer the proposed replacement. Assessing text quality contributes to that decision; it does not, on its own, cover data use and authority to act.
DUBSAR for Head of AI / AI Lead
Change record to complete
Use this record in your current documents and tools, without DUBSAR. Reuse references to existing material and identify anything missing.
Mission and review owner: …
Reference and proposed versions, with their references: …
Modification and reason for the change: …
| Field | Information to record |
|---|---|
| Possible effects on data | Affected sources, categories, volume or scope, recipients and copies; established facts and unknowns. |
| Possible effects on actions | Accessible operations, permissions, authorisation conditions and the meaning of responses. |
| Decisions and checks retained | Reference for each item and the verified reason why it remains valid after the change. |
| Targeted tests | Scenario, preconditions, expected result, observed result and evidence reference; state “not performed” where needed. |
| Authorisation required | Exact subject, empowered approver, information presented and the decision still awaited or actually made. |
| Reassessed scope | Operations and conditions actually examined; exclusions and remaining limitations. |
| Missions already in progress | Applicable version, affected actions and the decision needed before applying the change. |
| Conclusion and follow-up | Proceed within the agreed scope, revise or defer; decision-maker, date, outstanding checks and their owners. |
State when the decision takes effect and which missions it covers. Keep material from earlier versions accessible: a change must not reassign the old version’s tests or actions to the new one.
DUBSAR for Head of AI / AI Lead
Frequently asked questions
Do we need to retest everything whenever the model changes?
Choose tests according to the effects of the change. Identify checks that remain relevant, explain why they can be reused and retest the affected behaviour. An unclear impact should remain an open question, with someone responsible for resolving it.
Does good response quality allow us to keep every authorisation?
Text quality does not establish where data goes or which actions are accessible. Check whether the conditions of each authorisation still hold and seek a fresh decision when the change exceeds the scope already granted.
Can the change apply to missions already in progress?
Identify these separately from new missions in the record. Specify which version applies to their next action and which decisions remain valid. An authorisation granted for the previous configuration does not automatically extend to the new one.
DUBSAR for Head of AI / AI Lead
DUBSAR’s intended contribution
DUBSAR is developing a foundation to connect durable mission state, authorisations, human decisions and evidence. The aim is to help teams retrieve the framework that applies to the work and identify what needs reviewing when it changes. The proposed record assumes no DUBSAR engine for evaluation, versioning or automatic reassessment.
Selected components and workflows have been tested. Full B2B integration and validation are still underway. Pilots are not currently open.
DUBSAR for Head of AI / AI Lead