DUBSAR for Head of AI / AI Lead

Role · Head of AI / AI Lead

Heads of AI: what needs reassessing when a mission changes?

A change of model, instructions or tool can produce text that looks just as convincing as the previous version. It can also change the data sent, its recipient or the actions an agent can request.

Before proceeding with the change, the integrator and the client’s Head of AI need to establish what remains valid, what needs retesting and what requires fresh authorisation. A change record prepares that decision without rebuilding the entire mission record from scratch.

In brief

The question and the useful result.

Question
Which decisions remain valid after a change to a mission’s model, instructions, tools, data or permissions?
Useful result
A change record separating retained findings, targeted tests and authorisations needed before proceeding within a reassessed scope.

DUBSAR for Head of AI / AI Lead

Describe the difference between two identifiable versions

Identify the mission, its reference version and the proposed change. Keep the references actually available: model or service, instructions, workflow configuration, accessible tools, data scope and granted permissions. Mark any missing reference explicitly.

The purpose is to compare operating conditions as well as model responses. Broader instructions may change the data requested; a different tool may expose another operation; a new service may receive information previously sent to another recipient.

For each difference, identify the earlier decisions affected and the assumptions behind them. A decision may still apply if its scope and conditions remain established. Keeping the same label in a configuration is not enough to demonstrate that.

DUBSAR for Head of AI / AI Lead

Choose checks according to the effect of the change

The review should produce three useful categories:

  • Keep: unchanged elements and the reasons why earlier checks remain relevant.
  • Retest: affected behaviour, test conditions, the expected result and the observation needed to reach a conclusion.
  • Seek fresh authorisation: data, recipients or actions outside the scope already granted, with the person empowered to decide.

Match the checks to the change. Assess response quality when instructions change; examine the information actually sent when a connection changes; check accessible operations and the meaning of responses when a tool changes. Each finding remains tied to the conditions of its test.

An acknowledgement from the new tool may establish that it received a request without confirming the intended business effect. The review needs to distinguish the request, authorisation, attempt, acknowledgement and confirmed effect. A change that makes the response less informative may limit the acceptable scope, even if the generated text looks satisfactory.

DUBSAR for Head of AI / AI Lead

Example: the same summary, a more powerful tool

In this illustrative scenario, an agent prepares a summary from selected fields in a customer record. The integrator is considering replacing the lookup tool. The new tool returns similar information but can receive the full record and also supports an update operation. No available DUBSAR CRM integration is assumed.

Finding to examineWhat it does not establish by itselfFollow-up to record
The summary appears comparable to the previous version’s output.That the same data was sent to the same recipient.Compare actual inputs and identify the receiving service.
An update operation is accessible through the new tool.That the mission has permission to use it.Clarify the permitted scope and check how the connection implements it.
A test returns an acknowledgement.That the expected effect completed, or that it was authorised.Establish what the response means and how the outcome needs to be checked.

The decision might retain lookup within a verified scope, request a connection change or defer the proposed replacement. Assessing text quality contributes to that decision; it does not, on its own, cover data use and authority to act.

DUBSAR for Head of AI / AI Lead

Change record to complete

Use this record in your current documents and tools, without DUBSAR. Reuse references to existing material and identify anything missing.

Mission and review owner: …

Reference and proposed versions, with their references: …

Modification and reason for the change: …

FieldInformation to record
Possible effects on dataAffected sources, categories, volume or scope, recipients and copies; established facts and unknowns.
Possible effects on actionsAccessible operations, permissions, authorisation conditions and the meaning of responses.
Decisions and checks retainedReference for each item and the verified reason why it remains valid after the change.
Targeted testsScenario, preconditions, expected result, observed result and evidence reference; state “not performed” where needed.
Authorisation requiredExact subject, empowered approver, information presented and the decision still awaited or actually made.
Reassessed scopeOperations and conditions actually examined; exclusions and remaining limitations.
Missions already in progressApplicable version, affected actions and the decision needed before applying the change.
Conclusion and follow-upProceed within the agreed scope, revise or defer; decision-maker, date, outstanding checks and their owners.

State when the decision takes effect and which missions it covers. Keep material from earlier versions accessible: a change must not reassign the old version’s tests or actions to the new one.

DUBSAR for Head of AI / AI Lead

Frequently asked questions

Do we need to retest everything whenever the model changes?

Choose tests according to the effects of the change. Identify checks that remain relevant, explain why they can be reused and retest the affected behaviour. An unclear impact should remain an open question, with someone responsible for resolving it.

Does good response quality allow us to keep every authorisation?

Text quality does not establish where data goes or which actions are accessible. Check whether the conditions of each authorisation still hold and seek a fresh decision when the change exceeds the scope already granted.

Can the change apply to missions already in progress?

Identify these separately from new missions in the record. Specify which version applies to their next action and which decisions remain valid. An authorisation granted for the previous configuration does not automatically extend to the new one.

DUBSAR for Head of AI / AI Lead

DUBSAR’s intended contribution

DUBSAR is developing a foundation to connect durable mission state, authorisations, human decisions and evidence. The aim is to help teams retrieve the framework that applies to the work and identify what needs reviewing when it changes. The proposed record assumes no DUBSAR engine for evaluation, versioning or automatic reassessment.

Selected components and workflows have been tested. Full B2B integration and validation are still underway. Pilots are not currently open.