SignalsOperating intelligence
Open navigation

Operating question

Useful AI buying starts by knowing how work continues if a supplier changes, which expert clues shape a decision and which sources a current answer may use.

AI Operating Models

3 Things AI: The “What Are We Actually Buying?” Edition

3 Things AI 5 min3 sources

For

Leaders and workflow owners

You will leave with

3 operating decisions

Reading mode

5 min · 3 verified sources

Reading guide3 decisions · 4 sections+

Decision points

  1. 01Run a one-hour exit exercise on a low-risk AI workflow before a supplier change makes the test urgent.
  2. 02Start with one recurring alert an experienced employee can explain, and keep the consequential action with that person.
  3. 03Test a current-answer tool on known questions, restricted domains and explicit unknown responses before wider use.

Three practical ways to test a supplier exit, teach an alert assistant with experienced staff and evaluate current web answers before a wider rollout.

The harder part of buying AI is often not choosing the feature. It is knowing how work continues if a supplier changes, which expert knowledge the system needs and where a current answer came from. Three developments this week turn those questions into practical tests.

1. Test one way out before a supplier changes

A Canadian distributor uses one AI service to summarize contracts and draft customer replies. The work is useful, but the team does not know whether it can export its instructions, retrieve recent records or continue with another provider during an outage. The dependency becomes visible only when change is urgent.

IBM’s November 11 Digital Sovereignty Index reports that only 33% of surveyed enterprises regularly test their ability to demonstrate control when conditions change. The study covered 2,003 senior decision-makers in seven countries and found that AI-related issues were the most frequently reported sovereignty challenge. It did not include Canada or small businesses, and IBM sells technology and consulting related to the topic.

The benefit of an exit test is practical confidence: the team learns what can be exported, what must be rebuilt and which manual process can keep customers moving. The tradeoff is time spent testing a failure that may never happen, plus the risk of oversimplifying differences between providers.

Choose one low-risk AI workflow and run a one-hour exit exercise. Export its instructions and recent non-sensitive examples, revoke a test user’s access and try the manual fallback for one case. Record the missing files, permissions, owner and recovery time. Do not attempt this first on payroll, live customer communication or another consequential process. One exercise reveals obvious dependencies; it does not prove full resilience or legal compliance.

2. Start with the alert your expert can explain

A service company receives a warning that several customer appointments may be at risk. The dispatcher checks the schedule, weather, technician notes and earlier delays before deciding whom to call. A generic summary may save reading time, but the useful part is knowing which clue changes the decision.

Microsoft described how Korean telecom provider KT built AI agents that help experienced network operators investigate warnings before customers report a failure on November 11. The teams worked together on collecting, applying and rolling back expert feedback, and trained the system on operating procedures plus the practical knowledge of long-serving specialists. This is a Microsoft customer story about a large telecom network, not an independent evaluation or a result a small business can expect.

The benefit is a sensible starting point for AI: bring scattered clues together for a person who already understands the decision. The tradeoff is that expert feedback becomes part of the product. It takes time to capture, can reflect one person’s habits and needs a way to be corrected.

Choose one recurring warning that a senior employee handles well. Take twenty completed examples, remove personal details and record the inputs checked, the action chosen and what happened next. Test whether an assistant can rank the useful clues without taking the action. Keep the customer call or schedule change with the dispatcher. Twenty cases can expose gaps, not prove the assistant will handle every future exception.

3. Give a current-answer tool a short source list

A software team wants an internal assistant to answer, “Has this supplier changed its API or pricing this month?” A general model may be fluent but stale. A broad search may be current but return a reseller, an old forum answer or a page that only partly supports the claim.

AWS announced a direct Web Search API for Amazon Bedrock AgentCore on November 11. Existing applications can request ranked results with domain and publication-date filters, authorized through AWS identity permissions, without first creating an AgentCore Gateway. Results include an extract and, when available, a title, URL and publication time. AWS lists a price of $7 per 1,000 queries and notes that request logging is not enabled by default for the relevant data events.

The benefit is a smaller integration for a narrow current-information job. The tradeoff is that search results are evidence candidates, not verified answers. Missing dates, incomplete extracts and an allowed but weak page still need application checks. Cost and logging also require deliberate settings.

Test twenty questions your team already knows how to answer. Limit search to the supplier’s documentation and status domains, require a URL beside every statement and mark unsupported answers as unknown. Record accuracy, response time and queries per answer before connecting the tool to a customer or consequential workflow. The test measures only those questions and sources; it cannot establish reliability for the open web.

The bigger pattern

These developments make an AI purchase more concrete. A supplier is easier to depend on when you have tested one way out. An alert assistant is useful when it shows the clues an experienced person needs. Web search helps when the allowed sources and failure response are clear.

Which AI dependency in your business would be most revealing to test for one hour?

Verified sources

Continue your decision path

Move from understanding to action.

02 · Go deeper

3 Things AI: The “Where Did the Time and Money Go?” Edition

Three practical ways to connect AI cost to an outcome, test a difficult handoff before scaling it and keep creative judgment visible in faster work.

Read next
03 · Assess

Apply this signal to your architecture.

Identify the workflow, context, and controls to structure first.

Open Architecture Assessment