What We Learned From Agent Down
iluminr’s first Frontier AI Risk research microsimulation asked what happens when an AI capability financial services organizations depend on becomes unavailable. The main contingency was people.

This week also marked the release of iluminr’s first Frontier AI Risk microsimulation report, Agent Down. The exercise explored what happens when an AI capability that financial services organizations have come to depend on becomes unavailable.
One finding stood out to me. When participants considered how to maintain critical services during an AI disruption, the overwhelming contingency was people.
Alternative AI providers were rarely a practical solution within the timeframes required by critical service-level agreements. Organizations would need employees to step back into processes increasingly supported by automation.
The cumulative effect of strategic decisions
That raises an interesting question about the cumulative effect of strategic decisions. An AI investment might reduce operating costs. A workforce restructuring might improve margins. A decision to concentrate technology with one provider might simplify operations. Each could make sense independently.
Together, they may create an operating model that depends on capabilities the organization can no longer reproduce when circumstances change.
And disruption is only one dimension. A model provider might change its technology, pricing, or contractual terms. A capability that once differentiated a business might become widely available. The economics of a product or service could change without a security incident or technology failure.
These developments are connected to strategy, technology, workforce planning, procurement, and operational risk. Yet the decisions governing them are often made in different parts of the organization.
What it means for boards, adopters, and providers
That was also a theme in Vela’s AI governance workshop at Dorsey & Whitney in Dallas. Boards need to understand how management decisions interact, which assumptions underpin the business model, and what evidence exists that those assumptions will hold under different conditions.
For organizations adopting AI, it means testing the dependencies and human capabilities on which their operating models rely. For providers, it raises questions about how customers maintain continuity, manage concentration risk, and adapt when technology changes.
The longer-term challenge is developing organizations that can continue creating value as the technologies and assumptions behind their operating models evolve.
Disclosure: Eudai founder Paula Fontana serves as iluminr’s CMO and a board director.