The Automation Illusion: Thousands of tests still don’t prove the system is safe
Speaker
Target Audience
Basic (no prior knowledge)
Description
Every year, my organisation delivers more than 13,000 releases in a highly regulated environment where quality matters, audits are constant, and failure can be expensive.
Delivery teams have built extensive automation, optimized for their own services and releases, while a separate team focuses on system-wide testing.
We are now expanding the role of delivery team automation to embed system-level confidence directly within the teams building the software. This brings system testing into the development process, keeping it more up to date and reliable.
As we expand the role of team-owned automation, locally optimized tests need to work together to represent the health of the system, not just individual parts.
Achieving this requires alignment, shared practices, and clear ownership of system outcomes.
This talk explores the risks we identified in this transition, and how we are addressing them.
We will look at how to use team-owned automation to build system-level confidence, and what it takes to bring testing into the development process without losing trust in the system as a whole.
You will learn
Why more automated tests do not automatically create system-wide confidence
How team-owned automation only scales when teams align around shared quality signals and system outcomes
That automated tests only create confidence when the signals they produce are trusted