What is the Smallest Informative Test?

Moving to Kansas and spending several million dollars seems like a big step but actually qualifies as a small test of change.

In order to be able to test changes to information and money exchange in the regulated and mostly conservative U.S. banking industry, Suchitra Padmanabhan and Suresh Ramamurthi bought a tiny community bank in Kansas a couple of years ago.

Owning a functioning bank enables the couple to understand constraints and impact of improvement that has the potential to scale to larger banks and bank systems—and the bank enables them to test software innovations in a real laboratory. As Mr. Ramamurthi cogently observed, his bank is “…like a bonsai tree. It’s still a full tree. It’s just smaller.”

The test and background is described in the New York Times, 13 December 2014.

The point of testing is to gain insight and to increase degree of belief in future system states given specified conditions, thus reducing uncertainty. 

As George Box reminded us, “to find out what happens to a system when you interfere with it you have to interfere with it (not just passively observe it).” ("The Use and Abuse of Regression," Technometrics, Vol 8, no. 4, 1966, p. 629).

To move from talking to action, I usually ask my clients to think about the smallest informative test they can carry out to learn what happens when they make one or more changes.

(In working with my clients, I use the Model for Improvement--three questions connected to the Plan-Do-Study-Act cycle, developed by Associates in Process Improvement.   The size of a test of a change idea gets set in the Plan step of a PDSA cycle).