Web based A/B testing goes a long way towards eliminating dumb mistakes, making the best choice, creating a discipline around innovative activity, and encouraging change, and has been made far easier in a whole range of areas by the data collection capabilities of the net.
But what happens when you cannot test, when you are doing something so completely new that the frame of reference necessary for good test results does not exist?
Try testing the Model T in 1890, only a few would have seen the possibilities because the horse was the frame of reference, the early cubic paintings of Picasso, art that so broke the rules as to be outrageous, or the calculations of Copernicus demonstrating the earth was not the centre of the universe, something catholic church felt pretty strongly about.
At some point testing becomes a redundant tool, you simply cannot test everything, and you have to rely on the guts, instinct, and insight of the few outliers who see things differently to make meaningful change
