Jun 11, 2024 | Analytics, Governance
Goodhart’s law is a much quoted adage that states: When a measure becomes a target, it ceases to be a good measure’.
When we see numbers cited as evidence, we tend to instinctively give them more credibility than they may deserve. Without an examination of the source and scope of the numbers, just believing them on face value can lead to very bad choices.
Charles Goodhart is a prominent British monetary economist. His public profile started as a footnote to a 1975 article and read: ‘any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes’. It took on its now well-known form, above, when restated by an anthropologist seeking to link the idea to a much wider context than economics.
In almost every business I visit, I see examples of Goodhart’s law. The most common is the use of EBIT as a measure of success. The reality however, is that it is simply a financial measure of the outcome on the myriads of smaller actions and decisions that have been taken in the deployment of resources.
Two of the most public failures of measurement should make us all wary of using quantitative targets as a measure of performance.
The 2010 Volkswagen software scam. VW installed software in a range of cars with their turbocharged direct injection diesel engines that ensured that when being tested, the cars made the emissions standard of the US EPA.
The Vietnam War body count. The US defence department sought to justify the billions of dollars and thousands of lives expended in the defence of a corrupt government in South Vietnam. The measure became the body count of VC fighters and North Vietnamese regulars, which led to wholesale slaughter. These imaginary and bloody numbers were used for years as the basis for increasing commitment. It finally became obvious that they did not in any way reflect the ability or willingness of much of the population to fight what they saw as aggression by the US.
There are thousands of examples. If you looked, you would see them every day, especially when politicians open their mouths and quote numbers.
Header Photo: Charles Goodhart. Professor Emeritus at the London School of economics.
PS. When you look at the header, Dr. Goodhart is looking directly into your eyes. You will tend to believe anything he tells you.
Jun 5, 2024 | Governance, Leadership, Management
The word ‘argument’ has many meanings, depending on the context. It can mean a friendly difference of opinion, a negotiation point, a statement of reasoning a lawyer might use, to an expression in a mathematical formula.
A quarrel is far more specific, requiring a disagreement, the cause of which is often lost in the chaos of emotion a quarrel elicits. The only other meanings of the word I can think of is as a collective noun for a group of energetic and opinionated mammals noisily exchanging insults, such as monkeys, squirrels, cooks, and lawyers. It also refers to the tip of a crossbow bolt.
There is a standard three step formula for making an argument stick in the minds of the receiver. It is evident in every news cast you ever heard, the ‘newsreaders secret formula.’
- Tell them what you’re going to tell them. This is always called ‘the headline’.
- Tell them. The story, or series of stories.
- Tell them what you told them. Restate the headline, and any conclusion or resulting actions that emerged.
To win an argument, as you would a negotiation, debate, or in court, you need to modify the news readers trick by adding a step.
That step is analysis of a guiding fact, or set of facts.
This enables you to analyse those facts in a way that leads you to the conclusion you are arguing for.
For the sake of ease of use you can break this into a pneumonic ‘CRAC’
- Conclusion. State your conclusion.
- Rule. Identify the fact or facts upon which your conclusion is based.
- Analysis. Provide an analysis of how that rule makes any conclusion other the one you’ve reached invalid.
- Conclusion. Restate the conclusion.
This CRAC process was used very effectively recently by an acquaintance chairing a community group that was protesting a pending building approval decision of their local council.
She stated that the approval, if it was to proceed, was in defiance of the councils own regulations.
She then cited the specific regulations.
She then pointed out the specific parts of the pending approval that was in breach of the regulations, and why they breached them.
For good measure she also pointed out 2 other proposals similar to the one that appeared to be about to be approved, that had been rejected on the basis of the specific parts of the regulations stated previously.
She then repeated the conclusion that the project was in defiance of the council’s own regulations, and therefore should not proceed.
It was an impressive performance, well planned, well executed, and ultimately successful after some embarrassing back downs by several councillors.
With a bit of practise, it is easy to use, and always better than resorting to a quarrel.
Header cartoon credit: Scott Adams and his mate Dilbert.
Jun 3, 2024 | AI, Change, Leadership
My time is spent assisting SME’s to improve their performance. This covers their strategic, marketing, and operational performance. Deliberately, I initially try and downplay focus on financial performance as the primary measures, as they are outcomes of a host of other choices made throughout every business.
It is those choices around focus, and resource allocation that need to be examined.
Unfortunately, the financial outcomes are the easiest to measure, so dominate in every business I have ever seen.
When a business is profitable, even if that profit is less that the cost of capital, management is usually locked into current ways of thinking. Even when a business is marginal or even unprofitable, it is hard to drive change in the absence of a real catalyst, such as a creditor threatening to call in the receivers, or a keystone customer going elsewhere.
People are subject to their own experience and biases, and those they see and read about in others.
Convention in a wider context, status quo in their own environment.
Availability bias drives them to put undue weight in the familiar, while dismissing other and especially contrary information.
Confirmation bias makes us unconsciously seek information that confirms what we already believe, while obscuring the contrary.
Between them, these two forces of human psychology cements in the status quo, irrespective of how poor that may be.
Distinguishing between convention and principle is tough, as you need to dismiss these natural biases that exist in all of us. We must reduce everything back to first principles, incredibly hard, as we are not ‘wired’ that way.
The late Daniel Kahneman articulated these problems in his book ‘Thinking fast and Slow’ based on the data he gathered with colleague Amos Tversky in the seventies. This data interrogated the way we make decisions by experimentation, which enables others to quantitively test the conclusions, rather than relying on opinion.
That work opened a whole new field of research we now call ‘Behavioural Economics’ and won Kahneman the Nobel prize. Sadly however, while many have read and understand at a macro level these biases we all feel, it remains challenging to make that key distinction between convention, the way we do it, the way it has always been done, and the underlying principles that should drive the choices we make.
As Richard Feynman put it: “The first principle is that you must not fool yourself—and you are the easiest person to fool. So, you have to be very careful about that.”
May 29, 2024 | AI, Governance
Most BBQ conversations about the future of AI end up as a discussion about jobs being replaced, new jobs created the balance between the two, and the pain of those being replaced by machine.
It is difficult to forecast what those new jobs will be, we have not seen them before, the circumstances by which they will be created are still evolving.
18 months ago, a new job emerged that now appears to be everywhere.
‘Prompt engineer’.
Yesterday it seems, there was no such thing as a ‘prompt engineer’. Nobody envisaged such a job, nobody considered the capabilities or training necessary to become an effective prompt engineer. Now, if you put the term into a search engine there are millions of responses, thousands of websites, guides, and courses have popped up from nowhere. They promise riches for those who are skilled ‘prompt engineers’ and training for those who hop onto the gravy train.
What is the skill set required to be a prompt engineer?
There are no traditional education courses available, do you need to be an engineer, a copywriter, marketer, mathematician?
This uncertainty makes recruiting extremely difficult. The usual guardrails of qualifications and past experience necessary to fill a role are useless.
How do you know if the 20-year-old with no life experience and limited formal education might be an effective and productive prompt engineer?
How many job descriptions will emerge over the next couple of years that are currently not even under any sort of consideration?
Recruiting rules no longer play a role. We need to hire for curiosity, intellectual agility, and some form of conceptual capability that I have no word for.
The challenging task faced by businesses is how they adjust the mix of capabilities to accommodate this new reality.
Do they proactively seek to build the skills of existing employees which requires investment? Do they clean house and start again, losing corporate memory and costing a fortune? Do they try and find some middle path?
Where and how do you find the personnel capable of building for a future that is undefined?
May 27, 2024 | AI, Governance
Nvidia 2 years ago was a stock nobody had heard of. Now, it has a market valuation of $US2.7 trillion. Google, Amazon, and Microsoft from the beginning of this year have invested $30 billion in AI infrastructure, seen their market valuations accelerate, and there are hundreds of AI start-ups every week.
Everybody is barking up the same tree: AI, AI, AI…..
Warren Buffett, the most successful investor ever, is famous for saying he would not invest in anything he did not understand.
He conceded many opportunities have passed him by, but he gets many right. Berkshire is the single biggest investor in Apple, a $200 billion investment at current market value that cost a small fraction of that amount.
Does anyone really understand AI?
Are we able to forecast its impact on communities and society?
We failed miserably with Social media, why should AI be any different?
Even the experts cannot agree on some simple parameters. Should there be regulatory controls? Should the infrastructure be considered a ‘public utility’? when, and even if, will sentience be achieved?
Bubbles burst, and many investors get cleaned out, but when you look in detail, there are always elements of the bubble that remain, and prosper.
The 2000 dot com bubble burst, and many lost fortunes. However, there are a number of businesses that at the time looked wildly overvalued, that are now dominating the leaderboards: Apple, Amazon, and Google for example.
The tech was transformative, and at any transformative point, there are cracks that many do not see, so stumble. From the rubble, there always emerges some winners, often unexpected and unforecastable.
Is AI just another bubble, or is it as transformative as the printing press, steam, electricity, and the internet?
Header cartoon courtesy of an AI tool.
