The great trap of metrics

The great trap of metrics

 

Goodhart’s law is a much quoted adage that states: When a measure becomes a target, it ceases to be a good measure’.

When we see numbers cited as evidence, we tend to instinctively give them more credibility than they may deserve. Without an examination of the source and scope of the numbers, just believing them on face value can lead to very bad choices.

Charles Goodhart is a prominent British monetary economist. His public profile started as a footnote to a 1975 article and read: ‘any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes’. It took on its now well-known form, above, when restated by an anthropologist seeking to link the idea to a much wider context than economics.

In almost every business I visit, I see examples of Goodhart’s law. The most common is the use of EBIT as a measure of success. The reality however, is that it is simply a financial measure of the outcome on the myriads of smaller actions and decisions that have been taken in the deployment of resources.

Two of the most public failures of measurement should make us all wary of using quantitative targets as a measure of performance.

The 2010 Volkswagen software scam. VW installed software in a range of cars with their turbocharged direct injection diesel engines that ensured that when being tested, the cars made the emissions standard of the US EPA.

The Vietnam War body count. The US defence department sought to justify the billions of dollars and thousands of lives expended in the defence of a corrupt government in South Vietnam. The measure became the body count of VC fighters and North Vietnamese regulars, which led to wholesale slaughter. These imaginary and bloody numbers were used for years as the basis for increasing commitment. It finally became obvious that they did not in any way reflect the ability or willingness of much of the population to fight what they saw as aggression by the US.

There are thousands of examples. If you looked, you would see them every day, especially when politicians open their mouths and quote numbers.

Header Photo: Charles Goodhart. Professor Emeritus at the London School of economics.

PS. When you look at the header, Dr. Goodhart is looking directly into your eyes. You will tend to believe anything he tells you.

 

 

Is it an argument or a quarrel?

Is it an argument or a quarrel?

 

The word ‘argument’ has many meanings, depending on the context. It can mean a friendly difference of opinion, a negotiation point, a statement of reasoning a lawyer might use, to an expression in a mathematical formula.

A quarrel is far more specific, requiring a disagreement, the cause of which is often lost in the chaos of emotion a quarrel elicits. The only other meanings of the word I can think of is as a collective noun for a group of energetic and opinionated mammals noisily exchanging insults, such as monkeys, squirrels, cooks, and lawyers. It also refers to the tip of a crossbow bolt.

There is a standard three step formula for making an argument stick in the minds of the receiver. It is evident in every news cast you ever heard, the ‘newsreaders secret formula.’

  • Tell them what you’re going to tell them. This is always called ‘the headline’.
  • Tell them. The story, or series of stories.
  • Tell them what you told them. Restate the headline, and any conclusion or resulting actions that emerged.

To win an argument, as you would a negotiation, debate, or in court, you need to modify the news readers trick by adding a step.

That step is analysis of a guiding fact, or set of facts.

This enables you to analyse those facts in a way that leads you to the conclusion you are arguing for.

For the sake of ease of use you can break this into a pneumonic ‘CRAC’

  • Conclusion. State your conclusion.
  • Rule. Identify the fact or facts upon which your conclusion is based.
  • Analysis. Provide an analysis of how that rule makes any conclusion other the one you’ve reached invalid.
  • Conclusion. Restate the conclusion.

This CRAC process was used very effectively recently by an acquaintance chairing a community group that was protesting a pending building approval decision of their local council.

She stated that the approval, if it was to proceed, was in defiance of the councils own regulations.

She then cited the specific regulations.

She then pointed out the specific parts of the pending approval that was in breach of the regulations, and why they breached them.

For good measure she also pointed out 2 other proposals similar to the one that appeared to be about to be approved, that had been rejected on the basis of the specific parts of the regulations stated previously.

She then repeated the conclusion that the project was in defiance of the council’s own regulations, and therefore should not proceed.

It was an impressive performance, well planned, well executed, and ultimately successful after some embarrassing back downs by several councillors.

With a bit of practise, it is easy to use, and always better than resorting to a quarrel.

 

Header cartoon credit: Scott Adams and his mate Dilbert.

 

 

 

 

How do we prepare for AI roles that do not exist? 

How do we prepare for AI roles that do not exist? 

 

 

Most BBQ conversations about the future of AI end up as a discussion about jobs being replaced, new jobs created the balance between the two, and the pain of those being replaced by machine.

It is difficult to forecast what those new jobs will be, we have not seen them before, the circumstances by which they will be created are still evolving.

18 months ago, a new job emerged that now appears to be everywhere.

‘Prompt engineer’.

Yesterday it seems, there was no such thing as a ‘prompt engineer’. Nobody envisaged such a job, nobody considered the capabilities or training necessary to become an effective prompt engineer. Now, if you put the term into a search engine there are millions of responses, thousands of websites, guides, and courses have popped up from nowhere. They promise riches for those who are skilled ‘prompt engineers’ and training for those who hop onto the gravy train.

What is the skill set required to be a prompt engineer?

There are no traditional education courses available, do you need to be an engineer, a copywriter, marketer, mathematician?

This uncertainty makes recruiting extremely difficult. The usual guardrails of qualifications and past experience necessary to fill a role are useless.

How do you know if the 20-year-old with no life experience and limited formal education might be an effective and productive prompt engineer?

How many job descriptions will emerge over the next couple of years that are currently not even under any sort of consideration?

Recruiting rules no longer play a role. We need to hire for curiosity, intellectual agility, and some form of conceptual capability that I have no word for.

The challenging task faced by businesses is how they adjust the mix of capabilities to accommodate this new reality.

Do they proactively seek to build the skills of existing employees which requires investment? Do they clean house and start again, losing corporate memory and costing a fortune? Do they try and find some middle path?

Where and how do you find the personnel capable of building for a future that is undefined?

 

 

 

 

Are we in an AI bubble?

Are we in an AI bubble?

 

 

Nvidia 2 years ago was a stock nobody had heard of. Now, it has a market valuation of $US2.7 trillion. Google, Amazon, and Microsoft from the beginning of this year have invested $30 billion in AI infrastructure, seen their market valuations accelerate, and there are hundreds of AI start-ups every week.

Everybody is barking up the same tree: AI, AI, AI…..

Warren Buffett, the most successful investor ever, is famous for saying he would not invest in anything he did not understand.

He conceded many opportunities have passed him by, but he gets many right. Berkshire is the single biggest investor in Apple, a $200 billion investment at current market value that cost a small fraction of that amount.

Does anyone really understand AI?

Are we able to forecast its impact on communities and society?

We failed miserably with Social media, why should AI be any different?

Even the experts cannot agree on some simple parameters. Should there be regulatory controls? Should the infrastructure be considered a ‘public utility’? when, and even if, will sentience be achieved?

Bubbles burst, and many investors get cleaned out, but when you look in detail, there are always elements of the bubble that remain, and prosper.

The 2000 dot com bubble burst, and  many lost fortunes. However, there are a number of businesses that at the time looked wildly overvalued, that are now dominating the leaderboards: Apple, Amazon, and Google for example.

The tech was transformative, and at any transformative point, there are cracks that many do not see, so stumble. From the rubble, there always emerges some winners, often unexpected and unforecastable.

Is AI just another bubble, or is it as transformative as the printing press, steam, electricity, and the internet?

Header cartoon courtesy of an AI tool.

 

 

 

A Future Made in Australia: Does feeding ourselves count?

A Future Made in Australia: Does feeding ourselves count?

 

 

The recent declaration of “A Future Made in Australia” by the Prime Minister has put the future shape of the nation’s manufacturing sector back on the agenda.

There was however, nothing specific on the importance of agricultural innovation and value adding through the manufacturing sector, or the strategic value of food security.

The decline in Australian owned manufacturing in the food industry has been close to total. The FMCG manufacturing industry has seen input prices increase by 49% over the decade to 2020, while the wholesale prices received have increased by only 24% over the same period (Source: AFGC Sustaining Australia Food and Grocery manufacturing 2030 report) This downturn, and the 20 years prior which display similar trends has seen locally owned businesses either go bankrupt, or become subsidiaries of foreign conglomerates, relegating them to mere outposts.

From an era where medium-sized businesses thrived across various product categories, employing significant numbers in quality, engineering, the trades, and R&D, today these businesses have largely disappeared. This transition has been marked by a shift towards centralisation of product development and scientific research abroad, leaving Australian operations with minimal operational and decision-making authority.

This trend raises critical questions of how we feed ourselves, and make a useful contribution to the global food supply.

Notwithstanding the international ownership of most of food and beverage manufacturing, it contributes 6.5% of GDP, 32% of total manufacturing output, and employs 240,000 people, 40% of which are in regional areas. (source AFGC)

By any measure, the food manufacturing sector is profoundly important to Australians. Its future resilience and growth of sovereign capability should be paramount.

The lack of sovereign control of the resources and capital needs to generate growth is disturbing.

Central to an innovative and resilient manufacturing industry is the capacity to generate intellectual capital that translates into manufactured product. The progressive ‘internationalisation’ of company R&D noted above, has been matched by a progressive emasculation of the sovereign capability to generate the Intellectual capital necessary for long term growth. There is a significant number of SME’s in the sector, but collectively they contribute very little to the total of manufactured product. They are typically mixing often imported ingredients in low tech environments with a few employees and casuals. Distribution is largely through secondary channels like farmers markets, and local retailers and food service. They do not have the resources to compete with the R&D capability of multinationals, and the previously available intellectual assistance from federal and state institutions has been removed.

Take for example the CSIRO that in the past worked closely with business. Often this was in an informal and personal collaboration between individuals that enabled a thriving environment for problem solving and innovation. CSIRO’s sites in North Ryde, Werribee, and Canon Hill have either been downsized or sold off, and skilled, experienced  employees made redundant. Contributing to this erosion of the collaboration that in the past generated much of the ‘ideation’ that sets the stage for innovation, has been the demands of successive governments for a ‘productivity dividend’. This was typically 2% annually which compounds quickly to a killer blow to capability. It is code for removing those informal but fundamental creative collaborations with domestic companies, and encouraging the multinationals to centralise R&D elsewhere.

The power of the supermarket chains, currently under scrutiny has also played a key role in this process. SME’s simply do not have the deep pockets required to generate and maintain traction through the retail FMCG oligopoly.

To be successful, SME’s need to be able to absorb the reality of this gross power imbalance with retailers. Financial capital is necessary to enable the generation of the Intellectual Capital that underpins genuine innovation. Further investment is required to design, build and install the equipment to produce the innovative product. Deep pockets are then required to meet the retail trading term and promotional demands, as well as investment in the advertising necessary to attract consumers to a new product. As the power of the retailers has overwhelmed the diminishing group of domestic suppliers, we have been left with multinational suppliers and retailer house-brands, themselves often manufactured offshore.

The focus of government policies remains short-term, driven by electoral cycles rather than the decades required to bridge the gap between science and commercial success. Differing jurisdictions follow their own nose, resulting in a siloed and fragmented effort across the country, rather than a coherent and coordinated effort. The outcome is a mix of differing priorities, investment plans and initiatives around the country, sometimes used as incentives for business location. The commercial equivalent would be if a conglomerate allowed divisions and locations to compete for resources with declining levels of investment in the total absence of a coherent strategy. No sensible commercial board of directors would put up with such a self-defeating arrangement.

Grant programs send the wrong message and encourage behaviour that rarely delivers the outcomes touted in the press releases.

Culturally and politically risk is toxic to the body politic. However, the acknowledgement and management of risk is a fundamental element in successful innovation.

Successful risk management becomes a function of the extent to which a whole range of data, combined with qualitative assessment of what the future will look like is considered. Removing the capacity to make those assessments severely compromises the value of any conclusion reached.

The only potential solution to those institutional blockages to innovation in manufacturing industries generally is a confronting one.

Government needs to ‘upskill’ itself to be in a position to substitute early equity funding for grant funding.

Such a change requires a cohort of skills and experience not currently available within government and bureaucracies, but selectively available in industry. The early equity would be recoverable by those that are successful at a pre-agreed point, at a pre-agreed rate.  This removes the inertia and rent seeking evident in grant funding, replacing it with a modified form of Venture Capital.

In addition, FIRB needs to adjust the guidelines that currently rely on an intense focus on the economics of ‘Comparative advantage’. These rely on projections of current and past quantitative models of industries that usually bear little resemblance to what ultimately evolves. They never reflect the strategic value of sovereign manufacturing.

In the absence of meaningful strategic change, what remains of the domestically owned food manufacturing industry of any scale will disappear, and current and new SME’s will have no hope of replacing them.

Notes.

  1. The budget delivered on Tuesday night included a number of measures that should serve to give manufacturers some confidence that the government has recognised there is a problem, and that action was long overdue.
  2. A slightly edited (and improved) version of this post was published on Wednesday morning on the AuManufacturing website and Linkedin group.

 

Political Strategy: Choose between the now and the future.

Political Strategy: Choose between the now and the future.

 

 

Our current politics is an intensely adversarial, short-term, zero-sum game.

Is this what is best for the country?

The federal budget is due in a few days.

Based on the selective leaking and conversations happening, the budget will be focused, or at least the political narrative will be focused, on cost of living, housing, male violence against females, and the build-up of national security assets, military and technology.

All worthy topics, demanding attention, understanding, and investment.

However, if anybody in Canberra chose to take a helicopter view of the strategy of Australia Inc, as it would be in a business, there are only two questions that should be the framework that drives the tactical choices that are made every year in the budget.

  • What are we building that will deliver long term capacity, resilience, and innovation to the economy?
  • What are we doing now to optimise the way we invest resource is against those long-term priorities, and the shorter-term tactical investments necessary to achieve them?

The first is a drag on current expenditure that is designed to deliver a long-term outcome.

The second is an imposition on the long-term outcome to deliver in the short term.

At the best of times this is a delicate balance, and you never have all the right answers.

However, in the absence of asking the question, there will be no answers other than a knee jerk response to whatever happens to be in the headlines today.

Let’s not worry about our children and grandchildren, they will find a way to recover the can we have so solidly kicked down the road.

On that can are the words: we have a revenue problem.

This means tax, as that is the only way governments have the resources to deliver to the country.

Unfortunately, Tax is a noxious three letter word, and no politician who desires to remain one, (they almost all do) will touch, unless accompanied by the word ‘Cut’. Besides, no politician is short of a bob, superannuation entitlement, negatively geared investments, and the largess of party donors. They live comfortably on the teat, while often complaining about how hard the job is, which is no doubt true for those few who are trying, easy for those who are just seat warmers.

The header is courtesy of DALL-E, my artful helper.