Back to News
July 25, 2026

Why cyber security awareness training fails | the research

Most organisations run a similar approach to cyber security awareness training. Send your own people a fake scam email in the form of a phishing test, see who clicks, send whoever clicked to a training module.

I've spent a long time watching that cycle, and I kept coming back to one question. Does it work?

This blog includes the findings from some of the older and new research, MITRE, ETH Zurich, UC San Diego and others, more than 50,000 employees between them, going back over a decade.

The answer is not encouraging. And the story of how we got here starts in 2004, with the US Army and a cannon.

It started with a cannon

One of the first well-documented friendly phishes did not come from a software vendor. It came from the US Army.

In 2004, West Point ran an experiment called Carronade. A fake email from a made-up colonel went to 512 randomly selected cadets, evenly spread across all four year groups. More than 400 of them clicked. Among the freshmen, who had spent four hours that semester on information assurance and network security, the figure was 90 per cent.

A carronade is a short-range naval cannon, and the name was chosen deliberately. The exercise was built as a wake-up call. The original design included consequences for cadets who clicked, and the university's chief information officer objected, so the consequences came out. The purpose was to show the security officers where to focus, not to catch anyone out.

It seems the industry kept the first half of that idea and dropped the second and the 'phishing test + punish with learning' approach was born. Key features being: trick your staff, log who failed, punish them with learning, threaten their bonus or their email access if they fail it or never finish.

Two decades later, that is still the default.

Interesting fact: No major framework names phishing simulation as a control. ISO 27001:2022, PCI DSS v4.0, NIST CSF 2.0, NIS2 and APRA CPS 234 all require that personnel are made aware of social engineering, are trained to recognise and report it, and that you can evidence it.

What the research has found

The whole loop has now been studied for more than a decade.

Does it work?

MITRE, 2014. Across three embedded training trials, researchers found no reliable reduction in susceptibility, and most employees never read the training. Staff,

"expressed concern that the training webpage might have been part of the spear phishing attempt; consequently, many participants closed the training page without reading any text on the page".

ETH Zurich, 2022. Fifteen months, more than 14,000 employees, training delivered the moment someone clicked. The authors found it

"can have unexpected side effects that can make employees even more susceptible to phishing".

The same study found something that did work. Give people a simple way to report a suspicious message and they use it, and the reports are accurate enough to be operationally useful.

UC San Diego, 2025. 19,500 people, the phishing test, a 1.7 percentage point effect. Between 37 and 51 per cent of training sessions were closed immediately, with no engagement at all.

Does it last?

No. Whatever the module teaches after a failed test is largely gone within about five months. Run it annually and your workforce is uncovered for most of every year.

Can it measure itself?

A 2026 preprint analysing 19,341 employees across 17 campaigns found that training is only ever assigned to the people who clicked, which biases every effectiveness claim built on the loop. It also found that employees who spent time on the education page went on to click more than those who dismissed it. That second finding is correlational and the two groups selected themselves, so treat it as a reason to doubt the loop rather than proof against it.

So what does work?

A 2024 review of 42 studies found the factors that improve detection are training intensity, active learning and feedback. The test and module loop offers none of them.

The metric grades its own homework

A click rate is not a property of your workforce. It is a property of the test.

NIST built the Phish Scale in 2020 for exactly this reason. Click rates rise and fall with the difficulty of the lure, so the same workforce produces a low rate on an easy phish and a high rate on a hard one. In NIST's own words, the data

"can create a false sense of security if click rates are analyzed on their own without understanding the phishing email's difficulty."

Now look at who controls the difficulty. The vendor writes the lures. The vendor schedules the campaigns. The vendor publishes the benchmark you are measured against, generated from tests run on its own platform. And the vendor's renewal depends on your click rate going down. Nobody in that loop is rewarded for sending a harder test.

So a falling click rate can mean your people got sharper, or it can mean the tests got softer. The report does not tell you which.

The same softening happens inside programs that write their own tests. A security team includes one hard phish, built the way the real ones are built, and the complaints arrive within the hour. Not fair. Nobody could have spotted that. The next campaign is gentler, and the program settles into testing what people can already do, which is spotting a phishing email that looks like a phishing email.

The threat has outrun the model

The model was designed for a workforce at a desk, reading email, hunting for spelling mistakes and odd greetings. AI has removed those tells, and it has made convincing voice and video cheap.

Scams now arrive by SMS, voice call, QR code and video, on personal phones, to people who have never had a corporate inbox. An email test does not touch any of it.

Nobody learns while they are being shamed

"I love our phishing tests," said no one, ever.

In 20 years of employee engagement surveys, learning needs analyses and feedback, qualitative and quantitative, I have never once seen it.

Just this morning, someone asked me:

"Why would my employer threaten me with taking away my email access if I don't complete the cyber learning module, is that a promise?"

Being tricked by your own employer feels unsafe. Nobody learns while they are being shamed, humiliated and then punished with learning.

If you are running this model today, none of this is a verdict on you. It was a rational buy when the threat arrived mainly through the inbox and compliance was the measure that mattered, and it does some good. It keeps awareness up and it encourages reporting. The threat has changed underneath it.

What the evidence points to instead

The 2024 scoping review pooled 42 studies and named the factors associated with better detection:

"training intensity, active approaches to learning, the provision of detailed feedback, and supplementing attentional awareness skills-based training with traditional cue-based approaches"

In plain English: practise often, learn by doing, find out how you went, and learn to read the underlying signals rather than only memorising the familiar tricks.

The review is not against training. It found that

"improved user resilience to phishing emails confirms the utility of training as an important defensive mechanism".

It is blunt about where we have got to, though.

"Current approaches continue to leave trainees at risk",

and while the near-term impact of training is well documented,

"evidence on the success of programs in driving sustained behavioral change is limited".

So the gap is not whether to train people. It is how. Frequent practice against threats that look like the real ones, in a place where being wrong costs nothing and teaches something.

Social engineering fuelled by AI has made two skills essential: critical thinking, and spotting the signs of social engineering. Both are built by doing, not by reading.

And so that is what we built.

‍

FAQ

Why does security awareness training fail?

The traditional model teaches through failure. A simulated phishing test catches who clicks, and whoever clicks is sent to a module. Three large independent studies of that loop, from MITRE, ETH Zurich and UC San Diego, found little to no effect on susceptibility, and in ETH Zurich's case a risk of making some people worse. Most people never read the module: between 37 and 51 per cent of sessions in the UC San Diego study were closed immediately.

There is also a design trap. A phishing test cannot use the hardest real examples, because tricking your own staff with the real thing causes stress and complaints, so the tests stay soft and the skill never gets built.

Do phishing simulations reduce risk?

The published evidence says the effect is small at best. The UC San Diego study of 19,500 people measured 1.7 percentage points. The ETH Zurich study of 14,000 employees over 15 months found the training component did not help. Simulations do produce a number for your board, and they do keep the topic visible, which is not nothing.

How long does security awareness training last?

Research on anti-phishing awareness found the effect largely decays within about five months. An annual cycle therefore leaves most of the year uncovered.

Is phishing simulation required for compliance?

No major framework names phishing simulation as a control. ISO 27001:2022, PCI DSS v4.0, NIST CSF 2.0, NIS2 and APRA CPS 234 all require that personnel are made aware of social engineering, are trained to recognise and report it, and that you can evidence it. The one exception is DORA, whose threat-led penetration testing regime for designated financial entities typically includes live social engineering.

What works better than phishing simulation?

The 2024 review of 42 studies points to training intensity, active learning and feedback. In practice that means frequent, short, active practice with immediate feedback, across the channels people actually get attacked on, in a setting where a wrong answer is safe.

Sources

Written by Stacey Edmonds, co-founder of Lively, a Learning Agency

Stacey is a social scientist, teacher, and multi-award-winning digital producer.

As Partner, Future of Learning at Deloitte, Director at Transport for NSW, and Senior Partner at Korn Ferry, she has spent two decades translating complex human behaviour into practical organisational change. She also serves as a juror at the New York Film Festival and International Business Awards.

Her focus for the past decade has been a specific and well-documented problem: phishing training that does not change behaviour. Research now shows it can make people more likely to click, not less. She approaches this through the lens of behavioural science rather than compliance, drawing on peer-reviewed evidence to make the case for what actually works.

‍

‍