Metrics don’t just paint a picture of the past; they provide a pathway to the future. Metrics, when used correctly can identify areas of training that require improvement and employees who need additional help.
In addition to providing ROI for information security expenditures, metrics are instrumental in:
Identifying major data breaches and other vulnerabilities.
Attracting high-quality security personnel.
Increasing customer trust and loyalty.
Ensuring compliance with legal and self-regulatory.
Why don’t people use metrics?
Collecting metrics in a constantly changing risk environment can be challenging, especially given the lack of universally accepted measurements. Many organisations with security awareness programs don’t collect metrics at all.
An absence of metrics effectively means no realistic ROI can be supplied, which may make executives reluctant to commit resources for an integrated security awareness program. Because haphazard efforts make documentation even more difficult, this leads to a vicious circle of a lack of data leading to no ROI. This leads to inadequate resources, which leads to no structured program, and so on.
However, the biggest single barrier to collecting meaningful metrics may be the difficulty of measuring actions. Does the security awareness program create more responsible behaviour by the employees? Metrics that note the employees’ participation in the security program’s initiatives must be accompanied by those that describe improvements in their actions. Measuring input is easy; measuring output is much more difficult.
The good news is that there are straightforward ways of obtaining accurate, useful metrics. Here are five ways of securing important metrics.
1. Phishing Training
Phishing training is a relatively easy way of obtaining metrics. Before the training, establish a baseline by recording the number of employees who fall prey to social engineering attacks, as well as how many file reports on suspicious emails. After training, run a test campaign using fake phishing emails. Keep in mind this is to test how well the employees absorbed the training, so try to vary the tactics and times of the fake emails. By comparing the behaviour of the employees before and after the training, you can measure its effectiveness. Testing also has the benefit of engaging the employees and reinforcing their training.
Beware of these common mistakes in phishing training:
Do not rely only on phishing metrics for your security awareness program.
Don’t ignore the wealth of information available in the test campaign. For example, record changes in the time from incident to detection. Identify effective phishing tactics, as well as the most vulnerable employees
Test everyone. Do not assume that security personnel or high-level executives are immune.
Remember that the goal is to change behaviour, not punish. Refer to employees who frequently fall prey to fraudulent emails as “repeat responders,” not “repeat offenders.”
Annual surveys provide significant benchmarks for the staff’s attitudes toward information security, as well as their understanding of organisational policies. The effectiveness of security awareness training can be demonstrated by comparing the responses on one year’s survey to another. If many employees display ignorance about a security problem, training can be adjusted.
Surveys also reinforce the security awareness training. For example, the question, “Do you know that you are accountable if someone else uses your workstation for illegal purposes?” reminds employees to lock their workstations when leaving for the night.
3. Behaviour Change Metrics
As mentioned above, the real trick to measuring the effectiveness of a security awareness program is tracking behaviour. Key behavioural metrics include:
Amount of reported lost or stolen devices.
Increase in phishing email reports.
Decrease in reaction time of incident response teams to reported phishing emails.
Hours spent by staff learning at voluntary events.
4. Videos training
People love videos, especially ones with strong production values. Video-influenced behaviour changes can be measured with before and after benchmarks, like the phishing testing described above. Engagement can be measured by the number of views, time spent viewing, and shares.
5. Live Trainings
Communication at live training should be a two-way street. Are the employees’ questions becoming more sophisticated over time? Are there some security areas that they are resistant to understanding, even after repeated trainings? Pay attention to what your employees are saying.
Hopefully you have found this useful and you can implement at least one of these within your own program. For more information and advice contact you account manager.