Written by Stuart Grimsey, Senior Consultant at Baines Simmons
Genesis of Event Risk Classification
The aviation industry recognised during the 2000s that conventional risk assessment methods did not necessarily address complex risks specific to them in the most effective manner. Between 2007 and 2010, the ARMS Working Group, made up mainly of safety practitioners from Airlines, developed a methodology and framework to try and reduce the subjectivity within risk assessment. From this work, two distinct tools were developed:
ARMS Event Risk Classification (ERC) tool – used to triage safety reports/events to identify if immediate intervention is required and indicate the appropriate level of further investigation.
ARMS Safety Issue Risk Assessment (SIRA) tool – a methodology to assess risk considering four factors rather than the classic likelihood versus severity used in most risk matrices.
Both tools are available for use free of charge and can be integrated into both organisations’ Management Systems and appropriate software systems. The methodology was based on the International Civil Aviation Organisation (ICAO) Standards and Recognised Practices (SARPs) circa 2008.
Although the Working Group was disbanded in 2010 and further updates have not been carried out, the approach remains broadly consistent with the latest ICAO SARPs and therefore a valid option for organisations to consider. In this article, we will look at ARMS ERC, keep a lookout on the Baines Simmons LinkedIn site and website for Part Two, where we will explore ARMS SIRA.
Key Outputs for ARMS ERC
ARMS ERC is a triage tool and is therefore used to assess past events to enable three key outputs:
1. Prioritisation. By placing incidents through the two-question approach, a position will be assigned on the matrix, with an associated colour and number (event risk index value). Colour coding indicates the urgency and depth of follow-up required, as per the table below.
2. Standardisation. ERC provides a consistent framework for initial event evaluation across different departments and personnel, reducing subjective interpretation and ensuring uniform risk assessment standards.
3. Statistical Analysis. By assigning an event risk index value, ERC provides quantifiable data that can be used for trend analysis and amalgamated with risk index values from similar events to develop a broader picture of risk development over time. This is an enabler for data-driven, risk-based decision making.
ARMS ERC: Process
Question 1 of the process asks the user to consider the Most Credible outcome if the incident had escalated into an accident. This is not dissimilar to the severity judgment in a classic risk matrix.
What is useful is that Most Credible is considered, not Worst Possible; Worst Possible thinking generally ends up with multiple fatalities for nearly all incidents. This is of limited use when resource decisions need to be made. In the case of ARMS ERC, if Worst Credible was used, all triaged events would end up in the top row of the matrix under Question 2. The use of the Most Credible in the methodology provides a more realistic outcome on which to make triage decisions.
The tool also allows for the wording or criteria to be changed. This allows organisations operating under different approvals to tailor the tool to work for them. With a change of focus to the Release of an Unairworthy Design/Product/Engine, Question 1 can easily be amended to be suitable for Part 21G/J or Part 145 organisations.
Question 2 asks the user to look at the remaining barriers and their effectiveness in preventing a final accident. Using Barrier Based Thinking (typified by BowTie Analysis) is a strength of the triage system, but also a limiting factor if your organisation has not adopted it.
This is the key difference from classic risk thinking, where you would be trying to assign Likelihood. By considering barriers, the user will still have to make a subjective judgement on the number and effectiveness of those that remain, but by focusing on this rather than the generic overall likelihood, it is reasoned in the methodology that a more granular result will be produced.
If your organisation does not use BowTie analysis, this shouldn’t prevent you from gaining utility from the methodology – if you can align your thinking towards barriers, you can still use the tool.
It is worth reiterating that this is a triage tool; the primary output of the process will be an indication of urgency and depth of response required for the specific incident. It also aims to provide that standardisation and data for statistical analysis, but data is just data and not risk management until you do something with it. That is where further analysis using your organisation's Safety Risk Management process, or within this methodology, the ARMS SIRA tool, comes in. We have seen organisations misusing ARMS ERC for Risk Assessment, something it is not designed to achieve in isolation.
ARMS ERC: Pros and Cons
We have already alluded to some of the pros and cons of ARMS ERC. ICAO have also looked at just that in the pithily titled “Safety Risk Management Methodologies (SRM) Aviation Risk Management Solutions (ARMS) and Event Risk Classification (ERC) version 1.0 January 2024”.
Their useful summary is reproduced below from that publication:
Conclusion
ARMS ERC represents some tangible benefits to the triage process of incoming incident information, whether that be via a safety report or less formal channels. It can provide some structure, and therefore standardisation and repeatability, to how an organisation enacts this stage. The aspiration of the methodology to enable less subjective and more granular decisions on the urgency and depth of subsequent response to incidents is certainly worthy of consideration by organisations. Allied to this, the provision of risk index values opens the potential for subsequent analysis and exploitation that could enhance data-driven, risk-based decision making by Accountable Managers.
The substitution of the Likelihood judgement with Barrier Based Thinking in Question 2 of the process may well be the alluring property for many organisations who have already embraced BowTie or similar thinking – it is a smart concept to approach this most subjective of Safety Risk Management areas. That strength is also the main impediment to many; if your organisation does not subscribe to Barrier-Based Thinking in its Safety Risk Management, then the tool may still be of benefit, but not deliver as comprehensively. The other main consideration is that the methodology no longer has an active “owner” and hasn’t since 2010. Currently, this represents little risk as it remains broadly aligned to ICAO thinking; indeed, many organisations, large and small, have adopted and are using ARMS ERC today very effectively. There remains the chance that ICAO SARPs will diverge from the 2008 view, and ARMS ERC could become less effective unless revised.
I hope you have found some benefit in the summary of ARMS ERC and that it has provided some food for thought. What has been your experience with advanced risk assessment methodologies in aviation? Why not share your thoughts on how tools like ARMS ERC are shaping the future of aviation Safety Risk Management? Be sure to follow Baines Simmons, leaders in aviation safety services, on LinkedIn and check the website for Part Two of this article, where we’ll delve into ARMS SIRA.
Sources:
Unless otherwise stated, images are from The ARMS Methodology for Operational Risk Assessment in Aviation Organisations, Developed by the ARMS Working Group, 2007-2010.