FROM LIBRARY: WHAT MAKES A GOOD AIR SAFETY RECOMMENDATION?

Incidents and accidents within the aviation industry require investigations by the national safety agency (NSA) to find the root cause and, when necessary, issue safety recommendations as stated in International Civil Aviation Organisation (ICAO) Annex 13 [1]. There are accidents that make headline news such as the Concorde crash in 2000 [2] or the uncontained engine failure on the Qantas A380 in 2010 [3] and others that the public does not hear about. An investigation must take place if an accident or incident results in death, serious injury, or indicates a defect in the aircraft.

Existing work suggests that, compared to the efforts concentrated on root cause investigation, there is less emphasis on the creation and implementation of safety recommendations [4]. Research has found that recommendations may be at risk of causing unintended consequences [5] and are deemed to be only 70% effective by investigators themselves [6].

It is by issuing and accepting safety recommendation that the origins of incidents are addressed with the aim of preventing recurrence
and developing the safety of the aviation industry. So how do investigators improve the likelihood of getting these recommendations
accepted?

Currently there is some guidance on what a safety recommendation should contain by ICAO in Annex 13 Chapter 6 [7] along with its Manual of Aircraft Accident and Incident Investigation [6, 8]. For member states of the European Aviation Safety Agency (EASA), 996/2010 Article 17 [9] also provides some guidance.

The NSAs, including the UK Air Accident Investigation Branch (AAIB), have their own manuals; however, these are not available to the public.

Using public domain information, the current status of safety recommendations from the AAIB is readily available. In this work, a sample of 25 incidents investigated by the AAIB that occurred on fixed-wing commercial aircraft between 2005 and 2015 were pooled, producing a total of 136 safety recommendations. The AAIB’s annual safety reports were used to identify the investigation body’s status of these safety recommendations, as shown in Figure 1.

		It is important to note that this uses the AAIB’s status of the safety recommendation, which may differ from the addressee’s status as it is based on the last response received. The European Safety Recommendation Information System (SRIS) has been introduced to track the status of recommendations, but this tool is not in the public domain. Investigators aim to close safety recommendations, and the adequate, partially adequate, and not adequate grading is dictated by how well the response from the addressee aligns with the intent of the safety recommendation. As shown in Figure 1, 70% of the safety recommendations in this study have been closed, including 61% adequately closed.
Figure 1: AAIB status of the 136 safety recommendations sampled.

To assess if the content influenced whether the safety recommendations would be accepted, they were split into three categories

• Regulation and legislation—covering the legal aspect.
• Review in-service product—covering the technical aspect.
• Notifications, training, policies, and procedures—covering the process aspect.

Figure 2 shows that those covering technical and process aspects show a similar trend, whereas the legal ones showed significant differences. The latter category has a lower percentage of “adequate—closed” safety recommendations and a higher percentage of “not adequate—closed” and “response awaited—open.”

This is not surprising, as “regulation and legislation” includes changes to certification and means of demonstrating compliance that influence the whole sector and involve a lot of organizations, creating obstacles to acceptance. The reason for the larger percentage of safety recommendations awaiting response may be due to the numerous people involved, which often increases the length of the process. There is also the factor that regulatory bodies are the largest party of recipients of safety recommendations and therefore must prioritize responses. In summary, this analysis shows that the content covered can affect the likelihood of acceptance.

This analysis looked only at AAIB- issued safety recommendations, but how can the global standard be analyzed? To address this, it is useful to look at an agency that is the recipient of safety recommendations from numerous NSAs. In this case, EASA was used. Its annual safety recommendation review shows the safety recommendations the agency has responded to for that particular year. Reviewing the 2014 and 2015 version of this document, it was clear to see there was a varied standard in safety recommendations.

_{Figure 2: Status of safety recommendations for each category as a percentage.}

 

Initially a SMART analysis was attempted to evaluate the standard; however, this proved too ambiguous. A better way was looking for certain criteria that created a good safety recommendation.
These included containing a specific addressee; clear objective; succinct, supporting
evidence; and finally stating the issue, not the solution. Interviews with industry experts validated this approach and added to the list of criteria detailed in Table 1.

The sample of 10 interviewees covered the perspectives of NSAs, regulators, and manufacturers as well as academia, as shown in Figure 3.

The interviews allowed additional information to be gathered and validation of the AAIB and EASA document analysis. They also acted as a means of overcoming barriers, such as not having access to AAIB’s investigator manual or the SRIS tool.

The general consensus was that communication has the greatest effect on whether the safety recommendation is accepted. It is important that this is in the drafting stage as it means the addressee and investigator can agree on content and wording of the safety recommendation, ensuring a clear understanding. Some agencies already do this; however, by implementing this process globally the variation in the standard of safety recommendations could be reduced.

Another useful observation from the interviews was the effect of fatalities. There were mixed views on whether fatalities affect the likelihood of accepting safety recommendations. There was agreement that in ideal practice it shouldn’t, but due partly to news media influence, it may well. This could be due to an increased focus, commitment of greater resources, or to lawsuits for compensation. Common
reasons given by the experts for not accepting a safety recommendation included

• badly formulated and poorly supported safety recommendations,
• different views between investigators and addressees, and
• results of a cost-benefit analysis.

When an incident results in fatalities, it is the latter of these that is affected most, unless the likelihood of the incident recurring is seen to be minimal. There was, however, agreement by all participants that NSAs should not have legal power to enforce safety recommendations.
Noting that the majority of the interviewees stated that investigators were not the best placed to assess unintended risk is important. This should fall to the addressees who are the subject-matter experts. This finding agrees with Lefevre’s study in which investigators realized they had a part to play in evaluating potential new risks but stated the main responsibility lies with the addressee [11].

_{Table 1: What Makes a Good Safety Recommendation?}

 

The global standard could be improved and variation reduced by incorporating the factors mentioned in Table 1 into guide lines for writing a safety recommendation. EASA is working on harmonization within Europe through ENCASIA Working Group 6.

The work includes developing the SRIS tracking system and recently elaborating a Safety Recommendations of Union Wide Relevance (SRUR) approach.

In summary, the results of analyzing the acceptance of AAIB safety recommendations and identifying key factors that contribute to a good safety recommendation could be used by NSAs to improve guidelines and process for investigators. The main conclusions were that safety recommendations covering regulation and legislation are harder to implement, the current global standard is very varied, and communication is key in getting them accepted.

_{*International Federation of Airworthiness (IFA) is not a regulatory body, but the participant’s experience was in that area.}

^{Figure 3: Participants in the interviews.}

It is a challenge for investigators to write good-quality safety recommendation, and NSAs need to provide guidance as best they can. ICAO and agencies such as EASA and the Federal Aviation Administration need to work together to assist NSAs with this challenge and increase global harmonization. This is crucial to ensuring that the outcome of investigations are effective in improving the safety and reliability of the aviation sector.

References

1. ICAO. (2010) Annex 13 to the Convention of International Civil Aviation, Aircraft Accident and Incident Investigation, Tenth Edition, Chapter 1—Definitions.
2. Bureau d’Enquêtes et d’Analyses. (2002) Accident to the Concorde registered F-BTSC operated by Air France occurrence on 07/25/00 at Gonesse. Available at: https://www.bea.aero/les-enquetes/les-evenements-notifies/detail/event/lors-du-decollage-de-la-piste-26-droite-de-laerodrome-de-paris-charles-de-gaulle-peu-avant-la-ro-1/ (accessed on Dec. 10, 2016).
3. Australian Transport Safety Bureau. (2013) In-flight uncontained engine failure Airbus A380-842, VH-OQA, overhead Batam Island, Indonesia, 4 November 2010. Available at: https://www.atsb.gov.au/publications/investigation_reports/2010/aair/ao-2010-089/ (accessed on Dec. 10, 2016).
4. Lundberg J, Rollenhagen C, Hollnagel E and Rankin A. (2012) Strategies for dealing with resistance to recommendations from accident investigations. Accident Analysis and Prevention, Vol 45, pg 455-467.
5. Lefevre A. (2013) Are Safety Recommendations at Risk of Creating Unintended Consequences? Cranfield University, MSc Thesis.
6. ICAO. (2012) Manual of Aircraft Accident and Incident Investigation—Part II Procedures and Checklists, First Edition, Doc 9756.
7. ICAO. (2010) Annex 13 to the Convention of International Civil Aviation, Aircraft Accident and Incident Investigation, Tenth Edition, Chapter 6—Final Report.
8. ICAO. (2003) Manual of Aircraft Accident and Incident Investigation—Part IV Reporting, First Edition, Doc 9756.
9. EASA. (2010) Regulation (EU) No 996/2010 of The European Parliament and of the Council of 20 October 2010 on the Investigation and Prevention of Accidents and Incidents in Civil Aviation and Repealing Directive 94/56/EC. Available at: https://www.easa.europa.eu/document-library/regulations/regulation-eu-no-9962010 (accessed on Dec. 3, 2016).
10. Network Rail (2012) Investigators’ Handbook, Part 3—The investigation report, Part 3A, Isssue 2.
11. Arthur I. (2014) Do UK AAIB Safety Recommendations have their Intended Effect? A focus on Commercial Aviation. Cranfield University, MSc Thesis.

Written By: Maria Gregson, Faculty of Engineering, University of Nottingham, UK

Published On: THE ISASI FORUM MAGAZINE,Volume 50, Number 4 – October-December 2017