Assured Autonomy Through Safety Performance Monitoring

Description

In this presentation, the authors discuss safety assurance in unmanned and autonomous vehicle operations at the organizational and NAS levels.  The SMS component of safety assurance requires reliable measurement and monitoring of safety performance in high-risk operations that is applicable to an organization's SMS as well as a State's SSP.  The authors will first discuss the application of safety performance monitoring and measurement as part of safety assurance in unmanned and autonomous vehicle operations. Then, safety performance monitoring and measurement at the State level will be discussed with a focus on the critical role NASA's ASRS program plays in identifying safety opportunities in the NAS.

The framework for safety performance monitoring and measurement at the organizational level is composed of safety objectives which drive an organization’s safety strategy, data acquisition which drives sound decision making, and safety performance indicators which form the mechanism by which safety performance is quantified.  When considering the complex question of how to measure safety performance, it is tempting to concentrate on the measurement of end states such as incidents and accidents because these serious events are noticeable and therefore easily measured.  While the measurement of outcomes is valuable, it can be misleading especially in domains where a relatively low number of high consequence negative outcomes occur.  

Simply not crashing a vehicle is not a socially responsible or sufficient measure of safety performance.  Consequently, indirect measures of safety performance must be implemented to look beyond the outcome and at the underlying processes, latent conditions, and hazards that have the potential to manifest themselves in negative outcomes.   The successful integration and social acceptance of autonomous / unmanned vehicles will rely upon organizations building regulator and consumer confidence that these systems are in fact safe and reliable.  ICAO’s outline is a proven methodology for effectively measuring and monitoring safety performance in high-risk environments.

At the level of the National Airspace System (NAS), monitoring and measuring outcomes or events has long been a mechanism used by regulators to determine performance of State risk controls (e.g., regulations). However, as in organizational safety performance monitoring, measuring precursor events is key. The establishment of the NASA ASRS program has been the primary mechanism for this in the traditional aviation segment since 1976 to help not only to resolve prior incidents but to mitigate future problems. As modernization continues, these systems also provide the FAA with the data necessary to proactively build in safety protections.

In the drone space, hundreds of thousands, if not millions, of new airspace users are becoming key stakeholders in the safe operation of the national aerospace system (NAS)—whether they know it or not. It’s time these new users had access to an aviation safety reporting system that considers the unique challenges users of remotely piloted, and sometimes automated, aerial vehicles face in the NAS. In partnership with NASA, the Unmanned Aircraft Safety Team (UAST) has developed just such a system.

In early 2021, the FAA, with leadership from the Drone Safety Team (DST) and in conjunction with NASA, elevated UAS incident safety reporting to the same level as in manned aviation as now stipulated in Advisory Circular 00-46F.  Join long time DST executive committee member, Ken Kranz, as we look back at the past year to gauge the impact of the changes to NASA’s ASRS on the UAS industry.

In this talk we will examine the mechanisms of safety performance monitoring and measurement at the organizational level and how the data provide guidance for risk mitigation in operations. We will also explore how well defined and connected safety objectives can enhance safety in operations. We will then discuss safety assurance at the NAS level through the lens of the NASA ASRS. We will look at who is submitting safety data, how the data are being collected, and what safety trends are starting to emerge.  In addition, we will discuss other potential areas of safety data that will be coming available.  Finally, we will discuss now the ASRS will play a pivotal role as drone operations begin to migrate to scheduled (Part 121) and on demand/charter (Part 135) operations as it applies to their aircraft certification, operations, and maintenance. 

Agenda:

In the workshop, the attendees will:

1. Learn the importance of organizational safety assurance in the context of safety management systems.
2. Learn about the use of safety objectives - and the measurement of those objectives - to improve safety performance.
3. Become familiar with lead and lag measures for determining risk control performance.
4. Become familiar with the NASA ASRS program and how it contributes to safety assurance at the NAS level.
5. Learn the benefits of the NASA ASRS program for UAS.
6. Learn how to contribute to the NASA ASRS program to improve the integration of UAS into the NAS.

Job Role:
Management,Government/Policy/Regulatory,Remote Pilot/Operator,Student,Trainer

Tags:
Safety,Autonomy,Public Acceptance & Trust