These examples show the benefits of AVs, as well as the risks. The vehicles themselves offer novel solutions to transportation issues worldwide, from simple things like food delivery to critical tasks including transporting sick individuals to hospitals and clearing areas where landmines have been buried. AVs can increase mobility for elderly or disabled persons like Miller, improve traffic efficiency and safety, and even reduce emissions if clean propulsion is used. ^{3 4} However, the complexities of driving mean that driverless safety is not clear-cut. While some technologies are made of one system or function, AV technologies involve numerous interdependent systems within a vehicle, including traditional vehicle systems (power, emissions, steering, climate control) and new autonomous systems for navigation and sensing, which rely on computer programming and predictive modeling. 

AVs create and update a map of their surroundings as they drive, integrating input from sensors with static (e.g., maps) and dynamic (e.g., real-time road hazard or weather updates) navigation information. Recognition systems note traffic, obstacles, signs, stop lights, and humans. A human or vehicle must trigger instant decisions to avoid safety incidents such as near hits, vehicle crashes, or vehicle mechanical failures. And as we’ve noted above, what is safe in one instance may be dangerous in another.

Several safety standards support safe motor vehicle operation, including the International Organization for Standardization standards ISO 26262, Road Vehicles — Functional Safety, for traditional, nonautonomous vehicles and ISO/PAS 21448, Road Vehicles — Safety of the Intended Functionality, for semiautonomous vehicles. Semiautonomous vehicles, which hand off responsibility between a driver and the autopilot, still rely on driver oversight to ensure the vehicles’ safe operation.⁹ In 2017, fully autonomous taxis debuted in San Francisco, introducing a host of new, nonhuman-mitigated safety threats into city driving. For the first time, human oversight of vehicular safety was completely removed from vehicle operation, which meant that the machine had to be entirely responsible for the safety of the vehicle, its passengers, and its surroundings. No safety standards existed to regulate completely autonomous vehicles for human and goods transportation. What would this mean for public safety? 

The safety case method shifts responsibility for risk management to manufacturers and emphasizes meeting safety goals and documenting how they are met. ⁷ Safety cases require authors to fully describe how a product is designed as a safe system, using evidence to justify their argument that the product is “safe enough.” UL 4600 does not set the bar for what is “safe enough,” instead relying on the manufacturers to set and explain their own thresholds for safety. Koopman, the safety expert who brought the suggestion of using a safety case standard for the complex AV problem to ULSE, said, “We’re not standardizing the product, we’re standardizing the safety case.” ²

In a safety case, manufacturers construct logical trees of reasoning, branching from safety claims and subclaims (e.g., premises or goals) to linked logical arguments about how each claim could be valid if specific evidence could be collected (Figure 1). Finally, the tree terminates in the corresponding data to support the claim. Data should be collected from the product’s safety performance indicators to support the claim. Hence, SPIs are metrics used to justify a claim. ⁵

In this way, the actual data of operation, which could be gathered from computer-monitored indicators, road test performance, or physical measurements such as temperatures, are used to demonstrate the item’s safety in its safety case. Safety cases do not include ethical and societally acceptable factors such as assigning liability or blame following a safety incident. However, case authors do need to justify their chosen threshold values for safety with technical and nontechnical explanations. Logical arguments are required, and spurious or nondeductive arguments are discouraged. Every claim must be falsifiable to be convincingly supported by evidence.

In addition to general operation, the safety case should justify safe functioning during real and imagined risks and describe how the system would mitigate any hazards. For example, sensors are ubiquitous on AVs to support real-time navigation and driving. To fully describe how an AV’s sensors would operate safely, UL 4600 specifically mandates that authors create a fault model to address how their AV will handle a slew of adverse sensor reactions such as sensor faults triggered by environmental conditions (rain, water splash, mud, ice, dirt, low and high temperatures, and low and high humidity), man-made issues (sensor defacement, alignment compromise, gouged optics, blunt force impact), regular use (vibration and mechanical wear), and other “unknown unknowns” that could arise during operation. Safety cases even require the authors to list why they did not consider or mitigate a particular fault, ensuring that the safety case is comprehensive of a maximum number of in-scope threats or deliberately articulating which threats are out of scope. 

To make a strong argument, manufacturers should use the claim, argument, and evidence method to specifically address hundreds of potential AV-related faults provided in the standard, including those related to interactions with people inside and outside of the vehicle (e.g., passengers, bicyclists, pedestrians, construction crews, and toll booth workers); autonomy functions and support; software and systems engineering processes; dependability; data and networking; verification, validation and testing; integrated tools including commercial off-the-shelf components; life cycle concerns; and maintenance concerns. The lists of prompts in UL 4600 are extensive to clearly describe everything that might be included in an AV, from hardware to software, and sensors to battery management. 

A safety case that addresses hundreds of pages of prompts with detailed evidence and arguments will be lengthy. To support authors in being comprehensive while prioritizing what to include, UL 4600 organizes prompts into five categories: requirement statements, overarching categories that must be fully addressed in the safety case; mandatory prompts, which will also be fully addressed; required prompts that can be avoided if shown to be intrinsically incompatible with the item; highly recommended prompt elements that could be plausibly excluded with rationale; and recommended prompt elements, which may be excluded without a rationale. This guidance indicates that if a prompt or fault is outside the operational design domain, it can be considered an element out of context, and the authors must indicate why they are not addressing it.

To encourage comprehensive safety considerations, UL 4600 includes examples for many of the prompts taken from real-life incidents and the authors’ reasonable extrapolation of possible safety incidents. These examples are intended to help extend authors’ thinking by articulating potentially relevant instances for AV use. For instance, prompts might remind authors that human passengers may need to use medical equipment in and outside of the vehicle and may require special considerations. Or, noting that vehicles may not always be used on well-regulated city streets, in fact, they could pass outside of the operational design domain in war zones, flooded areas, fire zones, police activity zones, or extreme weather areas.

To achieve UL 4600 certification, a manufacturer should write its safety case and submit it to an independent, but not necessarily external, conformance assessor. UL 4600 works in tandem with other related safety standards and codes, which each have their own procedures for testing and certification. Like many safety standards, UL 4600 is a living document that will change over time in response to emerging technologies. It has been updated twice since it was first published in 2020, each time involving the same technical committee of experts to reach consensus and publication after public comments. Version three expands the scope of the standard to include heavy commercial autonomous trucks that operate on public roads.