EP-3894973-B1 - HUMAN-SUPERVISED AUTONOMOUS SYSTEMS PERFORMANCE AND SAFETY MONITORING METHODS AND APPARATUS

Inventors

• KOOPMAN, PHILIP, J., JR.

Dates

Publication Date

20260506

Application Date

20191210

Claims (9)

1. A computer-implemented method (400) for detecting human supervisor failures in autonomous system operations comprising: obtaining perception sensor data input from a perception sensor (216) for measuring interactions between an external object and a safety envelope (410); augmenting the perception sensor data input with at least one of temporal offset output and image augmentation output from a perception evaluation system (236) for detecting perception algorithm weaknesses and failures, to produce an augmented perception sensor data input; detecting, within the augmented perception sensor data input, a violation of the safety envelope (410) by the external object (420); observing engagement of a human supervisor with the autonomous system and triggering a response to the violation of the safety envelope (430) with the response being based upon: an expected time to mitigate the violation of the safety envelope, and the observed engagement of the human supervisor with the autonomous system.
2. The method (400) of claim 1, further comprising logging the violation of the safety envelope (410).
3. The method (400) of claim 1, further comprising activating an automatic safety maneuver.
4. The method (400) of claim 1, further comprising warning the human supervisor.
5. The method (400) of claim 1, wherein the safety envelope (410) has predetermined dimensions that are based upon the expected time for the human supervisor to detect the violation of the safety envelope (410) and initiate an action to mitigate the violation of the safety envelope (410).
6. The method (400) of claim 1, further comprising configuring the sensor (216) to collect data relating to external objects.
7. A system (200) for detecting human supervisor failures in autonomous system operations comprising a memory or other data storage facility (212) and one or more processors (210) configured to perform the steps of any one of claims 1 to 6.
8. The system (200) of claim 7 wherein the one or more processors are configured to detect multiple violations of a plurality of safety envelopes (114, 116, 118).
9. The system (200) of claim 8, further comprising: a response system (226), wherein the response system (226)is configured to implement a different response for each of the plurality of safety envelopes (114, 116, 118).

Description

FIELD OF THE INVENTION The present invention relates to novel approaches for an Autonomous Vehicle Performance Monitoring System ("AVPMS"). More particularly, the invention relates to a system for detecting violations of safety envelopes to provide data on human safety supervisor alertness and effectiveness. US20180099678A1 relates to a mobile sensor platform including various hardware and software to perform sensing operations of various occupants in the mobile sensor platform. BACKGROUND OF THE INVENTION Autonomous vehicles (generically, AVs) use computer-based controls to automate at least some aspects of vehicle operation, such as speed control and lane keeping assist. In addition to more conventional sensors, such as speed sensors, AVs commonly rely upon data from sensors such as cameras (visible light, infrared), LIDAR, radar, ultrasound acoustic sensors, and other similar sensors to provide data to perception systems (e.g., detection, classification, and prediction of aspects of the operational environment such as vehicles, pedestrians, other objects, roadway, infrastructure, and weather). In addition to all these types of sensors, humans can function as part of the control system as well. They can participate either as part of moment-to-moment vehicle control (e.g., human steers while vehicle maintains speed) or as a supervisor of safety of one or more autonomous systems. In a combined AY-human system, the human supervisor is responsible for monitoring at least some aspects of AY operation to ensure safety. The human supervisor is generally able to intervene via autonomy disengagement, takeover, override, or other interaction with the vehicle, if necessary, to maintain safe vehicle operation. Failure of the human portion in a combined AV-human system can have tragic consequences. A recent NTSB investigation into a fatal collision between an AV and a pedestrian in Arizona revealed that one of the causes of the incident was the failure of the human supervisor to closely monitor the road and the operation of the AV. Ensuring safety using a human supervisor requires that the human be appropriately alert, able to identify risky situations, and able to successfully react to risky situations to ensure safety. Some available technologies to monitor the human include a perception-based Supervisor Performance Monitoring Systems (SPMS), such as the Mobileye Advanced Driver Assistance System, for real time collision avoidance as known in the art. Such systems employ similar perception capabilities to those in an AV to determine whether a human supervisor is operating the vehicle in a safe manner. For example, a SPMS might monitor the distance between a leading vehicle and the ego vehicle and take action if the following distance is too close to be safe. An example of such a following distance might be determined by a fixed rule-of-thumb such as the two-second or three-second following rule commonly taught in driver's license training materials. Alternately, the following distance might be enforced by a more specific algorithmic approach such as the Responsibility- Sensitive Safety (RSS) algorithm promoted by MOBILEYE®. MOBILEYE® is a registered trademark of Mobileye Vision Technologies Ltd. of Jerusalem, Israel. Enforcement of the following distance is performed via an alert to the human supervisor such as one or more of a flashing light, an audible alarm, a steering wheel shaking device, or the like. Enforcement of safety rules might also be performed by activating emergency vehicle controls such as an Automatic Emergency Braking (AEB) or similar vehicle safety and shutdown capability. Potential detection capabilities not necessarily all addressed by RSS include not only following distance, but also whether lane changes are safe, whether cross-traffic will be avoided at intersections, pedestrian avoidance, other obstacle avoidance, whether speed is appropriate for the road surface conditions, and so on. Another technology is using a camera, steering wheel motion sensor, pedal sensor, or other sensor data to infer whether a human supervisor is paying attention. A Supervisor Monitoring (SM) system uses sensor data to infer human supervisor attentiveness to the dynamic driving task. For example, pressure on the steering wheel might be sensed, or an inward-pointing camera might be used to monitor human supervisor gaze and human supervisor facial expressions to determine if the human supervisor is alert and attentive. While the aforementioned systems and technologies address some of the needs, what is still needed is additional ways to improve the control systems of AVs and to enhance the safety systems incorporated therein. Further, the ability to monitor the performance of human safety supervisors and to detect their failures or inattentiveness would be beneficial. Additionally, the ability to mitigate such failures is desired. SUMMARY OF THE INVENTION In a first aspect, there is provided a computer-implemented m