US-20260125079-A1 - INTEGRATING HUMAN AND AI PREFERENCES IN AUTONOMOUS VEHICLES

Abstract

A computer-implemented method, system, and computer program product for autonomous vehicle ethical decision-making. A dataset of human moral judgements regarding autonomous vehicle ethical dilemmas is obtained, such as via a moral machine framework. Furthermore, a reinforcement learning (RL) agent is trained using the dataset to determine a preferred ethical action in a given dilemma. As a result of such training, the trained RL agent is responsible for synthesizing the human-preferred choices from the dataset into a functional policy. The preferred ethical action in a given action that was determined by the trained RL agent is then executed to control the autonomous vehicle (AV). For example, the RL agent's ethically-informed decisions directly govern the AV's behavior, such as steering or braking. Such an execution of the preferred ethical action translates the theoretical moral policy trained on human preferences into an on-the-road control command that influences the vehicle's operation in real-time.

Inventors

• Heena Rathore
• Henry Griffith

Assignees

• TEXAS STATE UNIVERSITY

Dates

Publication Date

20260507

Application Date

20251106

Claims (20)

1. 1 . A computer-implemented method for autonomous vehicle ethical decision-making, the method comprising: obtaining a dataset of human moral judgements regarding autonomous vehicle ethical dilemmas, wherein said dataset is collected via a moral machine framework; training a reinforcement learning agent using said dataset to determine a preferred ethical action in a given dilemma; and executing said preferred ethical action determined by said trained reinforcement learning agent to control an autonomous vehicle.
2. 2 . The method as recited in claim 1 further comprising: quantifying human preference within said dataset by integrating a Bradley-Terry (BT) model within said moral machine framework to perform pairwise comparisons on moral scenarios and generate strength parameters for potential actions.
3. 3 . The method as recited in claim 2 further comprising: converting said generated strength parameters into credence values; and integrating said credence values into a reward function of said reinforcement learning agent to guide its decision-making process.
4. 4 . The method as recited in claim 1 further comprising: guiding a voting mechanism by human-preferred credence values derived from said dataset to influence decision-making of said reinforcement learning agent.
5. 5 . The method as cited in claim 1 , wherein said training of said reinforcement learning agent comprises: utilizing a large language model (LLM) to simulate complex moral reasoning based on said dataset by considering demographic distinctions in human preferences.
6. 6 . The method as recited in claim 5 , wherein said LLM simulation is guided by engineered prompts that direct said LLM to consider a plurality of ethical theories to enhance human-value alignment of ethical decisions.
7. 7 . The method as recited in claim 1 further comprising: quantifying an ethical outcome of potential actions under utilitarian and deontological theories by assigning numerical severity weights to different actions within specific moral scenarios.
8. 8 . A computer program product for autonomous vehicle ethical decision-making, the computer program product comprising one or more computer readable storage mediums having program code embodied therewith, the program code comprising programming instructions for: obtaining a dataset of human moral judgements regarding autonomous vehicle ethical dilemmas, wherein said dataset is collected via a moral machine framework; training a reinforcement learning agent using said dataset to determine a preferred ethical action in a given dilemma; and executing said preferred ethical action determined by said trained reinforcement learning agent to control an autonomous vehicle.
9. 9 . The computer program product as recited in claim 8 , wherein the program code further comprises the programming instructions for: quantifying human preference within said dataset by integrating a Bradley-Terry (BT) model within said moral machine framework to perform pairwise comparisons on moral scenarios and generate strength parameters for potential actions.
10. 10 . The computer program product as recited in claim 9 , wherein the program code further comprises the programming instructions for: converting said generated strength parameters into credence values; and integrating said credence values into a reward function of said reinforcement learning agent to guide its decision-making process.
11. 11 . The computer program product as recited in claim 8 , wherein the program code further comprises the programming instructions for: guiding a voting mechanism by human-preferred credence values derived from said dataset to influence decision-making of said reinforcement learning agent.
12. 12 . The computer program product as cited in claim 8 , wherein said training of said reinforcement learning agent comprises: utilizing a large language model (LLM) to simulate complex moral reasoning based on said dataset by considering demographic distinctions in human preferences.
13. 13 . The computer program product as recited in claim 12 , wherein said LLM simulation is guided by engineered prompts that direct said LLM to consider a plurality of ethical theories to enhance human-value alignment of ethical decisions.
14. 14 . The computer program product as recited in claim 8 , wherein the program code further comprises the programming instructions for: quantifying an ethical outcome of potential actions under utilitarian and deontological theories by assigning numerical severity weights to different actions within specific moral scenarios.
15. 15 . A system, comprising: a memory for storing a computer program for autonomous vehicle ethical decision-making; and a processor connected to the memory, wherein the processor is configured to execute program instructions of the computer program comprising: obtaining a dataset of human moral judgements regarding autonomous vehicle ethical dilemmas, wherein said dataset is collected via a moral machine framework; training a reinforcement learning agent using said dataset to determine a preferred ethical action in a given dilemma; and executing said preferred ethical action determined by said trained reinforcement learning agent to control an autonomous vehicle.
16. 16 . The system as recited in claim 15 , wherein the program instructions of the computer program further comprise: quantifying human preference within said dataset by integrating a Bradley-Terry (BT) model within said moral machine framework to perform pairwise comparisons on moral scenarios and generate strength parameters for potential actions.
17. 17 . The system as recited in claim 16 , wherein the program instructions of the computer program further comprise: converting said generated strength parameters into credence values; and integrating said credence values into a reward function of said reinforcement learning agent to guide its decision-making process.
18. 18 . The system as recited in claim 15 , wherein the program instructions of the computer program further comprise: guiding a voting mechanism by human-preferred credence values derived from said dataset to influence decision-making of said reinforcement learning agent.
19. 19 . The system as cited in claim 15 , wherein said training of said reinforcement learning agent comprises: utilizing a large language model (LLM) to simulate complex moral reasoning based on said dataset by considering demographic distinctions in human preferences.
20. 20 . The system as recited in claim 19 , wherein said LLM simulation is guided by engineered prompts that direct said LLM to consider a plurality of ethical theories to enhance human-value alignment of ethical decisions.

Description

TECHNICAL FIELD The present disclosure relates generally to autonomous vehicles, and more particularly to integrating human and artificial intelligence (AI) preferences in autonomous vehicles. BACKGROUND Autonomous vehicles (AVs), also known as driverless or self-driving cars, are vehicles that can operate with little or no human input. They use sensors, cameras, and complex software to perceive their environment, make driving decisions, and perform actions, such as steering, accelerating, and braking. This technology can be applied to a wide range of vehicles, from cars and shuttles to trucks and buses. The rapid advancement of autonomous vehicles presents a critical challenge in ensuring their ethical decision-making capabilities, particularly in scenarios involving moral uncertainty and high stakes. Current approaches to AV decision-making primarily rely on established ethical frameworks, such as utilitarianism (maximizing overall well-being) or deontology (adherence to rules and duties). However, these rule-based systems often struggle with nuanced human ethical preferences and lack the adaptability to handle morally complex situations that may involve demographic-based decision biases (e.g., differences based on age or gender). This limitation poses a significant hurdle to societal acceptance and trustworthiness of AV technology as the public expects transparent and ethically aligned decision-making. SUMMARY In one embodiment of the present disclosure, a computer-implemented method for autonomous vehicle ethical decision-making comprises obtaining a dataset of human moral judgements regarding autonomous vehicle ethical dilemmas, where the dataset is collected via a moral machine framework. The method further comprises training a reinforcement learning agent using the dataset to determine a preferred ethical action in a given dilemma. The method additionally comprises executing the preferred ethical action determined by the trained reinforcement learning agent to control an autonomous vehicle. Other forms of the embodiment of the computer-implemented method described above are in a system and in a computer program product. The foregoing has outlined rather generally the features and technical advantages of one or more embodiments of the present disclosure in order that the detailed description of the present disclosure that follows may be better understood. Additional features and advantages of the present disclosure will be described hereinafter which may form the subject of the claims of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS A better understanding of the present disclosure can be obtained when the following detailed description is considered in conjunction with the following drawings, in which: FIG. 1 illustrates the internal components of an autonomous vehicle in accordance with an embodiment of the present disclosure; FIG. 2 is a diagram of the software components used by the autonomous driving compute system to enhance moral decision-making capabilities of autonomous vehicles in accordance with an embodiment of the present disclosure; FIG. 3 is a flowchart of a method for enhancing more decision-making capabilities of autonomous vehicles in accordance with an embodiment of the present disclosure; and FIG. 4 is a flowchart of a method for training the reinforcement learning (RL) agent to determine a preferred ethical action in a given dilemma in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION As stated above, autonomous vehicles (AVs), also known as driverless or self-driving cars, are vehicles that can operate with little or no human input. They use sensors, cameras, and complex software to perceive their environment, make driving decisions, and perform actions, such as steering, accelerating, and braking. This technology can be applied to a wide range of vehicles, from cars and shuttles to trucks and buses. The rapid advancement of autonomous vehicles presents a critical challenge in ensuring their ethical decision-making capabilities, particularly in scenarios involving moral uncertainty and high stakes. Current approaches to AV decision-making primarily rely on established ethical frameworks, such as utilitarianism (maximizing overall well-being) or deontology (adherence to rules and duties). However, these rule-based systems often struggle with nuanced human ethical preferences and lack the adaptability to handle morally complex situations that may involve demographic-based decision biases (e.g., differences based on age or gender). This limitation poses a significant hurdle to societal acceptance and trustworthiness of AV technology as the public expects transparent and ethically aligned decision-making. The embodiments of the present disclosure provide a means for providing a novel, integrated framework that addresses this gap by directly embedding human moral preferences into machine learning models for AV decision-making. Specifically, in on