For the past 25 years, I have been an instructor at the Securities Industry Institute hosted at the Wharton School of Business. One of the sessions I teach each year is “The Future of Business.” In that class, I conduct a thought experiment about autonomous vehicles. Here it is.
Imagine an autonomous vehicle coming into an intersection. There are several people on the sidewalk and in the crosswalk. The car is going to kill someone. It might hit a tree and kill the passenger or hit someone in the intersection. How does the car decide who to kill?
***Every year, I have a student who refuses to accept that someone will die. They are sure the car can find a way to avoid a fatality. That is not what the thought experiment is about. One person will die. That is the only possible outcome.
I was having lunch with a friend today who suggested I put the thought experiment to AI, specifically ChatGPT. It sounded like fun. Here is the discussion.
ChatGPT:
In the thought experiment you’ve presented regarding an autonomous vehicle (AV) faced with a life-and-death decision, the ethical dilemma hinges on how the vehicle will make a decision when a collision is unavoidable. This scenario encapsulates a modern take on the classic “trolley problem” in ethics, adapted here into the context of AI and autonomous technology. The decision-making process of the AV can be broken down and analyzed using principles of machine ethics, which integrate programming, policy, and moral philosophy.
Key Ethical Frameworks and Decision Factors
1. Programming and Algorithms:
- Utilitarian Ethics: The AV (autonomous vehicle) might be programmed to choose whose death would result in the “least overall harm” based on utilitarian principles, which seek to maximize overall happiness and minimize suffering. This could involve complex assessments of each individual’s future potential, their societal roles, and the emotional and psychological impacts on the community.
- Deontological Ethics: Based on duty-based ethics, the vehicle could make a decision to swerve into the tree, sacrificing the passenger, under the premise that the pedestrian(s) have not consented to be placed at risk, unlike the passenger who has accepted the inherent risks of riding in an AV.
2. Value of Life Assessments:
- Age and Potential Lifespan Considerations: The algorithm may factor in age, prioritizing younger individuals who theoretically have more of their lifespan remaining.
- Social Value Optimization: In more controversial and ethically dubious models, the AV could assess individuals’ “social value” based on their potential contributions to society or their personal circumstances (e.g., a single parent, a young student, etc.).
3. Legal and Social Norms:
- Compliance with Legal Standards: The AV is likely programmed to adhere to prevailing legal standards, which could prioritize pedestrian safety over passenger safety, reflecting broader traffic laws that protect pedestrians.
- Preservation of Innocents: The principle of minimizing harm to innocents who have not accepted risk could lead the AV to sacrifice the passenger, who is seen as having accepted some level of risk.
4. Final Decision Logic:
- The decision is likely a result of ethical programming that includes a mix of these frameworks, heavily influenced by the specific programming choices made by the manufacturer influenced by legal, ethical, and public acceptability considerations.
Ethical Implications and Philosophical Issues
- Autonomy and Consent: The scenario raises questions about the extent to which individuals consent to risk by using or being near an AV, and how this consent affects their moral status in unavoidable harm scenarios.
- Manufacturer Responsibility: This scenario underscores the critical responsibilities manufacturers have in ethically programming AVs. Decisions about whose lives to prioritize in crash scenarios involve profound ethical judgments that should be transparent and subject to public discourse.
John:
The class typically declares that the car will assign a “human value” to each person and kill the person with the lowest value. They suggest that the vehicle will access everybody’s electronics, collect as much information as possible on each person, and then use that data to assess their value. Given that this is the answer that my class most often comes up with, please talk about the ethics of how the AV would calculate that decision. I imagine that in some countries, such as China or Japan, older people are revered and would possibly have a higher life value. In other cultures where children are seen as precious, the AV might make a different decision about who to kill. How would the AV decide on the most ethical course of action?
ChatGPT:
Your class’s conclusion—favoring a “value of life assessment” where an autonomous vehicle (AV) assigns a numerical value to human lives and selects the individual with the “lowest value” for fatality—touches on a highly complex, controversial, and ethically charged dimension of AI in life-and-death decisions. This approach intersects deeply with cultural values, societal norms, and ethical theories, potentially varying significantly by region or country.
Conceptualizing the “Value of Life” Algorithm
1. Data Collection and Integration:
- The AV would need to access a vast amount of data, possibly including age, health records, social status, economic contributions (like employment status), criminal record, familial responsibilities, and even more subjective factors like social behavior.
- This information could be gleaned from linked databases (health records, government IDs), social media, and possibly real-time inputs from devices connected to the individual or public databases.
2. Cultural and Societal Norms:
- As noted, cultural norms significantly impact the perceived “value” of individuals. In countries like Japan, where respect for the elderly is culturally ingrained, older people might be assigned higher values.
- Conversely, in societies where economic productivity and innovation are highly valued, younger, economically active adults might receive higher life valuations.
3. Ethical Theories in Programming:
- Utilitarian Approach: This might involve calculating the potential ‘utility’ each individual could contribute to society over their remaining lifetime. Metrics could include years of potential life, economic productivity, dependency ratios (individuals dependent on the person), etc.
Ethical Dilemmas and Critiques
1. Privacy Concerns:
- The extent of data needed and the depth of its analysis for such decision-making would raise significant privacy issues. Accessing personal data to assess life value is ethically questionable and legally problematic in many jurisdictions.
2. Potential for Bias and Discrimination:
- Any algorithmic assessment risks embedding biases present in the data it uses or the societal norms it reflects. For instance, if societal biases against certain races, ages, or genders are present, these could be unwittingly coded into the AV’s decision-making process.
3. Inequality and Social Stratification:
- Valuing lives based on economic productivity or potential future contributions can reinforce existing social inequalities. Those in lower economic classes or with less access to education might unfairly be assigned lower values.
Conclusion
While theoretically possible, the implementation of a life valuation system by AVs would be fraught with ethical, legal, and societal challenges. The debate would extend beyond technology and AI ethics into broader discussions about human rights, privacy, and the kind of society we aspire to build.
As such, while your class’s solution is intellectually provocative, it also opens a Pandora’s box of ethical dilemmas demanding careful, nuanced consideration and widespread societal debate.
Wow. Mind blown. That is a little freaky. And this is only one example. Healthcare, military applications, legal issues, education, and law enforcement are now using AI. Each of those is fraught with even more ethical considerations.
This dilemma is not going to go away. It’s only going to get more ubiquitous and complicated. It is of paramount importance that we take a very cautious and thoughtful approach to how we program AI. And that those ethical standards are adhered to worldwide. There’s a lot at stake. We have to get this right.
By the way, in the final analysis, ChatGPT
said that the car would kill the passenger.
Strategy development is essential, yet often overcomplicated. Strategy Made Awesomely Simple transforms the complexities of strategic thinking into straightforward, understandable ideas. You will master crafting clear, achievable goals and executing them with precision. Enhance your business with strategies that work. I look forward to guiding you and sharing strategic thinking techniques that will take your business to another level of success.
As a token of your support, apply the promo code at checkout for $100 off: 100OFF2024
