De Kai
AI Professor @ HKUST CSE / Berkeley ICSI / The Future Society
Table of Contents
Do not index
The Raising AI course companion for a common core university course on AI Ethics.
Grading scheme
- 24% exercises, quizzes, assignments
- 27% midterms
- 25% class participation
- 29% final project, presentation
Syllabus
Chapter 2: Our artificial children
Trolley problems everywhere!
Overview and orientation to topics of fairness, accountability, and transparency in society, AI and machine learning, the impact of AI and automation upon labor and the job market (IEEE foundation of methodologies to guide ethical research and design) CILO-1, 5, 8
LECTURE 1
Provocation:
"The Trolley Problem", The Good Place, s02e05
Required reading:
- RAI ch2
- EAD p9-35, "From Principles to Practice", "General Principles"
Suggested materials:
The Good Place might be just a sitcom, but excellent introductory ethics books have been based on it.
Exercises:
- How should the AIs in self-driving cars make life-and-death decisions when suddenly faced with unexpected real world emergencies?
- Can AIs be trusted to make those decisions?
- Statistics show that self-driving AIs are far less likely to injure or kill people than human drivers. Is it more ethical to allow or to prohibit self-driving cars? (Notice that this dilemma is itself yet another trolley problem!)
- If a self-driving car is at fault in an accident, who is accountable? The owner of the car? The responsible human in the car? The manufacturer of the car? The maker of the AI in the car? Society at large? Nobody?
LECTURE 2
Provocation:
Required reading:
- Edmond Awad et al. (2018). “The Moral Machine experiment,” Nature 563(7729): 59–64.
Suggested materials:
The creator of the Moral Machine, Iyad Rahwan, is interviewed on my podcast.
- De Kai, host (2025). “What have machines learned about human ethics? MIT Moral Machine creator Iyad Rahwan and De Kai”. De Kai on AI, podcast, s01.
Exercises:
- Can you give logical rules to describe how a self-driving AI should make decisions?
- What criteria and objectives should a self-driving AI align to in its decision making?
- Are those culturally dependent?
- What is fairness?
- What happens to human taxi drivers and truck drivers?
Preface: Look up
The sky is falling!
How societies underestimate disasster risks and ignore giant dagners: cognitive biases like normalcy bias, ostrich effect, availability heuristic, and optimism bias CILO-1, 5, 8
Provocation:
Don’t Look Up
Required reading:
Exercises:
- What’s the probability that some human pulls the trigger on the personal WMD they hacked up at home today?
- How much time does humanity have left?
Afterword: The toxic AI cocktail
AI and social disruption
Deepfakes, chatbots, and drones: how AI democratizes weapons of mass destruction and disrupts civilization with information disorder and lethal autonomous weapons CILO-1, 5, 6
Provocation:
Required reading:
- RAI Preface, Afterword
- EAD p68-89, "Well-being"
Suggested materials:
PDF of the following is available at
- Caitlin Andrews. 2025. European Commission withdraws AI Liability Directive from consideration. https://iapp.org/news/a/european-commission-withdraws-ai-liability-directive-from-consideration (retrieved 12 Feb 2025).
- Saad Siddiqui, Kristy Loke, Stephen Clare, Marianne Lu, Aris Richardson, Lujain Ibrahim, Conor McGlynn, and Jeffrey Ding. 2025. Promising Topics for US–China Dialogues on AI Safety and Governance. Technical report, Oxford Martin School, University of Oxford; Safe AI Forum. https://www.oxfordmartin.ox.ac.uk/publications/promising-topics-for-us-china-dialogues-on-ai-safety-and-governance
Exercises: https://forms.gle/kwP1s5QwarPRjaip7
- Discuss how the emergence of AI might alter analyses of Carl Schmitt’s (1932) advocacy for making a “friend-enemy distinction” in The Concept of the Political.
- Contrast how a deontological rule-based AI ethics would look, assuming (a) Schmitt’s “friend-enemy distinction” should be made, versus assuming (b) Schmitt’s “friend-enemy distinction” should not be made.
- Contrast how a consequentialist AI ethics would look, assuming (a) Schmitt’s “friend-enemy distinction” should be made, versus assuming (b) Schmitt’s “friend-enemy distinction” should not be made.
- Contrast how a virtue AI ethics would look, assuming (a) Schmitt’s “friend-enemy distinction” should be made, versus assuming (b) Schmitt’s “friend-enemy distinction” should not be made.
Midterm 1
Chapter 17: Lessons from the history of AI
Paradigms of AI ethics
Descriptive versus prescriptive and predictive ethics; relates classic philosophy of normative/comparative ethics and deontological/consequentialist/virtue ethics to the problem of AI ethics, and discusses why purely rule-based AI ethics will fail (IEEE goal of human rights; IEEE objective of legal frameworks) CILO-1, 2
Provocation:
Required reading:
- RAI ch17
- EAD p36-67, "Classical Ethics in A/IS"
- Fabio Morandín-Ahuerma (2023). “Twenty-three Asilomar Principles for Artificial Intelligence and the Future of Life”. Originally published in Spanish as “Veintitrés principios de Asilomar para la inteligencia artificial y el futuro de la vida. In: F. Morandín (ed.), Principios normativos para una ética de la inteligencia artificial 5–27. Concytep. By CC BY-NC-SA 4.0. https://osf.io/dgnq8/download/?format=pdf.
Suggested materials:
- Future of Life Institute (2017). “Asilomar AI Principles”. Beneficial AI. Asilomar, California. https://futureoflife.org/open-letter/ai-principles/ (retrieved 10 Feb 2025).
- Future of Life Institute (2019). Beneficial AGI. San Juan, Puerto Rico. https://futureoflife.org/event/beneficial-agi-2019/ (retrieved 10 Feb 2025).
- Christoph Salge and Daniel Polani (2017). “Empowerment As Replacement for the Three Laws of Robotics”. Frontiers in Robotics and AI 4, article 25. June.
Exercises:
- Suggest real-world examples of trolley problems where one or more of Asimov’s Laws of Robotics contradict each other.
- Suggest real-world examples of trolley problems where one or more of the Asilomar AI Principles contradict each other.
Final project
Due: 30 Nov 2026 at 11:59pm
Required texts
[RAI] Raising AI: An Essential Guide to Parenting Our Future by De Kai, from MIT Press and Penguin Random House. Hardcover: MIT Press, 13 May 2025, 978-0262049764. eBook: MIT Press, 3 Jun 2025, 978-0262383653. Paperback: MIT Press, 2 Jun 2026, 978-0262054324. Audiobook: Penguin Random House, 8 Sep 2026, 979-8217535613.
[EAD] Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems (1st edition), The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems, by IEEE. 2019.
Reference material
Exercises:
Written by