Ozumcan Demir-Caliskan, ³Ô¹ÏºÚÁÏ College Business School, with Lydia Paine Hagtvedt, Akamai Technologies, Sarah Harvey, University College London, and Henrik Hagtvedt, Boston College.
Problem
Most of what we know about how AI creators handle the ethical consequences of their work comes from after the fact: retrospective accounts of what went wrong, or studies of unrelated ethical choices. We do not know much about what happens inside the creative process itself, in the moment when a creator realises their idea could cause harm. That gap matters. If we want AI to be built more responsibly, we need to understand how the people building it actually think.
Science
We conducted a study of 69 professionals engaged in creative AI work, including software engineers, machine learning engineers and research scientists from both academia and industry. We interviewed each participant asking them about their creative process, their views on the future of AI and how they thought about the potential consequences of their work. We then analysed the transcripts.
Outcome
We found two distinct ways that AI creators navigate the moral consequences of their ideas and called them bright imagining and dark imagining. Creators who engaged in bright imagining focused on the positive potential of their work, distanced themselves from potential harms and addressed ethical concerns through general rules applied across projects, such as avoiding certain domains or calling for industry-wide regulation. Around 50% of creators in the study were inclined towards bright imagining while 30% had a strong preference for dark imagining. Creators who imagined harm used that discomfort as a design constraint, producing technologies that were more carefully built.
Change
Our research reframes the question of AI safety. The problem is not that creators ignore ethical consequences. The problem is that the way they engage, whether they distance harm or confront it, shapes the technologies that get built. Organisations developing AI should consider how they structure the creative process: whether creators are exposed to the real-world consequences of their work, and whether that exposure happens early enough to shape the technology itself. The research was conducted when AI ethical standards were still emerging, making it a foundational study for understanding how responsible AI development works in practice.