Who Gets to Decide What is Ethical Health AI?

Why communities, not just experts, must shape the values, trade-offs, and cultural norms that define what ethical and responsible Health AI really means.

An issue that makes health ethics difficult to pin-down is the fact that what is considered ethical is a societally shared construct. Ethics are always a representation of the value-judgement of the community on a given issue at any given time.

Consequently, when the experts, researchers, or health practitioners decide what is ethical for their patients or communities at large, all the while never really bothering to go to the patients and the communities to ask them what they think about these matters, it becomes a problem.

This is especially problematic because the ethics are ostensibly always to advance patient and community interests. And the “experts” and the “people” may have a different opinion on what a patient’s interests are. We have made such mistakes before.

A similar mistake could transpire again in Health Artificial Intelligence (Health AI). A lot of the global discourse on AI in general, and especially in health, centers around “Ethical or Responsible or Trustworthy AI”. Except that we still haven’t been able to decide what is ethical or responsible and what is not. There are synthetic frameworks of what is considered responsible or ethical or trustworthy AI. The issue is that they are mostly constructed from desk research and consensus building approaches amongst experts and researchers.

And almost always these frameworks and standards are constructed without enough consultation with the community. As a result, there are strong rhetorics around “privacy” and “algorithmic-soverignty” without so much as a question about what the real end-users, beneficiaries, and patients feel about these abstract constructs.

Constructs around autonomy, privacy, and sovereignty are socio-cultural constructs, and their normative standards are heavily influenced by the existing milieu these constructs are implemented in. Because the current debate on the ethics of AI has been so dominated by the prevailing western sociocultural norms and beliefs, western values and normative standards heavily influence these norms, values, standards and the resulting frameworks. As a result current frameworks on ethical or responsible AI may not adequately apply everywhere.

For example, in many south asian societies, patients and their families often prefer that a doctor share a life-threatening medical diagnosis with the family first so that they can best decide how best to share that information with the patient. In a culture such as this, norms and values around patient privacy are different from what they are in western societies, where medical information is not shared with anyone else without the patient’s express permission. It is hardly surprising then, that these two communities could have different ideas about ethical or responsible AI and how Health AI should be deployed.

People and communities need to make collective decisions on these issues based on their own values and context, in cognizance of the trade-offs and the value-choices that they are making. We cannot indeed foist a harmful piece of technology upon people, and at the same time we also can’t let the discourse on ethics become an exercise where people force their beliefs upon others.

If the role of ethics is ostensibly to protect people’s interests, people should have a say in deciding what their interests are, and by extension, what is ethical versus what is not.

---

---

Kiran Raj Pandey is a physician and a health services & systems researcher. Learn more about him and his work at kiranrajpandey.com.