New Strategic Design Approach Focuses on Turning AI Mistakes into User Benefits

More and more often, automated lending systems powered by artificial intelligence (AI) reject qualified loan applicants without explanation.

Even worse, they leave rejected applicants with no recourse.

People can have similar experiences when applying for jobs or petitioning their health insurance providers. While AI tools determine the fate of people in difficult situations daily, Upol Ehsan says more thought should be given to challenging these decisions or working around them.

Ehsan, a Georgia Tech explainable AI (XAI) researcher, says many rejection cases are not the applicant’s fault. Rather, it’s more likely a “seam” in the design process — a mismatch between what designers thought the AI could do and what happens in reality.