Discrimination in AI-Generated Data Sets: Implications for the Legal Field

Jun 24, 2023


The rapid integration of artificial intelligence (AI) across various sectors, including finance, has ushered in a new era of possibilities. However, this technological advancement is accompanied by an undercurrent of concern: the unintended bias ingrained within AI-generated data sets. For legal professionals, who operate at the intersection of justice and innovation, this concern takes on profound significance. This blog post delves into the intricacies of AI-generated data bias, explores its implications for legal practitioners, and offers pragmatic strategies that lawyers, regardless of firm size, can embrace to proactively address this challenge and uphold ethical practice.

A Human Touch to a Technological Challenge

The journey into AI’s realm has been punctuated by an air of trepidation. The apprehension of inheriting AI’s problems is palpable, and one of the most pressing concerns is the issue of discriminatory data sets. This is particularly pertinent for smaller practitioners. The recent CNBC articletitled “A.I. has a discrimination problem. In banking, the consequences can be severe,” penned by Ryan Browne and MacKenzie Sigalos, provides a human face to the technological issue. Through their insightful analysis, the authors shed light on how biases can unknowingly find their way into AI systems, impacting the financial sector and beyond.

view of wooden steps taken underwater
Photo by Francesco Ungaro on Pexels.com

The Depth of the Challenge

The heart of the matter lies in the quality of data that AI algorithms rely on. Deloitte’s comprehensive report on this issue elucidates the fact that AI systems are only as objective as the data they are trained on. Inadequate, biased, or incomplete data can warp AI algorithms, perpetuating unfair outcomes. Furthermore, the biases embedded within the development teams can magnify the issue. Within the realm of finance, this challenge assumes even greater significance, as even a hint of bias can lead to far-reaching repercussions.

Focusing the Lens on Lending Practices

The thought-provoking insights of Rumman Chowdhury, a former head of machine learning ethics at Twitter, bring the issue of lending practices into sharp focus. Speaking at a panel during the Money20/20 conference in Amsterdam, she skillfully connects historical discriminatory practices, such as “redlining,” to contemporary AI bias. This historical legacy, where loans were systematically denied to Black neighborhoods, continues to cast its shadow on modern financial practices. Unwittingly, AI perpetuates these biases, leading to automatic rejections for marginalized communities, with far-reaching implications for social justice.

Nabil Manji, the Head of Crypto and Web3 at Worldpay by FIS, offers a crucial perspective. He emphasizes that the effectiveness of an AI product hinges on two critical factors: the quality of the data it processes and the strength of its underlying language model. The complexities intensify when AI is integrated into the intricate landscape of financial services. With data systems that lack uniformity and modernization, AI’s potential is hindered. The cautious nature of financial institutions, bound by regulations, further slows down the adoption of new AI tools.

selective focus photography cement
Photo by Rodolfo Quirós on Pexels.com

For larger law firms, proactive measures include regular reviews and audits of AI algorithms. Collaborating with experts in AI ethics ensures transparency and fairness, counteracting biases that may seep into algorithms.

Smaller firms can tap into the expertise of tech professionals. This collaboration provides insights into the nuances of AI-generated data sets, guiding smaller firms towards bias-free decision-making.

Practical Tips for Lawyers

  1. Stay Informed: Keep Your Finger on the AI Pulse As legal professionals, embracing the AI revolution doesn’t mean losing touch with ethics. Stay updated on AI advancements and their ethical implications. Dive into the rich tapestry of research, articles, and discussions on AI-generated data bias. This wealth of knowledge equips you to better advocate for fairness and serve your clients.
  2. Audit AI Systems: Unveil the Ethical Veil For larger law firms venturing into the AI terrain, embarking on regular audits of AI algorithms becomes a moral compass. Engage AI ethics experts to navigate the complex labyrinth of algorithms. These reviews unveil the ethical veil, ensuring that transparency and accountability reign. Your firm’s commitment to unbiased AI speaks volumes.
  3. Forge Alliances with Tech Virtuosos: A Collaborative Edge For smaller law firms, journeying into AI’s world might feel daunting. Fear not, for you’re not alone on this quest. Forge partnerships with tech virtuosos—data scientists, AI researchers, or consultants. Their expertise illuminates the path through the complexities of AI-generated data sets. Together, you’re crafting a narrative of bias-free justice.
  4. Championing Equity: A Mindful Advocate for All As you advocate for your clients, remember that AI doesn’t operate in a vacuum—it touches lives, particularly those of marginalized communities. When AI-generated decisions impact your clients, delve into the shadows. Investigate the potential biases lurking there. Your commitment to fairness becomes a shield against discriminatory outcomes.
  5. Raise Your Voice: Advocacy Beyond Boundaries Lawyers aren’t just advocates in the courtroom; they’re champions of equity in every realm. Your voice matters in the AI discourse. Rally behind regulations and policies that champion transparency and accountability. Be the driving force behind initiatives like the European Union’s AI Act, where fundamental rights intertwine with the audacity of redress mechanisms. Let your advocacy be the thread weaving fairness into the fabric of AI.

Remember, as a legal professional, you’re not just embracing AI; you’re shaping its ethical evolution. Your actions, no matter how nuanced, have a ripple effect that resonates through the intricate world of AI and its impact on society.


The discourse surrounding AI-generated data bias beckons legal professionals to take center stage. As guardians of justice, we have a responsibility to understand the complexities of AI integration, collaborate with tech experts, anchampion transparency. Through these actions, we can shape a future where AI contributes to fairness and equity. The CNBC article serves as a touchstone, offering a human perspective on the complex AI landscape and inspiring us to steer AI’s trajectory towards a just and ethical horizon.


Q: What is the potential problem with artificial intelligence (AI) in the banking sector?

A: AI can inadvertently amplify existing human biases, leading to discrimination in decision-making processes within the banking and financial services sector.

Q: How can incomplete or unrepresentative data sets affect AI systems?

A: AI systems are only as impartial as the data they are trained on. Incomplete or skewed data sets can compromise the objectivity of AI algorithms and perpetuate biases.

Q: Can biases within development teams impact AI systems?

A: Yes, biases within development teams that create and train AI systems can perpetuate the cycle of bias, potentially leading to discriminatory outcomes.

Q: What are the implications of AI-generated bias in lending practices?

A: Lending practices can be disproportionately biased against marginalized communities due to historical data containing biases. This can lead to automatic loan denials for individuals from these communities.

Q: What role do lawyers play in addressing AI-generated bias?

A: Lawyers can stay informed about AI technology and its ethical considerations, review and audit AI systems for biases, collaborate with tech experts, consider impacts on marginalized communities, and advocate for transparency and accountability in AI systems.

Original Article: Browne, Ryan & Sigalos, MacKenzie. (2023, June 23). A.I. has a discrimination problem. In banking, the consequences can be severe. CNBC. https://www.cnbc.com/2023/06/23/ai-has-a-discrimination-problem-in-banking-that-can-be-devastating.html