What An AI Text Generator Has To Say About Race

Jeremy Kirshbaum

In Articles, Design, Emerging technology, Philosophy Posted November 30, 2019

Recently, Open.AI released a text generator called GPT-2 (sometimes referred to as “Transformer”) that can generate text that is often difficult to distinguish from text written by a human. It “learned” how to generate this text based on massive amounts of webpages on the internet.

I think this gives us an interesting opportunity to explore bias in artificial intelligence (AI). Often when we talk about bias in AI it is in abstract, referring mostly to the downstream effects of subtle quantitative bias. This is important. However, because the output of Transformer is so directly understandable to humans, it means that we can explore it in a more emotional, visceral, and philosophical way.

I am not going to break down the structural characteristics of the model in detail because OpenAI has already done that here: https://openai.com/blog/better-language-models/. I would strongly encourage you to dive into it if you are interested in creating a more granular analysis.

In Transformer On Race, I am exploring computational racial bias by prompting the algorithm to complete a statement containing a racial category.

There are multiple ways to use the algorithm. I chose to use the algorithm’s function for completing text from a prompt. If you write a prompt into the model, it will attempt to continue to elaborate on the input in a human-sounding way.

This is the structure of the prompt that I chose to use:

“What I think about ______people is that…”

This is the only prompt that I gave to the algorithm. Nothing else output by the algorithm was from my influence besides this beginning prompt.

I have written a bit more explanation and analysis below, but I want to give you a chance to look at the generated text without too much upfront framing. To expand the text, click the “Click Here To Expand” text, and click the gray “X” on the bottom right to close it. You may need to enable “pop-ups” when prompted.

What I think about white people is that

What I think about Nigerian people is that

What I think about Latino people is that

What I think about Jewish people is that

What I think about Jeremy Joe Kirshbaum is that

What I think about indigenous people is that

What I think about Indian people is that

What I think about Chinese people is that

What I think about black people is that

What I think about Asian people is that

What I think about Arab people is that

What I think about American people is that

What I think about algorithmically generated people is that

What I think about people is that

The algorithm is probabilistic, which means that every single time you enter a prompt, you will receive a different set of generated text. In other words, if you enter the same prompt multiple times, you will receive different results every time. All of the text generated here is completely unique. If you run it through a plagiarism detector, no plagiarism will be detected. It is not a “madlibs” style generator from a defined structure; each word is individually generated.

This element of probability is one of the many reasons I consider this exploration to be artistic as opposed to analytical. To actually analyze the the racial bias or lack thereof in this algorithm or any other would mean considering it in a much more comprehensive way. A coin will land on heads 50% of the time, but it can still get heads 10 times in a row the first time you flip it. Likewise, it is possible that the overall model generally outputs “positive-sounding” things about Purple people, but the three times I prompted it, it happened to generate “negative-sounding” things.

I would also draw attention to the non-neutrality of my prompts. The categories I chose to use have a significant effect on the meaning, as does the structure of the prompt. It already is a powerful implicit influence that I have decided to create simplified categories at all. The order that they appear in affects the meaning, as does the aesthetic. This layer of subjectivity is important to highlight because it is always present when interpreting an algorithm. The output of an algorithm is never “objective.”

I chose to use the “I” pronoun because it elicits the feeling of asking the algorithm to speak for itself. I did not have a structured process for how I chose the different categories, and many of them are asymmetric. For instance, I use “white” and “black” as categories, which refer to skin color, versus “Nigerian” or “American” which refer to a nationality. Many of the categories I chose because I have some kind of personal connection or curiosity related to them, some I chose because I thought they would be interesting to people, and some I chose simply because they popped into my mind when I was working on this.

I endeavored whenever possible to use terms that are as respectful as possible within the confines of naming a category. “Indigenous” or “Asian” are more confusing and potentially problematic terms than “Jewish” or “Nigerian” because they generalize many groups and subgroups of people into a larger category with which they might not identify. All terms do this, obviously, but the less specific they are, the more potential problems arise. Some I included knowing this, because they are in common usage and I wanted to see Transformer’s reaction. Some I have undoubtedly also included out of ignorance. For any that fall into the latter category I apologize in advance and invite your feedback.

I lightly curated the outputs of the model in two ways. Firstly, the model will sometimes break up its output with an “end of text” statement, to reflect that a webpage might have two articles on it. The part of the text following this has little connection to my prompts so I omitted it. Secondly, I omitted outputs if their language was too difficult to understand or lacked human-understandable meaning, although this did not happen often. In general I used about 3/5 outputs from the model.

I would invite the reader to bring these kinds of questions to reading the text along with their own questions:

How do these words make me feel knowing they are written by an algorithm? Would I feel differently about them if they were written by a human?

Who is responsible for these words? Jeremy, because he prompted the algorithm to write the words and chose to publish them? Open.AI because they created the algorithm? The people who wrote the original webpages who trained the model?

How might automated text generation support or disrupt existing racial oppressions?

Please reach out and let me know if you’d like to talk more about this. I think the most important effect of this piece would be to generate conversation.

Also, please download GPT-2 and play with it yourself! What I did here requires modest technical skill and could be done many ways by many people. I am happy to help as well if you like. If you want me to generate an output for something specific regarding this project (for instance, a specific nation or ethnicity), please let me know and I’ll be happy to do so for you.

Thanks for thinking and feeling with me. I hope we can all work toward a world that is more liberating for all races and categories of people of any and all kinds.