There is a story we are frequently told about artificial intelligence. It goes something like this: a brilliant system, trained on the sum of human knowledge, can now write your emails, summarise legal documents, explain complex medical conditions, and generate code faster than any human engineer. It is framed as a product. A tool. A service. Something that simply works.
What is often left out of that story is the one that precedes it. Before the product, there were people. Hundreds of them, in some cases thousands. Making decisions. Drawing lines. Holding opinions. Operating with blind spots they may not even recognise as their own.
This post is in two parts. The first covers what a large language model actually is and how it comes to exist. The second pulls back the curtain on the humans behind it, and why their choices, values, and biases are present in every output these systems produce, whether we acknowledge that or not.
When we strip the human factor out of how we talk about AI, we do not just lose nuance. We lose the capacity to ask the questions that matter most.
Part One: What Is a Large Language Model, and Where Does It Come From?
A large language model, or LLM, is a type of artificial intelligence trained to process and generate human language. When you type a question into ChatGPT, Claude, or Gemini and receive a coherent, fluent response, you are interacting with one of these systems. They can write, explain, summarise, translate, code, and converse. To many users, they feel almost magical.
The process that produces them is considerably less mysterious, but understanding it changes how you see the outputs.
Step 1: Someone decides what problem to solve
Every LLM begins with a motivation. Founders and executives decide to invest. Research leads set the technical direction. Product managers define target users and use cases. These are not neutral decisions. They reflect commercial priorities, competitive pressures, and particular visions of whose needs matter most.
The decision to build a model optimised for coding assistance, for example, rather than medical communication or accessibility for non-English speakers, is a choice with consequences. It signals which problems the builders considered pressing, and often, which users they imagined first.
Step 2: Collecting and cleaning the data
LLMs learn by reading text. Enormous quantities of it: books, websites, code repositories, academic papers, social media, and more. Before that text is used for training, it is filtered. Engineers remove spam, hate speech, and private information. Curators design instruction datasets to shape how the model responds.
These sound like purely technical tasks. They are not. Every filtering decision reflects a judgement about what language is acceptable, which voices are worth including, and what the model should consider normal. That data cleaning process is where many of the most consequential human choices are made, often invisibly.
Step 3: Training the model
The model is fed that processed text and learns to predict what word or phrase is most likely to come next, billions of times over. The result is a system that has developed a sophisticated statistical understanding of language patterns. At this stage, the model can continue text, but it does not yet know how to follow instructions or behave helpfully. It is, as researchers sometimes put it, a general language brain.
Step 4: Fine-tuning and teaching preferences
This is where the model is shaped into a product. Through a process called instruction tuning, it is trained on examples of prompts and ideal responses. A significant component involves a technique known as Reinforcement Learning from Human Feedback, or RLHF: human raters read pairs of model responses and indicate which they prefer. The model learns to produce outputs that humans judge as better.
This is also where the model learns what it will and will not do. Safety researchers test for harmful behaviour. Policy teams set rules for what the model may say. These guardrails are genuinely important, but as we will explore in Part Two, the people setting them are making choices that reflect particular values, not universal ones.
Step 5: Evaluation and release
Before release, models are tested against benchmarks: standardised assessments of reasoning, mathematics, language, and coding ability. Teams attempt to find failures through adversarial testing. A version is released, feedback is collected, and the cycle begins again.
What you receive as a finished product, the sleek chat interface, the confident responses, the seamless API, is the end of a long chain of human decisions. Understanding that chain is the starting point for understanding AI critically.
Part Two: The Humans Behind the Model, and Why Their Fingerprints Are on Everything
When AI tools are described in product terms, the humans who built them tend to disappear from the frame. The model becomes the subject: what it can do, how well it performs, how it compares to competitors. This framing is convenient for marketing purposes. It is less useful for anyone trying to think clearly about how these systems shape information, decisions, and society.
Here is what that framing tends to obscure.
The data is not neutral, and neither is the cleaning of it
Large language models learn from text that reflects the world as it has been documented, primarily in English, primarily from Western internet sources, with all of the biases, gaps, and distortions that entails. Research has documented consistently that models perform less accurately for speakers of languages other than English, reflect cultural assumptions embedded in the training corpus, and reproduce historical biases present in the source material.
The filtering process does not resolve this. Decisions about what counts as hate speech vary by language and cultural context. The choice of which instruction datasets to include, what counts as a high-quality response, and whose writing is treated as the standard all shape what the model learns to value. These choices are made by a relatively small group of people, and they scale to millions of users.
The preferences encoded in RLHF are someone’s preferences
The human raters who provide feedback during fine-tuning are frequently contractors, often based in countries where labour costs are lower, evaluating responses to prompts they may not have deep context for, using rubrics designed by someone else. Their cultural background, their assumptions about what a good answer looks like, their tolerance for ambiguity, all of this gets encoded into the model at scale.
There is a structural issue worth naming here. The model learns to produce responses that feel satisfying to the people rating them. That is not the same as learning to produce responses that are accurate, fair, or genuinely useful across diverse contexts. Fluent and confident can score highly even when subtly wrong. Appropriately hedged or genuinely complex may score lower because it is less immediately pleasing. This dynamic, where apparent quality is optimised over actual quality, has real consequences for users who rely on these systems.
The model learns to produce responses that feel satisfying. That is not the same as responses that are accurate, fair, or useful across diverse contexts.
Someone decided what the model will not say, and who decides is worth asking
Every major language model has a safety and policy layer. Teams set rules for what the model may and may not do. These are genuinely important functions, and the people doing this work are, in most cases, thinking carefully about real harms. But they are also exercising significant power.
The decisions made in these teams determine, in effect, what kinds of speech an AI will engage with, which topics get suppressed or softened, and how the system navigates contested political, social, and ethical terrain. When those decisions are made within a small number of organisations, without meaningful input from the communities most affected by the outputs, they represent a concentration of influence over public discourse that has no real historical precedent.
The reasoning behind these decisions is rarely published in full. The model presents as authoritative. The choices that made it that way are largely invisible to users.
Benchmarks measure what they measure, not everything that matters
Before release, models are assessed against standardised tests. These benchmarks are useful tools, but they have limits. A model can score highly on reasoning and language tasks while consistently producing outputs that are biased along lines of gender, ethnicity, or economic background. It can appear helpful in evaluation while failing users with non-standard language patterns or cultural contexts outside the testing set.
The gap between benchmark performance and real-world fairness is one of the most significant unresolved challenges in AI development. It matters because that gap is often invisible to the end user, who has no visibility into evaluation methodology and no reason to distrust a fluent, confident response.
The people building these systems are not representative of the people using them
The AI industry remains heavily concentrated. Demographically, geographically, and professionally. The teams making foundational decisions about how these systems work tend to share certain educational backgrounds, work in certain cities, and bring certain assumptions about what technology is for and who it is for.
This is not an accusation. It is a structural observation. Homogeneous teams making decisions that affect diverse populations produce systems that reflect that homogeneity. This is well-documented across many domains of technology design, and there is no reason to believe AI development is exempt.
Iteration is not the same as accountability
LLM development is cyclical. User feedback is collected, training methods are updated, new versions are released. This is presented, often, as a form of responsiveness. And it is, in a limited sense. But iteration within a closed loop is not the same as accountability to the public whose language, labour, and cultural production these systems have learned from.
The people whose data trained these models did not consent to that use in any meaningful sense. The communities most likely to be harmed by AI bias or misinformation are rarely the ones consulted during development. The feedback that shapes model updates tends to come disproportionately from users who are already well-served by the system.
Why This Matters Beyond the Technical
The reason to understand all of this is not to stop using these tools. They are, in many contexts, genuinely useful. The reason is to use them with eyes open.
When a language model tells you something, it is not a neutral oracle reporting objective truth. It is a statistical system shaped by human choices, trained on human-generated text that reflects human history, filtered and fine-tuned according to human preferences, and governed by rules set by humans operating within particular institutional constraints. All of that is present in the output, invisibly, every time.
That matters in high-stakes contexts: medical information, legal guidance, hiring decisions, credit assessments, content moderation. It matters in lower-stakes contexts too, because the cumulative effect of millions of interactions with systems that embed particular assumptions is a slow, quiet reshaping of what feels normal, authoritative, and true.
A language model is not a neutral oracle. It is a statistical system shaped by human choices, trained on human text, filtered according to human preferences, and governed by rules set by humans with particular institutional constraints. All of that is in the output, invisibly, every time.
The people building these systems are not, in the main, acting in bad faith. But good intentions do not neutralise structural blind spots. And the structural blind spots in AI development are significant.
Asking who built this, why, with whose data, according to whose values, and with what oversight is not technophobia. It is the basic critical literacy that this moment requires.
0 Comments