Where cultural context stands in AI's digital black box

When a group of Harvard researchers published a report last year examining ChatGPT's cultural alignment, they found that the LLM is accustomed to Western, Educated, Industrialised, Rich, and Democratic societies (WEIRD). Specifically, the chatbot was found to be similar to Germany, New Zealand and the U.K. 

Joycelyn David, a Canadian entrepreneur with multiple technology companies, was not surprised by this comparison. Her company, Tulong, works with organisations to further develop models by training on culturally intelligent data.

While it's no secret that AI can be used in functions from financial accounting to storyboard drafting, it's only as "intelligent" as its training data. Bias can still become apparent if the data the tech was trained on is skewed.

"Training on the open internet is that's not a bad thing when you're innovating and early," said David. "But as you scale, what we see is bias scaling at the speed of AI ...  As more clients are relying on those tools. We'd be like - actually, yeah, that tool is a good tool process-wise, but it's lacking in diverse data sets, which means garbage in, garbage out. So we became extremely busy with trying to work with clients who were obviously navigating AI and the use of it, but wanting to culturally fact check to ensure quality assurance."

The introduction of AI in business workflows has not gone without controversy. David pointed to generative AI's use in marketing campaigns as a key risk if the data used to train systems has bias.

Last month, luxury fitness chain Equinox pulled an AI-generated ad campaign amid controversy over its depiction of a disembodied head of an Asian woman on a stand with a finger between her lips, a marketing mistake that David said brought forth harmful hypersexualised associations.

The "Question Everything But Yourself" campaign launched in January 2026 by Los Angeles agency Angry Gods, intended to highlight the prevalence of digital fakery by contrasting AI creations with real, human photography.

This event, she said, brings forth the need for guidance on what data organisations are training their technology on.

"The only way to solve the problem is to ensure that we have some governance around the right data sets that need to be put in. Just like a black box in an airplane ... every AI should have its own equivalent to govern it and to ensure that for safety and for all that use the tool, they can feel a sense of comfort in knowing that those things will protect them at the end of the day."

Marzieh Fadaee, Head of Research at Cohere Labs, the research wing of the Canadian tech giant, led a study on the cultural impact of AI earlier this summer. Riffing off previous findings from a collective of European universities that found that as companies rapidly develop AI technology in the race to be the best, digitally underrepresented cultures are left out - which adds an increased risk of bias and misinformation. 

With 81 responses from over 22 countries, Cohere found that while most participants were fluent in English, over 80 per cent indicated they regularly communicate in other languages for a variety of daily tasks and important functions.

"Social media content generation, legal government advice, learning a new language, job application, medical health questions - all of these different domains we are excited and happy about how these frontier models are impacting and making a big difference, but then when you think about beyond English, the challenges there are still open and are still unexplored," said Fadaee.

From that same study, over 89 per cent reported switching from their primary language to English when they were interacting with AI. Questions posed about a job interview, for example, might give an answer in a native language, but compared to the English answer, it may fall short on quality and context.

"The problem is what is getting lost here. Maybe you were concerned about something specific about job applications in Persian-speaking countries, or there is maybe something different there that you don't end up having on the English answer side," Fadaee added.

Similarly, asking whether Tram 11 has issues in Kajkavian Croatian will get no answer, while in Toronto and English, asking about Line 2 using nicknames like the Green Line or Bloor Line will be understood.

A study by the University of Copenhagen and Korea Advanced Institute of Science & Technology this year examined this case and found that generative models' cultural reasoning is imbalanced, resulting in more confusion with less-represented cultures.

Cohere Labs reported that when prompted in Hindi or Punjabi, AI tends to generate responses according to Western-centric norms in social situations and communication styles. "It fails to capture context in code-switched languages, all of which can make responses feel unnatural and even impolite."

Back at home, Canada's National Artificial Intelligence Strategy: AI for All, released earlier this year, lists one pillar of the plan as "representing and including Canadian voices, languages, and culture." This includes supporting Indigenous leadership, supporting cultural expression and linguistic vitality, enhancing French language functions and advancing open-source AI.

This goal was supported by a CAD $500 million investment to expand and enhance the Regional Artificial Intelligence Initiative. Tulong received over CAD $500,000 of that earlier this month.

One way the company is consulting on culturally intelligent AI, for example, is with banks and other financial institutions to import more culturally representative datasets into systems that could realistically decline some applications if they assume bias.

"In the financial sector, showing bias against applicants because of their name, because of where they studied, or not having a credit history in North America because it's a data set that it doesn't understand - these are all inherent biases that affect, in that example, how people get credit."

David said a recent trip to a large car company in the U.S. really put the nuances of cultural understanding into perspective.

In a room full of executives, she revealed a chart showing the Hofstede cultural dimensions between the U.S. and Canada. The visualisation was nearly identical between the two countries - a near-perfect match on values of egalitarianism, while scoring an exact 68 on indulgence (placing a high value on leisure time, personal happiness, and enjoying life).

"I asked the room, 'Would you say that our culture in Canada and the U.S. is exactly the same?' And most people were like, 'No, no, no, no, we're not.' Absolutely, it's not the same because culture is a lot more dynamic than what studies and sociologists have traditionally done. Culture moves at the speed of likes and moves across borders. As immigration and multiculturalism have fostered multicultural families, multi-generational families, these indexes are not really accurate today."

For David, that room of executives captured the whole argument in miniature: two countries that look statistically alike on paper, and a shared conviction that once the finer-grained data was on the screen, they weren't alike at all.

"So when you look at culture, it's not one-dimensional," she said. "When we think about culture as businesses, culture drives how we make decisions."