Rachel Feltman: Recently there has been a lot of hype around artificial intelligence. Some companies want us to believe that machine learning is powerful enough to practically tell the future. But what about using AI to explore the past and talk to members of long-dead civilizations?
In fact American scientific‘s fast science I’m Rachel Feltman. My guest is Michael Varnum, head of social psychology and associate professor at Arizona State University. He is one of the authors of a recent opinion piece that proposes a relatively new use of tools like ChatGPT.
Michael, thank you so much for joining us today.
About supporting science journalism
If you like this article, please consider supporting our award-winning journalism subscribe. By purchasing a subscription, you’re helping to ensure a future of impactful stories about the discoveries and ideas that shape our world.
Michael Varnum: my pleasure Thank you for being with me.
Feltman: So you have this new role, a “machine machine” kind of vibe (laughs). Tell us a little about the problem you’re solving.
Defense: Yeah, so I’ve been interested in thinking about culture change for a long time, and I’ve done a lot of work in that area. But we encounter some limitations when we are trying to understand the mentality or behavior of people who are no longer with us. Of course we don’t have a time machine, do we? We cannot bring back the dead and ask them to participate in our experiments or run us through economic games.
And so usually what people like me have to do is use indirect proxies, right? Maybe we get archival data on things like marriage and divorce or crime, or we look at cultural products like the language people use in books and try to infer what kinds of values people might have or what feelings they might have. they had with different groups. But all this is indirect.
What would be amazing is if we could get the kinds of data that we get from people today, say, you know, from the ancient Romans or the Vikings or the medieval Persians. And one thing that really excited me in the last year or two is that people started realizing that they could at least simulate modern participants with programs like ChatGPT and surprisingly, and I think excitingly, replicate a lot of the classic effects in the behavioral sciences.
And so we thought, “Well, if we can do this based on these patterns created from the writings of modern people, maybe we can do it based on the writings of ancient people. This can open up a whole new world of possibilities.”
Feltman: Yeah, could you tell me a little bit more about some of these experiments that have replicated psychological phenomena using language learning models?
Defense: One of the most powerful set out to replicate 70 different large-scale survey experiments with simulated ChatGPT participants, and found that the results correlated about 0.9 with what people observed with real humans. And of course, this is not what anyone designed Llama or ChatGPT to do…
Feltman: Mm-hmm.
Defense: But in the process of making these models that are able to speak to us in a very natural way, they seem to have picked up quite a bit of human psychology.
Feltman: And you mention in the paper that some people are already using historical texts to work on large language models, so what kind of things are they doing so far?
Defense: So far it’s just baby steps.
Feltman: Mm-hmm.
Defense: People are trying to see, “Okay, if we train a model based on medieval European texts, what about the solar system or, you know, medicine or biology?” And they have the wrong number of planets. They believe in the four humors of the body.
So far, as far as I know, no one has really run these kinds of models adjusted through modern experiments or surveys, but I think that’s going to start happening soon, and I’m excited to see what people come up with. .
Feltman: yes So one thing that came to mind when I was reading your paper was the inherent bias that we see in the fossil record. You know, our sense of what life looked like in the past is influenced by what’s preserved, and that’s affected by all sorts of factors, like climate and the bodies of the organisms we’re talking about. And, you know, I imagine that in most times and places in history, certain people were overrepresented in written texts. So how do you suggest researchers navigate that to make sure we don’t get a really, you know, biased sense of what people were like?
Defense: That’s a very daunting challenge for this kind of proposal…
Feltman: Mm-hmm.
Defense: Because, well, for most of human history, no one was literate, right? Writing is relatively new…
Feltman: right
Defense: And by the time some societies had writing, very few people knew how to read and write. Even fewer wrote things that survive into the modern era. And so the data you’re getting is data that’s going to be skewed towards people who are more elite and more educated.
Feltman: Hmm
Defense: And we think there might be a couple of ways to deal with that, and they’re imperfect, right? But maybe if we use them in combination, we will still be able to resist a little bit of the trend that will come into these patterns.
One way is that we know quite a bit about how things like social class affect the psychology of modern populations…
Feltman: Hmm
Defense: So potentially we could fine-tune those models a little bit, or run them through experiments and surveys and then wait for their responses to try to account for that bias. You know that in some cases we have other sources of historical records and analysis. To the extent that they perhaps capture some of the mentalities or behavioral patterns of past populations, we could see whether the results of these large historical linguistic patterns are consistent with these kinds of findings. But it is difficult, for sure. This will be a real challenge to overcome.
Feltman: Yes, and of course, this would not be a challenge related to using historical data. This is a challenge we also see in training LLMs with modern data.
Defense: Oh, absolutely, right? And, you know, one of the things that inspired this idea is some of the work of people like Mohammad Atari and Yan Tao, that the current big language models look really CRAZY because they match the psychology of people in the West. and Anglophone populations than in many other parts of the world, and I mean, well, that makes sense, doesn’t it, given that training data overrepresents those societies. But it’s also exciting because it suggests that if you had a different kind of corpus, then you could capture that cultural zeitgeist and the specific cultural mentality of the people who produced it.
Feltman: Yeah, could you tell people what WEIRD means in this context? Because I think it’s a really good acronym (laughs), so…
Defense: Yes, so it’s an acronym that Joe Henrich developed about a decade and a half ago, and it means western, educated, industrialized, wealthy (and democratic). Therefore, the minority of humans today live in such societies.
Feltman: Mm-hmm.
Defense: But depending on how you slice it, the majority of behavioral science participants come from these samples.
And that matters because culture affects the way we think and act in so many ways, from the values we hold to the distance we like between people, to basic patterns of visual attention and cognition, rates. of cooperation It’s a very long list.
Feltman: No, and I mean, I can certainly imagine—you know, obviously it’s very suggestive to talk about ancient history, but I can certainly imagine researchers trying to use, you know, some, like, 19, 20, even 21. .the text of the 20th century, the text of underrepresented groups, you know, may have left large parts of the population to reexamine these psychological studies. out
Defense: Yes, I think it’s a very good idea. And somehow, the further you go back into the past, the easier it becomes to do this kind of research.
Feltman: yes
Defense: So while it’s exciting to think about going very, very far back, probably the beginning, you know, the starting point is, “Let’s look back 100 years or 150.”
Feltman: Mm, yeah… well, and speaking of that, you know, imagine for a moment that this whole idea takes off and we’re running some undead psychology projects, you know, what are yours? , dream use cases for that?
Defense: So I do a lot of research based on evolutionary psychology.
Feltman: Mm-hmm.
Defense: And sometimes we’ll do an experiment or a survey, and we’ll try to, you know, get data from every continent in the world, right, to see if some part of human psychology is universal. And when we find it, it’s really exciting, but we’re making a leap of inference from saying, “It’s universal, and it makes adaptive sense,” to “That’s how people thought in the past, and especially. in the deep past.”
And so being able to push back that time window…
Feltman: Mm-hmm.
Defense: You know, well, (Douglas) Kenrick and (David) Schmitt and others have found differences between men and women in their preferred sexual strategies: You know, do you want lots of partners and non-committal relationships or do you want more exclusive relationships and fewer partners? This seems to be true all over the world, but I think we could have a lot more confidence that these things are a fundamental part of human nature if we started looking at societies that lived hundreds or thousands of years ago.
Feltman: Totally
Defense: The idea is forward-looking and speculative, isn’t it? I, for one, don’t have any of these things ready to run on my computer, but Sachin Banker and colleagues recently published a paper in which the GPT-4 generated dozens of new hypotheses for social psychology research, and then actual social psychologists generated new ones. assumptions
Feltman: Hmm
Defense: And other social psychologists thought that AI produced more plausible and possibly truer ideas.
Feltman: Mm, interesting.
Defense: So in the future we can see AI being used not only to simulate participants or code data but also to generate ideas, and you can imagine these weird closed loops where people like me could be out of a job.
Feltman: (Laughs) Well, I hope not. I think, you know, there’s always going to be room for that unique human factor. But I think it’s great to think about the ways AI can be an interesting tool for us, so thank you so much for taking the time to come chat with us today.
Defense: Oh, thank you, Rachel. This was my pleasure. I enjoyed the interview.
Feltman: That’s all for this week’s Friday Fascination. We’ll be back on Monday with the weekly news roundup. And on Wednesday we’re talking about something as scary as AI ghosts: the psychology of Black Friday shopping.
Fast Science produced by me, Rachel Feltman, along with Fonda Mwangi, Kelso Harper, Madison Goldberg and Jeff DelViscio. Shayna Posses and Aaron Shattuck check out our show. Our theme music was composed by Dominic Smith. subscribe American scientific for more up-to-date and in-depth science news.
In fact american scientific this is Rachel Feltman. Have a great weekend!
