The Importance of Human Beliefs in Decision Making and Platform Design

When it comes to understanding how people make decisions, there are many factors at play. From cognitive biases to personal experiences, each individual's thought process is unique and influenced by their own set of beliefs. According to Celeste Wade, a researcher who has spent years studying human behavior, one of the most significant influences on decision making is the way information is presented.

"Words can be used in many different ways," says Celeste. "If you have a weird or deviant definition of something, that's like very different for other people in the population." This concept highlights the importance of considering how language and communication impact our understanding of complex concepts. When we hear someone using words in a way that we don't recognize, it can be confusing or even lead to misunderstandings. However, Celeste suggests that those who are more aware of deviant usage may actually have a better chance of grasping the concept.

This idea is closely related to the Kruger effect, which states that people are more likely to overestimate their own abilities and knowledge. When we believe that we're competent in a particular area, we tend to be more confident in our understanding of it. Conversely, if we feel incompetent, we may become less aware of areas where we need improvement. Celeste notes that this effect can have significant implications for decision making, as people who are overconfident in their abilities may make decisions without fully considering the potential consequences.

Another important aspect of human behavior is the way we form beliefs quickly. According to Celeste's research, people tend to form new beliefs very rapidly, often with high confidence. This means that even when we're presented with contradictory information, we can be resistant to changing our minds. As a result, it's essential to consider how people are motivated and influenced by the information they receive.

Celeste highlights an interesting point: when we start learning something new, we may not realize how much we don't know. This lack of awareness can actually be beneficial, as it allows us to approach new information with a fresh perspective. However, this dynamic is often overlooked in favor of more conventional approaches to decision making and platform design.

According to Celeste, the assumption that certain platforms or tools are "neutral" is often misleading. In reality, these platforms are designed to influence behavior and shape people's decisions. The way options are presented, the order in which information is displayed, and even the types of suggestions made can all impact how people make choices.

Ultimately, Celeste emphasizes the importance of treating human beliefs with respect and understanding. When we design systems or present information to others, we're not just influencing behavior; we're shaping people's fundamental beliefs. By recognizing this power dynamic, we can create more effective and responsible designs that take into account the complexities of human thought.

This idea is closely related to the concept of "transient" influences versus fundamentally altered beliefs. While some changes in behavior may be temporary or superficial, others are driven by deep-seated changes in our understanding of the world. By recognizing this distinction, we can develop more nuanced approaches to decision making and platform design.

In conclusion, human beliefs play a critical role in decision making, and their influence is often overlooked in favor of more conventional approaches. By acknowledging the complexities of human thought and the ways in which information is presented, we can create more effective and responsible designs that take into account the intricacies of human behavior.

"WEBVTTKind: captionsLanguage: enwelcome to the 1200 i podcast I'm your host Sam Charrington this week on the podcast I'm happy to share just a few of the nearly 20 interviews I recorded earlier this month at the 33rd annual nerves conference if you've been waiting for the twill pendulum to swing from workflow and deployment back over to AI and mo research this is your time we've got some great interviews in store for you over the upcoming weeks before we move on I want to send a huge thanks to our friends at Shell for their support of the podcast and their sponsorship of this nerd series shell has been an early adopter of a wide variety of AI technologies to support use cases across retail trading new Energy's refineries exploration and many more and is doing some really interesting things but don't take it from me Microsoft CEO Satya Nadella recently noted that what's happening at Shell is pretty amazing they have a very deliberate strategy of using AI right across their operation from the drilling operations to safety last year the company established the shell dot a eye residency program a two-year full-time program which allows data scientists and AI engineers to gain experience working on a variety of AI projects across all shell businesses if you're in a position to take advantage of an opportunity like this I'd encourage you to hit pause now and head over to shell dot a AI to learn more once again that shell dot a I and now on to the show all right everyone I am here in Vancouver at nerps and I've got the pleasure of being seated with Celeste kid Celeste is an assistant professor of psychology at UC Berkeley Celeste welcome to the 20th cast thank you so much I'm so excited to be here I am super excited to dive into this conversation you delivered and invited talked here yesterday that has been blowing up the Twitter's and I'm really looking forward to kind of chatting with you about it but before we do that tell us a little bit about your background you kind of operate in the intersection of psychology and machine learning and we were talking about you building models and stuff like that yeah what are you up to I I do I like pulling a lot of things from a lot of places my background was actually in investigative reporting which sounds not that relevant but I was interested in doing big data type analyses so my first real science was getting together public records and you know looking for looking for corruption I started at UC Santa Cruz and then I also liked computers so started in CS I ended up transferring and changing my degrees to linguistics and then I finished that the journalism degree and while I was doing those I got very lucky and happened upon science via some really amazing mentors and professors at USC and fell in love with it and liked that unlike journalism in science truth is on your side I think when you find the truth you win much more so than I was experiencing at that time in journalism I thought the journalism was gonna be like that but I mean hope terminal ISM is like that I think it's becoming more like that and that's actually an interesting discussion too because that's where it's going is much more machine learning relevant than it used to be when I was making the decision about whether to continue in journalism or to transition to science I knew I really loved science but there was still a part of me that thought that journalism was a kind of higher calling and at that time I had a bunch of stories that I had compose that I was very proud of but they contain things that my editors weren't expecting they weren't that's a fist akid like I hadn't taken formerly a statistics class but in that era if I brought something to my editor and it wasn't a pie chart or a bar graph they were confused and asked me to like go back and turn it into a pie chart or a bar graph and I was like regression analyses or not we're not things that editors would even like hear you out on so I think that's changed a whole lot data journalism is the whole thing now it's a thing in the era in which I was in college the the thing that was the version of that was obviously named in the 70s and was called computer-assisted reporting and the conventions were really really disappointing I liked pretty much what everybody was doing is there's all these workshops and there's reporters that have had profiles to high profile stories and the promise was what will tell you how to do something new and innovative and almost all of them it was like what I did is I got some public records for school bus drivers and I got registered sex offenders cross-reference on BAM story um now I got some priests and registered sex offenders cross-reference um story I was like that was the only thing we're really doing that should be like table stakes of hey doing some research right yeah yeah so I wasn't I was having a hard time finding inspiration in that field and then the stories I was producing often got the editor wasn't willing trick to run them as they were where they they wanted me to simplify them to the point that I thought it wasn't true anymore and nobody was paying investigative reporters I applied to grad school also had a job that it was very likely that I could have was like lined up that I was I was considering and I called my journalism mentor is expecting them to talk me out of going to grad school for science and every single one of them said this is a terrible time journalism plan ahead something else go do it yeah and in that era as it might my eat my grad school stipend those are not known for being generous but the journalism salaries are really abysmal as I'm in grad school stipend actually was higher while I'd taken the journalism jobs Wow yeah I'm happy to be here in science we can do all sorts of more sophisticated analyses than you know 2000 era journalism yeah so tell us about the focus of your lab at Berkeley we are very interested in belief formation we're interested in how people form their beliefs and we apply that to domains that I think people don't think of usually they don't use the term beliefs like they use terms like knowledge acquisition things that people think of as you learning and you're done probably that's not true so things like words if you know the word table you might think we talked about a kid who knows the word table if I say like hey show me a table they can point to it they produce the word table in the right instances but what is actually true but you can't observe directly because the mind is a black box is you're never done forming your concept of table with every new table you encounter your belief about the concept of table changes just a little bit and it updates if you move to a place where the tables look systematically different your concept will move in that direction too so we think about what other people might call knowledge in terms of beliefs and we think of beliefs in terms of being kind of packets of probabilistic expectations and thinking about this example of table like how do you experimentally validate those kinds of ideas oh I didn't get to do that much you didn't get to do as much in in in reporting so we do things like we ask people to we give them a choice as like we say here's a concept for example pick a political figure Donald Trump is Donald Trump more like Richard Nixon or more like Elizabeth Warren and people make a selection I was like we collect these compare persons across a whole set of examples and based on people's responses were able to use clustering algorithms in order to infer the true number of types of categories that exist in the population that's a good summary and it sounds like from what I've read about some of the work at your lab you're also in addition to kind of an experimental type of approach you're also building models and using machine learning to well you describe what's the connection between the model work that you do and the experimental work it depends on the particular type of project but in general we're building computational models that represents formal versions of classic theories from learning science so we take a lot of inspiration from people like Jean Piaget and Maria Montessori and Lev Vygotsky all of them had pretty similar ideas about what the relationship should be between what you currently understand and what you're interested in sampling from next door what you were most able to learn from next all of them said similar things about there being a just a right amount of information I was like you want to seek out stuff that's a little bit different from what you currently understand but not so different from what you currently understand that you can't gain any traction those ideas remain largely untested because of the black box problem you can't directly you know open up a kid's head and see see what's in there so we use models in order to represent a sort of formal version of those kinds of ideas like for that particular work because we have a set of research about infants and how they sample from the world we were interested in testing whether or not infants are generating probabilistic expectations in the absence of any specific goal and whether those probabilistic expectations were influencing their decisions about what to look at whether or not they should continue looking at something or they should cut and run find something else so we created a probabilistic model Dursley multinomial so pretty pretty standard stuff and then use that to compute the surprisal value for different actions in in a sequence I'm not sure if this is I was like now I'm taking a little bit so I was like I don't know if I'm going backwards is this understandable no no so the the surprisal value it is I forgot to tell you what they're looking at I should like oh yes we can get we can fill in the context now sir okay for these experiments people have been interested in how infants decide what to look at moment to moment for a long time for a number of reasons like one of them is you can't ask infants questions the only way that you can find out what they know is to look at what they're looking at what they're interested in and try to make inferences the most common format of infant experiments is you show the most in ulis a you showed them a stimulus B and you see if there's a difference in looking between those two and from that you try to draw very rich inferences about what is in that black box and as you can imagine like those are those are one bit experiments that's challenging you can't really learn anything from a single experiment what the people that are great in this this area do was like people like Lisp Elkie and Rene by a jean is they're not really drawing inferences from just just want experiments they do a whole series of experiments and they know other things in the background about one infants know so I was interested in that method that was a method that I used with Toby Mensa at USC and was sort of fascinated that scientists were inferring things like whether or not infants had object concepts like that's a pretty rich kind of representation that you're inferring just on the basis of a kid looking longer over here versus over here I was interested in that and wanted to know more about the linking function between expectations and infants interests and I had that idea and then it took a few years before I could figure out how you might be able to get out that I was watching a kid play whack-a-mole and objects pop out in some order and if you imagine whack of balls like imagine there's like three moles in three holes that's a perfect that's a perfect instance in which I can imagine I'm way to model what you think is likely in this very limited toy space so you walk up to the whack-a-mole machine at the very onset you haven't observed any data but if I ask you like how often do you think each one pops up people say like they're all equally likely okay great there's your prior now you put a corner in the Machine you see one mole pop up if I stop you right there and say like okay how likely do you think it is will a will be you it will see you only observe one pop up so it doesn't change your mind very much but if mole egg pops up there's like more data you come to that shifts your beliefs more so we're using that setup where this is a domain in which if we just care about the sequence of events we can actually quantify how predictable or surprising a particular event is that's that's a way of getting at that linking function question so we made a version sorry the linking function represents what the linking function represents the relationship between the connotations by some value yeah surprisal value for an event in a sequence an infant's interest and surprisal value is a formal way about trying to start thinking about expectations influencing infants beliefs which is obviously true but so that people weren't sure exactly what what that what that relationship looked like because nobody ever varies it on a continuum for infants and when you call this a linking function are you trying to actually define the function or get to okay yeah these are probably correlated or these are not correlated or something like that we are trying to understand how infants guide their search for information in the world I think it's maybe easy to forget this because we have limited attention but at every moment where you or an infant is looking at one thing you're necessarily not looking at everything else each decision that you make about where you're gonna put your attention or where you're gonna click or what you listen to or who you're going to talk to because the world isn't static comes with a huge opportunity cost I'm here in this room talking to you and I'm not at the conference seeing whatever is happening over there I'm not looking at a poster so the the linking function that we're interested we're linking interested the linking function because we're interested in understanding given how much richness and how vast all of the information is in the world how could an infant possibly get started trying to make the decisions about like where they should look and when they should terminate and how could you design a system that can go from possessing as little information as an infant has to eventually having not perfect but I was like a relatively sophisticated network of knowledge like an adult that's studying machine learning mm-hmm even calling these decisions suggests a higher level of processing then I might think is the case in a lot of you know particularly for infants and whether they're looking at the banana or the picture or something like that I don't want to when I use that word imply that I mean that they're conscious I think all of these things are happening automatically I use the word decisions sounds like you could use the word choice me and Ben Hayden have a paper in neuron that's about how we don't really care what words literally it's part of our our research program in the lab to work on yeah how people's concepts vary and how when two people use the same word they're not activating the same concept so yeah I don't I don't I don't worry I don't get in fights over my cue point is they're looking at one thing or another thing and that's what you're calling the decision well the mechanism that's that's correct and that I'm using the word decision or choice because I think it's very important that if you're going to have a smart intelligence selection attentional system if you want that you want the same general guiding principles to maybe explain where you put your eyes it's like your saccade your eye movements where are you going to look but then that same system should also guide other ways in which you might sample information from the world so what you click that's like what you're willing to pay for if you're buying movies on some of them some kind of streaming services like but there are decisions that are more or less conscious they happen at different time scales but if you were to design a smart system what it should do is seek out information that's valuable and what it means to be valuable is that it offers something new but you can integrate it with your existing representations so the idea of this infant work is trying to see whether or not probabilistic expectations guide infants looking at all they've been theorized to do that for a long time and if they do what is the relationship between a symmetric like surprisal and their interest and what we found is that like many people had suggested that it might be you get a u-shape relationship between infants interest and the surprisal value foreign events in a sequence meaning that infants are most likely to terminate their attention to events that are very low surprisal value so things they're very expected don't offer you much you thought that was gonna happen so you're not learning a lot but on the opposite end of the spectrum if it's too surprising if it's a high surprisal value you also terminate your attention infants are most interested at maintaining their attention when they encounter events in the sequence that are a little bit surprising given what they were expecting but not overly surprising and is the overly surprising results is that counterintuitive to you or surprising at all or is that expected I think it is to some people if you are just coming into these questions and you think like what should I do I'm gonna design a robot that's gonna search for places that it should learn in the world you might think like the most new information is the place that I should start I like to use the analogy of like you're gonna pick a book to read or a movie to watch if I go for the most new information that I could find maybe you pick like a book in a language you don't understand on the topic so and theoretically but really you can't because you're missing the base levels of representation to make sense of it if you know a lot about the topic you can probably pick up some of the words from a language you don't understand if you know the language and you don't know the topic you can make progress on the topic but trying to do two things simultaneously the intuition that Maria Montessori had is you're not going to make a lot of traction now you're not getting the attraction there you're not going to make a lot of progress and yeah that that there's something to that that's why that French version of Game of Thrones is sitting on my shelf only ten pages of having been read not enough I see yeah you don't learn about direwolves and you know University of French class yeah yeah most people have had the experience of trying to absorb information they like want to go wise at a high level but it's just like hard you stick on it that that's the point of all of this so what we're theorizing it's that you have built-in attentional mechanisms that are guiding you towards material that won't waste your time and if you are encountering something that is a little bit below where you're at it was like if it's overly redundant it's really hard to stay on task for those things if you encounter stuff that's beyond where you're at that should also be similarly difficult to focus on from the perspective of you're trying to not waste time moments moments you should seek out stuff where you're making progress but it's not over well it's not to overlap people you know and so the models your building is the idea or goal to advance or enhance machine learning by applying these traditional psychological learning models to you know create better machine learning models or more to try to validate concretely the things that you're observing experimentally you know with these models or both or neither yeah so it's it's multiple things our first goal is to just understand how human systems work from a basic science perspective I'm very interested in understanding how people come to know the things that they know and how beliefs that you form early influence the sampling process downstream I just talked about an instance in which your previous experiences shaped the knowledge and the beliefs that you have the beliefs that you have influenced what you're interested in next which means that little things that happen early could potentially have really profound downstream effects so we're interested in these systems in humans but we're also interested in understanding human belief formation because we're interested in making sure that people can design technologies that interface well with humans like if you design a technology without respect to what we know about humans form how humans form beliefs you run the risks of designing something that's pushing people away from access to reality there's ways in which you might push information to form beliefs that are not right that don't match the ground truth one of the things that I talked about in the talk was the relationship between your subjective sense of certainty and your willingness to seek out new information and also encode it I was like even even if you're not choosing it if if you become very certain a rational agent shouldn't waste time there right as like I talked about that in infants the problem is that sometimes people become certain when they shouldn't be John justified certainty is a thing too and if people become certain their curiosity goes way down like if we present them information after they're certain they don't weight it in the same way they don't attend to it it counts for a whole lot less hmm so it's like if you're designing a system that delivers information to people it's really important that you're aware of that a lot of platforms make decisions to optimize engagements it makes sense that you'd push content to people that they appear to once as indicated by you know them reading it or clicking it or whatever but it's potentially dangerous if in doing that you're giving people more confirmatory evidence than they would encounter if they sampled randomly from the world our systems were not designed to have information presented that optimized our interests as like our systems were designed to forage for information in the world so second reason we're interested in this is is kind of from a cautionary perspective it's very important that we understand how human belief formation works so that we can design technologies that don't mess it up in ways that are bad for individuals and society is like it's very bad if somebody you know logs on to Facebook not sure of whether or not they should vaccinate their kids and walks away thinking that they know they like people yeah I'm not going to I'm not going to be what should definitely vaccinate their kids that will go on record saying that yes so no I was saying if they walked away with the impression that no one that was a bad thing right yes yes those are actually yeah I didn't there's something that was in the talk and then I had to cut for time you're referring to an inference about what is true for other people in the population and I did have a slide about that but I didn't presented in the talk so that when somebody enters a search pretty neutral they form beliefs very quickly we use the example of searching for activated charcoal if you search for activated charcoal trying to figure out whether or not that's like a useful thing to use like they'll start by being equally likely to say like maybe it's good maybe it's bad but in just three clicks of about two to three videos people or all the way up at like 80 90 percents for thinking it's probably a great thing for wellness and the slide that I cut not only are they forming that belief from like I'm not sure to like okay I think this is probably right pretty quickly they're also drawing social inferences about the prevalence of that belief world given the the over-representation of you know super scientific materials on all of the streaming platforms this is this is potentially dangerous because it runs the risk of people becoming certain before they have the chance to encounter disconfirming evidence that was number two sure one this is like they're like we're like not near this yet the goal down downstream way downstream in the future if you want to design truly truly intelligence artificial intelligence you want to understand how human systems work because we fail sometimes because sometimes we form strong beliefs that are not justified given reality I was like sometimes we make bad decisions you want to understand what the pitfalls are in human belief formation so that you can in design an intelligent system that doesn't doesn't have those kind of speaking AI safety types of research or research directions I mean in a sense an aspect of what you described if we're heading in the direction we'll rebuilding AGI what are the how do we build safeguards into the AGI so that we protect ourselves as humans I guess I'm not so much thinking of that as I'm thinking of situations in which people don't use the data in the way they really should so for example it's well-documented that doctors although well-intentioned have a gender and racial biases that prevent them from seeing the evidence objectively black women are much more likely to die giving birth to a child then white women are the reason for that is because when they report that they are in pain when they report the same symptoms they're not taken as seriously because of racial biases about black women complaining we are increasingly looking to AI to make decisions in the medical field people do that badly and as we are introducing AI into these processes we ideally do not want to replicate those bad parts of the way humans make those decisions so those are the three driving goals for your work in your talk you listed or reviewed five conclusions of your work would you call those conclusions of the lessons lessons lessons walk us through those number one lesson is that people are continuously forming probabilistic expectations they are constantly monitoring the statistics of their environments and using those statistics to inform what they're looking at what they're listening to what they're integrating into their new representations you don't learn the concept for something and then you're done right we open with that that was the table example and so what were you mentioned that you've got graphs supporting all this what were the graphs that told that story that one is the the infant work showing that you can use you can compute surprisal over sequential displays and you get a u-shaped trend between there look away behavior and the surprise elect you got it yeah okay second point we also covered is that certainty diminishes interest so the the evidence for that I just picked one example from a study by by cherlene where we look at people's certainty as they're generating answers to trivia questions and that the take-home message is that when you're very certain that you know the right answer you're not curious you don't want that information a little bit worse than that is like if we present that information you're less likely to integrate that once you're certain you cut and run you move on to something else and that is problematic because sometimes people are certain when they should not be so I was like that this is a potential explanation for why people sometimes get stuck with stubborn beliefs that aren't justified in the world if you're very certain it's really hard to get people to go back and reconsider the third thing was that certainty is driven by feedback so this work actually is the the project I think we've talked about all of the the main lines of research except for this one this evidence comes from work by Luis Marti in which we try to figure out when you feel very certain where that subjective sense of certainty is coming from that's an interesting question yeah it is an interesting question it did not come out the way that we were expecting he did not come out the way that we were expecting at all when we first what is that where what can even mean coming from like within the brain or something else when you feel so just like how certain do you feel does that influence your behavior is your certainty some reflection of how certain you should be given the strength of the evidence there are these rational models that show that how certain you should be given the evidence or good at predicting people's accuracy when they're learning new concepts these these studies were done in which you asked people to learn new concepts by just observing evidence so I give them you say you're gonna learn whether or not something is Daxi and then you give them examples of things that are Daxi or not sometimes the concept is whether something is what Daxi they're making up a new concept rather these experiments we like not bringing about the like complexities or the were all the redesign in yeah experimental psych just like make up something new that doesn't have any of the confidence or problems or easiness of real-world data okay so so we ask people just to figure out what's taxi and then we give them examples and how you learn in these tasks is we just ask you a moment-to-moment to say like is this taxi or is this not yeah I'm gonna show you some shapes and they vary along some numbers of dimensions sounds like there's gonna be different colors different sizes and different different shapes and the concept you're gonna try to infer just by guessing so at the very onset is this taxi yes or no you can't possibly know so you take a shot and you say like maybe yes and then you get a second and you get feedback you get a second example and you keep going and what's the original studies that use these paradigms showed was that the complexity of the concept made a difference in terms of people's accuracy as you might expect so if the concept is simple if it's something like red just varies along one dimension people are pretty good at learning that pretty quickly if the concept is something more complex like it could be something like triangle and red and small or big and blue and whatever so I think that those those the more logical operators the more difficult it was for people to infer the concepts in the worst their accuracy in those cases these models were pretty good at predicting people's accuracy and you might have dated that was required before you could figure out what taxi nets but those same models that we were hoping would be a good predictor of how certain you feel we were expecting that people might be more certain than they should be they may feel more certain than they should be given the evidence but instead what we found is that how certain you feel about whether or not the concept you have in mind being right it was pretty divorced from the from the strength of the evidence instead the best predictor of how certain you are was whether or not you're getting stuff right which is a little disturbing because if you're just saying yes or no it's pretty easy just by chance to get a string of birth a string of answers correct if you get a string of answers correct whatever idea you had in mind when you got that string of answers correct you gain high confidence and the reason why that's problematic is you don't keep sampling I was like if we give you the option of like moving on to something else you do you leave the task and you don't actually figure out what Daxi means if we create a circumstance in which you like keep collecting evidence you don't weight it the same as you did before you were certain so we think that's this is a part of the puzzle in understanding how people sometimes have very stubborn beliefs that aren't justified given evidence in the world mm-hmm you start with some idea if you get a few pieces of feedback that are consistent with that you may develop a high degree of certainty and once you've done that it may be hard to go back and revise but this one links in with like reason number two why don't we do this kind of research this may be was less common when you were walking around the world sampling from I know I hesitate to use the term natural environment but an environment that's not optimized what you want to I like that yeah and now if you're going online to form your beliefs so if you have some kind of idea and you watch some YouTube video maybe you think that's maybe the earth is flat let me search for that you get a few videos that are consistent with that the risk is that you could develop a high degree of confidence that that's correct and once you develop a high degree of confidence you feel very very certain your curiosity your interest in revising plummets and you may get stuck with that wrong belief do you think that these kind of models for belief formation are inherent and therefore kind of unchangeable or can we possibly as humans adapt to these new environments that were in that are kind of optimizing around our attention and priors that is an excellent question that I would like to know the answer to my whole lab is actually this is a question that one of the lab members led one of the people that's now in the lab working on this stuff to to the lab one of the people in the lab that's a research scientist is someone named Adam Conover who is predominantly his background is in science communication and also comedy and he was very interested in how people might have a shot at not forming bad beliefs online an idea that we plan to test but are still working out right now is whether or not these tendencies people have may be able to be mitigated if they're aware of human belief formation processes or maybe if they're aware that the systems that are giving them information or it the same as the the natural world outside I have extended family members that do not understand that when they go on Facebook this is not a true representative sample of opinions in the world I was like they you know we might think they should know that as like they should know that they're only seeing content from people that they opted into seeing they don't understand the algorithms the fact this is they don't understand that what they hang on longer influences what they're more likely to see next so instead they're drawing inferences under an incorrect assumption which is that what they see represents what's true in the world that's how our systems are made it's possible that understanding either something about your internal system you could maybe make those bad tendencies click different beliefs that are not justified maybe you could help mitigate some of those it's also it's also possible knowing something about the back-end system you could adjust but we really don't know there's there's a little visiting like a routine or an experiment that uses the results of surprisal to kind of train people to shift their internal belief distribution or something like that right yes people are sensitive to they're capable of comprehending information about distributions and drawing correct inferences under different assumptions like replace versus not so the fact that we as scientists are capable of learning statistics it's obviously possible how much of a difference it could make in these day to day moment to moment decisions you're forming beliefs constantly yeah so he's yet to be is yet to be seen but there's some precedent from the implicit bias literature if you are trying to make people less racist the worst thing that they can do is say like I'm not racist I'm a racist I don't see color I was like I'm not considering it if you point out to people that they may have implicit biases they're not conscious but they're influencing their decision-making processes it doesn't undo the biases but they're their lessons just through that knowledge so it's possible that people knowing something about how they work and how they form beliefs could make a difference but we don't know all right so we're on number three oh yeah four I talked about the influence of feedback and feedback being the primary driver of how certain you feel and why this is problematic for forming beliefs everywhere but especially on the internet point number four is that in the absence of feedback maybe you think like the feedbacks problematic we'll just remove it but that's probably not right either in the absence of feedback people seem to be overconfident and our evidence for that is we look at situations in which people don't get very much feedback we wanted to do something naturalistic and more kind of real worldly this is work that was led by Luis Marti that was published in an open mind very recently the domain we came up with is when two people use a word are they activating this the same concept so this is one relating to the concept stuff that I talked about I talked about how the first set of findings is about how when two people use a word they have different concepts in mind so I was I mean two people don't usually have the same concept in mind for abstract political kinds of concept like Joe Biden but they also don't have the same kind of concept for table either I was like we were expecting to find like more disagreement about abstract things like you know people in politics and and less disagreement about concrete objects because you can observe them but even for things like table and chair those things also mean different things to different people hmm so so many questions that I'm not going to ask due to lack of time so it looks like people have there's more than one concept in the population and I was like we tried to we use tools from ecology and various various clustering techniques for trying to infer the true distribution of clusters of beliefs in the population and when you do that you get something like five to ten for most concepts although they're not all the same like there's more disagreement about other examples of the five to ten beliefs people have about tables I'm also the data from this you know the question that I went you know how how do we know that it's not just differences in the way we describe things as opposed to the fundamental inherent belief about this thing because we control the context actually I don't know if that I started saying that then I'm not sure if that's actually the answer to your question mmm we get this data we didn't want people we told first of all that we don't ask about words that have synonyms because it could be that it looks like two different concepts but people that's like a different weird situation we're asking about things like penguins or people in politics or concrete objects or or abstract concepts depending upon the the particular experiment we're controlling the context to minimize the chance that when people appear to have two different concepts it's because they're imagining a different situation like instead we're getting this data from like here's Donald Trump who is he more similar to and you just pick one or the other so we're fitting the data into binary vectors and we're thinking about like the concrete example like a table you know I might describe it functionally is the thing that you put things on I might describe it structurally it's a thing that has legs that variation I would say falls under the context in which our thinking of a table and we're controlling for that by saying just pick is a table more similar to a glass or a tape-recorder so we're getting around that by just having you make a judgment with respect to the other objects to control for exactly well that problem and then also the like people imagining a different situation yeah well we wouldn't want to conclude that people have two different concepts of table because somebody's picturing a table that's like sitting there and somebody else is picturing the table that like someone is standing on so it's more like a stage right or something like that so we control the contacts by just having you judge how similar a table is to like a glass and then hopefully everybody has like pretty much the same concept in mind I should also maybe add that for this work in the past most people have thought about concepts like table as being relatively stable it's less important exactly how many concepts there are and more important that there's not one like so the way people usually think about these things is just one instead your concept of table is slightly changed by every table you encounter in your life and what that means is that what you'd accept as a table as opposed to a stage maybe table is a weird example unless you're going to make a distinction between like our coffee table in like a dining table high top right yeah a cup of cup and bowlers is a classic example I mean I'm imagining many conversations I've had with my wife who grew up in a different area refers to things slightly differently and you know we have these circular commerce sale that's not a XYZ that's a you know ABC no it's XYZ yes yeah that's exactly there were some classic studies in psychology that thought of like cups and bowls is happening on a continuum so what counts as a cup and what counts as a bowl maybe depends partially on the material but then also like the width versus the height but also like the absolute size or if all the cups are dirty then my daughter is known to have drunk out of a bowl correct now it's a bowl that's called of all the bowls are dirty now this cup is a bowl and I've also I've also pulled that one with small children so yes so with the where you draw the line you might expect people had pretty similar places but it looks like it's it's more different than you might expect even for the concrete objects so where I would draw the line at the cup Bowl distinction depending on the relationship of absolute size and height and width is probably different from some substantial portion of the population and where people draw the line along all of these dimensions they you can you can cluster them together and try to come up with an estimate for roughly how many how many overall concepts there are okay so yeah big point is there's not one even things like penguin and Cup and Bowl and the question we were most interested in asking is given that there is variation when two people use one word they're not necessarily activating the same concept are people aware of that possibility as I do I know when I say something you may not have the same thing in mind and the answer is like two points to three right the answer the answer is no there's a really fun like sub finding which is people generally overestimate the degree to which they believe that their concept will align with somebody else's but if you have a weirdo deviant definition of something if you're activating a concept it's like very different for other people in the population you actually have a better chance at being aware of that which is not what I would have guessed if you talk to somebody that's like using words weirdly that's like you might expect it's because they are unaware of that deviate usage I was like they're more likely to be aware that that is a weird way of that's a weird concept that they have so is all this related to the I forget the name of the law like people who you know experts perceive they're very yeah it's possible there's definitely connection to this is the den Creek or Kruger effect is if you are very competent you're more likely to be aware of the areas in which you have incompetence but if you're totally clueless you are not aware of all the words if you're estimating your own confidence it doesn't it doesn't go well given given those dynamics I don't know it's not exactly the same thing but that is an interesting point you might think that it's a bad design if people their estimates of their own incompetence are not well matched to how incompetent they actually are but surely and cherlene Wade's research from my lab showing that people are most curious when they believe that they're about to know the answer when they believe that they're about to know everything those two things taken together might actually indicate that that's a feature not not a bug if you at the beginning of learning something new were acutely aware of how incompetent you were like what our data from the lab would say is that you would not be motivated to take that first step right you may never try to approach it it actually may be a good thing from a motivational standpoint that when you first start out learning something new you don't know like how much you're going to have your competence revealed incrementally we would predict is the gives you the best chance of continuing yeah yeah and so very clear that that 5th point and that fifth point people form beliefs very quickly so it's possible for people to go from completely undecided to believing that they should buy the like weird black smoothie bagel those things that are like expensive coffee shops that have charcoal added to them for I didn't unclear unclear pseudo-scientific wellness purposes it doesn't take very long or perform the belief form that belief with high confidence and so when we're we're designing systems that offer information to people it's really important that we keep that in mind when somebody certain they stop searching when they see disconfirming evidence after that point it doesn't count for the same so I think the takeaway of all of this the big point that I wanted to make in the talk was that when you hear people say that this tool or this platform is neutral that's I would say dishonest we know that there are ways of making decisions behind the scenes that influence people's behavior I was like all of us design things that change behavior I was like whether or not you're making different decisions about how to present different options of suggested items that people might purchase or whether you are trying to keep people on your your site for longer if people's behavior is changing as a function of different design decisions you make or different types of things that you optimize it's important to remember that the mediating variable there is human beliefs you're messing with human beliefs and you're messing with what people know and they walk away into the real world to make real-world decisions with those change beliefs so there is no neutral platform I think that's a really ill conceived way of thinking about these problems I think it's irresponsible it's really important to appreciate that the way you present information the order in which you present information it really matters and it has an important result like I think we tend to think about you know the kind of influences you're describing is just influencing discrete behaviors or actions as opposed to fundamentally beliefs right and they're very different and you can is transient one is yeah you can't change behavior without altering people's beliefs and it's important to keep that in mind and to treat that Duty with the respect it deserves right really interesting stuff we could go on for another hour I well Celeste thanks so much for you know taking some time out of your busy nerves to come chat with us and share a bit about what you're up to of course thank you so much for having me thank you alright everyone that's our show for today for more information on today's guest or our nerps podcast series head over to twilly Icom slash nerves 2019 thanks once again to shell for sponsoring this week's series check out the shell dot a I residency program by typing Shelley I into your browser's address bar thanks so much for listening happy holidays and catch you next timewelcome to the 1200 i podcast I'm your host Sam Charrington this week on the podcast I'm happy to share just a few of the nearly 20 interviews I recorded earlier this month at the 33rd annual nerves conference if you've been waiting for the twill pendulum to swing from workflow and deployment back over to AI and mo research this is your time we've got some great interviews in store for you over the upcoming weeks before we move on I want to send a huge thanks to our friends at Shell for their support of the podcast and their sponsorship of this nerd series shell has been an early adopter of a wide variety of AI technologies to support use cases across retail trading new Energy's refineries exploration and many more and is doing some really interesting things but don't take it from me Microsoft CEO Satya Nadella recently noted that what's happening at Shell is pretty amazing they have a very deliberate strategy of using AI right across their operation from the drilling operations to safety last year the company established the shell dot a eye residency program a two-year full-time program which allows data scientists and AI engineers to gain experience working on a variety of AI projects across all shell businesses if you're in a position to take advantage of an opportunity like this I'd encourage you to hit pause now and head over to shell dot a AI to learn more once again that shell dot a I and now on to the show all right everyone I am here in Vancouver at nerps and I've got the pleasure of being seated with Celeste kid Celeste is an assistant professor of psychology at UC Berkeley Celeste welcome to the 20th cast thank you so much I'm so excited to be here I am super excited to dive into this conversation you delivered and invited talked here yesterday that has been blowing up the Twitter's and I'm really looking forward to kind of chatting with you about it but before we do that tell us a little bit about your background you kind of operate in the intersection of psychology and machine learning and we were talking about you building models and stuff like that yeah what are you up to I I do I like pulling a lot of things from a lot of places my background was actually in investigative reporting which sounds not that relevant but I was interested in doing big data type analyses so my first real science was getting together public records and you know looking for looking for corruption I started at UC Santa Cruz and then I also liked computers so started in CS I ended up transferring and changing my degrees to linguistics and then I finished that the journalism degree and while I was doing those I got very lucky and happened upon science via some really amazing mentors and professors at USC and fell in love with it and liked that unlike journalism in science truth is on your side I think when you find the truth you win much more so than I was experiencing at that time in journalism I thought the journalism was gonna be like that but I mean hope terminal ISM is like that I think it's becoming more like that and that's actually an interesting discussion too because that's where it's going is much more machine learning relevant than it used to be when I was making the decision about whether to continue in journalism or to transition to science I knew I really loved science but there was still a part of me that thought that journalism was a kind of higher calling and at that time I had a bunch of stories that I had compose that I was very proud of but they contain things that my editors weren't expecting they weren't that's a fist akid like I hadn't taken formerly a statistics class but in that era if I brought something to my editor and it wasn't a pie chart or a bar graph they were confused and asked me to like go back and turn it into a pie chart or a bar graph and I was like regression analyses or not we're not things that editors would even like hear you out on so I think that's changed a whole lot data journalism is the whole thing now it's a thing in the era in which I was in college the the thing that was the version of that was obviously named in the 70s and was called computer-assisted reporting and the conventions were really really disappointing I liked pretty much what everybody was doing is there's all these workshops and there's reporters that have had profiles to high profile stories and the promise was what will tell you how to do something new and innovative and almost all of them it was like what I did is I got some public records for school bus drivers and I got registered sex offenders cross-reference on BAM story um now I got some priests and registered sex offenders cross-reference um story I was like that was the only thing we're really doing that should be like table stakes of hey doing some research right yeah yeah so I wasn't I was having a hard time finding inspiration in that field and then the stories I was producing often got the editor wasn't willing trick to run them as they were where they they wanted me to simplify them to the point that I thought it wasn't true anymore and nobody was paying investigative reporters I applied to grad school also had a job that it was very likely that I could have was like lined up that I was I was considering and I called my journalism mentor is expecting them to talk me out of going to grad school for science and every single one of them said this is a terrible time journalism plan ahead something else go do it yeah and in that era as it might my eat my grad school stipend those are not known for being generous but the journalism salaries are really abysmal as I'm in grad school stipend actually was higher while I'd taken the journalism jobs Wow yeah I'm happy to be here in science we can do all sorts of more sophisticated analyses than you know 2000 era journalism yeah so tell us about the focus of your lab at Berkeley we are very interested in belief formation we're interested in how people form their beliefs and we apply that to domains that I think people don't think of usually they don't use the term beliefs like they use terms like knowledge acquisition things that people think of as you learning and you're done probably that's not true so things like words if you know the word table you might think we talked about a kid who knows the word table if I say like hey show me a table they can point to it they produce the word table in the right instances but what is actually true but you can't observe directly because the mind is a black box is you're never done forming your concept of table with every new table you encounter your belief about the concept of table changes just a little bit and it updates if you move to a place where the tables look systematically different your concept will move in that direction too so we think about what other people might call knowledge in terms of beliefs and we think of beliefs in terms of being kind of packets of probabilistic expectations and thinking about this example of table like how do you experimentally validate those kinds of ideas oh I didn't get to do that much you didn't get to do as much in in in reporting so we do things like we ask people to we give them a choice as like we say here's a concept for example pick a political figure Donald Trump is Donald Trump more like Richard Nixon or more like Elizabeth Warren and people make a selection I was like we collect these compare persons across a whole set of examples and based on people's responses were able to use clustering algorithms in order to infer the true number of types of categories that exist in the population that's a good summary and it sounds like from what I've read about some of the work at your lab you're also in addition to kind of an experimental type of approach you're also building models and using machine learning to well you describe what's the connection between the model work that you do and the experimental work it depends on the particular type of project but in general we're building computational models that represents formal versions of classic theories from learning science so we take a lot of inspiration from people like Jean Piaget and Maria Montessori and Lev Vygotsky all of them had pretty similar ideas about what the relationship should be between what you currently understand and what you're interested in sampling from next door what you were most able to learn from next all of them said similar things about there being a just a right amount of information I was like you want to seek out stuff that's a little bit different from what you currently understand but not so different from what you currently understand that you can't gain any traction those ideas remain largely untested because of the black box problem you can't directly you know open up a kid's head and see see what's in there so we use models in order to represent a sort of formal version of those kinds of ideas like for that particular work because we have a set of research about infants and how they sample from the world we were interested in testing whether or not infants are generating probabilistic expectations in the absence of any specific goal and whether those probabilistic expectations were influencing their decisions about what to look at whether or not they should continue looking at something or they should cut and run find something else so we created a probabilistic model Dursley multinomial so pretty pretty standard stuff and then use that to compute the surprisal value for different actions in in a sequence I'm not sure if this is I was like now I'm taking a little bit so I was like I don't know if I'm going backwards is this understandable no no so the the surprisal value it is I forgot to tell you what they're looking at I should like oh yes we can get we can fill in the context now sir okay for these experiments people have been interested in how infants decide what to look at moment to moment for a long time for a number of reasons like one of them is you can't ask infants questions the only way that you can find out what they know is to look at what they're looking at what they're interested in and try to make inferences the most common format of infant experiments is you show the most in ulis a you showed them a stimulus B and you see if there's a difference in looking between those two and from that you try to draw very rich inferences about what is in that black box and as you can imagine like those are those are one bit experiments that's challenging you can't really learn anything from a single experiment what the people that are great in this this area do was like people like Lisp Elkie and Rene by a jean is they're not really drawing inferences from just just want experiments they do a whole series of experiments and they know other things in the background about one infants know so I was interested in that method that was a method that I used with Toby Mensa at USC and was sort of fascinated that scientists were inferring things like whether or not infants had object concepts like that's a pretty rich kind of representation that you're inferring just on the basis of a kid looking longer over here versus over here I was interested in that and wanted to know more about the linking function between expectations and infants interests and I had that idea and then it took a few years before I could figure out how you might be able to get out that I was watching a kid play whack-a-mole and objects pop out in some order and if you imagine whack of balls like imagine there's like three moles in three holes that's a perfect that's a perfect instance in which I can imagine I'm way to model what you think is likely in this very limited toy space so you walk up to the whack-a-mole machine at the very onset you haven't observed any data but if I ask you like how often do you think each one pops up people say like they're all equally likely okay great there's your prior now you put a corner in the Machine you see one mole pop up if I stop you right there and say like okay how likely do you think it is will a will be you it will see you only observe one pop up so it doesn't change your mind very much but if mole egg pops up there's like more data you come to that shifts your beliefs more so we're using that setup where this is a domain in which if we just care about the sequence of events we can actually quantify how predictable or surprising a particular event is that's that's a way of getting at that linking function question so we made a version sorry the linking function represents what the linking function represents the relationship between the connotations by some value yeah surprisal value for an event in a sequence an infant's interest and surprisal value is a formal way about trying to start thinking about expectations influencing infants beliefs which is obviously true but so that people weren't sure exactly what what that what that relationship looked like because nobody ever varies it on a continuum for infants and when you call this a linking function are you trying to actually define the function or get to okay yeah these are probably correlated or these are not correlated or something like that we are trying to understand how infants guide their search for information in the world I think it's maybe easy to forget this because we have limited attention but at every moment where you or an infant is looking at one thing you're necessarily not looking at everything else each decision that you make about where you're gonna put your attention or where you're gonna click or what you listen to or who you're going to talk to because the world isn't static comes with a huge opportunity cost I'm here in this room talking to you and I'm not at the conference seeing whatever is happening over there I'm not looking at a poster so the the linking function that we're interested we're linking interested the linking function because we're interested in understanding given how much richness and how vast all of the information is in the world how could an infant possibly get started trying to make the decisions about like where they should look and when they should terminate and how could you design a system that can go from possessing as little information as an infant has to eventually having not perfect but I was like a relatively sophisticated network of knowledge like an adult that's studying machine learning mm-hmm even calling these decisions suggests a higher level of processing then I might think is the case in a lot of you know particularly for infants and whether they're looking at the banana or the picture or something like that I don't want to when I use that word imply that I mean that they're conscious I think all of these things are happening automatically I use the word decisions sounds like you could use the word choice me and Ben Hayden have a paper in neuron that's about how we don't really care what words literally it's part of our our research program in the lab to work on yeah how people's concepts vary and how when two people use the same word they're not activating the same concept so yeah I don't I don't I don't worry I don't get in fights over my cue point is they're looking at one thing or another thing and that's what you're calling the decision well the mechanism that's that's correct and that I'm using the word decision or choice because I think it's very important that if you're going to have a smart intelligence selection attentional system if you want that you want the same general guiding principles to maybe explain where you put your eyes it's like your saccade your eye movements where are you going to look but then that same system should also guide other ways in which you might sample information from the world so what you click that's like what you're willing to pay for if you're buying movies on some of them some kind of streaming services like but there are decisions that are more or less conscious they happen at different time scales but if you were to design a smart system what it should do is seek out information that's valuable and what it means to be valuable is that it offers something new but you can integrate it with your existing representations so the idea of this infant work is trying to see whether or not probabilistic expectations guide infants looking at all they've been theorized to do that for a long time and if they do what is the relationship between a symmetric like surprisal and their interest and what we found is that like many people had suggested that it might be you get a u-shape relationship between infants interest and the surprisal value foreign events in a sequence meaning that infants are most likely to terminate their attention to events that are very low surprisal value so things they're very expected don't offer you much you thought that was gonna happen so you're not learning a lot but on the opposite end of the spectrum if it's too surprising if it's a high surprisal value you also terminate your attention infants are most interested at maintaining their attention when they encounter events in the sequence that are a little bit surprising given what they were expecting but not overly surprising and is the overly surprising results is that counterintuitive to you or surprising at all or is that expected I think it is to some people if you are just coming into these questions and you think like what should I do I'm gonna design a robot that's gonna search for places that it should learn in the world you might think like the most new information is the place that I should start I like to use the analogy of like you're gonna pick a book to read or a movie to watch if I go for the most new information that I could find maybe you pick like a book in a language you don't understand on the topic so and theoretically but really you can't because you're missing the base levels of representation to make sense of it if you know a lot about the topic you can probably pick up some of the words from a language you don't understand if you know the language and you don't know the topic you can make progress on the topic but trying to do two things simultaneously the intuition that Maria Montessori had is you're not going to make a lot of traction now you're not getting the attraction there you're not going to make a lot of progress and yeah that that there's something to that that's why that French version of Game of Thrones is sitting on my shelf only ten pages of having been read not enough I see yeah you don't learn about direwolves and you know University of French class yeah yeah most people have had the experience of trying to absorb information they like want to go wise at a high level but it's just like hard you stick on it that that's the point of all of this so what we're theorizing it's that you have built-in attentional mechanisms that are guiding you towards material that won't waste your time and if you are encountering something that is a little bit below where you're at it was like if it's overly redundant it's really hard to stay on task for those things if you encounter stuff that's beyond where you're at that should also be similarly difficult to focus on from the perspective of you're trying to not waste time moments moments you should seek out stuff where you're making progress but it's not over well it's not to overlap people you know and so the models your building is the idea or goal to advance or enhance machine learning by applying these traditional psychological learning models to you know create better machine learning models or more to try to validate concretely the things that you're observing experimentally you know with these models or both or neither yeah so it's it's multiple things our first goal is to just understand how human systems work from a basic science perspective I'm very interested in understanding how people come to know the things that they know and how beliefs that you form early influence the sampling process downstream I just talked about an instance in which your previous experiences shaped the knowledge and the beliefs that you have the beliefs that you have influenced what you're interested in next which means that little things that happen early could potentially have really profound downstream effects so we're interested in these systems in humans but we're also interested in understanding human belief formation because we're interested in making sure that people can design technologies that interface well with humans like if you design a technology without respect to what we know about humans form how humans form beliefs you run the risks of designing something that's pushing people away from access to reality there's ways in which you might push information to form beliefs that are not right that don't match the ground truth one of the things that I talked about in the talk was the relationship between your subjective sense of certainty and your willingness to seek out new information and also encode it I was like even even if you're not choosing it if if you become very certain a rational agent shouldn't waste time there right as like I talked about that in infants the problem is that sometimes people become certain when they shouldn't be John justified certainty is a thing too and if people become certain their curiosity goes way down like if we present them information after they're certain they don't weight it in the same way they don't attend to it it counts for a whole lot less hmm so it's like if you're designing a system that delivers information to people it's really important that you're aware of that a lot of platforms make decisions to optimize engagements it makes sense that you'd push content to people that they appear to once as indicated by you know them reading it or clicking it or whatever but it's potentially dangerous if in doing that you're giving people more confirmatory evidence than they would encounter if they sampled randomly from the world our systems were not designed to have information presented that optimized our interests as like our systems were designed to forage for information in the world so second reason we're interested in this is is kind of from a cautionary perspective it's very important that we understand how human belief formation works so that we can design technologies that don't mess it up in ways that are bad for individuals and society is like it's very bad if somebody you know logs on to Facebook not sure of whether or not they should vaccinate their kids and walks away thinking that they know they like people yeah I'm not going to I'm not going to be what should definitely vaccinate their kids that will go on record saying that yes so no I was saying if they walked away with the impression that no one that was a bad thing right yes yes those are actually yeah I didn't there's something that was in the talk and then I had to cut for time you're referring to an inference about what is true for other people in the population and I did have a slide about that but I didn't presented in the talk so that when somebody enters a search pretty neutral they form beliefs very quickly we use the example of searching for activated charcoal if you search for activated charcoal trying to figure out whether or not that's like a useful thing to use like they'll start by being equally likely to say like maybe it's good maybe it's bad but in just three clicks of about two to three videos people or all the way up at like 80 90 percents for thinking it's probably a great thing for wellness and the slide that I cut not only are they forming that belief from like I'm not sure to like okay I think this is probably right pretty quickly they're also drawing social inferences about the prevalence of that belief world given the the over-representation of you know super scientific materials on all of the streaming platforms this is this is potentially dangerous because it runs the risk of people becoming certain before they have the chance to encounter disconfirming evidence that was number two sure one this is like they're like we're like not near this yet the goal down downstream way downstream in the future if you want to design truly truly intelligence artificial intelligence you want to understand how human systems work because we fail sometimes because sometimes we form strong beliefs that are not justified given reality I was like sometimes we make bad decisions you want to understand what the pitfalls are in human belief formation so that you can in design an intelligent system that doesn't doesn't have those kind of speaking AI safety types of research or research directions I mean in a sense an aspect of what you described if we're heading in the direction we'll rebuilding AGI what are the how do we build safeguards into the AGI so that we protect ourselves as humans I guess I'm not so much thinking of that as I'm thinking of situations in which people don't use the data in the way they really should so for example it's well-documented that doctors although well-intentioned have a gender and racial biases that prevent them from seeing the evidence objectively black women are much more likely to die giving birth to a child then white women are the reason for that is because when they report that they are in pain when they report the same symptoms they're not taken as seriously because of racial biases about black women complaining we are increasingly looking to AI to make decisions in the medical field people do that badly and as we are introducing AI into these processes we ideally do not want to replicate those bad parts of the way humans make those decisions so those are the three driving goals for your work in your talk you listed or reviewed five conclusions of your work would you call those conclusions of the lessons lessons lessons walk us through those number one lesson is that people are continuously forming probabilistic expectations they are constantly monitoring the statistics of their environments and using those statistics to inform what they're looking at what they're listening to what they're integrating into their new representations you don't learn the concept for something and then you're done right we open with that that was the table example and so what were you mentioned that you've got graphs supporting all this what were the graphs that told that story that one is the the infant work showing that you can use you can compute surprisal over sequential displays and you get a u-shaped trend between there look away behavior and the surprise elect you got it yeah okay second point we also covered is that certainty diminishes interest so the the evidence for that I just picked one example from a study by by cherlene where we look at people's certainty as they're generating answers to trivia questions and that the take-home message is that when you're very certain that you know the right answer you're not curious you don't want that information a little bit worse than that is like if we present that information you're less likely to integrate that once you're certain you cut and run you move on to something else and that is problematic because sometimes people are certain when they should not be so I was like that this is a potential explanation for why people sometimes get stuck with stubborn beliefs that aren't justified in the world if you're very certain it's really hard to get people to go back and reconsider the third thing was that certainty is driven by feedback so this work actually is the the project I think we've talked about all of the the main lines of research except for this one this evidence comes from work by Luis Marti in which we try to figure out when you feel very certain where that subjective sense of certainty is coming from that's an interesting question yeah it is an interesting question it did not come out the way that we were expecting he did not come out the way that we were expecting at all when we first what is that where what can even mean coming from like within the brain or something else when you feel so just like how certain do you feel does that influence your behavior is your certainty some reflection of how certain you should be given the strength of the evidence there are these rational models that show that how certain you should be given the evidence or good at predicting people's accuracy when they're learning new concepts these these studies were done in which you asked people to learn new concepts by just observing evidence so I give them you say you're gonna learn whether or not something is Daxi and then you give them examples of things that are Daxi or not sometimes the concept is whether something is what Daxi they're making up a new concept rather these experiments we like not bringing about the like complexities or the were all the redesign in yeah experimental psych just like make up something new that doesn't have any of the confidence or problems or easiness of real-world data okay so so we ask people just to figure out what's taxi and then we give them examples and how you learn in these tasks is we just ask you a moment-to-moment to say like is this taxi or is this not yeah I'm gonna show you some shapes and they vary along some numbers of dimensions sounds like there's gonna be different colors different sizes and different different shapes and the concept you're gonna try to infer just by guessing so at the very onset is this taxi yes or no you can't possibly know so you take a shot and you say like maybe yes and then you get a second and you get feedback you get a second example and you keep going and what's the original studies that use these paradigms showed was that the complexity of the concept made a difference in terms of people's accuracy as you might expect so if the concept is simple if it's something like red just varies along one dimension people are pretty good at learning that pretty quickly if the concept is something more complex like it could be something like triangle and red and small or big and blue and whatever so I think that those those the more logical operators the more difficult it was for people to infer the concepts in the worst their accuracy in those cases these models were pretty good at predicting people's accuracy and you might have dated that was required before you could figure out what taxi nets but those same models that we were hoping would be a good predictor of how certain you feel we were expecting that people might be more certain than they should be they may feel more certain than they should be given the evidence but instead what we found is that how certain you feel about whether or not the concept you have in mind being right it was pretty divorced from the from the strength of the evidence instead the best predictor of how certain you are was whether or not you're getting stuff right which is a little disturbing because if you're just saying yes or no it's pretty easy just by chance to get a string of birth a string of answers correct if you get a string of answers correct whatever idea you had in mind when you got that string of answers correct you gain high confidence and the reason why that's problematic is you don't keep sampling I was like if we give you the option of like moving on to something else you do you leave the task and you don't actually figure out what Daxi means if we create a circumstance in which you like keep collecting evidence you don't weight it the same as you did before you were certain so we think that's this is a part of the puzzle in understanding how people sometimes have very stubborn beliefs that aren't justified given evidence in the world mm-hmm you start with some idea if you get a few pieces of feedback that are consistent with that you may develop a high degree of certainty and once you've done that it may be hard to go back and revise but this one links in with like reason number two why don't we do this kind of research this may be was less common when you were walking around the world sampling from I know I hesitate to use the term natural environment but an environment that's not optimized what you want to I like that yeah and now if you're going online to form your beliefs so if you have some kind of idea and you watch some YouTube video maybe you think that's maybe the earth is flat let me search for that you get a few videos that are consistent with that the risk is that you could develop a high degree of confidence that that's correct and once you develop a high degree of confidence you feel very very certain your curiosity your interest in revising plummets and you may get stuck with that wrong belief do you think that these kind of models for belief formation are inherent and therefore kind of unchangeable or can we possibly as humans adapt to these new environments that were in that are kind of optimizing around our attention and priors that is an excellent question that I would like to know the answer to my whole lab is actually this is a question that one of the lab members led one of the people that's now in the lab working on this stuff to to the lab one of the people in the lab that's a research scientist is someone named Adam Conover who is predominantly his background is in science communication and also comedy and he was very interested in how people might have a shot at not forming bad beliefs online an idea that we plan to test but are still working out right now is whether or not these tendencies people have may be able to be mitigated if they're aware of human belief formation processes or maybe if they're aware that the systems that are giving them information or it the same as the the natural world outside I have extended family members that do not understand that when they go on Facebook this is not a true representative sample of opinions in the world I was like they you know we might think they should know that as like they should know that they're only seeing content from people that they opted into seeing they don't understand the algorithms the fact this is they don't understand that what they hang on longer influences what they're more likely to see next so instead they're drawing inferences under an incorrect assumption which is that what they see represents what's true in the world that's how our systems are made it's possible that understanding either something about your internal system you could maybe make those bad tendencies click different beliefs that are not justified maybe you could help mitigate some of those it's also it's also possible knowing something about the back-end system you could adjust but we really don't know there's there's a little visiting like a routine or an experiment that uses the results of surprisal to kind of train people to shift their internal belief distribution or something like that right yes people are sensitive to they're capable of comprehending information about distributions and drawing correct inferences under different assumptions like replace versus not so the fact that we as scientists are capable of learning statistics it's obviously possible how much of a difference it could make in these day to day moment to moment decisions you're forming beliefs constantly yeah so he's yet to be is yet to be seen but there's some precedent from the implicit bias literature if you are trying to make people less racist the worst thing that they can do is say like I'm not racist I'm a racist I don't see color I was like I'm not considering it if you point out to people that they may have implicit biases they're not conscious but they're influencing their decision-making processes it doesn't undo the biases but they're their lessons just through that knowledge so it's possible that people knowing something about how they work and how they form beliefs could make a difference but we don't know all right so we're on number three oh yeah four I talked about the influence of feedback and feedback being the primary driver of how certain you feel and why this is problematic for forming beliefs everywhere but especially on the internet point number four is that in the absence of feedback maybe you think like the feedbacks problematic we'll just remove it but that's probably not right either in the absence of feedback people seem to be overconfident and our evidence for that is we look at situations in which people don't get very much feedback we wanted to do something naturalistic and more kind of real worldly this is work that was led by Luis Marti that was published in an open mind very recently the domain we came up with is when two people use a word are they activating this the same concept so this is one relating to the concept stuff that I talked about I talked about how the first set of findings is about how when two people use a word they have different concepts in mind so I was I mean two people don't usually have the same concept in mind for abstract political kinds of concept like Joe Biden but they also don't have the same kind of concept for table either I was like we were expecting to find like more disagreement about abstract things like you know people in politics and and less disagreement about concrete objects because you can observe them but even for things like table and chair those things also mean different things to different people hmm so so many questions that I'm not going to ask due to lack of time so it looks like people have there's more than one concept in the population and I was like we tried to we use tools from ecology and various various clustering techniques for trying to infer the true distribution of clusters of beliefs in the population and when you do that you get something like five to ten for most concepts although they're not all the same like there's more disagreement about other examples of the five to ten beliefs people have about tables I'm also the data from this you know the question that I went you know how how do we know that it's not just differences in the way we describe things as opposed to the fundamental inherent belief about this thing because we control the context actually I don't know if that I started saying that then I'm not sure if that's actually the answer to your question mmm we get this data we didn't want people we told first of all that we don't ask about words that have synonyms because it could be that it looks like two different concepts but people that's like a different weird situation we're asking about things like penguins or people in politics or concrete objects or or abstract concepts depending upon the the particular experiment we're controlling the context to minimize the chance that when people appear to have two different concepts it's because they're imagining a different situation like instead we're getting this data from like here's Donald Trump who is he more similar to and you just pick one or the other so we're fitting the data into binary vectors and we're thinking about like the concrete example like a table you know I might describe it functionally is the thing that you put things on I might describe it structurally it's a thing that has legs that variation I would say falls under the context in which our thinking of a table and we're controlling for that by saying just pick is a table more similar to a glass or a tape-recorder so we're getting around that by just having you make a judgment with respect to the other objects to control for exactly well that problem and then also the like people imagining a different situation yeah well we wouldn't want to conclude that people have two different concepts of table because somebody's picturing a table that's like sitting there and somebody else is picturing the table that like someone is standing on so it's more like a stage right or something like that so we control the contacts by just having you judge how similar a table is to like a glass and then hopefully everybody has like pretty much the same concept in mind I should also maybe add that for this work in the past most people have thought about concepts like table as being relatively stable it's less important exactly how many concepts there are and more important that there's not one like so the way people usually think about these things is just one instead your concept of table is slightly changed by every table you encounter in your life and what that means is that what you'd accept as a table as opposed to a stage maybe table is a weird example unless you're going to make a distinction between like our coffee table in like a dining table high top right yeah a cup of cup and bowlers is a classic example I mean I'm imagining many conversations I've had with my wife who grew up in a different area refers to things slightly differently and you know we have these circular commerce sale that's not a XYZ that's a you know ABC no it's XYZ yes yeah that's exactly there were some classic studies in psychology that thought of like cups and bowls is happening on a continuum so what counts as a cup and what counts as a bowl maybe depends partially on the material but then also like the width versus the height but also like the absolute size or if all the cups are dirty then my daughter is known to have drunk out of a bowl correct now it's a bowl that's called of all the bowls are dirty now this cup is a bowl and I've also I've also pulled that one with small children so yes so with the where you draw the line you might expect people had pretty similar places but it looks like it's it's more different than you might expect even for the concrete objects so where I would draw the line at the cup Bowl distinction depending on the relationship of absolute size and height and width is probably different from some substantial portion of the population and where people draw the line along all of these dimensions they you can you can cluster them together and try to come up with an estimate for roughly how many how many overall concepts there are okay so yeah big point is there's not one even things like penguin and Cup and Bowl and the question we were most interested in asking is given that there is variation when two people use one word they're not necessarily activating the same concept are people aware of that possibility as I do I know when I say something you may not have the same thing in mind and the answer is like two points to three right the answer the answer is no there's a really fun like sub finding which is people generally overestimate the degree to which they believe that their concept will align with somebody else's but if you have a weirdo deviant definition of something if you're activating a concept it's like very different for other people in the population you actually have a better chance at being aware of that which is not what I would have guessed if you talk to somebody that's like using words weirdly that's like you might expect it's because they are unaware of that deviate usage I was like they're more likely to be aware that that is a weird way of that's a weird concept that they have so is all this related to the I forget the name of the law like people who you know experts perceive they're very yeah it's possible there's definitely connection to this is the den Creek or Kruger effect is if you are very competent you're more likely to be aware of the areas in which you have incompetence but if you're totally clueless you are not aware of all the words if you're estimating your own confidence it doesn't it doesn't go well given given those dynamics I don't know it's not exactly the same thing but that is an interesting point you might think that it's a bad design if people their estimates of their own incompetence are not well matched to how incompetent they actually are but surely and cherlene Wade's research from my lab showing that people are most curious when they believe that they're about to know the answer when they believe that they're about to know everything those two things taken together might actually indicate that that's a feature not not a bug if you at the beginning of learning something new were acutely aware of how incompetent you were like what our data from the lab would say is that you would not be motivated to take that first step right you may never try to approach it it actually may be a good thing from a motivational standpoint that when you first start out learning something new you don't know like how much you're going to have your competence revealed incrementally we would predict is the gives you the best chance of continuing yeah yeah and so very clear that that 5th point and that fifth point people form beliefs very quickly so it's possible for people to go from completely undecided to believing that they should buy the like weird black smoothie bagel those things that are like expensive coffee shops that have charcoal added to them for I didn't unclear unclear pseudo-scientific wellness purposes it doesn't take very long or perform the belief form that belief with high confidence and so when we're we're designing systems that offer information to people it's really important that we keep that in mind when somebody certain they stop searching when they see disconfirming evidence after that point it doesn't count for the same so I think the takeaway of all of this the big point that I wanted to make in the talk was that when you hear people say that this tool or this platform is neutral that's I would say dishonest we know that there are ways of making decisions behind the scenes that influence people's behavior I was like all of us design things that change behavior I was like whether or not you're making different decisions about how to present different options of suggested items that people might purchase or whether you are trying to keep people on your your site for longer if people's behavior is changing as a function of different design decisions you make or different types of things that you optimize it's important to remember that the mediating variable there is human beliefs you're messing with human beliefs and you're messing with what people know and they walk away into the real world to make real-world decisions with those change beliefs so there is no neutral platform I think that's a really ill conceived way of thinking about these problems I think it's irresponsible it's really important to appreciate that the way you present information the order in which you present information it really matters and it has an important result like I think we tend to think about you know the kind of influences you're describing is just influencing discrete behaviors or actions as opposed to fundamentally beliefs right and they're very different and you can is transient one is yeah you can't change behavior without altering people's beliefs and it's important to keep that in mind and to treat that Duty with the respect it deserves right really interesting stuff we could go on for another hour I well Celeste thanks so much for you know taking some time out of your busy nerves to come chat with us and share a bit about what you're up to of course thank you so much for having me thank you alright everyone that's our show for today for more information on today's guest or our nerps podcast series head over to twilly Icom slash nerves 2019 thanks once again to shell for sponsoring this week's series check out the shell dot a I residency program by typing Shelley I into your browser's address bar thanks so much for listening happy holidays and catch you next time\n"