**Revolutionizing Early Computer Science Education**

The way we approach early computer science education is undergoing a significant transformation. Historically, it was often challenging for children to understand complex concepts, with educators and parents feeling that they were at a loss for how to explain these ideas. However, with the advent of new technologies and approaches, it's now possible to break down even the most intricate topics into manageable pieces that can be grasped by young minds.

One such example is the field of deep reinforcement learning, which involves building agents that can play complex games like Snake. Initially, this task was approached by hand-coding the rules and using simple neural networks to give the agent examples. However, with the advent of more advanced techniques, it's now possible to have children work on deep learning projects, where they're given a set of examples and encouraged to explore and learn through experimentation.

This approach is not only accessible but also empowering for young learners. By taking complex concepts like deep reinforcement learning and breaking them down into manageable pieces, we can create educational experiences that are both engaging and effective. As one researcher noted, "if you just break something down enough, children can get it right." This confidence in the ability of young minds to learn and understand new concepts is a crucial step towards revolutionizing early computer science education.

**Beyond Deep Learning: Expanding the Scope of AI Education**

While deep reinforcement learning is an exciting area of research, there are many other areas where AI has the potential to transform educational experiences. For instance, one researcher in the field is working on developing an AI ethics curriculum for children, which would help them understand the ethical implications behind every AI decision. This approach recognizes that education should not only focus on teaching technical skills but also on promoting critical thinking and responsible behavior.

Another area of research involves creating online platforms and resources that make it easy for teachers to integrate AI into their classrooms. One such platform is a dream-like online space where children can learn about AI in a meaningful way, with the goal of making it accessible to schools and communities across the country.

**A Call to Action: Partnerships and Support**

As one researcher noted, there's a need for partnerships between educators, researchers, and industry experts to create educational resources that are both effective and engaging. One potential approach is to work with AI professionals who feel comfortable with teaching these concepts to others. By providing them with the tools and support they need, we can empower teachers to take on this role and create a new generation of learners who are equipped with the skills and knowledge needed to thrive in an increasingly complex world.

In addition, there's also a need for accessible resources that parents and educators can use to try out AI-based educational activities with their children. Fortunately, there are already some excellent resources available, including short activities based on Scratch that can be tried by anyone with a child between the ages of 7 or older. By providing these kinds of tools and support, we can create a community of learners who are excited about exploring the possibilities of AI.

**Conclusion**

The field of early computer science education is undergoing a significant transformation, driven in part by advances in deep reinforcement learning and other areas of AI research. By breaking down complex concepts into manageable pieces and providing accessible resources for teachers and parents, we can create educational experiences that are both engaging and effective. As one researcher noted, "it's not just about teaching children to code – it's also about helping them understand the world in a new way." With the right approach and support, we can empower young learners to become the next generation of innovators and leaders.

"WEBVTTKind: captionsLanguage: enhello and welcome to another episode of twimble talk the podcast why interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam Cherrington a quick update on our deep learning study group as you know we're huge fans of the fast not a I courses and we recently had our second group of deep learning learners complete the fast a I deep learning for coders course back in December I'm excited to announce that we'll be hosting our third group of students taking the part one course starting this Saturday morning February 2nd this study group will run seven weeks finishing just in time for participants to jump right into the deep learning for coders part two course which is set to start in mid-march for details on the study groups or to get registered visit to Amelia comm slash meetup while it NURBS this past December I had the pleasure of attending the second annual black in AI workshop and dinner which brings in participants from all over the world to showcase their research share experiences and support one another I was fortunate enough to spend the day at the workshop and I'm excited to share with you over the course of this month conversations with just a few of the great members of this community to keep up with the series visit to Amelia comm slash black in AI 19 to get this series kicked off we're joined by Randy Williams PhD student at the MIT Media Lab at the black and AI workshop randy presented her research on pop pots an early childhood AI curriculum which is geared toward teaching preschoolers the fundamentals of artificial intelligence in our conversation we discussed the origins of the project the three AI concepts that are taught in the program and the goals that Randy hopes to accomplish with her work this is a fun conversation it was super thought-provoking enjoy all right everyone I am on the line with Randy Williams randy is a PhD student at the MIT Media Lab Randy welcome to this week in machine learning and AI hi thank you Sam for inviting me I'm happy to be here to talk to everyone about my work absolutely so we had an opportunity to meet at nurbs recently in fact he presented at the black and AI workshop there and I was really fascinated by the work you're doing in teaching preschool children about artificial intelligence what what sparked your interest in in doing that and teaching yes young children about AI yeah thanks the workshop that was definitely incredible and I was happy to share my work with the people there so I am a PhD student at the Media Lab I've been working on this project for about three years and when it started it wasn't about AI and it wasn't necessarily about preschool children either it was about computational thinking and how do we help students who might not have access to fancy robotic tool kits or to teachers how do we help them start to learn about these things so how do we like spread the influence of the whole craze about CS into different populations so I personally am from well Prince George's County Maryland but my family's from Baltimore and I went to school in Baltimore and while I was an undergrad I spent a lot of time working with like inner-city children and you know doing maker spaces Arduino workshops that kind of thing and what was really awesome about it was how engaged students were with you know learning these different things but it also made me a bit sad that you know so many of my classmates in school they were like oh yeah I've been programming since I was seven and you know these students you know they were like high schoolers and they're just getting started with this so I just felt really strongly that you know there needs to be more done to help everyone have a chance to learn about these things early on and what was really difficult in Baltimore was that there just weren't a lot of teachers when a lot of people who knew about the field to come and you know teach the kids and show the expertise so then the group that I started working in in the MIT Media Lab it's called the personal robots group and my professor is Cynthia Brazil and she's really passionate about how AI and robotics can help us flourish as human beings and so a lot of our work has been about education and how robots play a role in that and so I started out just building a robot that could help children learn how to program and sort of like be the fun interactive learning companion to help them you know figure things out and push them to solve new problems and things like that so the absence of a trained teacher how can a robot help children learn about these kinds of things and it was really fun I started thinking about you know like so what makes the most sense you know having some experience in Baltimore is like well no one's gonna go buy a two hundred dollar robot to do this so how do we make something less expensive so the paw pods project that I'm working on it's mostly based around a mobile phone so the mobile phone is the intelligent robot that children program and then I was also thinking well how do we break away from computer sciences you know solving mazes or doing puzzles and really open it up to different interests that students might have so the robot you know can become a character you can make it look like whatever you want it can control the lights around your room and things like that um so it's sort of like opening doors for you know art and theater and music even and then somewhere along the way I was having a conversation with Cynthia one day about the project in the direction and she's like you know Randy it work is great but you should really think about a high because AI is the next big thing and no one's really doing like AI education and you know as a student like okay well I've taken AI classes in college I don't really know how I'm gonna teach children AI but yeah sure you know go ahead and try it so that I quickly pivoted and how did I end up with preschool children well there were a lot of like robot cakes Bryleigh seven to 10-year old age and I just I guess enjoy not sleeping and like solving really crazy problems so I was like I'm gonna go like right below that I'm gonna do the five to seven-year-olds and that's it worked out I guess so we have a five to seven year old preschool AI school kid that I'm working on that's awesome so is the new lab is that its own department or are you affiliated with computer science or robotics department and kind of what I'm also curious about here is have you also received any formal training in education and how do you kind of think about the interdisciplinary nature of your project yeah so the Media Lab is this weird crazy place I work in the robotics lab the people next to me they do devices that go underneath your skin to help monitor your health and the people on the other side and made your neurobiology so it's a department with pretty much anyone who has a crazy idea that doesn't fit into like normal science or engineering department you know that's the place that you go to do the work so as a result I get to do this project that's very interdisciplinary I get to think about art I get to go to schools and do education work I get some robots and you know I also have lots of resources around the lab where people do all of these things and can sort of help contribute to the project and help me grow my ideas so I don't actually have formal training and education um however a lot of the work that I'm doing is built on this program called scratch so scratch is this website where children ages 7 and up can go and learn about CS and it was I don't know so currently the leader of the lab is Mitchel Resnick but I believe it was started by Seymour Papert like you know logo turtles if you don't know what that is it's these turtles where they you know Seymour Papert back in the 60s and cycle all kids should know how program and all kids should you know be able to do this and this is when like computers weren't even very popular said it was like dude you're crazy but you know he's like yeah I'm gonna do it so he started building this programming language for children and you know generations later there's this online portal portal where literally children millions of children all over the world are learning how to program so even though I didn't have the background the CS education background to know how to do it I got to work with Mitch and I took his class and I learned from the students and can start to pull those things in I also didn't necessarily have a background in robotics I did Computer Engineering and undergrad but mostly like building little devices not things that were like big and interacted with people so I learned a lot from my group and then everyone at the Media Lab there kind of just you know like artists so I'm not an artist an engineer but I became an art isn't it's able to pull that in so really with awesome about interdisciplinary work is that you get to pull in from all these different fields you know talking to developmental psychologists Ken kids understand AI are they ready to do that yet what are the right ways to translate the information so that it makes sense to them I really actually been inspired by all the people who have gotten to work with so when we think about teaching AI to preschool kids yeah obviously we're not trying to teach gradient descent like how do you how do you kind of break down or maybe take it from the other direction what are your goals and trying to teach AI to children at this age level yeah I would say my primary goal is to give children agency and the world around them so before I even you know put out this whole kid started like actually testing with children and building things I did a whole series of studies with other people in my group around what do children think about AI so we would have them interact with toys if you look at kids toys now they're like amazing they're really cool so they have like these little robot things called Cosmo which like they move around and they can play games against you and they're super cute and you can program them and stuff too but you know it's like really I that is being marketed to children then there is all this controversy about this Barbie doll I'm a Barbie that they talk to you and every kid in the world I think has had a conversation with Syria or Alexa you know not every kid in the world but quite a few have so it's interesting to see how in our time you know computers were just kind of like coming to be and the internet was just coming to be and children are growing up in a world now where it's like yeah yeah it's kind of a thing like it's normal to see that so I was like okay so when a child interacts with this thing that's not a life not a human it can talk to them seems kind of smart you know what are they thinking what's going on in their head and oftentimes they're like kind of just figuring it out they're like okay it talks to me like you know let's poke it with see if it can answer questions about dinosaurs or sloths or does it know what's in the grocery store down the street things like that and then they're like well can I break it hey do you want this Apple you know asking these kinds of things to computers and just to see what it will say or hey do you have a boyfriend you have a girlfriend funny things like that but even more so they kind of didn't really understand how it worked and to me that's like an opportunity in a challenge for today because we don't really want children to have toys that they can't pick apart and understand you know they're like like so won't answer my question does she not like me and it's like no Alexis you know and I'll P algorithm just is a program for young children's voices because it was made by adults who thought only adults would be using it so you know that's what's going on but if you say that to a kid they'll look at you like what are you saying yeah I have no idea what's going on so the goals of the curriculum or related to help children break those kinds of ideas down like Oh Alexa isn't working because Alexa was trained a certain way and if you try and have lessons do things outside of the way that she was trained then she's not gonna get it like that's kind of the right level that I think any child should have it makes me think a little bit of a couple of examples one that comes to mind is when children like you know who are raised on iPads see magazines and start wrapping it wanting it to do something or the other example that comes to mind is knowing how to really effectively search Google it's it's a powerful skill but you know both of these things I think illustrate you know like mental models that are created over time about this thing that you're interacting with that you know in the case of an iPad it's just like this piece of glass but you kind of develop this model about like how you know flat things work I guess or you know in the case of a search box like you know how do you can really effectively use you know this world that's behind the search box is part of your work here trying to shift the mental model that kids have about a I actually really love that framing and I might have to yeah so that is literally what I'm doing so a part of the actual pop bot study that I did so I have children interact with the AI and not just interact with it also have them like building algorithms from scratch not the whole thing they're not writing the programming but pretty much they have a lot of control over the way the algorithm works so we do that at the end of the day they have this finished AI product that seems intelligent and before and after they're learning about AI ask them what do you think about this thing that's a funny do you think it's alive is that a person is it a toy as an adult isn't a child is it smarter than you are you smarter than it and I'm asking like these questions because I expect a mental model change to happen when children are learning about AI and I'm wondering you know how can we make sure that you know there's so many privacy insecurities like safety concerns around having something that's always recording you in the house so what are children's mental models and how does learning about AI impact that how does it change the way that they want to interact with these things and it Beecher like and what were some of the results you saw in before and after surveys mostly very strange thing so before well the studies that I did like two years ago I interviewed four to ten year olds about AI and the like older kids they like eight to ten girls they were like solid they knew exactly what they thought about AI and you know they're like it's not a person it's not quite a toy it's somewhere in the middle like they knew what they were doing but the younger children they kept telling me I don't know I don't know I don't know and I was getting frustrated you know three six I'm like I don't know it's not gonna get me a paper and so what I found before it was pretty much the same children were like I'm not really sure what this thing is like you know we're all over the place people saying yes people saying no people saying oh it's said my name so I guess it's pretty smart or it didn't say my name or it doesn't know my favorite song about the train so it's not smart you know just things that were very hard to understand which is interesting because parents are also looking at this and they're like I'm not sure what my child thinks about Alexa you know they keep calling it their best friend's so what is this relationship but afterwards there was some interesting differences so I split my age grouping into like pre-k and kindergarten children and so after they had learned about AI the pre-k children were like oh now I understand it so yeah I would say this thing's pretty smart and the kindergarteners like oh I don't think it's smart anymore like I thought of a smart before but now that I get how it works it's like nope like okay and then I also created like assessments like very simple multiple-choice questions to ask how much do children really understand about the activities that I had given them and how much more thing understanding about the AI concepts so the ones who did the best and the AI concepts were like you know not that I played with this thing I'm like yeah it's kind of like a person like it thinks sometimes in ways that I think and yeah it could be smarter than me to it could learn they can get better and better but interestingly the children who didn't understand activities very well where the opposite they're like nope not smarter than me it's just a toy you know it's fun but it's it doesn't seem to be as alive or as human to me and so I'm still undecided about what conclusions I want it drop of that but I definitely think it's interesting that there is a big difference like the children who understood AI versus the children who didn't understand I saw the technology in very different ways so at the very least it sounds like teaching children about AI does cause something interesting to happen something interesting and hopefully not negative so it seems like a good motivator to continue with the work and continue to explore this I can't help but think that teaching adults about AI would have the same positive Thanks you know you think about kind of some of the the mass media coverage of AI and some of the you know hysterias that you read about they often kind of Bulai this is just lack of understanding and have you thought about creating an adult version of your curriculum so I haven't thought about it myself but sort of you know even to discuss it to you we're mentioning about the more that people understand the less scared they might be I think that the way that you know AI conversations are just playing out in society right now it's kind of like a little bit of both so there are some people I guess like myself and people in my lab who are like you know look at AI look at how much of good it can do like we can use it to build this you know this good thing that could make this good thing and I would say that we're experts but then there are also people who are extremely wary of AI so they're like they see this technology coming and they see all the negative ways that it can be used and there like cautious too like you know completely repulsed by it like no we even if a I can do good things we shouldn't build it because in the wrong hands it can just be too powerful and too destructive and so I think that any adult education AI thing would probably have an interesting time trying to wrestle with that you know not trying to push people in any particular direction but to let them come up with their own interpretation of how they think the technology should continue yeah that's a really interesting point I definitely get where you're coming from you know it's almost like the thing that you're afraid of is it really the thing it's not that there's nothing to be afraid of or not even afraid but worried about there are there are genuine concerns but you know it's it's not necessarily kind of this Terminator scenario and the next like the thing where it's like all online like that's a bit more unlikely but I wish you guys know terminators let's maybe talk about the curriculum that you developed as part of pop bots what are the core AI concepts that you're trying to teach these children so the three that I started with where knowledge based expert systems supervised machine learning and generative AI and I started with these because they seem to be the most relevant to what children were experiencing so a lot of their simple toys we're using these you know kinds of concepts and them and I could easily make connections between when your toy does this this is what's happening underneath it and to me that seemed like the most important things to teach children at first so you know knowledge-based systems let's go through each of those and maybe you can provide an example of the way a child might experience that in kind of everyday toys and how you introduce that in the curriculum knowledge-based systems for rule-based expert systems or expert systems they have multiple names often come up in natural language processing now a lot of NLP also uses you know deep learning but you know in the past you know back in the good old days it came up a lot in natural language processing as well as medical diagnosis and even the video game characters that you know aren't the main characters but the ones that I don't know if you play a lot of our PG games when you're battling the random townsman like those kinds of characters were all controlled by knowledge-based expert systems rule-based systems and so it was easy to you know talk to children about it's like oh when you're playing tic-tac-toe against your smart computer or when you're doing that video game or when you're talking to Alexa there's probably a bit of this point on underneath and the way that we did the activity was we played rock paper scissors so rock paper scissors unique knowledge like their rules which is you know rock beats scissors paper beats rock etc and so children would literally program these rules into the robot all of the interfaces are a completely picture based because I was working with children who were so young that they couldn't read so it was an interesting design challenge to say okay powerful AI absolutely no words whatsoever and like nomads and stuff like that so children pretty much put like pictures of a paper hand and then an area like greater than sign and then a rock which means paper beats rock and so they put in all the rules and now the robot has its knowledge base and it can use that to make decisions about what to play so when the child is actually playing against the robot then the robot will kind of keep track of their moves and it'll use that to say oh well I think that you're gonna play paper next and you told me that scissors beats paper so I'm gonna play scissors and then it's revealed you know did the child actually play a paper and a lot of times you know after a couple rounds of the game yeah there abouts pretty good at guessing and they're like oh my god the robot got so smart but what's really cool about that is it didn't start off smart it actually starts off losing a lot and as it keeps going it gets smarter so children start to see how this intelligence didn't disappear it was learned over time also the children gave the robot the rules to the game so the children actually played a part in helping the robot become intelligent with even more funny is that the kids will be like I'm gonna cheat and they'll like switch all the rules around backwards and the robot will be like well I guess that paper beats scissors so does that mean that I went and the child's like laughing their head off like ha ha ha but that's actually interesting point because you know expert Bayes rule systems one of the ethical issues is what if the rule that you're teaching isn't correct so children even get to explore that idea like ok if I have a car that's driving by itself and I teach it the wrong rules what happens and you can see like this look of realization like you know move over the child's face and they're like oh my god that would be so bad I'm like yes like you're understanding like you know the impact the real-world impact of AI and then also children will teach the robot how to react to winning losing and getting a tie so you know like the robot can be a sore winner and like every time it wins it's like ha ha I win you lose and like it makes this farting sound so that's like one innovation you can have it when humbly it's like oh that was a good game so children are also you know it's kindergarten preschool children also need to have some way that you know social interaction social learning is coming into this as well so that's a part of it too I'm curious in this first part the rules based systems when you when you the child was programming the robot teaching the robot how to respond via these rules were you also introducing some notion of probabilistic systems or responses so in the sense that the robot was learning over time what the child is most likely to do next yes it was a little bit tricky because probability doesn't come up for in like early education but we would make these rule trackers or these game trackers rather where the children would write down you know what moves they put and the robot would say well I think you're gonna put paper next because you put paper like three times out of the five times that we just played and the child can look back at what they did and they can start to see oh yeah like one two three like those at times name picks paper so yeah that's that's the sense in which I envisioned it yeah yeah it was like it's not like super deep but yeah that's about how far you can go but you kind of raised that as a like that was made explicit in in the curriculum thinking about you know these you know percentages or frequency types of numbers yeah absolutely and the entire time the robot is like saying these things that's pointing out what its knowledge is and it's explaining why it's making decisions so that the child can understand it and I think my hope is that you know when they get older if they have another AI class they can revisit these ideas and actually learn about probability and it all starts to make sense so like even reinforcing those ideas later and so just a point of clarification you are referring back to this idea of a robot is this an entirely software based robot just on the smartphone or is there hardware component as well ah yes I should have explained that so the robot is a mobile phone with this social robot technology that we've developed in our lab so it can talk it has all these really cute fun animations I can listen to you it has a camera but around it I've built a Lego body so there's two different bodies than working on now one uses Lego we do which is like this $200 motor kit and you can use normal Legos and then you can add these motors to it so now the robot can move and but because it's Lego you can change the way that it looks so sometimes it's a car some children really wanted to play with one that's spun around so they can sort of have total control over the way that their robot is so they're programming it they're building it like all of it is brought down to the child's level and then the other one I imagine is our Dino so now I'm like you know thinking about you know slightly older kids and even more fun things to build so building an Arduino platform for it too so then back to these concept we just talked about the rules based types of systems you mentioned supervised machine learning as well yeah so supervised machine learning comes up in YouTube kids which surprisingly a lot of children were interacting with so you know I would ask them you know how does YouTube know which movie you want to watch this and they're like oh that just you know it it just picks whatever is random and I'm like no it's not random it usually picks things that are kind of like the video you just watched like oh yeah I guess you're right and so we can talk about supervised machine learning so that's when you label some things for example is good and bad so in the case of YouTube kids children are labeling things as things that I want to watch by watching it and then they can also give extra feedback thumbs up thumbs down so that the robot sorry not the robot YouTube's algorithm can learn better but it's often used for like recommender system so YouTube it can be Netflix children don't have email yet but sometimes they can kind of get what I mean if I throw that in there but for children what we do is we sort foods into healthy and unhealthy groups so rock-paper-scissors is nice because it has two rules but if you want to teach a robot about which foods are healthy and unhealthy you know I have children think about like how many foods are there like how many foods would you have to teach the robot about and after they kind of like hit 2:30 they backs out and they're like oh my god that's so many foods so I'm like okay there's a better way we can give the robot a few examples and it can learn to make guesses on its own so sort of on the back end something I do beforehand the robot has this database where it has like the color of foods with food group it's and how much sugar it has all of these different and then it's going to use the k-nearest neighbours algorithm to sort of say well this food has this many features similar to this other food so maybe these to our nearest neighbors as opposed to this other food so a good example is like bananas would be more similar to lemons than chocolate so what I have children do they have this like list of twenty foods and I say we're gonna label two in them so they label you know either both of them good or both of bad take whatever food you want and then we're say okay now let's ask the robot to guess like whether this food is healthy or not and so they start to see like okay if I tell the robot that strawberries and tomatoes are healthy and then ask about chocolate it's gonna think chocolate is healthy too because I haven't given it any bad examples so I have to do better okay let's teach the robot the chocolate is not healthy now we have strawberries and tomatoes and the good chocolate in the bed let's ask it about ice cream and the robot can say oh well ice cream is probably closer to chocolate than it is to the other things because they both you know are in the sweet section they have a lot of sugar so it's chocolate unhealthy too and boom like magically the robot seems intelligent it seems like it's learning it ends up being that after we teach the robot about like five foods then it can sort of guess the other 15 foods that remain but it all depends on how good the training set is so I don't use the word training set with a five-year-old right the idea is still the same it's like so we only told it about these five foods and it learned about these 15 foods like what if all the five foods have been good would it do a good job what if they were all bad that do a good job and they can start to seeing how the robot you know needs certain examples a certain quality like if we only teach it about red foods and we asked it about blue foods it's probably gonna be a bit confused because like you haven't given it a good enough training set that one's usually really fun and then again of course children when it trick the robot it's like well I like chocolate so I'm putting chocolate the good side like okay well you can do that of course that impact the robot and then we can discuss that as well and then the last activity is generative AI and this was one that I thought was really important because AI doesn't just follow rules and it doesn't just classify things and make rules sometimes it is creative and it can be used in art this particular activity is about music so first children give the robot parameters about different emotions and how they would sound is music so happy music sounds you know kind of fast and upbeat and it also sort of goes up in court progression so like rather than going like turn turn turn no it's gonna go out din din din so they teach the robot that by sliding these two bars like chord progression up and music fast and then they do sad they're like chord progression down music maybe a little bit slow and then excited so chord progression up music bass or scared chord progression down but music fast and they teach the robot about what different emotions should sound like a songs and then I drive all the teachers and there I'm crazy we start playing music with the robots so children have this piano where they can play a song and then the robot will take whatever song they make and it will remix it according to the different emotions so it's just like really noisy but a lot of fun is like children are like you know playing songs and hearing the robot play the song back and it's sort of like going back and forth with this turn taking and then you know we ask questions like so the robots songs sound like your song so if you tell it not to change anything yes if you tell it to go faster it'll just change the progression a little bit if you tell this go slower it'll you know make it a bit slower sometimes it'll add new notes if you tell the chord progression to go up but you play a Down chord progression and then how does that impact the emotions it's like oh well it seems like it's kind of picking randomly but all the happy songs kind of start to sound the same so it's really cool to watch children sort of do this less structured like you know it's not right wrong answer it's how does it sound and they're making music and then they get up and they play a class Orchestra and then we turn the top after all of that stimulation so to be clear in this this third concept how are you getting at the AI element of of what's happening so examples of this AI come up like if you look at some of Google's AI experiments they have this piano where you can play music along with a computer but the question is how does that computer know what to play and so in this activity children are actually setting parameters for how the computer should change the song to make it sound a particular way so if I give you if they give the robot rather an input with three nodes CDE and they tell it to make it faster and happier than the robot should return something according to the parameters faster and maybe it'll go up ceg like even higher than the child's output so they start to see how they can use AI to create and to create new things and then you mentioned previously the some of the surveys you did before and after was that before and after children go through this curriculum or there are there additional observations that you made about their experiences and what they learned having gone through this curriculum beyond what we've already talked about yeah so there was the before and after about how children feel about AI and about robots I also did it before and after about how children feel about Engineers so one of my I guess pet pieces an engineer is that you know there's a lot of emphasis on science and mathematics and stem but often technology and engineering it's a lot harder to do so it gets less attention so as I went through the curriculum I was hoping that children would have a better sense of what engineers do and why engineering is fun and unfortunately they did it and I think that you know a lot of that is because it was a very new concept to them so to tell her hilarious story the person they don't went in I was like okay who here knows when the engineers and you know and this classroom of 20 like two children raise their hand I was like okay that's good we have to do better than that and so I you know pointed to one of the children and I said okay you tell me what is an engineer tell us all he's like an engineer is someone who drives a train and further than I even thought we were so you know part of doing this research that I find you know like personally enjoyable is that I get to say well I'm an engineer I'm an engineer because I build things and I build things to help people and starting to have children you know think about that as a different new career path I kind of wish going back that I had built more but into this curriculum that I built so you know these activities were fun and they were playing games but at the end of the day they didn't get to see how the things they were building could be useful or how they could help other people or how they could bring joy to other people and I think that's why you know at the end when I was like okay who wants to be an engineer I still got crickets because it's like alright we need to do a better job of helping children sort of like see themselves as this but also see the value of it in society and then also just like I said I did a sort of AI assessment so how much did children learn about these things the AI assessment was like ten questions all about the different activities and some of them are kind of tricky it was like I think I mentioned before if you only teach the robot about good foods where will it take chocolate goes for a five-year-old you know like of course everyone knows chocolates unhealthy but it's very difficult for them to see like okay wait but this AI algorithm only knows foods that I taught it only foods that I've labeled so to always use those labels to make its guesses so it's really cool to see like a lot of children start to get those things right because it kind of like blew the the development psychology literature you know out of the water they were like I'm not sure until Jen can do this kind of reasoning yet know might know they did it it was awesome that's fantastic yeah so yeah anyways ten questions some of them a bit tricky and I think the median score was like 70% so I mean obviously we shouldn't assess children to have a lead that's probably not healthy for them but they understood a good amount of what was presented in front of them and I think that's really encouraging and important it's I mean the same with like early computer science education I was very easy to say children can't understand this it's too complex but the way that something is designed it can be made accessible to children so you know right now I'm working on a deep reinforcement learning activity so pretty much we're gonna build agents that can play snake so first we're gonna hand code it and then we're gonna use a simple neural net where we like give it a bunch of examples and then we're gonna have it do deep learning and I'm feeling very confident that these children can understand it because you know if you just are able to break something down enough they can get it wrong so if you're a deep RL person I'm gonna have some five-year-olds coming for your job pretty so get ready this is awesome work where do you go from here so this was really the the center of your master's thesis where do you see it going beyond that there are so many different things that I want to do and so much work to be done so some really things that are going on that there are others in my lab who I was kind of like the person to go and try this and see if it we're doing that so now that it works other people in my lab are also trying out their own experiments one of my lab mates veena is doing this work around you know when children are learning with AI how can that impact their creativity so they're not just learning about AI anymore they're also learning to be more creative and to be explorative as they're learning which will have like huge benefits from the for them you know beyond just learning a particular skill another student my group Blakely not student my group like I'm their professor another one of my lab mates Blakely is working on AI ethics curriculum so really helping children be able to understand the ethics behind every AI decision so that they can critically evaluate the things that around them but also when they're building things you know why teach children to build something if they don't know how to build it ethically like at the same time actually be thinking about both of these things so I'm really excited about that work personally I've gotten a lot of feedback from teachers like this is great I have no idea about anything it has so can you teach me so I'm trying to figure out the problem of how can we actually make this something that teachers can like use and feel empowered to use and not scared of so that it can really get into classrooms and get into the spaces in Baltimore that I used to work in and then I think also just making more cool activities like in my dream of Dreams this will become like this big online platform and children everywhere can learn about AI and ways that are meaningful to them so you know there has to be a lot more content behind it beyond these three activities what other things can children learn and what are other metaphors that make sense for them right so yeah that's all the things I'm gonna do awesome awesome and are there are there things that you have identified that you need meaning if there's you know some potential partner out there that you know someone in our listening community might you know be connected to anything come to mind in that regard I mean yes I can visit a plug for things so I think one thing that would be really awesome is to have an AI person someone who feels comfortable with AI who's really passionate about teaching this who I can sort of help get started with their own activities so I'm doing that with teachers right now and I think the biggest problem is that they're not comfortable with AI and there's a lot of work that I have to do to get them there so I'm wondering how might it be different if I take an AI person and start to give them tools to be teachers that would be really cool also if you kind of just want to try things out with your kids and experiment and ask questions some of the papers that are linked and the website that we have have actually I resources that parents can go on and try it right now I'm just like short activities based on scratch so unfortunately you have to have a kid between the ages of 7 or maybe over there but they're already things that exist that people can try that I highly recommend them try and give us feedback on well Randy thanks so much for taking the time to share what you're working on with us is really cool stuff and I'm looking forward to seeing how it evolves yeah thank you I really appreciate the opportunity thanks alright everyone that's our show for today for more information on randy or any of the topics covered in the show visit Twilio comm slash talks last two to five for more information on the black and AI series visit twimble AI comm / black in AI 19 as always thanks so much for listening and catch you next timehello and welcome to another episode of twimble talk the podcast why interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam Cherrington a quick update on our deep learning study group as you know we're huge fans of the fast not a I courses and we recently had our second group of deep learning learners complete the fast a I deep learning for coders course back in December I'm excited to announce that we'll be hosting our third group of students taking the part one course starting this Saturday morning February 2nd this study group will run seven weeks finishing just in time for participants to jump right into the deep learning for coders part two course which is set to start in mid-march for details on the study groups or to get registered visit to Amelia comm slash meetup while it NURBS this past December I had the pleasure of attending the second annual black in AI workshop and dinner which brings in participants from all over the world to showcase their research share experiences and support one another I was fortunate enough to spend the day at the workshop and I'm excited to share with you over the course of this month conversations with just a few of the great members of this community to keep up with the series visit to Amelia comm slash black in AI 19 to get this series kicked off we're joined by Randy Williams PhD student at the MIT Media Lab at the black and AI workshop randy presented her research on pop pots an early childhood AI curriculum which is geared toward teaching preschoolers the fundamentals of artificial intelligence in our conversation we discussed the origins of the project the three AI concepts that are taught in the program and the goals that Randy hopes to accomplish with her work this is a fun conversation it was super thought-provoking enjoy all right everyone I am on the line with Randy Williams randy is a PhD student at the MIT Media Lab Randy welcome to this week in machine learning and AI hi thank you Sam for inviting me I'm happy to be here to talk to everyone about my work absolutely so we had an opportunity to meet at nurbs recently in fact he presented at the black and AI workshop there and I was really fascinated by the work you're doing in teaching preschool children about artificial intelligence what what sparked your interest in in doing that and teaching yes young children about AI yeah thanks the workshop that was definitely incredible and I was happy to share my work with the people there so I am a PhD student at the Media Lab I've been working on this project for about three years and when it started it wasn't about AI and it wasn't necessarily about preschool children either it was about computational thinking and how do we help students who might not have access to fancy robotic tool kits or to teachers how do we help them start to learn about these things so how do we like spread the influence of the whole craze about CS into different populations so I personally am from well Prince George's County Maryland but my family's from Baltimore and I went to school in Baltimore and while I was an undergrad I spent a lot of time working with like inner-city children and you know doing maker spaces Arduino workshops that kind of thing and what was really awesome about it was how engaged students were with you know learning these different things but it also made me a bit sad that you know so many of my classmates in school they were like oh yeah I've been programming since I was seven and you know these students you know they were like high schoolers and they're just getting started with this so I just felt really strongly that you know there needs to be more done to help everyone have a chance to learn about these things early on and what was really difficult in Baltimore was that there just weren't a lot of teachers when a lot of people who knew about the field to come and you know teach the kids and show the expertise so then the group that I started working in in the MIT Media Lab it's called the personal robots group and my professor is Cynthia Brazil and she's really passionate about how AI and robotics can help us flourish as human beings and so a lot of our work has been about education and how robots play a role in that and so I started out just building a robot that could help children learn how to program and sort of like be the fun interactive learning companion to help them you know figure things out and push them to solve new problems and things like that so the absence of a trained teacher how can a robot help children learn about these kinds of things and it was really fun I started thinking about you know like so what makes the most sense you know having some experience in Baltimore is like well no one's gonna go buy a two hundred dollar robot to do this so how do we make something less expensive so the paw pods project that I'm working on it's mostly based around a mobile phone so the mobile phone is the intelligent robot that children program and then I was also thinking well how do we break away from computer sciences you know solving mazes or doing puzzles and really open it up to different interests that students might have so the robot you know can become a character you can make it look like whatever you want it can control the lights around your room and things like that um so it's sort of like opening doors for you know art and theater and music even and then somewhere along the way I was having a conversation with Cynthia one day about the project in the direction and she's like you know Randy it work is great but you should really think about a high because AI is the next big thing and no one's really doing like AI education and you know as a student like okay well I've taken AI classes in college I don't really know how I'm gonna teach children AI but yeah sure you know go ahead and try it so that I quickly pivoted and how did I end up with preschool children well there were a lot of like robot cakes Bryleigh seven to 10-year old age and I just I guess enjoy not sleeping and like solving really crazy problems so I was like I'm gonna go like right below that I'm gonna do the five to seven-year-olds and that's it worked out I guess so we have a five to seven year old preschool AI school kid that I'm working on that's awesome so is the new lab is that its own department or are you affiliated with computer science or robotics department and kind of what I'm also curious about here is have you also received any formal training in education and how do you kind of think about the interdisciplinary nature of your project yeah so the Media Lab is this weird crazy place I work in the robotics lab the people next to me they do devices that go underneath your skin to help monitor your health and the people on the other side and made your neurobiology so it's a department with pretty much anyone who has a crazy idea that doesn't fit into like normal science or engineering department you know that's the place that you go to do the work so as a result I get to do this project that's very interdisciplinary I get to think about art I get to go to schools and do education work I get some robots and you know I also have lots of resources around the lab where people do all of these things and can sort of help contribute to the project and help me grow my ideas so I don't actually have formal training and education um however a lot of the work that I'm doing is built on this program called scratch so scratch is this website where children ages 7 and up can go and learn about CS and it was I don't know so currently the leader of the lab is Mitchel Resnick but I believe it was started by Seymour Papert like you know logo turtles if you don't know what that is it's these turtles where they you know Seymour Papert back in the 60s and cycle all kids should know how program and all kids should you know be able to do this and this is when like computers weren't even very popular said it was like dude you're crazy but you know he's like yeah I'm gonna do it so he started building this programming language for children and you know generations later there's this online portal portal where literally children millions of children all over the world are learning how to program so even though I didn't have the background the CS education background to know how to do it I got to work with Mitch and I took his class and I learned from the students and can start to pull those things in I also didn't necessarily have a background in robotics I did Computer Engineering and undergrad but mostly like building little devices not things that were like big and interacted with people so I learned a lot from my group and then everyone at the Media Lab there kind of just you know like artists so I'm not an artist an engineer but I became an art isn't it's able to pull that in so really with awesome about interdisciplinary work is that you get to pull in from all these different fields you know talking to developmental psychologists Ken kids understand AI are they ready to do that yet what are the right ways to translate the information so that it makes sense to them I really actually been inspired by all the people who have gotten to work with so when we think about teaching AI to preschool kids yeah obviously we're not trying to teach gradient descent like how do you how do you kind of break down or maybe take it from the other direction what are your goals and trying to teach AI to children at this age level yeah I would say my primary goal is to give children agency and the world around them so before I even you know put out this whole kid started like actually testing with children and building things I did a whole series of studies with other people in my group around what do children think about AI so we would have them interact with toys if you look at kids toys now they're like amazing they're really cool so they have like these little robot things called Cosmo which like they move around and they can play games against you and they're super cute and you can program them and stuff too but you know it's like really I that is being marketed to children then there is all this controversy about this Barbie doll I'm a Barbie that they talk to you and every kid in the world I think has had a conversation with Syria or Alexa you know not every kid in the world but quite a few have so it's interesting to see how in our time you know computers were just kind of like coming to be and the internet was just coming to be and children are growing up in a world now where it's like yeah yeah it's kind of a thing like it's normal to see that so I was like okay so when a child interacts with this thing that's not a life not a human it can talk to them seems kind of smart you know what are they thinking what's going on in their head and oftentimes they're like kind of just figuring it out they're like okay it talks to me like you know let's poke it with see if it can answer questions about dinosaurs or sloths or does it know what's in the grocery store down the street things like that and then they're like well can I break it hey do you want this Apple you know asking these kinds of things to computers and just to see what it will say or hey do you have a boyfriend you have a girlfriend funny things like that but even more so they kind of didn't really understand how it worked and to me that's like an opportunity in a challenge for today because we don't really want children to have toys that they can't pick apart and understand you know they're like like so won't answer my question does she not like me and it's like no Alexis you know and I'll P algorithm just is a program for young children's voices because it was made by adults who thought only adults would be using it so you know that's what's going on but if you say that to a kid they'll look at you like what are you saying yeah I have no idea what's going on so the goals of the curriculum or related to help children break those kinds of ideas down like Oh Alexa isn't working because Alexa was trained a certain way and if you try and have lessons do things outside of the way that she was trained then she's not gonna get it like that's kind of the right level that I think any child should have it makes me think a little bit of a couple of examples one that comes to mind is when children like you know who are raised on iPads see magazines and start wrapping it wanting it to do something or the other example that comes to mind is knowing how to really effectively search Google it's it's a powerful skill but you know both of these things I think illustrate you know like mental models that are created over time about this thing that you're interacting with that you know in the case of an iPad it's just like this piece of glass but you kind of develop this model about like how you know flat things work I guess or you know in the case of a search box like you know how do you can really effectively use you know this world that's behind the search box is part of your work here trying to shift the mental model that kids have about a I actually really love that framing and I might have to yeah so that is literally what I'm doing so a part of the actual pop bot study that I did so I have children interact with the AI and not just interact with it also have them like building algorithms from scratch not the whole thing they're not writing the programming but pretty much they have a lot of control over the way the algorithm works so we do that at the end of the day they have this finished AI product that seems intelligent and before and after they're learning about AI ask them what do you think about this thing that's a funny do you think it's alive is that a person is it a toy as an adult isn't a child is it smarter than you are you smarter than it and I'm asking like these questions because I expect a mental model change to happen when children are learning about AI and I'm wondering you know how can we make sure that you know there's so many privacy insecurities like safety concerns around having something that's always recording you in the house so what are children's mental models and how does learning about AI impact that how does it change the way that they want to interact with these things and it Beecher like and what were some of the results you saw in before and after surveys mostly very strange thing so before well the studies that I did like two years ago I interviewed four to ten year olds about AI and the like older kids they like eight to ten girls they were like solid they knew exactly what they thought about AI and you know they're like it's not a person it's not quite a toy it's somewhere in the middle like they knew what they were doing but the younger children they kept telling me I don't know I don't know I don't know and I was getting frustrated you know three six I'm like I don't know it's not gonna get me a paper and so what I found before it was pretty much the same children were like I'm not really sure what this thing is like you know we're all over the place people saying yes people saying no people saying oh it's said my name so I guess it's pretty smart or it didn't say my name or it doesn't know my favorite song about the train so it's not smart you know just things that were very hard to understand which is interesting because parents are also looking at this and they're like I'm not sure what my child thinks about Alexa you know they keep calling it their best friend's so what is this relationship but afterwards there was some interesting differences so I split my age grouping into like pre-k and kindergarten children and so after they had learned about AI the pre-k children were like oh now I understand it so yeah I would say this thing's pretty smart and the kindergarteners like oh I don't think it's smart anymore like I thought of a smart before but now that I get how it works it's like nope like okay and then I also created like assessments like very simple multiple-choice questions to ask how much do children really understand about the activities that I had given them and how much more thing understanding about the AI concepts so the ones who did the best and the AI concepts were like you know not that I played with this thing I'm like yeah it's kind of like a person like it thinks sometimes in ways that I think and yeah it could be smarter than me to it could learn they can get better and better but interestingly the children who didn't understand activities very well where the opposite they're like nope not smarter than me it's just a toy you know it's fun but it's it doesn't seem to be as alive or as human to me and so I'm still undecided about what conclusions I want it drop of that but I definitely think it's interesting that there is a big difference like the children who understood AI versus the children who didn't understand I saw the technology in very different ways so at the very least it sounds like teaching children about AI does cause something interesting to happen something interesting and hopefully not negative so it seems like a good motivator to continue with the work and continue to explore this I can't help but think that teaching adults about AI would have the same positive Thanks you know you think about kind of some of the the mass media coverage of AI and some of the you know hysterias that you read about they often kind of Bulai this is just lack of understanding and have you thought about creating an adult version of your curriculum so I haven't thought about it myself but sort of you know even to discuss it to you we're mentioning about the more that people understand the less scared they might be I think that the way that you know AI conversations are just playing out in society right now it's kind of like a little bit of both so there are some people I guess like myself and people in my lab who are like you know look at AI look at how much of good it can do like we can use it to build this you know this good thing that could make this good thing and I would say that we're experts but then there are also people who are extremely wary of AI so they're like they see this technology coming and they see all the negative ways that it can be used and there like cautious too like you know completely repulsed by it like no we even if a I can do good things we shouldn't build it because in the wrong hands it can just be too powerful and too destructive and so I think that any adult education AI thing would probably have an interesting time trying to wrestle with that you know not trying to push people in any particular direction but to let them come up with their own interpretation of how they think the technology should continue yeah that's a really interesting point I definitely get where you're coming from you know it's almost like the thing that you're afraid of is it really the thing it's not that there's nothing to be afraid of or not even afraid but worried about there are there are genuine concerns but you know it's it's not necessarily kind of this Terminator scenario and the next like the thing where it's like all online like that's a bit more unlikely but I wish you guys know terminators let's maybe talk about the curriculum that you developed as part of pop bots what are the core AI concepts that you're trying to teach these children so the three that I started with where knowledge based expert systems supervised machine learning and generative AI and I started with these because they seem to be the most relevant to what children were experiencing so a lot of their simple toys we're using these you know kinds of concepts and them and I could easily make connections between when your toy does this this is what's happening underneath it and to me that seemed like the most important things to teach children at first so you know knowledge-based systems let's go through each of those and maybe you can provide an example of the way a child might experience that in kind of everyday toys and how you introduce that in the curriculum knowledge-based systems for rule-based expert systems or expert systems they have multiple names often come up in natural language processing now a lot of NLP also uses you know deep learning but you know in the past you know back in the good old days it came up a lot in natural language processing as well as medical diagnosis and even the video game characters that you know aren't the main characters but the ones that I don't know if you play a lot of our PG games when you're battling the random townsman like those kinds of characters were all controlled by knowledge-based expert systems rule-based systems and so it was easy to you know talk to children about it's like oh when you're playing tic-tac-toe against your smart computer or when you're doing that video game or when you're talking to Alexa there's probably a bit of this point on underneath and the way that we did the activity was we played rock paper scissors so rock paper scissors unique knowledge like their rules which is you know rock beats scissors paper beats rock etc and so children would literally program these rules into the robot all of the interfaces are a completely picture based because I was working with children who were so young that they couldn't read so it was an interesting design challenge to say okay powerful AI absolutely no words whatsoever and like nomads and stuff like that so children pretty much put like pictures of a paper hand and then an area like greater than sign and then a rock which means paper beats rock and so they put in all the rules and now the robot has its knowledge base and it can use that to make decisions about what to play so when the child is actually playing against the robot then the robot will kind of keep track of their moves and it'll use that to say oh well I think that you're gonna play paper next and you told me that scissors beats paper so I'm gonna play scissors and then it's revealed you know did the child actually play a paper and a lot of times you know after a couple rounds of the game yeah there abouts pretty good at guessing and they're like oh my god the robot got so smart but what's really cool about that is it didn't start off smart it actually starts off losing a lot and as it keeps going it gets smarter so children start to see how this intelligence didn't disappear it was learned over time also the children gave the robot the rules to the game so the children actually played a part in helping the robot become intelligent with even more funny is that the kids will be like I'm gonna cheat and they'll like switch all the rules around backwards and the robot will be like well I guess that paper beats scissors so does that mean that I went and the child's like laughing their head off like ha ha ha but that's actually interesting point because you know expert Bayes rule systems one of the ethical issues is what if the rule that you're teaching isn't correct so children even get to explore that idea like ok if I have a car that's driving by itself and I teach it the wrong rules what happens and you can see like this look of realization like you know move over the child's face and they're like oh my god that would be so bad I'm like yes like you're understanding like you know the impact the real-world impact of AI and then also children will teach the robot how to react to winning losing and getting a tie so you know like the robot can be a sore winner and like every time it wins it's like ha ha I win you lose and like it makes this farting sound so that's like one innovation you can have it when humbly it's like oh that was a good game so children are also you know it's kindergarten preschool children also need to have some way that you know social interaction social learning is coming into this as well so that's a part of it too I'm curious in this first part the rules based systems when you when you the child was programming the robot teaching the robot how to respond via these rules were you also introducing some notion of probabilistic systems or responses so in the sense that the robot was learning over time what the child is most likely to do next yes it was a little bit tricky because probability doesn't come up for in like early education but we would make these rule trackers or these game trackers rather where the children would write down you know what moves they put and the robot would say well I think you're gonna put paper next because you put paper like three times out of the five times that we just played and the child can look back at what they did and they can start to see oh yeah like one two three like those at times name picks paper so yeah that's that's the sense in which I envisioned it yeah yeah it was like it's not like super deep but yeah that's about how far you can go but you kind of raised that as a like that was made explicit in in the curriculum thinking about you know these you know percentages or frequency types of numbers yeah absolutely and the entire time the robot is like saying these things that's pointing out what its knowledge is and it's explaining why it's making decisions so that the child can understand it and I think my hope is that you know when they get older if they have another AI class they can revisit these ideas and actually learn about probability and it all starts to make sense so like even reinforcing those ideas later and so just a point of clarification you are referring back to this idea of a robot is this an entirely software based robot just on the smartphone or is there hardware component as well ah yes I should have explained that so the robot is a mobile phone with this social robot technology that we've developed in our lab so it can talk it has all these really cute fun animations I can listen to you it has a camera but around it I've built a Lego body so there's two different bodies than working on now one uses Lego we do which is like this $200 motor kit and you can use normal Legos and then you can add these motors to it so now the robot can move and but because it's Lego you can change the way that it looks so sometimes it's a car some children really wanted to play with one that's spun around so they can sort of have total control over the way that their robot is so they're programming it they're building it like all of it is brought down to the child's level and then the other one I imagine is our Dino so now I'm like you know thinking about you know slightly older kids and even more fun things to build so building an Arduino platform for it too so then back to these concept we just talked about the rules based types of systems you mentioned supervised machine learning as well yeah so supervised machine learning comes up in YouTube kids which surprisingly a lot of children were interacting with so you know I would ask them you know how does YouTube know which movie you want to watch this and they're like oh that just you know it it just picks whatever is random and I'm like no it's not random it usually picks things that are kind of like the video you just watched like oh yeah I guess you're right and so we can talk about supervised machine learning so that's when you label some things for example is good and bad so in the case of YouTube kids children are labeling things as things that I want to watch by watching it and then they can also give extra feedback thumbs up thumbs down so that the robot sorry not the robot YouTube's algorithm can learn better but it's often used for like recommender system so YouTube it can be Netflix children don't have email yet but sometimes they can kind of get what I mean if I throw that in there but for children what we do is we sort foods into healthy and unhealthy groups so rock-paper-scissors is nice because it has two rules but if you want to teach a robot about which foods are healthy and unhealthy you know I have children think about like how many foods are there like how many foods would you have to teach the robot about and after they kind of like hit 2:30 they backs out and they're like oh my god that's so many foods so I'm like okay there's a better way we can give the robot a few examples and it can learn to make guesses on its own so sort of on the back end something I do beforehand the robot has this database where it has like the color of foods with food group it's and how much sugar it has all of these different and then it's going to use the k-nearest neighbours algorithm to sort of say well this food has this many features similar to this other food so maybe these to our nearest neighbors as opposed to this other food so a good example is like bananas would be more similar to lemons than chocolate so what I have children do they have this like list of twenty foods and I say we're gonna label two in them so they label you know either both of them good or both of bad take whatever food you want and then we're say okay now let's ask the robot to guess like whether this food is healthy or not and so they start to see like okay if I tell the robot that strawberries and tomatoes are healthy and then ask about chocolate it's gonna think chocolate is healthy too because I haven't given it any bad examples so I have to do better okay let's teach the robot the chocolate is not healthy now we have strawberries and tomatoes and the good chocolate in the bed let's ask it about ice cream and the robot can say oh well ice cream is probably closer to chocolate than it is to the other things because they both you know are in the sweet section they have a lot of sugar so it's chocolate unhealthy too and boom like magically the robot seems intelligent it seems like it's learning it ends up being that after we teach the robot about like five foods then it can sort of guess the other 15 foods that remain but it all depends on how good the training set is so I don't use the word training set with a five-year-old right the idea is still the same it's like so we only told it about these five foods and it learned about these 15 foods like what if all the five foods have been good would it do a good job what if they were all bad that do a good job and they can start to seeing how the robot you know needs certain examples a certain quality like if we only teach it about red foods and we asked it about blue foods it's probably gonna be a bit confused because like you haven't given it a good enough training set that one's usually really fun and then again of course children when it trick the robot it's like well I like chocolate so I'm putting chocolate the good side like okay well you can do that of course that impact the robot and then we can discuss that as well and then the last activity is generative AI and this was one that I thought was really important because AI doesn't just follow rules and it doesn't just classify things and make rules sometimes it is creative and it can be used in art this particular activity is about music so first children give the robot parameters about different emotions and how they would sound is music so happy music sounds you know kind of fast and upbeat and it also sort of goes up in court progression so like rather than going like turn turn turn no it's gonna go out din din din so they teach the robot that by sliding these two bars like chord progression up and music fast and then they do sad they're like chord progression down music maybe a little bit slow and then excited so chord progression up music bass or scared chord progression down but music fast and they teach the robot about what different emotions should sound like a songs and then I drive all the teachers and there I'm crazy we start playing music with the robots so children have this piano where they can play a song and then the robot will take whatever song they make and it will remix it according to the different emotions so it's just like really noisy but a lot of fun is like children are like you know playing songs and hearing the robot play the song back and it's sort of like going back and forth with this turn taking and then you know we ask questions like so the robots songs sound like your song so if you tell it not to change anything yes if you tell it to go faster it'll just change the progression a little bit if you tell this go slower it'll you know make it a bit slower sometimes it'll add new notes if you tell the chord progression to go up but you play a Down chord progression and then how does that impact the emotions it's like oh well it seems like it's kind of picking randomly but all the happy songs kind of start to sound the same so it's really cool to watch children sort of do this less structured like you know it's not right wrong answer it's how does it sound and they're making music and then they get up and they play a class Orchestra and then we turn the top after all of that stimulation so to be clear in this this third concept how are you getting at the AI element of of what's happening so examples of this AI come up like if you look at some of Google's AI experiments they have this piano where you can play music along with a computer but the question is how does that computer know what to play and so in this activity children are actually setting parameters for how the computer should change the song to make it sound a particular way so if I give you if they give the robot rather an input with three nodes CDE and they tell it to make it faster and happier than the robot should return something according to the parameters faster and maybe it'll go up ceg like even higher than the child's output so they start to see how they can use AI to create and to create new things and then you mentioned previously the some of the surveys you did before and after was that before and after children go through this curriculum or there are there additional observations that you made about their experiences and what they learned having gone through this curriculum beyond what we've already talked about yeah so there was the before and after about how children feel about AI and about robots I also did it before and after about how children feel about Engineers so one of my I guess pet pieces an engineer is that you know there's a lot of emphasis on science and mathematics and stem but often technology and engineering it's a lot harder to do so it gets less attention so as I went through the curriculum I was hoping that children would have a better sense of what engineers do and why engineering is fun and unfortunately they did it and I think that you know a lot of that is because it was a very new concept to them so to tell her hilarious story the person they don't went in I was like okay who here knows when the engineers and you know and this classroom of 20 like two children raise their hand I was like okay that's good we have to do better than that and so I you know pointed to one of the children and I said okay you tell me what is an engineer tell us all he's like an engineer is someone who drives a train and further than I even thought we were so you know part of doing this research that I find you know like personally enjoyable is that I get to say well I'm an engineer I'm an engineer because I build things and I build things to help people and starting to have children you know think about that as a different new career path I kind of wish going back that I had built more but into this curriculum that I built so you know these activities were fun and they were playing games but at the end of the day they didn't get to see how the things they were building could be useful or how they could help other people or how they could bring joy to other people and I think that's why you know at the end when I was like okay who wants to be an engineer I still got crickets because it's like alright we need to do a better job of helping children sort of like see themselves as this but also see the value of it in society and then also just like I said I did a sort of AI assessment so how much did children learn about these things the AI assessment was like ten questions all about the different activities and some of them are kind of tricky it was like I think I mentioned before if you only teach the robot about good foods where will it take chocolate goes for a five-year-old you know like of course everyone knows chocolates unhealthy but it's very difficult for them to see like okay wait but this AI algorithm only knows foods that I taught it only foods that I've labeled so to always use those labels to make its guesses so it's really cool to see like a lot of children start to get those things right because it kind of like blew the the development psychology literature you know out of the water they were like I'm not sure until Jen can do this kind of reasoning yet know might know they did it it was awesome that's fantastic yeah so yeah anyways ten questions some of them a bit tricky and I think the median score was like 70% so I mean obviously we shouldn't assess children to have a lead that's probably not healthy for them but they understood a good amount of what was presented in front of them and I think that's really encouraging and important it's I mean the same with like early computer science education I was very easy to say children can't understand this it's too complex but the way that something is designed it can be made accessible to children so you know right now I'm working on a deep reinforcement learning activity so pretty much we're gonna build agents that can play snake so first we're gonna hand code it and then we're gonna use a simple neural net where we like give it a bunch of examples and then we're gonna have it do deep learning and I'm feeling very confident that these children can understand it because you know if you just are able to break something down enough they can get it wrong so if you're a deep RL person I'm gonna have some five-year-olds coming for your job pretty so get ready this is awesome work where do you go from here so this was really the the center of your master's thesis where do you see it going beyond that there are so many different things that I want to do and so much work to be done so some really things that are going on that there are others in my lab who I was kind of like the person to go and try this and see if it we're doing that so now that it works other people in my lab are also trying out their own experiments one of my lab mates veena is doing this work around you know when children are learning with AI how can that impact their creativity so they're not just learning about AI anymore they're also learning to be more creative and to be explorative as they're learning which will have like huge benefits from the for them you know beyond just learning a particular skill another student my group Blakely not student my group like I'm their professor another one of my lab mates Blakely is working on AI ethics curriculum so really helping children be able to understand the ethics behind every AI decision so that they can critically evaluate the things that around them but also when they're building things you know why teach children to build something if they don't know how to build it ethically like at the same time actually be thinking about both of these things so I'm really excited about that work personally I've gotten a lot of feedback from teachers like this is great I have no idea about anything it has so can you teach me so I'm trying to figure out the problem of how can we actually make this something that teachers can like use and feel empowered to use and not scared of so that it can really get into classrooms and get into the spaces in Baltimore that I used to work in and then I think also just making more cool activities like in my dream of Dreams this will become like this big online platform and children everywhere can learn about AI and ways that are meaningful to them so you know there has to be a lot more content behind it beyond these three activities what other things can children learn and what are other metaphors that make sense for them right so yeah that's all the things I'm gonna do awesome awesome and are there are there things that you have identified that you need meaning if there's you know some potential partner out there that you know someone in our listening community might you know be connected to anything come to mind in that regard I mean yes I can visit a plug for things so I think one thing that would be really awesome is to have an AI person someone who feels comfortable with AI who's really passionate about teaching this who I can sort of help get started with their own activities so I'm doing that with teachers right now and I think the biggest problem is that they're not comfortable with AI and there's a lot of work that I have to do to get them there so I'm wondering how might it be different if I take an AI person and start to give them tools to be teachers that would be really cool also if you kind of just want to try things out with your kids and experiment and ask questions some of the papers that are linked and the website that we have have actually I resources that parents can go on and try it right now I'm just like short activities based on scratch so unfortunately you have to have a kid between the ages of 7 or maybe over there but they're already things that exist that people can try that I highly recommend them try and give us feedback on well Randy thanks so much for taking the time to share what you're working on with us is really cool stuff and I'm looking forward to seeing how it evolves yeah thank you I really appreciate the opportunity thanks alright everyone that's our show for today for more information on randy or any of the topics covered in the show visit Twilio comm slash talks last two to five for more information on the black and AI series visit twimble AI comm / black in AI 19 as always thanks so much for listening and catch you next time\n"