**The Role of Humans and AI in Decision-Making**

It seems to me that utilizing humans for something that they are not particularly effective at, and using AI and AI tools for things that require creativity or unbelievable dreaming of things that don't exist, is misusing. AI is a phenomenal tool that tells us what's already out there, what's known, and what kind of patterns are implicit that we may not have recognized as important yet. And sharpening and shaping the roles in organization that people have can unleash enormous amounts of productivity that has not been tapped into.

**Automation and Creative Work**

If you've got routine decisions, that's something you want to automate away. If you've got something that requires creativity, you want to get all your best humans onto that and get them being creative and having interesting thoughts. In terms of who gets involved in creating guardrails, it means getting all the stakeholders together to talk about stuff. Who's going to be in charge of setting up a guard rail? This depends on the project, but let me take the example of Creative Commons for example.

**Stakeholders in Guardrail Creation**

In the case of Creative Commons, you had content providers as stakeholders, but then you also had those that were involved in the design of large platforms online platforms like Wikipedia, that utilized a lot of content, as stakeholders. Then you had some of the large search engines like Google for example, being part of it because you want to be able to find Creative Commons content easily and straightforwardly. It's findability, which is important. Then you want to have some of the regulators and lawyers at the table, so that you're not constructing something that is against the law or has other regulatory problems.

**The Importance of User Communities**

Then you create a user community that is quite heterogeneous, with people from different backgrounds and expertise coming together to contribute and engage with each other. When we talk about Creative Commons, a lot of people say oh yeah, that's that that that licensing scheme, or they say oh yeah, Creative Commons, that's the huge repository of content that can be easily used and reused. But the truth is, it's not just the hardware, but also the wetwear around it - conferences, online discussion groups, forums, the hundreds of thousands of hours of volunteers that people put in to evolve the Creative Commons licenses over 20 years for example.

**Inclusivity and Collaboration**

And what was quite interesting in the Creative Commons process is that they were quite inclusive, welcoming NGO groups but also welcoming industry stakeholders as long as the goal was clear and remained clear. To create an easy and straightforward copyright licensing scheme that was machine readable, standardized, and enabled the reuse of intellectual property. The inclusion of diverse voices and perspectives has been crucial to the development of guardrails like Creative Commons.

**Digital Tools for Communication**

We have a lot of wonderful digital tools that enable us to do this - vickies, forums, zoom, and lots of other ways by which we can engage with each other. And of course, we can also engage with each other face-to-face in actual conferences and workshops and symposia. This is how we create a user community that is engaged, motivated, and committed to creating good guardrails.

**Getting Started**

I hope people got some good ideas on how to get going with creating guardrails. And my final advice for anyone who's interested in this is just to get started, get going. The one thing you can make is not start because then you will never have a good guardrail. You'll fail half a dozen times or a dozen times doesn't matter. Get up, dust off your guard rail, and continue doing it. Every good guardrail framework started like that. And I think we don't have any other chance than to have good guardrails to improve our decision-making so that we can better face the challenging and perhaps even existential questions that we face as a species.

"WEBVTTKind: captionsLanguage: enso when Whenever there is a very straightforward uh efficient decision to be made then that can be Auto uh automatized but whenever it gets more complicated whenever the context is changing whenever goals are more muddy uh sometimes contested or battling with each other then I think it is perhaps better to keep the human involved not just in the loop as sort of at the end pressing the button but to be really involved in the decision making process I think there is also a good division of labor if you want between Ai and and the humans uh AI is very efficient uh for alerting us and focusing us uh on a solution that is already available humans are perhaps better positioned to come up with novel Solutions hi Victor thank you for joining me on the show hi Richie great to be here uh wonderful so uh since we're talking about guard rails today to begin with can you explain what is a guard rail a guard rail is a a guidepost it helps us in our decision- making I mean we make lots of decisions every day literally thousands of them but some of them are really consequential and uh if we make the wrong decision that could have uh huge consequences that could mean difference between life and death uh and so making good decisions is really helpful and having somebody that can provide some guidance on how to make good decisions is even better wonderful so I like the idea of tools to help you make decisions uh so can you give me uh some examples of how guard rails might be used in a business context sure uh but let me give you an example of what what a good guard rail is first uh outside of the business context just so that you understand and the the listeners understand what's the difference between a good guard rail that's flexible enough and adaptable enough and and something that's really not good so really good guardrail is for example uh the rule that we should drive on the right side of the road if you're in North America or on the left side of the road if you're in the UK um that's a guard wh um it helps us avoid accidents uh it's hugely efficient and beneficial um but we can break that guardrail if we want to for example when we want to overtake a car and there's no other car coming our way we can actually change to the other side and overtake that car uh the guard the real good guard rail uh helps us guides us in our decision- making uh but we are still in the driver's seat uh we still make the decisions uh it doesn't Nanny us it doesn't tell us what we do it empowers us to make good decisions um but it uh but it doesn't go beyond that that's a good guardrail so in the business context EXT for example if we now bring it back to the business context uh a good guardrail is one uh that uh that helps the manager that helps an organization uh to uh reach its goals to reach its aims um and uh uh uh that may come in all kinds of forms and shapes standard operating procedures uh in companies are guardrails uh in fact standard operating procedures in airplanes are guard rails too and Pilots should keep through the standard operating procedures but there are exceptional circumstances where they can break them and the same is true in the business context of course as well okay so it sounds like um there's quite a range then from just sort of simple rules of thumb in order to give you good advice through to um processes and procedures up to I guess the laws counters guard rails as well absolutely and some laws are quite strict and you can only do a certain thing and and other laws quite flexible as well and and keep in mind there's a lot that we cannot speed on the highway Beyond in the US 55 or 65 or sometimes 75 miles per hour um but we can still speed we just have to live with the consequences of a speeding ticket for example uh in other words that is a a pretty good guardrail because it contains and it and it it has that flexibility built in uh and similarly uh in uh in a commercial context a good guard rail helps decision makers but doesn't disempower them doesn't take decision um Power away from them okay so it's like this is a really good idea but you have the freedom to break it if you deem fit EX uh you mentioned that sometimes making decisions can be a life or death situation um do you have any examples where a lack of guard rails has caused some sort of problem yes um and uh if you forgive I'll I'll I'll offer a uh an example of the from the aviation history again uh we open our book guardwell with this very Stark story um it's about 20 years ago 20 some years ago um at that time there was a new uh device or relatively new device that was built into commercial airliners to avoid a head-on collisions it was a a collision warning system and uh these two um boxes and two airplanes that got too close to each other they would negotiate and one would order the crew to ascend and the other would order the other crew to descend thereby avoiding the the head-on crash um and when these boxes came around initially there was no good guardrail no good rule that mandated that pilots in all or almost all circumstances should follow the direct directive of the machine uh to avoid the Collision um and so two uh airplanes uh over uh the Swiss German border uh got very close to each other uh both had those boxes in there but only one airplane had a standard operating Airline had a standard operating procedure and the pilots complied with it uh and thereby uh descended uh the other side did not leave the machine uh did not have standard operating procedures not guard rails in place descended as well a crash and 100 people died uh that's about as bad as it gets in terms of outcomes there so that that's a pretty tragic event so uh yeah I can certainly see having um a procedure and making sure that other people are following that procedure as well uh can be incredibly important um all right I think we need I think we need a happier story to to balance this so do you have any examples of when guard rail have been helpful and there been a positive outcome absolutely um and and you see already um as I'm drawing in examples from the aviation industry from from from uh Automobiles and and and and driving uh all the way to uh the commercial context uh guard rails really are everywhere um one of the the happy outcomes in a way with respect to guard rails is uh from Africa where there is um often times s there are often times contested resources just like water uh and in West Africa a number of small Nations depended on uh the Walter river water supply um but uh uh some Nations took more water than the others um and so they came together uh and in a very inclusive uh process uh came up with guard whales with a framework on how to deviate up that limited resource um and and then ran by it it was not a fixed framework it was somewhat flexible it also needed to be adapted to changing uses and changing context and also climate change of course uh but overall uh it was a very positive story as it uh enabled not just a limited number of people to get access to good water uh but a a very large uh group of people that transcend a particular jurisdiction or a particular Nation okay yeah so certainly different nations agreeing on how to share resources feels like a quite a tremendous thing it doesn't happen very often the more lik to argue or go to war with each other over over resources so and what's really interesting is I think that you know um you would never think that this was easy or possible but it actually happened so we are capable as as a species to come up with quite flexible but also quite pragmatically operative efficient guard rails okay uh so I you've convinced that guard rails can be very useful uh now I'm curious as to what makes a good guard rail so you mentioned before that uh they need to be empowering of individuals uh are there any other principles like that that make a good guard rail if I know exactly what I want to achieve um then I can design the guard ra I can design the the the regulatory mechanism to achieve that goal the problem is is uh what if the goal is somewhat wobbly or what if the context may change think about artificial intelligence regulation these days uh um and and and how do we go about this it seems to me that the European Union uh with its AI act recently passed um seemed to understand perfectly what the problem was a and then create the perfect solution for the problem except what if they didn't understand the problem very well then the mechanism to achieve the the goal uh May ultimately turn out to be quite inefficient and so in other words uh if we don't really know what we want to achieve or we don't completely understand the problem we need some guard Wells that are perhaps a little more flexible but most importantly help us to learn uh to learn from the decisional mistakes that we make and also help us to then adapt the guardrail itself uh so that it is uh better suited uh for the context in which we're in uh and that's really hard that's unbelievably difficult to do uh especially for for classical State Regulators or multinational Regulators who want to lay everything down uh in in great detail uh so that there is no ambiguity whatsoever uh orus and I in our book argue that when we don't really understand the problem completely yet uh one of the designed principles is to uh build some flexibility into the guardrail and some ability for the guardrail to be adapted but also to create guardrails that help people learn learn from their decisions that they make so that they don't make the stupid mistakes again okay uh so that sounds sful it also sounds a bit like um you've got like the agile software approach coming to creation of guardrails as well like you need to do something and you know a few weeks later you review it and change your mind about what you're doing next yeah yeah and and and what is interesting Richard that you bring this up um when we talk to programmers or people in the software industry they say yeah exactly that's what I do a lot with agile uh programming and all that because oftentimes I don't really understand my user perfectly yet and I need to adapt I need to also adapt to to to new user um wants and new user preferences uh and all that the the problem is really that The Regulators haven't understood this The Regulators have over the last 150 years gone from relatively flexible rules to ever more detail and tight rules uh which squeeze all the flexibility out of the system uh actually that's interesting you mention flexibility and having very detailed rules so my understanding of like uh very lame understanding of lawers like in the US you have these very deep detailed rules and in uh you tend to have like broad rules that sort of covers everything but um to a lesser extent it is there like is one better than the other when it comes to guard rails do you want um deepness or broadness you know the truth is much more sobering um the truth is yes in the United States we have case law and that is quite detailed plus we also have statutory law uh stat statutes rules that have been enacted lots and lots of them and by the way in Europe we have pretty much the same thing um so when you look at the the number of laws that have been passed either uh by legislatures in the United States or by legislator in Europe whether it's the UK or the European Union you pick um it has grown almost um by Leaps and Bounds over the last two decades or so uh legislators have gotten incredibly productive in enacting laws uh which of course limits the ability to be flexible uh and to stick to flexible rules uh and uh it it it it may reduce ambiguity and interpretative um flexibility but it also creates uh the potentiality for statutes that are outdated and no longer are in sync with the reality on the ground okay uh that's interesting um and I had no idea that there were just dramatically more laws than there were a few decades ago um all right so the example you gave early on was about um speed limits in cars and how there's a sort of law there that says you can't go past this but individual drivers have the ability to speed if they want to um so it seems like a good guard rail is one that people are actually going to follow at least most of the time so how do you design your guard rails to make sure they are actually followed mostly by keeping your customers in mind much like a good user interface designer keep your customers in mind uh and and and their goals what do they want to achieve if the a guard rail helps people ultimately to achieve their goals and if you can communicate that then they will stick to it um if if if you tell people that um at at the speed limit of 65 rather than 85 um they they they still reach their um destination within just a couple of minutes later than they would otherwise uh but with 20% less fatal crashes and accidents uh then that might be the sensible thing for a lot of people to do um and you may not ever reach 100% compliance but you don't need to uh if 90% or 95% the of the people uh follow the rules that's pretty dar good uh and uh that uh reduces for example in in in in uh car traffic reduces uh accident rates dramatically um so what you are trying to do is basically look at what people want to achieve try to empower them and try to communicate how you empowered them uh and hope that you did it well okay so really it's about like making sure that people are aware of like what the guard rail is so I guess in that case yeah it's the it's the signpost that tell you what the speed limit is and also educating people as to why the guard rail exists so if they know why it's there they're going to want to you know I I'm a rock climber and a lot of times when I climb uh there is fixed ropes uh and you clip in the fixed rope and then you climb up and and and and so you're always secure because you're you're clipped into uh that that rope um but there are sometimes there are people who just come from behind and want to overtake they can un clip and they can overtake you and then continue on uh but it's it's their own decision uh and they have to live with the consequences and I think that's the important point we don't want to Nanny people we don't want to disempower them we don't want to take the responsibility but also their freedom and liberty away from them um through the guard whe through well-designed guard Wheels uh we want to empower them to make better decisions uh without actually caging them I was getting m terrified about people unclipping themselves from a rck face and and trying to overtake that just sound incredibly dangerous uh so but yeah I can see how uh you'd want it to be their own decision to do that rather than uh the uh standard procedure all right so um I'd like to talk a bit about decisionmaking uh so uh you mentioned like one of the goals around guard rails is to help people make better decisions could you just talk me through how this works for good decision making um we basically need two elements one we need to have the right information available um in order to to make our decisions and then two we actually have to somehow weigh and calculate and balance all the information that we have and uh factoring our preferences in order to come to a decisional option and that's the hard part the the you know if you think at getting all the information in place is the hard part uh just think about the fact that we have learned over the last 40 years or so that we have numerous cognitive biases that uh that that shape our decision making and often times hinder Us in in making the right rational decision whether it's um uh confirmation Biers or availability bioses all kinds of bioses that we are as humans susceptible to and there is no easy way to untrain those biases and to get rid of them um so what cognitive Psych ologists have said is that rather than often times trying to find the best of bit or the better between two mediocre decision options it is better to broaden your decision space so that you have more options available uh which ultimately may lead to to better outcomes uh we are not particularly good at choosing with our biases between two options but we are as humans pretty good in coming up with lots and lots of decision options if we try uh we we can kind of um dream up new decision options relatively well uh and so a good decision- making should therefore provide us with uh good information but then also help us to broaden our decision space and then to navigate that decision space appropriately okay yeah um that certainly seems to be a good idea to help you um compensate for all these sort of human biases we have like uh yeah the tricky decisions do you want the salad or the french fries it's like you're going to be biased towards one um okay uh so uh it does seem like there's often a lot of uncertainty in decision making and you mentioned that sometimes you don't know quite what your goal is or what your users want uh can guard rails be used to help U with this level of uncertainty uh yes they can now we need to understand that uh there is no certainty there uncertainty means that uh even if you have guard rails that work 80% of the time it means 20% of the time they're off um uh but but there are sort of a good starting point if you want um Atul Gand uh a an author and medical doctor in the United States uh had a wonderful book out a couple of years ago um called the list and it was a a a list of standard procedures in the emergency rooms uh for in US hospitals and what he showed was that if people stick if doctors stick to the list of you know the the seven or 10 things that need to be done in this order when a new patients comes in um it significantly improves the chances for a positive outcome for most patients not all patients because you have those odd cases that un fortunately the the the list doesn't cover and may make the situation actually worse that's why you need the empowered uh physician who can then kind of discard the list and say uh in this case I need to do something else um but uh but but overall um guard rails help us these standard operating procedures this list help us to uh to cover uh the the most obvious cases and that's why they are useful okay I say I'm a big fan of Che list I'm quite often forgetting things particularly admin tasks where I can't hold them in my brain so having those checklists has helped me go through and make sure that I'm doing absolutely everything I should be doing um all right so um you mentioned one of the big principles of this is about making sure that you have enough flexibility around uh the guard rail so that people can be empowered like so what's your strategy for deciding like is this guard rail too strict or too lenient um how do you get that balance right these rules of thumb that I I was alluding to whether it's the rule of thumb that a guardwell should enable learning or whe it's the rule of thumb that a guardwell um should facilitate the empowerment of individual decision makers uh those are um more like design principles they are not fixed rules themselves but they're they they are kind of principles that should guide our designing of uh of the guardrails and so the designing uh process is not a scient ific one it it it is Artistic in a way uh and it requires a lot of trial and error uh which by the way is very difficult for Regulators as I mentioned before because Regulators usually want to regulate and then they want to forget about the regulation for the next 10 years at least uh and that's not what um flexible good guard Wells are uh they need to be constantly checked and fine-tuned H and adapted to changing environments um and and so um what we require is not just a good guard rail a well-designed guard wheel but what we require is an Institutional structure around it that can kind of revisit it constantly or or frequently uh and adjust it uh and to set that up is even harder than to come up with a flexible guardrail a flexible guardwell doesn't help you if you have no um in institutional structure around it that can then step in and increase flexibility or sort of rejig the guard rail uh one way or the other um so you always need to think about the sort of institutional environment the context in which a guardrail uh exists um and and and and how to make sure that the that that they stay on on top of um keeping it uh uh flexible and adapt okay so um it's not just the guard rail itself it's it's the all sort of the institutional framework to say is this actually going to work or not all right um this is getting slightly abstract so maybe we need a concrete example can can you just talk me through some examples of guard rails in business that are used for helping you make decisions absolutely uh think about environmental laws for example you can try and limit the amount of carbon dioxide that is emitted by factories and you just have a law that says uh uh no Factory can emit more than x number of tons of carbon dioxide a year that's a very inflexible very fixed rule but it's very clear doesn't have any room for interpretation or very little room for interpretation uh think about a very alternative concept where you say look um every uh uh every company uh every Factory that's emitting carbon dioxide uh gets a uh emission certificate um and if you don't uh emit as much carbon dioxide as you have the certificate for you can trade that certificate on an emissions Market an emissions certificate market and every year we cut 10% off the total amount of certificates that are available um so uh what we do is we sort of set the goal but we do not Define predefine the pathways that individual companies set to reach that goal um and that's a pretty flexible guard rail in a way um because it leaves open uh a lot of different ways to achieve the the the envisioned goal uh and it creates incentives for Innovation it creates incentives for companies to even go beyond what the the the current limit is because that gives them an opportunity to trade their emission uh certificates and make extra money of it uh all these kind of things this is a a pretty flexible frame work and a pretty flexible setup it uses the market uh as an incentive structure uh it uses Innovation and human Ingenuity uh that's built into it um uh and and it is uh capable of sort of lowering uh the limit of carbon dioxide emissions over time while the hard rule just sets a limit and sticks to that limit okay that's interesting because it seem like the focus there is less about having um rules in place to enforce stuff uh in the same way the laws are and this is more about the focus is on this is the goal and these the incentives to make you adhere to this goal yeah and it helps individual decision makers in the companies to make the right choices because it creates incentives for them to uh make certain choices rather than others okay um now you mentioned that um you don't always get these uh guard rails right the first time so you need some process of uh well having feedback to make sure you can improve the guard rail later can you talk me through first of all how do you test whether or not your guard rail is actually working absolutely and that's one of the hardest parts right you have a guard rail um and the a guard rail is working if it helps people uh achieve their goals more effectively than before um whatever their goals are we're not going to judge the goals uh here this is just uh uh the the the guard whale is a a mechanism to achieve a an exogenous goal um now measuring that is actually not easy uh and uh and when you start measuring it just in terms of economic impact uh a sort of cost benefit analysis of some sort then you are starting to measure something um but you're not Capt in of course the full uh comprehensive picture you're just capturing what can be economically captured in data um so it's a good first step but it's not complete and comprehensive um and what we need to do is to therefore uh develop better measuring tools uh to to measure the effectiveness uh of guardrails um we are at the beginning of that but we have made quite some Headway going Beyond uh simple cost benefit analysis uh capturing longer term externalities and these type of things um uh but but there is a long way to go however having said all of this the truth of the matter is I I've advised politicians and policymakers over the last 25 years and the truth is the ugly truth is that they don't even do most of the time a simple cause benefit analysis even that would be better uh than what they are doing it's um oftentimes shoot from the hip so even small steps towards um getting um some assessment of Effectiveness would go a long way to improve uh guard rail design okay yeah it does seem like if you're um proposing a law you should do some kind of thinking about what of the costs and what of the benefits abely um I I advised the German government and that was one of our recommendations uh and it was the one recommendation that immediately got binned um uh that's that's a little bit worrying um okay so uh it seems like uh a lot of the key then to being able to test whether a guard rail is good or not is is the goal that the guard rail is for well defined so I let's talk like smart goals where they're like testable and all that kind of stuff so uh if you got a well- defined goal then hopefully uh you should be able to test us what's happening all right uh so uh I think we've mostly been talking about guardrail so far in the context of like helping humans but now ai is sort of reaching a point where it can automate some human decisions um so can you talk me through like um first of all like when should um artificial intelligence replace human decision making and when should it complement it that's the $100 million question at the end isn't it at least or or even more than that I I think um the the answer is relatively straightforward it may be surprising but it's just relatively straightforward if the decision making is very routine um and and and doesn't require change context or anything like that then there is no reason why the machine shouldn't make the decision if you enter an elevator and you press the button for the third floor you let them machine make the decisions of closing the door and getting you to the third floor uh that is a very efficient process and you don't need to question that never mind artificial intelligence here um uh so when Whenever there is um uh a very straightforward uh efficient decision to be made uh then that can be AO automatized um but whenever it gets more complicated whenever the context is changing whenever goals are more muddy uh sometimes contested or um battling with each other uh then I think it is perhaps better to keep the human involved not just in the loop as sort of at the end pressing the button but to be really involved in the decision-making process not because the human makes the better choice between two mediocre options as I imagine as I said but perhaps because the human might be able to come up with a third or fourth or fifth options we haven't yet considered that is actually better uh than the options that are that are already on the table and so in that sense I think there is also good division of labor if you want between Ai and and the humans uh AI is very efficient uh for alerting us and and and and and uh focusing us uh on a solution that is already available uh humans are perhaps better positioned to come up with novel Solutions okay um I like that so if you need some kind of creativity or there's some novelty uh in the situation then probably humans are going to perform better than Ai and it's it's it's often times it's important to then keep humans in the loop even for relatively routine decisions um I I'll give you another example from the aviation industry um some years ago uh Asian commercial airliners were always using autopilot to Autoland their their uh airplanes uh when they came to the US um because it was very smooth and everything uh but then uh one day the Autoland function at San Francisco Airport was down and a very large Asian Airline I had to land uh uh manually beautiful weather everything uh but they crashed aircraft and people died um and uh that led uh airline companies all around the world to mandate that their pilots have to continue to land by hand so that they continue to train experience learn uh what they're doing and how to land the aircraft uh so that they know what they need to do when they have to do it that's again quite a horrific tragedy there um and I can certainly see how having that continuous training is going to keep people you know keep their brain sharp and uh make sure that the the quality of the landing when they have to do it uh is going to better as um like the the similar thing in the data world is like generative AI can write your code for you but and it's fine till you don't have access to it and then you need to write your own code um okay uh so um if you are designing guard rails at for AI then what would those look like are they going to be different to guard rails for humans or is it just the same sort of thing no um if you design guard Wheels uh in the context of AI or how AI should be used um it's important to keep in mind the same design principles that we mentioned before for example to empower individual decision- making so rather than delegate away a uh a complex decision from the human because it's too complex for the human uh that's not a good idea we should keep it with the human and we should provide some uh guidance uh on on how options could be generated uh how additional how the option space could be uh broadened um uh we should also uh understand that uh guard rail should be designed to uh enable human learning rather than AI learning um it turns out that cultural learning is actually really powerful and has uh propelled our species from a relatively middling mammal species uh about 100,000 years ago or even 25,000 years ago uh to to something that can actually land on the moon uh and uh there is no other mammal that even aspires to do that at least to my knowledge um and so in that sense um learning and cultural learning is hugely important and we can design AI systems to enhance and enable and facilitate uh human learning uh rather than take that chore away from the human and embed it in the ey okay uh that's kind of interesting and I like the idea of cultural knowledge and you can sort of build on what uh other people have done before you yeah there aren't many of the Apes going to the Moon I suppose um okay uh so I think this leads to the idea of like transparency of how decisions are made so certainly once you got AI in there uh sometimes up this black box so um when do you need to worry about how decisions are made rather than just the answer is a big question it's a big question because behind it lurks uh an even larger question about causality and the role of causality in our understanding the world um and of course we are um we are living beings that make sense of the world through cause and effect um and and and that means uh we always want to peek in the black box into the black box um even when it is not possible um now what is really interesting is um the truth is a lot of times in our own lives we cannot we um other people's minds or our own sometimes is a black box we may arrive at a decision uh not completely knowing why we came to that decision in the first place and then we take that decision and then we post- rationalize um and the the the goal of the post- rationalization of course is to tell us a there is cause and effect and B uh we knew it all and calculated it accordingly um and that gives us that warm and fuzzy feeling that we are in charge uh as a post- rationalization I not sure that actually we are uh so um cleare eyed uh in fact I think we are quite often um uh deciding based on black boxes so maybe therefore dare I say but that's tentative dare I say it's less important to peek into the black box of AI then to understand uh and make sure what role the AI box fulfills in the decision- making process so what is what is the what what what is the tool for rather than how does the tool work that's interesting uh now I can certainly see how you got that sort of um confirmation bias way something goes right I was think yeah def definitely that I did that on purpose but um if it goes wrong then we sort of quietly forget about it um or blame others oh blame mother yes that's that's another good trick um okay so you think uh you just need to Define what the role of thei is within the decision-making process okay uh can you make that concrete have you got like a real example of how uh that might how you might go about doing that um I I think I'm I I um I I mentioned that before uh already uh and that has to do for example with um uh positioning the AI as something that does not take the decision power away from a human being uh away from a decision maker uh but rather facilitate uh the the ability of the decision maker to for example broaden the option space so Concrete in concrete terms um if uh if if I'm uh facing a decision I shouldn't ask chat gbt uh which of the two choices should I go for but as chat GPT tell me all possible options and that gives me a sort of a a lay of the land a a a sense of the option space available and then use that as a uh as a jumping board to come up with even further additional decisional options uh that perhaps others haven't considered that is uh taking the AI Tool uh and utilizing it giving it the role of um facilitating and enabling my coming up with additional decision options uh rather than delegating my decision- making to the AI box I like the idea of an option space it reminds me of U when you have small children you say do you want to eat the carrot or the broccoli and they never think about the third option that they could ask for ice cream so it ends up encouraging them in the right direction to eat vegetables and and by the way Richie if if if I may add um one of the interesting things is that uh that children uh uh uh hone over the first couple of years from about year one to year four or five hone those whatif skills their their skills of sort of thinking in larger option spaces what if I don't choose broccoli or carrot and go for something else eventually by the by the age of three or so they will be there okay uh they they they learn the scam very quickly okay all right so um just going back to the relationship between Ai and humans in decision making um if when you have these systems that are sort of partially automated and you have um some uh model or AI saying okay the answer is no it's very easy for a computer for a human to say oh computer says no and they don't do any thinking so how does using AI change the accountability in decision making yeah exactly that's a um great question in a way I allude to it when I gave that example of the U of the uh Pilots that did not know how to land the aircraft anymore because they were so riant on their technology it wasn't AI but it was a a similar technology that kind of took their decision power away from them um that they forgot how to do it in the first place that they had unlearned uh that particular skill in a way uh and decision making making decision is a uh powerful but also a hard skill to learn and it requires constant practice we are lazy very lazy uh beings um and so we don't like to make decisions uh we it's easiest if you can delegate decisions uh and don't have to make them particularly hard ones and that's why we have to practice them uh and so uh what we need to design um AI systems for uh and in general uh decision assistance systems for um is to um Force us to continue to practice our decision- making um and to help us get better at it um to to to learn to experiment to help us experiment to an extent um but but but not to Nanny us or to take the decision making power away uh and uh and that's a very tall order uh because of course um uh there there is a huge temptation to say we're going to use a machine because the machine is going to reduce fatality rates accidents rates whatever um bad decision rates um significantly um but the problem is that uh if we use the machine to make decisions uh um and we use generative AI That's trained on training data of actual decisions that generates new decision data that then the AI can be trained with again if we do that over and over and over again the option space is narrowing because the AI will always make the same decisions so the training data will always confirm an Ever narrowing band of decisional options um that only works if the reality doesn't change if none of our goals none of our preferences of the context ever changes if it does we require diversity in our decision space uh which the machine cannot provide us easily that's why humans are better have a leg up there we kill each other once in a while making bad decisions but overall experimentation has furthered the species ah this is like very close to the idea of um model drift in terms of machine learning where it's like the the reality of what you need to make decision about is changed from what theel model is predicting and the C model decreases over time okay interesting all right so in business suppose you're a manager you're now convinced that guard rails a really good idea what do you do in order to start uh increasing the number of guard rails or implementing them in your own team or company uh start an inclusive process um and and bring the the various people and stakeholders in because you're like you're designing um user interface you are trying to bring the stakeholders in and trying to get them to accept some guardrails uh and so what they need to have is a say in how they're designed uh and and what they're aimed to do uh to the extent that you do that you have a higher chance of compliance um you have a higher chance of acceptance um and uh you may also end up with higher quality guardrails uh at the end of the day um so that's very important the process of generating the guardwell uh is is is an important one it takes time it takes effort it's costly um but uh at the end of the day you come up with something that's quite flexible in the book uh we uh conclude the book in the Final Chapter with the rule of St Benedict uh the monks uh and the Order of St Benedict uh these rules are thousand years old uh and they were designed to be so flexible um to start with that they're still in place today um because they have had flexibility built in and they had a lot of buyin as well uh over the years and generations of monks that were involved in it fine-tuning it uh that's um an example for a good Buy in um if you want the design of the creative common licenses or some of the sort of Wikipedia setup or also good buyin um designs uh the Volta River resource example that I gave earlier is another one but keep in mind in all of these cases uh this wasn't a cheap uh overnight process it took real effort uh to get all the people involved at the table uh to get them to discuss and clarify the goals the preferences to make sure that the mechanisms the means uh built into the guardrails are robust enough and are are effective enough to uh to further the goal in mind uh all of that doesn't come easy uh or doesn't come cheap I like this idea of just um speaking to all the stakeholders like figuring out who's going to care about this and getting their opinion on what they actually want I think that's going to help like Define the goal more clearly uh and sort of probably um yeah you'll find out if they G made any problems up front before you put this card R in place but um since that sounds like it might be a fairly sort of timec consuming process and like quite difficult to um set initially are there any examples of like really simple guard rails that make like a good first guard rail project um yeah I wish I I think there is actually a very pragmatic way to to to to start and that is to come up with a guard rail but give it a sunset Clause um and say okay we'll we'll we'll use that guard ra for the next X amount of time uh and then we'll we'll take stock um we experiment uh I think that in itself is incredibly helpful uh because it also tells everyone around the table uh that this is now in place but it's not edged in stone um it's something that we'll try out uh and we'll then um huddle and reconsider if it was the wrong choice and it is perfectly okay if it was the wrong choice and we get rid of it um that's just how we learn um so um also have uh a bit of an arrow culture in your organization is useful Spotify is famous for its Arrow wall um I I think that's a a fabulous way where you know of a sub team at Spotify makes a mistake especially if it was a costly one they have to write it on a little piece of paper and put it on the mistake wall uh so that others can read it uh the idea is that you shouldn't make the you should always make an error a mistake uh it's uh it it just happens but you shouldn't make it not make it twice uh and that's a way to learn from other people's mistakes uh and uh that that's not a way or um a strategy to blame others uh but a strategy to learn from others there we are again cultural learning standing on the shoulders of of Giants and if you have Sunset Clauses uh for for guard rails um then what you're signaling is um this is not permanent this is just temporary and we're just starting off with this but we understand that we may need something very different okay um I like that idea of the uh the era wall I think like I probably need to need to get a bigger office just to feel mistakes on it uh but yeah uh I I can see how you want to make sure that it's very clear that this isn't a Wall of Shame it's a it's a learning tool because yeah there's a possibility that could get very misinterpreted um okay so uh I like trying to get an easy first project on the flip side are there any um are there any guard rail projects that like you to have a high impact on your business like where's where's the the big money in guard rails in oron in most organizations the big money in guard rails is when you look at the role of people uh and what the the role of people is if the role of people is to make standardized decisions or relatively standard decisions uh then um that is not a particularly good um utilization of human Ingenuity uh of human fantasy of human dreaming um and then you need to rejig the role that that for for for for this particular post or position um it seems to me um because otherwise uh you are utilizing humans for something that they are not particularly effective at um and similarly um if you are using Ai and AI tools for things that are uh that that require creativity or unbelievable dreaming uh of things that don't exist uh then you are um misusing if not abusing uh ai ai is a phenomenal tool to tell us what's already out there what's known uh what kind of um pattern are uh implicit that we may not have recognized as uh uh important yet um and and so in a way uh sharpening and shaping the the roles in organ organization that people have uh can unleash enormous amounts of productivity uh that I think uh uh has not been tapped into that latent productivity okay I like idea is if you've got um routine decisions then that's something you want to automate away if you've got something that requires creativity you want to get all your best humans onto that and get them being creative and having interesting thoughts all right um and in terms of who gets involved in creating guard rails you mentioned that um you probably want to get all the stakeholders together to talk about stuff what's that mean in practice like who's going to be in charge of setting up a guard rail and can you give me like a have you got a concrete example of like um who might be a stakeholders for a particular guard rail obviously that the uh depends on the project but let me take the example of Creative Commons for example um and with Creative Commons uh you had um cont providers as stakeholders but then you also had uh those that uh were involved in the design of large platforms online platforms like Wikipedia uh that utilized a lot of content uh as as stakeholders then you had some of the large search engines like Google for example uh being part of it because you want to be able to find creative commment content easily and straightforwardly that's findability uh it's important uh then you want to have some of the regulators and lawyers at the table so that you're not constructing something that is against the law uh or has other regulatory problems um and uh and at the end you create a user Community um that is quite heterogeneous um uh and when you talk about creative commment uh a lot of people say oh yeah that's that that licensing scheme uh or they say oh yeah Creative Commons uh that's the the kind of huge repository of content that can be easily used and reused but the truth is that that's just the the hardware the wetwear around it are the conferences the online discussion groups the forums uh the hundreds of thousands of hours of volunteers that people put in uh in order to uh evolve uh the Creative Commons licenses over 20 years for example uh and a and what what was quite interesting in the creative commment processes that they were quite inclusive welcoming NGO groups but also welcoming industry stakeholders uh as long as the goal was clear uh and remained clear uh to create uh an easy and straightforward uh copyright licensing scheme that was machine readable standardized and enabled the ReUse uh of the the the the cost-free you reuse of intellectual property okay um I like that um it's not just um people getting together as being the sort of method of feedback You' also got you've got like discussion forums and reports and all that kind of stuff and there's a lot of sort of uh communication infrastructure going around that then as well exactly and and and we have a lot of the wonderful digital tools that enable us to do that you know you have uh vickies and you have uh forums and then you have zoom and then there's lots of ways by which we can engage with each other and then of course we can also engage with each other uh face Toof face uh in in actual conferences and workshops and symposia okay wonderful all right so uh I'm hoping people got some good ideas on uh how to get going with uh creating guard rails then uh do you have any final advice for anyone who's interested in this just get started get going uh the one U mistake that you can make is not start because then you will never have a good guard ra um only when you start yeah you'll fail you'll fail half a dozen times or a dozen times doesn't matter uh get up dust off uh rejig your guardrail uh and continue doing it we all have started like that uh every good guardrail framework started like that um uh but uh I think we don't have any other chance than to have good guardrails to improve our decision- making so that we can better face the challenging and perhaps even existential questions that we we we we face as a species all right super uh I like that just get started that's uh good motivation excellent all right uh thank you for your time Victor thank you very much Richie wonderful to talk to youso when Whenever there is a very straightforward uh efficient decision to be made then that can be Auto uh automatized but whenever it gets more complicated whenever the context is changing whenever goals are more muddy uh sometimes contested or battling with each other then I think it is perhaps better to keep the human involved not just in the loop as sort of at the end pressing the button but to be really involved in the decision making process I think there is also a good division of labor if you want between Ai and and the humans uh AI is very efficient uh for alerting us and focusing us uh on a solution that is already available humans are perhaps better positioned to come up with novel Solutions hi Victor thank you for joining me on the show hi Richie great to be here uh wonderful so uh since we're talking about guard rails today to begin with can you explain what is a guard rail a guard rail is a a guidepost it helps us in our decision- making I mean we make lots of decisions every day literally thousands of them but some of them are really consequential and uh if we make the wrong decision that could have uh huge consequences that could mean difference between life and death uh and so making good decisions is really helpful and having somebody that can provide some guidance on how to make good decisions is even better wonderful so I like the idea of tools to help you make decisions uh so can you give me uh some examples of how guard rails might be used in a business context sure uh but let me give you an example of what what a good guard rail is first uh outside of the business context just so that you understand and the the listeners understand what's the difference between a good guard rail that's flexible enough and adaptable enough and and something that's really not good so really good guardrail is for example uh the rule that we should drive on the right side of the road if you're in North America or on the left side of the road if you're in the UK um that's a guard wh um it helps us avoid accidents uh it's hugely efficient and beneficial um but we can break that guardrail if we want to for example when we want to overtake a car and there's no other car coming our way we can actually change to the other side and overtake that car uh the guard the real good guard rail uh helps us guides us in our decision- making uh but we are still in the driver's seat uh we still make the decisions uh it doesn't Nanny us it doesn't tell us what we do it empowers us to make good decisions um but it uh but it doesn't go beyond that that's a good guardrail so in the business context EXT for example if we now bring it back to the business context uh a good guardrail is one uh that uh that helps the manager that helps an organization uh to uh reach its goals to reach its aims um and uh uh uh that may come in all kinds of forms and shapes standard operating procedures uh in companies are guardrails uh in fact standard operating procedures in airplanes are guard rails too and Pilots should keep through the standard operating procedures but there are exceptional circumstances where they can break them and the same is true in the business context of course as well okay so it sounds like um there's quite a range then from just sort of simple rules of thumb in order to give you good advice through to um processes and procedures up to I guess the laws counters guard rails as well absolutely and some laws are quite strict and you can only do a certain thing and and other laws quite flexible as well and and keep in mind there's a lot that we cannot speed on the highway Beyond in the US 55 or 65 or sometimes 75 miles per hour um but we can still speed we just have to live with the consequences of a speeding ticket for example uh in other words that is a a pretty good guardrail because it contains and it and it it has that flexibility built in uh and similarly uh in uh in a commercial context a good guard rail helps decision makers but doesn't disempower them doesn't take decision um Power away from them okay so it's like this is a really good idea but you have the freedom to break it if you deem fit EX uh you mentioned that sometimes making decisions can be a life or death situation um do you have any examples where a lack of guard rails has caused some sort of problem yes um and uh if you forgive I'll I'll I'll offer a uh an example of the from the aviation history again uh we open our book guardwell with this very Stark story um it's about 20 years ago 20 some years ago um at that time there was a new uh device or relatively new device that was built into commercial airliners to avoid a head-on collisions it was a a collision warning system and uh these two um boxes and two airplanes that got too close to each other they would negotiate and one would order the crew to ascend and the other would order the other crew to descend thereby avoiding the the head-on crash um and when these boxes came around initially there was no good guardrail no good rule that mandated that pilots in all or almost all circumstances should follow the direct directive of the machine uh to avoid the Collision um and so two uh airplanes uh over uh the Swiss German border uh got very close to each other uh both had those boxes in there but only one airplane had a standard operating Airline had a standard operating procedure and the pilots complied with it uh and thereby uh descended uh the other side did not leave the machine uh did not have standard operating procedures not guard rails in place descended as well a crash and 100 people died uh that's about as bad as it gets in terms of outcomes there so that that's a pretty tragic event so uh yeah I can certainly see having um a procedure and making sure that other people are following that procedure as well uh can be incredibly important um all right I think we need I think we need a happier story to to balance this so do you have any examples of when guard rail have been helpful and there been a positive outcome absolutely um and and you see already um as I'm drawing in examples from the aviation industry from from from uh Automobiles and and and and driving uh all the way to uh the commercial context uh guard rails really are everywhere um one of the the happy outcomes in a way with respect to guard rails is uh from Africa where there is um often times s there are often times contested resources just like water uh and in West Africa a number of small Nations depended on uh the Walter river water supply um but uh uh some Nations took more water than the others um and so they came together uh and in a very inclusive uh process uh came up with guard whales with a framework on how to deviate up that limited resource um and and then ran by it it was not a fixed framework it was somewhat flexible it also needed to be adapted to changing uses and changing context and also climate change of course uh but overall uh it was a very positive story as it uh enabled not just a limited number of people to get access to good water uh but a a very large uh group of people that transcend a particular jurisdiction or a particular Nation okay yeah so certainly different nations agreeing on how to share resources feels like a quite a tremendous thing it doesn't happen very often the more lik to argue or go to war with each other over over resources so and what's really interesting is I think that you know um you would never think that this was easy or possible but it actually happened so we are capable as as a species to come up with quite flexible but also quite pragmatically operative efficient guard rails okay uh so I you've convinced that guard rails can be very useful uh now I'm curious as to what makes a good guard rail so you mentioned before that uh they need to be empowering of individuals uh are there any other principles like that that make a good guard rail if I know exactly what I want to achieve um then I can design the guard ra I can design the the the regulatory mechanism to achieve that goal the problem is is uh what if the goal is somewhat wobbly or what if the context may change think about artificial intelligence regulation these days uh um and and and how do we go about this it seems to me that the European Union uh with its AI act recently passed um seemed to understand perfectly what the problem was a and then create the perfect solution for the problem except what if they didn't understand the problem very well then the mechanism to achieve the the goal uh May ultimately turn out to be quite inefficient and so in other words uh if we don't really know what we want to achieve or we don't completely understand the problem we need some guard Wells that are perhaps a little more flexible but most importantly help us to learn uh to learn from the decisional mistakes that we make and also help us to then adapt the guardrail itself uh so that it is uh better suited uh for the context in which we're in uh and that's really hard that's unbelievably difficult to do uh especially for for classical State Regulators or multinational Regulators who want to lay everything down uh in in great detail uh so that there is no ambiguity whatsoever uh orus and I in our book argue that when we don't really understand the problem completely yet uh one of the designed principles is to uh build some flexibility into the guardrail and some ability for the guardrail to be adapted but also to create guardrails that help people learn learn from their decisions that they make so that they don't make the stupid mistakes again okay uh so that sounds sful it also sounds a bit like um you've got like the agile software approach coming to creation of guardrails as well like you need to do something and you know a few weeks later you review it and change your mind about what you're doing next yeah yeah and and and what is interesting Richard that you bring this up um when we talk to programmers or people in the software industry they say yeah exactly that's what I do a lot with agile uh programming and all that because oftentimes I don't really understand my user perfectly yet and I need to adapt I need to also adapt to to to new user um wants and new user preferences uh and all that the the problem is really that The Regulators haven't understood this The Regulators have over the last 150 years gone from relatively flexible rules to ever more detail and tight rules uh which squeeze all the flexibility out of the system uh actually that's interesting you mention flexibility and having very detailed rules so my understanding of like uh very lame understanding of lawers like in the US you have these very deep detailed rules and in uh you tend to have like broad rules that sort of covers everything but um to a lesser extent it is there like is one better than the other when it comes to guard rails do you want um deepness or broadness you know the truth is much more sobering um the truth is yes in the United States we have case law and that is quite detailed plus we also have statutory law uh stat statutes rules that have been enacted lots and lots of them and by the way in Europe we have pretty much the same thing um so when you look at the the number of laws that have been passed either uh by legislatures in the United States or by legislator in Europe whether it's the UK or the European Union you pick um it has grown almost um by Leaps and Bounds over the last two decades or so uh legislators have gotten incredibly productive in enacting laws uh which of course limits the ability to be flexible uh and to stick to flexible rules uh and uh it it it it may reduce ambiguity and interpretative um flexibility but it also creates uh the potentiality for statutes that are outdated and no longer are in sync with the reality on the ground okay uh that's interesting um and I had no idea that there were just dramatically more laws than there were a few decades ago um all right so the example you gave early on was about um speed limits in cars and how there's a sort of law there that says you can't go past this but individual drivers have the ability to speed if they want to um so it seems like a good guard rail is one that people are actually going to follow at least most of the time so how do you design your guard rails to make sure they are actually followed mostly by keeping your customers in mind much like a good user interface designer keep your customers in mind uh and and and their goals what do they want to achieve if the a guard rail helps people ultimately to achieve their goals and if you can communicate that then they will stick to it um if if if you tell people that um at at the speed limit of 65 rather than 85 um they they they still reach their um destination within just a couple of minutes later than they would otherwise uh but with 20% less fatal crashes and accidents uh then that might be the sensible thing for a lot of people to do um and you may not ever reach 100% compliance but you don't need to uh if 90% or 95% the of the people uh follow the rules that's pretty dar good uh and uh that uh reduces for example in in in in uh car traffic reduces uh accident rates dramatically um so what you are trying to do is basically look at what people want to achieve try to empower them and try to communicate how you empowered them uh and hope that you did it well okay so really it's about like making sure that people are aware of like what the guard rail is so I guess in that case yeah it's the it's the signpost that tell you what the speed limit is and also educating people as to why the guard rail exists so if they know why it's there they're going to want to you know I I'm a rock climber and a lot of times when I climb uh there is fixed ropes uh and you clip in the fixed rope and then you climb up and and and and so you're always secure because you're you're clipped into uh that that rope um but there are sometimes there are people who just come from behind and want to overtake they can un clip and they can overtake you and then continue on uh but it's it's their own decision uh and they have to live with the consequences and I think that's the important point we don't want to Nanny people we don't want to disempower them we don't want to take the responsibility but also their freedom and liberty away from them um through the guard whe through well-designed guard Wheels uh we want to empower them to make better decisions uh without actually caging them I was getting m terrified about people unclipping themselves from a rck face and and trying to overtake that just sound incredibly dangerous uh so but yeah I can see how uh you'd want it to be their own decision to do that rather than uh the uh standard procedure all right so um I'd like to talk a bit about decisionmaking uh so uh you mentioned like one of the goals around guard rails is to help people make better decisions could you just talk me through how this works for good decision making um we basically need two elements one we need to have the right information available um in order to to make our decisions and then two we actually have to somehow weigh and calculate and balance all the information that we have and uh factoring our preferences in order to come to a decisional option and that's the hard part the the you know if you think at getting all the information in place is the hard part uh just think about the fact that we have learned over the last 40 years or so that we have numerous cognitive biases that uh that that shape our decision making and often times hinder Us in in making the right rational decision whether it's um uh confirmation Biers or availability bioses all kinds of bioses that we are as humans susceptible to and there is no easy way to untrain those biases and to get rid of them um so what cognitive Psych ologists have said is that rather than often times trying to find the best of bit or the better between two mediocre decision options it is better to broaden your decision space so that you have more options available uh which ultimately may lead to to better outcomes uh we are not particularly good at choosing with our biases between two options but we are as humans pretty good in coming up with lots and lots of decision options if we try uh we we can kind of um dream up new decision options relatively well uh and so a good decision- making should therefore provide us with uh good information but then also help us to broaden our decision space and then to navigate that decision space appropriately okay yeah um that certainly seems to be a good idea to help you um compensate for all these sort of human biases we have like uh yeah the tricky decisions do you want the salad or the french fries it's like you're going to be biased towards one um okay uh so uh it does seem like there's often a lot of uncertainty in decision making and you mentioned that sometimes you don't know quite what your goal is or what your users want uh can guard rails be used to help U with this level of uncertainty uh yes they can now we need to understand that uh there is no certainty there uncertainty means that uh even if you have guard rails that work 80% of the time it means 20% of the time they're off um uh but but there are sort of a good starting point if you want um Atul Gand uh a an author and medical doctor in the United States uh had a wonderful book out a couple of years ago um called the list and it was a a a list of standard procedures in the emergency rooms uh for in US hospitals and what he showed was that if people stick if doctors stick to the list of you know the the seven or 10 things that need to be done in this order when a new patients comes in um it significantly improves the chances for a positive outcome for most patients not all patients because you have those odd cases that un fortunately the the the list doesn't cover and may make the situation actually worse that's why you need the empowered uh physician who can then kind of discard the list and say uh in this case I need to do something else um but uh but but overall um guard rails help us these standard operating procedures this list help us to uh to cover uh the the most obvious cases and that's why they are useful okay I say I'm a big fan of Che list I'm quite often forgetting things particularly admin tasks where I can't hold them in my brain so having those checklists has helped me go through and make sure that I'm doing absolutely everything I should be doing um all right so um you mentioned one of the big principles of this is about making sure that you have enough flexibility around uh the guard rail so that people can be empowered like so what's your strategy for deciding like is this guard rail too strict or too lenient um how do you get that balance right these rules of thumb that I I was alluding to whether it's the rule of thumb that a guardwell should enable learning or whe it's the rule of thumb that a guardwell um should facilitate the empowerment of individual decision makers uh those are um more like design principles they are not fixed rules themselves but they're they they are kind of principles that should guide our designing of uh of the guardrails and so the designing uh process is not a scient ific one it it it is Artistic in a way uh and it requires a lot of trial and error uh which by the way is very difficult for Regulators as I mentioned before because Regulators usually want to regulate and then they want to forget about the regulation for the next 10 years at least uh and that's not what um flexible good guard Wells are uh they need to be constantly checked and fine-tuned H and adapted to changing environments um and and so um what we require is not just a good guard rail a well-designed guard wheel but what we require is an Institutional structure around it that can kind of revisit it constantly or or frequently uh and adjust it uh and to set that up is even harder than to come up with a flexible guardrail a flexible guardwell doesn't help you if you have no um in institutional structure around it that can then step in and increase flexibility or sort of rejig the guard rail uh one way or the other um so you always need to think about the sort of institutional environment the context in which a guardrail uh exists um and and and and how to make sure that the that that they stay on on top of um keeping it uh uh flexible and adapt okay so um it's not just the guard rail itself it's it's the all sort of the institutional framework to say is this actually going to work or not all right um this is getting slightly abstract so maybe we need a concrete example can can you just talk me through some examples of guard rails in business that are used for helping you make decisions absolutely uh think about environmental laws for example you can try and limit the amount of carbon dioxide that is emitted by factories and you just have a law that says uh uh no Factory can emit more than x number of tons of carbon dioxide a year that's a very inflexible very fixed rule but it's very clear doesn't have any room for interpretation or very little room for interpretation uh think about a very alternative concept where you say look um every uh uh every company uh every Factory that's emitting carbon dioxide uh gets a uh emission certificate um and if you don't uh emit as much carbon dioxide as you have the certificate for you can trade that certificate on an emissions Market an emissions certificate market and every year we cut 10% off the total amount of certificates that are available um so uh what we do is we sort of set the goal but we do not Define predefine the pathways that individual companies set to reach that goal um and that's a pretty flexible guard rail in a way um because it leaves open uh a lot of different ways to achieve the the the envisioned goal uh and it creates incentives for Innovation it creates incentives for companies to even go beyond what the the the current limit is because that gives them an opportunity to trade their emission uh certificates and make extra money of it uh all these kind of things this is a a pretty flexible frame work and a pretty flexible setup it uses the market uh as an incentive structure uh it uses Innovation and human Ingenuity uh that's built into it um uh and and it is uh capable of sort of lowering uh the limit of carbon dioxide emissions over time while the hard rule just sets a limit and sticks to that limit okay that's interesting because it seem like the focus there is less about having um rules in place to enforce stuff uh in the same way the laws are and this is more about the focus is on this is the goal and these the incentives to make you adhere to this goal yeah and it helps individual decision makers in the companies to make the right choices because it creates incentives for them to uh make certain choices rather than others okay um now you mentioned that um you don't always get these uh guard rails right the first time so you need some process of uh well having feedback to make sure you can improve the guard rail later can you talk me through first of all how do you test whether or not your guard rail is actually working absolutely and that's one of the hardest parts right you have a guard rail um and the a guard rail is working if it helps people uh achieve their goals more effectively than before um whatever their goals are we're not going to judge the goals uh here this is just uh uh the the the guard whale is a a mechanism to achieve a an exogenous goal um now measuring that is actually not easy uh and uh and when you start measuring it just in terms of economic impact uh a sort of cost benefit analysis of some sort then you are starting to measure something um but you're not Capt in of course the full uh comprehensive picture you're just capturing what can be economically captured in data um so it's a good first step but it's not complete and comprehensive um and what we need to do is to therefore uh develop better measuring tools uh to to measure the effectiveness uh of guardrails um we are at the beginning of that but we have made quite some Headway going Beyond uh simple cost benefit analysis uh capturing longer term externalities and these type of things um uh but but there is a long way to go however having said all of this the truth of the matter is I I've advised politicians and policymakers over the last 25 years and the truth is the ugly truth is that they don't even do most of the time a simple cause benefit analysis even that would be better uh than what they are doing it's um oftentimes shoot from the hip so even small steps towards um getting um some assessment of Effectiveness would go a long way to improve uh guard rail design okay yeah it does seem like if you're um proposing a law you should do some kind of thinking about what of the costs and what of the benefits abely um I I advised the German government and that was one of our recommendations uh and it was the one recommendation that immediately got binned um uh that's that's a little bit worrying um okay so uh it seems like uh a lot of the key then to being able to test whether a guard rail is good or not is is the goal that the guard rail is for well defined so I let's talk like smart goals where they're like testable and all that kind of stuff so uh if you got a well- defined goal then hopefully uh you should be able to test us what's happening all right uh so uh I think we've mostly been talking about guardrail so far in the context of like helping humans but now ai is sort of reaching a point where it can automate some human decisions um so can you talk me through like um first of all like when should um artificial intelligence replace human decision making and when should it complement it that's the $100 million question at the end isn't it at least or or even more than that I I think um the the answer is relatively straightforward it may be surprising but it's just relatively straightforward if the decision making is very routine um and and and doesn't require change context or anything like that then there is no reason why the machine shouldn't make the decision if you enter an elevator and you press the button for the third floor you let them machine make the decisions of closing the door and getting you to the third floor uh that is a very efficient process and you don't need to question that never mind artificial intelligence here um uh so when Whenever there is um uh a very straightforward uh efficient decision to be made uh then that can be AO automatized um but whenever it gets more complicated whenever the context is changing whenever goals are more muddy uh sometimes contested or um battling with each other uh then I think it is perhaps better to keep the human involved not just in the loop as sort of at the end pressing the button but to be really involved in the decision-making process not because the human makes the better choice between two mediocre options as I imagine as I said but perhaps because the human might be able to come up with a third or fourth or fifth options we haven't yet considered that is actually better uh than the options that are that are already on the table and so in that sense I think there is also good division of labor if you want between Ai and and the humans uh AI is very efficient uh for alerting us and and and and and uh focusing us uh on a solution that is already available uh humans are perhaps better positioned to come up with novel Solutions okay um I like that so if you need some kind of creativity or there's some novelty uh in the situation then probably humans are going to perform better than Ai and it's it's it's often times it's important to then keep humans in the loop even for relatively routine decisions um I I'll give you another example from the aviation industry um some years ago uh Asian commercial airliners were always using autopilot to Autoland their their uh airplanes uh when they came to the US um because it was very smooth and everything uh but then uh one day the Autoland function at San Francisco Airport was down and a very large Asian Airline I had to land uh uh manually beautiful weather everything uh but they crashed aircraft and people died um and uh that led uh airline companies all around the world to mandate that their pilots have to continue to land by hand so that they continue to train experience learn uh what they're doing and how to land the aircraft uh so that they know what they need to do when they have to do it that's again quite a horrific tragedy there um and I can certainly see how having that continuous training is going to keep people you know keep their brain sharp and uh make sure that the the quality of the landing when they have to do it uh is going to better as um like the the similar thing in the data world is like generative AI can write your code for you but and it's fine till you don't have access to it and then you need to write your own code um okay uh so um if you are designing guard rails at for AI then what would those look like are they going to be different to guard rails for humans or is it just the same sort of thing no um if you design guard Wheels uh in the context of AI or how AI should be used um it's important to keep in mind the same design principles that we mentioned before for example to empower individual decision- making so rather than delegate away a uh a complex decision from the human because it's too complex for the human uh that's not a good idea we should keep it with the human and we should provide some uh guidance uh on on how options could be generated uh how additional how the option space could be uh broadened um uh we should also uh understand that uh guard rail should be designed to uh enable human learning rather than AI learning um it turns out that cultural learning is actually really powerful and has uh propelled our species from a relatively middling mammal species uh about 100,000 years ago or even 25,000 years ago uh to to something that can actually land on the moon uh and uh there is no other mammal that even aspires to do that at least to my knowledge um and so in that sense um learning and cultural learning is hugely important and we can design AI systems to enhance and enable and facilitate uh human learning uh rather than take that chore away from the human and embed it in the ey okay uh that's kind of interesting and I like the idea of cultural knowledge and you can sort of build on what uh other people have done before you yeah there aren't many of the Apes going to the Moon I suppose um okay uh so I think this leads to the idea of like transparency of how decisions are made so certainly once you got AI in there uh sometimes up this black box so um when do you need to worry about how decisions are made rather than just the answer is a big question it's a big question because behind it lurks uh an even larger question about causality and the role of causality in our understanding the world um and of course we are um we are living beings that make sense of the world through cause and effect um and and and that means uh we always want to peek in the black box into the black box um even when it is not possible um now what is really interesting is um the truth is a lot of times in our own lives we cannot we um other people's minds or our own sometimes is a black box we may arrive at a decision uh not completely knowing why we came to that decision in the first place and then we take that decision and then we post- rationalize um and the the the goal of the post- rationalization of course is to tell us a there is cause and effect and B uh we knew it all and calculated it accordingly um and that gives us that warm and fuzzy feeling that we are in charge uh as a post- rationalization I not sure that actually we are uh so um cleare eyed uh in fact I think we are quite often um uh deciding based on black boxes so maybe therefore dare I say but that's tentative dare I say it's less important to peek into the black box of AI then to understand uh and make sure what role the AI box fulfills in the decision- making process so what is what is the what what what is the tool for rather than how does the tool work that's interesting uh now I can certainly see how you got that sort of um confirmation bias way something goes right I was think yeah def definitely that I did that on purpose but um if it goes wrong then we sort of quietly forget about it um or blame others oh blame mother yes that's that's another good trick um okay so you think uh you just need to Define what the role of thei is within the decision-making process okay uh can you make that concrete have you got like a real example of how uh that might how you might go about doing that um I I think I'm I I um I I mentioned that before uh already uh and that has to do for example with um uh positioning the AI as something that does not take the decision power away from a human being uh away from a decision maker uh but rather facilitate uh the the ability of the decision maker to for example broaden the option space so Concrete in concrete terms um if uh if if I'm uh facing a decision I shouldn't ask chat gbt uh which of the two choices should I go for but as chat GPT tell me all possible options and that gives me a sort of a a lay of the land a a a sense of the option space available and then use that as a uh as a jumping board to come up with even further additional decisional options uh that perhaps others haven't considered that is uh taking the AI Tool uh and utilizing it giving it the role of um facilitating and enabling my coming up with additional decision options uh rather than delegating my decision- making to the AI box I like the idea of an option space it reminds me of U when you have small children you say do you want to eat the carrot or the broccoli and they never think about the third option that they could ask for ice cream so it ends up encouraging them in the right direction to eat vegetables and and by the way Richie if if if I may add um one of the interesting things is that uh that children uh uh uh hone over the first couple of years from about year one to year four or five hone those whatif skills their their skills of sort of thinking in larger option spaces what if I don't choose broccoli or carrot and go for something else eventually by the by the age of three or so they will be there okay uh they they they learn the scam very quickly okay all right so um just going back to the relationship between Ai and humans in decision making um if when you have these systems that are sort of partially automated and you have um some uh model or AI saying okay the answer is no it's very easy for a computer for a human to say oh computer says no and they don't do any thinking so how does using AI change the accountability in decision making yeah exactly that's a um great question in a way I allude to it when I gave that example of the U of the uh Pilots that did not know how to land the aircraft anymore because they were so riant on their technology it wasn't AI but it was a a similar technology that kind of took their decision power away from them um that they forgot how to do it in the first place that they had unlearned uh that particular skill in a way uh and decision making making decision is a uh powerful but also a hard skill to learn and it requires constant practice we are lazy very lazy uh beings um and so we don't like to make decisions uh we it's easiest if you can delegate decisions uh and don't have to make them particularly hard ones and that's why we have to practice them uh and so uh what we need to design um AI systems for uh and in general uh decision assistance systems for um is to um Force us to continue to practice our decision- making um and to help us get better at it um to to to learn to experiment to help us experiment to an extent um but but but not to Nanny us or to take the decision making power away uh and uh and that's a very tall order uh because of course um uh there there is a huge temptation to say we're going to use a machine because the machine is going to reduce fatality rates accidents rates whatever um bad decision rates um significantly um but the problem is that uh if we use the machine to make decisions uh um and we use generative AI That's trained on training data of actual decisions that generates new decision data that then the AI can be trained with again if we do that over and over and over again the option space is narrowing because the AI will always make the same decisions so the training data will always confirm an Ever narrowing band of decisional options um that only works if the reality doesn't change if none of our goals none of our preferences of the context ever changes if it does we require diversity in our decision space uh which the machine cannot provide us easily that's why humans are better have a leg up there we kill each other once in a while making bad decisions but overall experimentation has furthered the species ah this is like very close to the idea of um model drift in terms of machine learning where it's like the the reality of what you need to make decision about is changed from what theel model is predicting and the C model decreases over time okay interesting all right so in business suppose you're a manager you're now convinced that guard rails a really good idea what do you do in order to start uh increasing the number of guard rails or implementing them in your own team or company uh start an inclusive process um and and bring the the various people and stakeholders in because you're like you're designing um user interface you are trying to bring the stakeholders in and trying to get them to accept some guardrails uh and so what they need to have is a say in how they're designed uh and and what they're aimed to do uh to the extent that you do that you have a higher chance of compliance um you have a higher chance of acceptance um and uh you may also end up with higher quality guardrails uh at the end of the day um so that's very important the process of generating the guardwell uh is is is an important one it takes time it takes effort it's costly um but uh at the end of the day you come up with something that's quite flexible in the book uh we uh conclude the book in the Final Chapter with the rule of St Benedict uh the monks uh and the Order of St Benedict uh these rules are thousand years old uh and they were designed to be so flexible um to start with that they're still in place today um because they have had flexibility built in and they had a lot of buyin as well uh over the years and generations of monks that were involved in it fine-tuning it uh that's um an example for a good Buy in um if you want the design of the creative common licenses or some of the sort of Wikipedia setup or also good buyin um designs uh the Volta River resource example that I gave earlier is another one but keep in mind in all of these cases uh this wasn't a cheap uh overnight process it took real effort uh to get all the people involved at the table uh to get them to discuss and clarify the goals the preferences to make sure that the mechanisms the means uh built into the guardrails are robust enough and are are effective enough to uh to further the goal in mind uh all of that doesn't come easy uh or doesn't come cheap I like this idea of just um speaking to all the stakeholders like figuring out who's going to care about this and getting their opinion on what they actually want I think that's going to help like Define the goal more clearly uh and sort of probably um yeah you'll find out if they G made any problems up front before you put this card R in place but um since that sounds like it might be a fairly sort of timec consuming process and like quite difficult to um set initially are there any examples of like really simple guard rails that make like a good first guard rail project um yeah I wish I I think there is actually a very pragmatic way to to to to start and that is to come up with a guard rail but give it a sunset Clause um and say okay we'll we'll we'll use that guard ra for the next X amount of time uh and then we'll we'll take stock um we experiment uh I think that in itself is incredibly helpful uh because it also tells everyone around the table uh that this is now in place but it's not edged in stone um it's something that we'll try out uh and we'll then um huddle and reconsider if it was the wrong choice and it is perfectly okay if it was the wrong choice and we get rid of it um that's just how we learn um so um also have uh a bit of an arrow culture in your organization is useful Spotify is famous for its Arrow wall um I I think that's a a fabulous way where you know of a sub team at Spotify makes a mistake especially if it was a costly one they have to write it on a little piece of paper and put it on the mistake wall uh so that others can read it uh the idea is that you shouldn't make the you should always make an error a mistake uh it's uh it it just happens but you shouldn't make it not make it twice uh and that's a way to learn from other people's mistakes uh and uh that that's not a way or um a strategy to blame others uh but a strategy to learn from others there we are again cultural learning standing on the shoulders of of Giants and if you have Sunset Clauses uh for for guard rails um then what you're signaling is um this is not permanent this is just temporary and we're just starting off with this but we understand that we may need something very different okay um I like that idea of the uh the era wall I think like I probably need to need to get a bigger office just to feel mistakes on it uh but yeah uh I I can see how you want to make sure that it's very clear that this isn't a Wall of Shame it's a it's a learning tool because yeah there's a possibility that could get very misinterpreted um okay so uh I like trying to get an easy first project on the flip side are there any um are there any guard rail projects that like you to have a high impact on your business like where's where's the the big money in guard rails in oron in most organizations the big money in guard rails is when you look at the role of people uh and what the the role of people is if the role of people is to make standardized decisions or relatively standard decisions uh then um that is not a particularly good um utilization of human Ingenuity uh of human fantasy of human dreaming um and then you need to rejig the role that that for for for for this particular post or position um it seems to me um because otherwise uh you are utilizing humans for something that they are not particularly effective at um and similarly um if you are using Ai and AI tools for things that are uh that that require creativity or unbelievable dreaming uh of things that don't exist uh then you are um misusing if not abusing uh ai ai is a phenomenal tool to tell us what's already out there what's known uh what kind of um pattern are uh implicit that we may not have recognized as uh uh important yet um and and so in a way uh sharpening and shaping the the roles in organ organization that people have uh can unleash enormous amounts of productivity uh that I think uh uh has not been tapped into that latent productivity okay I like idea is if you've got um routine decisions then that's something you want to automate away if you've got something that requires creativity you want to get all your best humans onto that and get them being creative and having interesting thoughts all right um and in terms of who gets involved in creating guard rails you mentioned that um you probably want to get all the stakeholders together to talk about stuff what's that mean in practice like who's going to be in charge of setting up a guard rail and can you give me like a have you got a concrete example of like um who might be a stakeholders for a particular guard rail obviously that the uh depends on the project but let me take the example of Creative Commons for example um and with Creative Commons uh you had um cont providers as stakeholders but then you also had uh those that uh were involved in the design of large platforms online platforms like Wikipedia uh that utilized a lot of content uh as as stakeholders then you had some of the large search engines like Google for example uh being part of it because you want to be able to find creative commment content easily and straightforwardly that's findability uh it's important uh then you want to have some of the regulators and lawyers at the table so that you're not constructing something that is against the law uh or has other regulatory problems um and uh and at the end you create a user Community um that is quite heterogeneous um uh and when you talk about creative commment uh a lot of people say oh yeah that's that that licensing scheme uh or they say oh yeah Creative Commons uh that's the the kind of huge repository of content that can be easily used and reused but the truth is that that's just the the hardware the wetwear around it are the conferences the online discussion groups the forums uh the hundreds of thousands of hours of volunteers that people put in uh in order to uh evolve uh the Creative Commons licenses over 20 years for example uh and a and what what was quite interesting in the creative commment processes that they were quite inclusive welcoming NGO groups but also welcoming industry stakeholders uh as long as the goal was clear uh and remained clear uh to create uh an easy and straightforward uh copyright licensing scheme that was machine readable standardized and enabled the ReUse uh of the the the the cost-free you reuse of intellectual property okay um I like that um it's not just um people getting together as being the sort of method of feedback You' also got you've got like discussion forums and reports and all that kind of stuff and there's a lot of sort of uh communication infrastructure going around that then as well exactly and and and we have a lot of the wonderful digital tools that enable us to do that you know you have uh vickies and you have uh forums and then you have zoom and then there's lots of ways by which we can engage with each other and then of course we can also engage with each other uh face Toof face uh in in actual conferences and workshops and symposia okay wonderful all right so uh I'm hoping people got some good ideas on uh how to get going with uh creating guard rails then uh do you have any final advice for anyone who's interested in this just get started get going uh the one U mistake that you can make is not start because then you will never have a good guard ra um only when you start yeah you'll fail you'll fail half a dozen times or a dozen times doesn't matter uh get up dust off uh rejig your guardrail uh and continue doing it we all have started like that uh every good guardrail framework started like that um uh but uh I think we don't have any other chance than to have good guardrails to improve our decision- making so that we can better face the challenging and perhaps even existential questions that we we we we face as a species all right super uh I like that just get started that's uh good motivation excellent all right uh thank you for your time Victor thank you very much Richie wonderful to talk to you\n"