[DataFramed AI Series #4] Building AI Products with ChatGPT (with Joaquin Marques)

The Complexity of AI: A Discussion with [Name]

In a recent conversation, [Name] discussed the challenges of creating intelligent machines that can think and act like humans. The topic was centered around the idea of pruning dead ends in AI systems, which is crucial for preventing them from taking unwanted turns or generating nonsensical responses.

The problem of AI's "hallucinations" was brought up, where the system produces answers that are not supported by facts. [Name] explained that this occurs when the possibilities are almost endless, and it becomes impossible to account for every single one of them. To overcome this, a dynamic policy that calculates everything maybe two or three moves ahead from any point is necessary. This means pruning just a small part of the tree and then pruning again at the next step.

To achieve this, [Name] suggested using a causal end or model that identifies certain things as impossible due to natural laws. For example, in fantasy or science fiction novels, certain actions may be impossible because they violate established rules. This can serve as a policy criteria for the AI system, allowing it to check for breaking natural laws and avoid taking actions that are not possible.

The idea of implementing this concept was also discussed. [Name] suggested using a combination of a large language model like GBT with a chess engine-like component to prune decisions and limit scope. Additionally, having a factual engine or data-based approach to fact-checking can provide a better AI experience. However, [Name] cautioned that creating sets of cause-and-effect rules is too complex for general applications but may be feasible for specific domains.

The importance of constraint in chatbot situations was also highlighted. [Name] explained that having less freedom is often better than giving an AI system too much room to maneuver. For example, if a chatbot is fed incorrect information or "crap" data, it will learn from it and generate nonsensical responses. Therefore, it's essential to constrain the system to a specific domain and establish rules for cause-and-effect.

For organizations looking to adopt generative AI, [Name]'s final advice was to get as much experience up front as possible. They recommended that researchers should strive to gain real-life experience, including multiple failures, to learn from them and avoid pitfalls. It's also essential to listen to feedback and be willing to say "no" when necessary.

In conclusion, the development of intelligent machines that can think and act like humans is a complex task that requires careful consideration of various factors. By understanding the challenges of pruning dead ends, establishing cause-and-effect rules, and constraining systems, organizations can create more effective AI solutions that provide valuable insights and services.

"WEBVTTKind: captionsLanguage: enone thing that we've all served is this idea it was just an idea um that there would be emerging Behavior if you fetted enough data okay it was not obvious that would occur but the mechanism itself the Transformer is is easy to understand but what's difficult to understand is if you feel enough data at one point it starts coming back with responses that surprised all right Joaquin thank you for joining us on data frames thank you for inviting it's a pleasure to be here uh yeah I'd like to dive straight in so uh let's talk a little bit about your work creating uh generative AI applications so can you tell me what are the most common use cases your customers have for generative AI yes certainly um right now I'm devoted to chatbots because of care gbt um it represents a unique opportunity because the chatbots that we did before Chad gbt did not use the narrative AI we sort of created a different deep scenarios for dialogue and try to account for all the scenarios we could conceive of and have rules uh in special content for each scenario so it was quite involved thankful that has become much easier now and more creative and we are not teaching hold on specific scenarios a specific sets of questions so now it's it's much more natural but we have other challenges so uh chat box most recently and then before that I use the narrative AI to generate equal queries to answer English like questions basically you ask a question it translates it into SQL and go through the database gets the answer submits it to the chatbot to sort of surround you with some nice context before answering back um and one other project that I was involved in that involved computer vision um and what I was doing was basically taking video from cameras where older people lived in their houses and if for example one of them fell okay to detect the fault and be able to react and have the software called so that's one where the generative AI was basically data generating video because video is very expensive and you cannot have old people falling on purpose it's hard to get actors that will go for that so um you can generate some specific scenarios and then have the software learn what fall looks like and you know it's very useful because indicator in the neural network to recognize it I think all those three um examples you know the chat Bots the natural language interfaces to SQL and then the the computer vision they're all fascinating examples of using AI um maybe we'll start with the the chat bots so um you said that uh using church gbt this has made it easier to create chat Bots because you don't need to worry about the scenarios does it also have a benefit for the end users as well of the chatbots yes it is much more natural um for that for example one of the big challenges and it combined two of the scenarios that I was talking about uh the data that the user is interested in is generally not the generic data that was used to train that UV team for example this specific data to their business um and therefore you cannot ask TBD for you um and on the other hand you cannot feed your daily routine because of privacy um concerns so you sort of have have a local database that you can query but then the trick is how do you do it do you have jbt make the call for you then you have to supply the chat GPT with the context of the question so that he understands where you're coming from and what is it that you refer to with certain specific words um and then that helps it to generate the SQL query including the specific points that you want to make for example you could ask give me sales for March 2000 2019. so where March 2019 goes in the query it needs to make that link that is fantastic to do that it's a very difficult problem to solve otherwise and it's all for you it's just a matter of okay using your imagination and you may need two or three questions back and forth a small piece of dialogue which at UT before you get the answer that you need um and so there therein lies the The Secret of getting these chat box to work the way that you expect them to work that's absolutely fascinating that I'd love to get into the details of like how you've been dealing with this sort of um these privacy issues and using local data to get around that um maybe we'll talk about that later but uh for now just can you tell me a bit more about the different Technologies you're using to build these chatbots so you mentioned um GPT but are there any other technologies that you think are important here I'm currently using land chain which allows you to create basically a sequence or back and forth and get several uh pieces of software involved not just yeah tdd um told that for example if you um don't want any bad feedback therefore you are supposed not to use that word then you could filter the bad words up front and announcement struck cat GPD not to use any bad language um so the the involvement of these other tools has to do with filtering with correction um with preparing the context to make the query okay and then the rgbt comes back with how you need to make that query and then you make it of your local database now chat gbt so try to keep the ends up putting together the question exactly the right way that a programming for example or SQL specialist will do okay so that's one uh one other interesting aspect is I have to do a lot of reading so um I've put together a system by which I I get the PDFs especially uh technical articles um I feed them into a vector database together with your usual text and then um I can ask okay well in the last three years give you the list of all the articles um on a specific topic and it comes back with uh I have some background on this because in the 90s I was G4 detect from infosage which was an IBM product the world's first um customized news in a different article type of search but you would give it the topics you're interested in the context um and it will come back with articles with news items that concern that topic and that specific Focus that you may have um so it's the same idea but now we can do it much more sophisticated and let's say for example that I'm investigating something like for real I'm now into x i x AI explainability there are hundreds of Articles already so create a survey of them I feed them and then I ask specific topics and it uses those articles as the source and it's outsides of charity okay so it's called a lean bearings or vector digits okay uh that's that's very cool stuff um so I like the idea that you are using AI to do research for you in order to build more AI applications a good sort of virtuous circle there but uh yeah so it seems like Lang chain and uh Vector databases seem to be like the big technologies that are important here they're all fine pieces also excellent and I think um so a lot of companies at the moment they they really want to get involved in in generative AI but they're not sure whether they need to um use these other pieces of software that um other companies are making or whether they should build things themselves so I guess um in terms of of your work why do organizations come to you as a consultant rather than building things in-house well when they realize that it's not just having the university trade but also uh the experience um thankfully our universities are graduating thousands of data scientists but um it's hard to find somebody with more than maybe 10 years experience I've been lucky enough that I started doing this in the eight so I've learned through experience and I've been exposed to all of these Technologies and absorb them on my own as opposed to having that experience at a university and having that many years experience here's me uh perspective because for example when we were doing that was state of the art in the in the 80s and 90s has been completely superseded some of the ideas are still there but they're much easier to implement than back then back then we had to write everything from scratch including the vector databases and expert systems for example the neural network back then we know about them we have a good idea how to put them together not the modern models that have been discovered in the past after a thousand years but some simple amounts and they were impossible to run due to memory limitations of the hardware okay and they were too slow across okay um so I'm glad things it was on from the 1980s it's got the uh making progress absolutely but one of the things that clients realize is they may try one or two projects many of the AI projects fail because um you know you're asked to do something that if you know it's very very hard to do uh but you don't have the experience to know it then you go ahead and do it try it and uh AIC not ready to solve that particular problem that that particular way but if you have the experience then you can and I've told the clients that's not possible to do it needs research um so it may be possible to do in six months but currently it's not so you have to know the current technology you have to know um what the hurdles are for each type of Technology as well but what can be done what cannot be done okay so just related to that um how do you go about deciding um what makes a good AI project like um how do you how do you plan this how do you decide whether this is high priority whether it's feasible I would say a combination of knowledge and experience um the clients um hang on to everything that they read in the news okay and then people have a tendency to think that they can get away with a lot more than than it's possible given the technology um so for example there are people that won't charge EBT to do some reasoning for them well it turns out yeah TBT does not do any reason what it does is predict as they say the next word but not only that but predict the context and if you give it some context before you create the prompt it helps it quite a bit to constrain the possibilities right um and this context becomes quite uh complicated depending on how you ask the questions the sequence and all of that so for example if you ask a question in one context and then you go and ask another question in completely different context and you don't set it up before that before changing context um then you get the wrongness because it assumes that history has part of the dialogue you go through okay so if you want to change but the way that you are thinking about something you need to help it and that's part of the prompt engineering challenge that we have currently and this chat jbd in a way it's a black box even though we understand the mechanisms of the Transformers um it is a black box Because by the time you feed those billions of documents into it create a specialized model um that contains all of that information you really don't have a very good idea of what could come up okay um in the past we use neural networks like the endless long short term memory okay um and that's the end then we're capable of memorizing uh what had just come before so in a sentence if you were parsing one word it knew what had gone before three or four or five words before okay but that's not enough because you could continue in a crazy way that was completely out of context okay and in some science fiction novels or comedies or whatever you can never predict what comes afterwards but with Transformers you read not just a whole sentence you could be reading in parallel a whole paragraph so you get both the context of what happened before and you get the context of what's going to happen later so it uses these attention mechanism to do that and it makes it so much more powerful because it's like having a second reading all the same text you've already been there now you go through it again but doing it all at once and you get much better results and that's partially part of the Improvement uh in one thing that we've observed is this idea it was just an idea um that there would be emerging Behavior if you fed it enough data okay it was not obvious that would occur but the mechanism itself the Transformer is is easy to understand but what's difficult to understand is if you feel enough data at one point it starts coming back with responses that surprise you okay so that makes it incredibly interesting because all of a sudden you realize oh I I thought for example that it would get that question wrong and it turns out he gets it right or maybe you didn't ask it correct so you need to work on your own problem so this is really interesting um the idea that you need to um so the first of all you need to worry about your prompts and but also um sometimes the responses you get are going to be surprising so in general how can you evaluate the quality of responses from uh text generation model well for example let's say a client um has a specific type of this and um you want to supply your chatbot or managers of the stores for that life that have let's say a high school education so they usually don't have a course in statistics to understand you know the intervals of confidence um in the data that they are getting back I try to solve that and partially solved it a few years ago doing visual statistics reading curves and showing where they were in the set of curves so that they could say okay we are above the medium but below 75 percent or we are above the 75 percentile but below the 95 percentile but if you are anywhere between 2.5 and 97.5 you are within the usual high confidence inter so you can see your curse where you are today and Visually you say I'm doing great if you are certain about a certain and you don't have to understand the details well it turns out a chat about someone better let's get a question in English gets back to you it tells you how you're doing how your business is doing at a moment and if it's not doing well one of the things that we do is to feedback a predictive answer for example tomorrow is going to be Laos and also prescriptive like the answer which is what you have to do is start selling coupons right now okay or put an uh announce something or call a nearby business let's say if you are a parking lot and you know you're not going to do enough business tomorrow you call some nearby places that will attract customers and split the parking with them okay that's fascinating the way you think about like how do you interact and provide more information she's saying it's better to just give it more information rather than give it a specific task to do yeah exactly or you feed this with the the information that contact information and unbeknownst to the user so from the user question you interpreted oh okay he's talking about this particular context using some keywords therefore I need to give captivity this preamble with the information that it needs for example in the case of the SQL databases we don't feed the data we only feed feed the um the schema for the tables and then it figures out how to do the SQL from the scheme uh so you're not providing any actual data to the AI you just provide this is the structure of the databases what's in each table and then they can figure things out from there um that's very interesting um so does it make a difference when you're building these AI products if you're building a complete AI product from scratch or if you're just adding AI features to an existing product yeah it makes quite a difference um the the product that I designed for IBM in the 90s to um two years to do okay to design and to code and all of that we had to do everything including the networking software um and design the databases the vector databases to do this and create new technology I got three NLP packets out of it it was quite an effort nowadays I don't need to do that a lot of the products are open source I can just use them okay my only um criteria is that I use them in a commercial product or do I have to ask the client to buy or license that's the only thing um so nowadays things are much easier uh the problems we have to solve are at a different level okay um and as I said it's a question of prompting but also architectural design because you cannot ask details of your business again from chat GPT you need to get your activity to figure out how to ask your local database for the details okay so that's really interesting so I'm saying well from things obviously a bit of a a challenge but also figuring out the architectural design are there any other uh big problems that you need to consider when you're trying to um devise an AI project yes um first I had to think about it for a day or two then I need to figure out are there components are of the intermediate problems tall already and what components will solve okay and how do I need to strain them together um and if if I can figure that out then I tell the client yes go ahead and I believe this is the effort if I see that there is a gap somewhere and it needs research then I tell decline right now we need to cover this Gap and until we fill in all the gaps you know but I tell them up front so that they know some of them decide okay we'll wait six months and see what happens um or others say okay we'll we'll pay you to do the research and put it put it together all right um that's kind of interesting so um can you give me some examples of success stories where you've built an AI product for customer and it's had a big impact for them yes um when I was head of Consulting in the chief data scientists at the Oracle Latin American um we were challenged with some problems with uh it's not that we were not selling we didn't know what we were selling and in what quantity because we were selling so much so things got lost um this was a while and basically um what I did was to travel to several places talk to the sales people um talk to the marketing people took a look at the last four years of sales and then using at the time some a technical svm support vector fiend using that technique I basically was looking okay when we sell a set of products because I've been at that I'm in Oregon I tells usually sets of products for a corporation it's not just one and for example databases are sold with almost everything because they are required uh and we were targeting okay what's going on with the pencil database um the the sales people were taking quite a while they dare the materials they needed to approach the potential person and let's say they they used four and a half days to prepare for day 201 presented to the customer The Proposal um I was able to reduce by identifying patterns for example what products sold in combination and what products if you saw one you never saw me up those types of patterns okay um I was able to decrease the preparation time to basically zero because it was based on previous sales experience and I could tell them through the AI do you need to bring a technical team with you for this particular customer and combinations of products or it can go on your own being a salesperson or do you need backup of some sort and then what what is the best way to engage the customer if they decide to go ahead you'll bring a team or a series of meetings before a final commitment or or guitar okay and we were able to basically reduce the time that it took to prepare um is the customer to only half of they visit the customer and the AI just gave them you know a list of what happened so what we were selling and what else we could offer if they went for it okay and also the strategy that you needed to to use in um pointers to previous sales so you can't could contact the sales people and get their wisdom about this particular customer so that was the first we had a predicted part we predicted which customers would be more receptive and the prescriptive part which is how should you approach the customer okay um that that was quite a success and what else I would say I really love that story because when you started I thought oh you didn't say you built a recommendation engine but actually it was improving business processes so just speeding up the operations and also finding more effective ways for you to sell things as well so that sounds like it's had like an impact on many different levels yes yes it did and it was quite successful excellent uh just on the flip side though have you ever had any cases where you've had a customer that's been very excited about generative Ai and then after talking you've decided actually that's not the right technology for their problem yes and mostly in the in the past um I worked at cognitive systems which was you know come for the uh not shoot of Yale Department of artificial intelligence um and we were doing natural language processing back in the eight um and we created systems that would basically for example at a bank would receive the bills of Landing which are documents that the ships arrive with saying this is the cargo that we bring and these are the permissions you need to get and these are the charges and how to extract the cargo from um and but there were many different types of bills so we created a system to categorize them and put them in specific bins um to be um read and either accepted or rejected by specific people that had a particular knowledge of that okay um so that that was an interesting one um and it was challenging because for example we had to have several thousand documents and we use techniques that are they are called case-based reasons that that was at the time that's been superseded completely um other um aspects more recent is for example to to use word which is a type of neural network to do redaction okay um chatbot.gbd can also do it nowadays um and um a lot of people for example they they do redaction by just erasing words okay and um I discovered that instead of doing that if you use special characters surrounding a label the Social Security ID goes here blanking it out and then uh if you use the output that has those labels into a vector database or even chat Unity itself you can you could fine tune it uh then it starts looking at the patterns in terms of okay um Social Security ideas could go in this position for that position or that position not just blanks okay um so it would start seeing behaviors of where people tend to use your Social Security IDs for example in their correspondence okay and then from there um what was the use of that was this just to provide better redaction then or this month yes and then um you know to be able to predict when you are not sure that these uh sequence of numbers is some International ID or a social security number if you know the parents in which a social security number pops up you may be able to do a rotation okay this is not associated about you in Target 40. okay oh so it might be like a phone number instead but you need the context it was attached metadata when you see something and we know and for example if you can redacted then you cross it over but you know it was a social security now that would be useful later on if you do this day in and day out you'll recognize the path that's like a better level of knowledge that one team packing to catch you so as I talk a little bit about how generative AI um can um democratize access to data so I guess one of the big sort of potential benefits of this technology is that some tasks are usually performed by a data team can now be performed by and mom like even if they don't have a dative background so have you seen any examples of this well I've heard and read about so many samples but they are limited to particular contexts okay um is that everybody is trying to sell these tools that supposedly don't need any code they are fantastic if you apply them to use cases that everybody has saw okay because that's what the tools are for but if you have anything even slightly complicated then they need to program you need to add your own code into it to make it to what you wanted to do and not what somebody thought you would want to do always um go I have not seen anything that's impressive from this point of view now the the democratizing of the data there's several different layers because for example if you are talking about personal identifiable information like EI uh then you need to determine okay you can look at your own personal data but you cannot look at anybody else okay or if you have your higher um in the talking Bowl let's say at a bank you may be able to use somebody else's personal datability depends on your rank and your particular function within an organization all of those rub issues need to be set up um even before you can make the quick all right so what you get is what is maybe due to the security process you go through rather than the chatbot itself so the chatbot has no this it assumes okay you ask it a question you already have the right to ask okay um so by the time you get to it you have to go through all of security layers you know um now the democratization of data in terms of for example you could go and completely obliterate all the data that's pii no matter what and then you feed it or you put those wings okay like I was saying Social Security number personal phone number or cell number uh name of a company without mentioning the company and so on that's democratization anybody can read it you're not damaging anyone some people could guess who you are talking about okay from the context um but you know that's one aspect where human intelligence it still exceeds anything that chat GDP can do um so in that sense yes you can democratize data you can actually exchange information without giving away any Secret um so well related to this as you've been working on uh natural language interfaces uh for SQL queries and as that helped provide access to data for people that couldn't normally get access to those databases yes because they will need to be experts in SQL you know and have the access rights to go into the database to start and then know how to actually see and in some of these queries get very very complex um so yes now um as far as further democratization um I things that very soon will be able to um exchange information to companies between people in such a way that we are basically guaranteeing that no piis being of any time or for example that they fully they exchange fully complies with Deepa or some other um so I believe these things are going to be possible very short that's really interesting the idea that um you can safely exchange data from one company to the next uh and not have to worry about those data privacy issues um in general um do you think or where do you think organizations should make use of generative AI just to improve their data capabilities well one of the problems everybody has is having enough examples of data to be able to train the neural network specific tasks not related to catchy and so let's say for example how one of my questions is how many different ways to chat GPT can you ask about what was the most profitable month of the year for a specific how many different ways can you ask that and instead of me spending days or weeks you know writing down everything that occurs to me um I have charge APD generally that data and then I take all those 100 200 questions whatever they are all asking the same thing and now I go and use it in my chatbot make sure that it comes up with the same right SQL query every time and usually it doesn't so I need then to take care of um because language isn't most of the time is not specific enough it's indeed in that ambiguity and it depends on how you ask the question that'll be given to your script if you don't get rid of it you don't get the right word Ah that's interesting so if everyone said this correctly you're generating hundreds of different prompt variations in order to make sure that regardless whatever a user inputs into your chat bot you're going to get a consistent correct answer yes and if there are ways of asking that always get gets it wrong then one of the things that I do is I check let's say using a Pinecone Vector database um are you asking this type of question and if the answer is yes it comes back he said I'm going to charge it please say um would you be more precise okay okay and related to this do you have to um do any like testing of prompt quality like do you do a b testing on your prompt or anything like that yes yeah depending on the needs of the client yes so we prepare a whole test that we can apply but again you know the the labeling and the data generational that I I had a a client um an insurance company in Los Angeles uh that they want to be able to get the readings from dongles in The Limousines and taxis uh that they insured so that they could see first whether the drivers were behaving themselves while driving okay and secondary if there was um an accident that we would be able to detect it but after more than ten thousand messages we only found 13 of them that were two actions so every time I try to train the neural network you will say well it's just 13 of them let's assume they don't exist so it would never classify anything as uh accident so this was a huge problem and it's very hard to fake data from an action okay so I'm in your door uh in the car it's 4G right there and there it's the same as for example a bike bumping against the door um so how can you tell the difference and if you can't how can you produce more accidents um so I went to the website at the government website where they test the cars and bought their accident data transformed it into the right format as it became from a dunbook and then fit but those added a few dozen examples not enough um so in that case I was lucky because I used a mathematical technique called a fast Fourier transform which is used in physics and the Fast Food Network form changed it from uh you know acceleration versus time to acceleration versus frequency the frequency gave it a Telltale sign of the accident okay there were some Peaks a certain particular places the Roady occurred in accidents then I fed that result from the fast food that was formed into the neural network and you could easily tell one from here I was working because nowadays um we could use a generative AI to actually produce more examples of the accident that without having to be lucky and get a transformation that gave UPS absolutely so um fast Fourier transformed uh it's like detect accident data that just sound like quite quite an intuitive or not intuitive like a novel sort of leave for the imagination for how to solve that but yeah um uh so more generally it just seem like uh generative air is really good for creating synthetic data and I do like the idea that you can use that if you do have a problem with class imbalance where you've got you're trying to detect rare events okay well that that's one of the the big plus results um so does it change um the kind of skills that you need uh to work with data now the fact that you can do different things with AI yes um but if you are working at a different level you know in order to take full advantage of the engine let's say cat gbd you have to speak its language use it for what it was intended to be it's not like there's a lot of apis you can go through really how you set up your phone by context avoiding certain terminologies um and not making it too complicated um one other thing that I've noticed is for example if the database has too many tables then it even takes too long what it gets is wrong because there are a lot of joints okay and that takes time by itself if you take the query yourself but it takes time to set up correctly so catch it is not as good so um you would make the table simpler a data analytics type and for the relational database into Data workhouse type with with lots of attributes together the thickness in the table the tables as possible and and you get better results soon you have to make accommodations for the limitations of the chat okay I think that's a useful tip to know is that if you're trying to generate these SQL queries with lots of joins then it's not going to work so at least at the moment unique is lucky and it works if it has a very closely example in this memory but if it doesn't which far chances are not then easily get them wrong or more sensitive to changes in the wording the English wording behind English query you know ah yes okay so the the language used in the database matters perhaps because GPT in general is better with English rather than other languages for example uh if you're asking something about a specific record in the database then I found that using the word simple it's crucial you say similarly those is One Direction if you say something else or avoid the word syndrome it may give thumb of a particular cup because things are moving pretty fast in so many developments going on right now it's quite hard to keep track of everything so are there any generative AI projects that or tools that you're particularly excited about at the moment well as I said right now I'm into activity in Lion King but I'm also looking at other aspects I have a strong interest in as I said in xai um explain a little um and for example in the case of explainability the algorithms that work best in explaining themselves are things like decision trees force that type of thing but if you think about the complexity of certain problems um and we already have examples of those types of problems in math that's a math problems that have been solved with computers that no human understands we just take it for granted because people have checked specific portions of the code um but nobody will ever understand them because they involve thousands of steps um and the same occurs with a decision tree or a force um in uh that it may have thousands of decision points to give you an answer and the best that you can do is well if you find different variable between these range and that range and then you repeat that a hundred times who's going to understand you know if you have 100 there absolutely I can make them incredibly difficult to explain what's going on in particularly complicated models yeah and we may have to settle for more generic answers um if the only way that we can explain it even with the most explainable algorithms is by providing hundreds of thousands of decision points and so related to this I get the theme that um over the next few months there can be a lot of um sort of things that claim to be exciting AI but maybe aren't so do you have a way of deciding like or any heuristics for deciding like what is a good quality AI tool or company versus um what's uh just I don't know cashing in on the hype well usually I read the scientific papers um in arcade but not the Press I mean I also read the press to see but um I don't believe it until I see the background to the idea research and they have a certain level of confidence that this will work and then it's a question of reproducing it um and if the results are dive with the clients in in the Articles then I'll try okay and as I said we use this much open source as possible um open source has been fantastic for everyone but occasionally there are those gaps that need to have a client that's interesting things works um and as I said the The Avengers for them is the agreement Secret Ary okay uh yeah so I do find that interesting because that you mentioned having to read all these um these Journal papers because sometimes if you're not involved in Academia then you think well it's just something that happens in the background uh universities but actually this is a really important part of your like your research for building business products then yes and and also I read articles on other types of products and the level of confidence that they have um but I I do tend to read the literature um and um for example uh when I was at IBM DJ Watson labs they had all the papers there I I researched as much as there was before I started inventing things with my team um that's the way to do it you don't want to be then want to say step on shoulders of giants absolutely um all right so is there anything you're working on right now that you're excited about well as far as the explainability okay um it is a big concern of mine because it's going to come up very quickly with charity okay uh in general can explain when you ask explain it to me uh what the last answer is meant to do um but checking it is a different type of piece because you cannot use their gbt to check um many times you can make it hallucinate not as often okay um as it may otherwise by restraint and making sure that you design your queries so that it's well restained you for example there is one parameter called temperature that if you set it to zero will only give you answers that's 100 sure are correct in other words that answer is somewhere in one of the texts sorry okay that's still not a guarantee but yet we live with it a lot of scientific papers um are not of the best quote and that has been proven so we live with uncertainty uh and we may need to check multiple sources here and there um ppt4 can give you sources so you can go and check with and make sure that yes indeed that in subjects with what people already know um but in order to create guard bridge to keep a generative AI in general not just yeah TV but Valley and others from going off the rent you have you need two things you need um a set of rules okay of policy that keep you within about it it has it as an expert outside that reads the answer now this means cannot give you the answer word by word because it needs to take the whole answer before giving it to make sure this one this will add additional time okay so that's a consideration you don't get your answer you may want to go for coffee and come back all right uh but you have an independent check on the answer we are sure that he judge by all the guardrails that you've set up in this the second piece is a plan um what do I mean by a planner basically something that will say Okay um we have these facts and they want to know if we can meet these goals starting from those facts uh entering the plan and make sure every step of the plan advise by all the rules all right um let me give you a small example you know the cannibals and missionary problems I don't know oh Cannibal's in missionary I I think I might have heard with different people crossing the river but go on tell the story you know it never happened but uh it's uh three missionaries and three cannibals on one of the river's margins with the boat on that margin of the river and basically if you at any time had either in one of the margins or in the boat itself more cannibals than missionaries the cannibal the missionaries will be gone after a while um so you have to solve the problem by always keeping them in equal numbers both cannibals and missionaries or more missionaries right um and imagine that you were to draw all the different ways in which this can happen you start with three three and one margin but in other words the six people is one margin and with an empty vote on that March um one possibility is that the boat is in the other margins of gable they can't do anything um but other possibilities lead to situations where you have more animals than insurance so imagine you have all the possible you cross out every time the rule is violent okay so you prune the tree um and you use that tree to make the plan you are guaranteed that you'll always succeed because all of the Dead ends have been cut up all the time okay now this is a very simple problem the thought but what happens if the possibilities are basically ended and you do not know how it's going to progress like in a game of chess there are so many possibilities it's inconceivable that you could account for them all and prune all the ones are wrong so you need to set up a dynamic policy that calculates everything maybe two or three moves ahead from any point okay and then you proceed according to the answer by that policy you in other words you prone just a small part of the tree and then you prune again at the next step and so on and so on okay um we would need to do something like this in chat gbt to keep it from hallucinating and taking it in the wrong direction you also need a causal end or model that for example says okay certain things are impossible because of natural loss all right so that you can also use that as a policy criteria because you need to check for breaking natural laws when you do things of course it would be terrible if you are interpreting for example satisfaction novel or fantasy novel because it would violate but that you could have your own rules okay in an imaginary world saying this is valid and this is not valid and then have it interpreted accordingly okay that's one of the big advantages but if we were to implement the combination of these I believe that we could keep captivity within reasonable balance and it will not take you off in that one Chase okay um there was a lot to think in let me make sure I've understood this so um you're saying that if we use uh say uh gbt as uh or another large language model as part of a bigger um AI system where you would have some sort of like chess engine type thing where you're pruning decisions uh in order to limit the scope and also maybe have some kind of factual engine um yeah maybe something Based on data like uh or from alpha or even like just some sort of checking of uh like fact-checking thing then that would provide a better AI experience is that correct yes all right brilliant okay I I think that you've just solved uh the problem of AI That's brilliant um well no it's going to be quite a test this is a noise solution come up with it but it's worth uh um there have been attempts to creating sets of uh cause and effect rules of causal effect to interpret the work right um but it's too complex to be capturing food but you might be able to do it for a specific domains and it's worth it because then the feedback you get from an engine like chat GPT will be um purely reasonable impossible and it will obey the rules that you set up okay that would be good or he will tell you it's impossible it will explain why okay uh I I think I see how this leads back to chat Bots because in in a chatbot situation you really want it to be constrained until the answers it's giving you and so having less freedom is often better in a business situation than having a really broad AI that can say anything because acid resistancy of people even people working with uh chatbots say oh if they only throw more facts into the equation to train the of the engines like channel 2bd that would eventually uh you know another emerging property all of a sudden it will gain reason uh you want if you feed it crap it will learn it okay it will take it for granted and will say well something marvelous happened and then all of a sudden you were in a and you are in B where you wanted to because of this matter that makes no sense so depending on the problem you need to constrain it to the right domain and then come up with the rules the cause and effect rules and the planning engine that will allow you to go from A to B within that that context fantastic um before we wrap up Giovanni final advice for any organizations wanting to adopt generative AI um well that they they should um get as much experience up front as possible which advice as possible to avoid going in the wild case um the the researchers nowadays that they are well trained but they don't have that level of real life experience they haven't had you know multiple failures like we all have and learn from them more than successes and to know what to avoid and what to go for and tell you know their the companies they work for these may lead you nowhere it's not a guarantee um and be listened to as well so if you can tell a client no I've done that three times and against my advice it hasn't worked okay um not that I don't make mistakes I still do but I think different mistakes all right uh uh it's nice to know that everyone makes mistakes regardless of how much experience they have uh so uh thank you very much for coming on the show working uh I hope you enjoyed the experience thank you thank youone thing that we've all served is this idea it was just an idea um that there would be emerging Behavior if you fetted enough data okay it was not obvious that would occur but the mechanism itself the Transformer is is easy to understand but what's difficult to understand is if you feel enough data at one point it starts coming back with responses that surprised all right Joaquin thank you for joining us on data frames thank you for inviting it's a pleasure to be here uh yeah I'd like to dive straight in so uh let's talk a little bit about your work creating uh generative AI applications so can you tell me what are the most common use cases your customers have for generative AI yes certainly um right now I'm devoted to chatbots because of care gbt um it represents a unique opportunity because the chatbots that we did before Chad gbt did not use the narrative AI we sort of created a different deep scenarios for dialogue and try to account for all the scenarios we could conceive of and have rules uh in special content for each scenario so it was quite involved thankful that has become much easier now and more creative and we are not teaching hold on specific scenarios a specific sets of questions so now it's it's much more natural but we have other challenges so uh chat box most recently and then before that I use the narrative AI to generate equal queries to answer English like questions basically you ask a question it translates it into SQL and go through the database gets the answer submits it to the chatbot to sort of surround you with some nice context before answering back um and one other project that I was involved in that involved computer vision um and what I was doing was basically taking video from cameras where older people lived in their houses and if for example one of them fell okay to detect the fault and be able to react and have the software called so that's one where the generative AI was basically data generating video because video is very expensive and you cannot have old people falling on purpose it's hard to get actors that will go for that so um you can generate some specific scenarios and then have the software learn what fall looks like and you know it's very useful because indicator in the neural network to recognize it I think all those three um examples you know the chat Bots the natural language interfaces to SQL and then the the computer vision they're all fascinating examples of using AI um maybe we'll start with the the chat bots so um you said that uh using church gbt this has made it easier to create chat Bots because you don't need to worry about the scenarios does it also have a benefit for the end users as well of the chatbots yes it is much more natural um for that for example one of the big challenges and it combined two of the scenarios that I was talking about uh the data that the user is interested in is generally not the generic data that was used to train that UV team for example this specific data to their business um and therefore you cannot ask TBD for you um and on the other hand you cannot feed your daily routine because of privacy um concerns so you sort of have have a local database that you can query but then the trick is how do you do it do you have jbt make the call for you then you have to supply the chat GPT with the context of the question so that he understands where you're coming from and what is it that you refer to with certain specific words um and then that helps it to generate the SQL query including the specific points that you want to make for example you could ask give me sales for March 2000 2019. so where March 2019 goes in the query it needs to make that link that is fantastic to do that it's a very difficult problem to solve otherwise and it's all for you it's just a matter of okay using your imagination and you may need two or three questions back and forth a small piece of dialogue which at UT before you get the answer that you need um and so there therein lies the The Secret of getting these chat box to work the way that you expect them to work that's absolutely fascinating that I'd love to get into the details of like how you've been dealing with this sort of um these privacy issues and using local data to get around that um maybe we'll talk about that later but uh for now just can you tell me a bit more about the different Technologies you're using to build these chatbots so you mentioned um GPT but are there any other technologies that you think are important here I'm currently using land chain which allows you to create basically a sequence or back and forth and get several uh pieces of software involved not just yeah tdd um told that for example if you um don't want any bad feedback therefore you are supposed not to use that word then you could filter the bad words up front and announcement struck cat GPD not to use any bad language um so the the involvement of these other tools has to do with filtering with correction um with preparing the context to make the query okay and then the rgbt comes back with how you need to make that query and then you make it of your local database now chat gbt so try to keep the ends up putting together the question exactly the right way that a programming for example or SQL specialist will do okay so that's one uh one other interesting aspect is I have to do a lot of reading so um I've put together a system by which I I get the PDFs especially uh technical articles um I feed them into a vector database together with your usual text and then um I can ask okay well in the last three years give you the list of all the articles um on a specific topic and it comes back with uh I have some background on this because in the 90s I was G4 detect from infosage which was an IBM product the world's first um customized news in a different article type of search but you would give it the topics you're interested in the context um and it will come back with articles with news items that concern that topic and that specific Focus that you may have um so it's the same idea but now we can do it much more sophisticated and let's say for example that I'm investigating something like for real I'm now into x i x AI explainability there are hundreds of Articles already so create a survey of them I feed them and then I ask specific topics and it uses those articles as the source and it's outsides of charity okay so it's called a lean bearings or vector digits okay uh that's that's very cool stuff um so I like the idea that you are using AI to do research for you in order to build more AI applications a good sort of virtuous circle there but uh yeah so it seems like Lang chain and uh Vector databases seem to be like the big technologies that are important here they're all fine pieces also excellent and I think um so a lot of companies at the moment they they really want to get involved in in generative AI but they're not sure whether they need to um use these other pieces of software that um other companies are making or whether they should build things themselves so I guess um in terms of of your work why do organizations come to you as a consultant rather than building things in-house well when they realize that it's not just having the university trade but also uh the experience um thankfully our universities are graduating thousands of data scientists but um it's hard to find somebody with more than maybe 10 years experience I've been lucky enough that I started doing this in the eight so I've learned through experience and I've been exposed to all of these Technologies and absorb them on my own as opposed to having that experience at a university and having that many years experience here's me uh perspective because for example when we were doing that was state of the art in the in the 80s and 90s has been completely superseded some of the ideas are still there but they're much easier to implement than back then back then we had to write everything from scratch including the vector databases and expert systems for example the neural network back then we know about them we have a good idea how to put them together not the modern models that have been discovered in the past after a thousand years but some simple amounts and they were impossible to run due to memory limitations of the hardware okay and they were too slow across okay um so I'm glad things it was on from the 1980s it's got the uh making progress absolutely but one of the things that clients realize is they may try one or two projects many of the AI projects fail because um you know you're asked to do something that if you know it's very very hard to do uh but you don't have the experience to know it then you go ahead and do it try it and uh AIC not ready to solve that particular problem that that particular way but if you have the experience then you can and I've told the clients that's not possible to do it needs research um so it may be possible to do in six months but currently it's not so you have to know the current technology you have to know um what the hurdles are for each type of Technology as well but what can be done what cannot be done okay so just related to that um how do you go about deciding um what makes a good AI project like um how do you how do you plan this how do you decide whether this is high priority whether it's feasible I would say a combination of knowledge and experience um the clients um hang on to everything that they read in the news okay and then people have a tendency to think that they can get away with a lot more than than it's possible given the technology um so for example there are people that won't charge EBT to do some reasoning for them well it turns out yeah TBT does not do any reason what it does is predict as they say the next word but not only that but predict the context and if you give it some context before you create the prompt it helps it quite a bit to constrain the possibilities right um and this context becomes quite uh complicated depending on how you ask the questions the sequence and all of that so for example if you ask a question in one context and then you go and ask another question in completely different context and you don't set it up before that before changing context um then you get the wrongness because it assumes that history has part of the dialogue you go through okay so if you want to change but the way that you are thinking about something you need to help it and that's part of the prompt engineering challenge that we have currently and this chat jbd in a way it's a black box even though we understand the mechanisms of the Transformers um it is a black box Because by the time you feed those billions of documents into it create a specialized model um that contains all of that information you really don't have a very good idea of what could come up okay um in the past we use neural networks like the endless long short term memory okay um and that's the end then we're capable of memorizing uh what had just come before so in a sentence if you were parsing one word it knew what had gone before three or four or five words before okay but that's not enough because you could continue in a crazy way that was completely out of context okay and in some science fiction novels or comedies or whatever you can never predict what comes afterwards but with Transformers you read not just a whole sentence you could be reading in parallel a whole paragraph so you get both the context of what happened before and you get the context of what's going to happen later so it uses these attention mechanism to do that and it makes it so much more powerful because it's like having a second reading all the same text you've already been there now you go through it again but doing it all at once and you get much better results and that's partially part of the Improvement uh in one thing that we've observed is this idea it was just an idea um that there would be emerging Behavior if you fed it enough data okay it was not obvious that would occur but the mechanism itself the Transformer is is easy to understand but what's difficult to understand is if you feel enough data at one point it starts coming back with responses that surprise you okay so that makes it incredibly interesting because all of a sudden you realize oh I I thought for example that it would get that question wrong and it turns out he gets it right or maybe you didn't ask it correct so you need to work on your own problem so this is really interesting um the idea that you need to um so the first of all you need to worry about your prompts and but also um sometimes the responses you get are going to be surprising so in general how can you evaluate the quality of responses from uh text generation model well for example let's say a client um has a specific type of this and um you want to supply your chatbot or managers of the stores for that life that have let's say a high school education so they usually don't have a course in statistics to understand you know the intervals of confidence um in the data that they are getting back I try to solve that and partially solved it a few years ago doing visual statistics reading curves and showing where they were in the set of curves so that they could say okay we are above the medium but below 75 percent or we are above the 75 percentile but below the 95 percentile but if you are anywhere between 2.5 and 97.5 you are within the usual high confidence inter so you can see your curse where you are today and Visually you say I'm doing great if you are certain about a certain and you don't have to understand the details well it turns out a chat about someone better let's get a question in English gets back to you it tells you how you're doing how your business is doing at a moment and if it's not doing well one of the things that we do is to feedback a predictive answer for example tomorrow is going to be Laos and also prescriptive like the answer which is what you have to do is start selling coupons right now okay or put an uh announce something or call a nearby business let's say if you are a parking lot and you know you're not going to do enough business tomorrow you call some nearby places that will attract customers and split the parking with them okay that's fascinating the way you think about like how do you interact and provide more information she's saying it's better to just give it more information rather than give it a specific task to do yeah exactly or you feed this with the the information that contact information and unbeknownst to the user so from the user question you interpreted oh okay he's talking about this particular context using some keywords therefore I need to give captivity this preamble with the information that it needs for example in the case of the SQL databases we don't feed the data we only feed feed the um the schema for the tables and then it figures out how to do the SQL from the scheme uh so you're not providing any actual data to the AI you just provide this is the structure of the databases what's in each table and then they can figure things out from there um that's very interesting um so does it make a difference when you're building these AI products if you're building a complete AI product from scratch or if you're just adding AI features to an existing product yeah it makes quite a difference um the the product that I designed for IBM in the 90s to um two years to do okay to design and to code and all of that we had to do everything including the networking software um and design the databases the vector databases to do this and create new technology I got three NLP packets out of it it was quite an effort nowadays I don't need to do that a lot of the products are open source I can just use them okay my only um criteria is that I use them in a commercial product or do I have to ask the client to buy or license that's the only thing um so nowadays things are much easier uh the problems we have to solve are at a different level okay um and as I said it's a question of prompting but also architectural design because you cannot ask details of your business again from chat GPT you need to get your activity to figure out how to ask your local database for the details okay so that's really interesting so I'm saying well from things obviously a bit of a a challenge but also figuring out the architectural design are there any other uh big problems that you need to consider when you're trying to um devise an AI project yes um first I had to think about it for a day or two then I need to figure out are there components are of the intermediate problems tall already and what components will solve okay and how do I need to strain them together um and if if I can figure that out then I tell the client yes go ahead and I believe this is the effort if I see that there is a gap somewhere and it needs research then I tell decline right now we need to cover this Gap and until we fill in all the gaps you know but I tell them up front so that they know some of them decide okay we'll wait six months and see what happens um or others say okay we'll we'll pay you to do the research and put it put it together all right um that's kind of interesting so um can you give me some examples of success stories where you've built an AI product for customer and it's had a big impact for them yes um when I was head of Consulting in the chief data scientists at the Oracle Latin American um we were challenged with some problems with uh it's not that we were not selling we didn't know what we were selling and in what quantity because we were selling so much so things got lost um this was a while and basically um what I did was to travel to several places talk to the sales people um talk to the marketing people took a look at the last four years of sales and then using at the time some a technical svm support vector fiend using that technique I basically was looking okay when we sell a set of products because I've been at that I'm in Oregon I tells usually sets of products for a corporation it's not just one and for example databases are sold with almost everything because they are required uh and we were targeting okay what's going on with the pencil database um the the sales people were taking quite a while they dare the materials they needed to approach the potential person and let's say they they used four and a half days to prepare for day 201 presented to the customer The Proposal um I was able to reduce by identifying patterns for example what products sold in combination and what products if you saw one you never saw me up those types of patterns okay um I was able to decrease the preparation time to basically zero because it was based on previous sales experience and I could tell them through the AI do you need to bring a technical team with you for this particular customer and combinations of products or it can go on your own being a salesperson or do you need backup of some sort and then what what is the best way to engage the customer if they decide to go ahead you'll bring a team or a series of meetings before a final commitment or or guitar okay and we were able to basically reduce the time that it took to prepare um is the customer to only half of they visit the customer and the AI just gave them you know a list of what happened so what we were selling and what else we could offer if they went for it okay and also the strategy that you needed to to use in um pointers to previous sales so you can't could contact the sales people and get their wisdom about this particular customer so that was the first we had a predicted part we predicted which customers would be more receptive and the prescriptive part which is how should you approach the customer okay um that that was quite a success and what else I would say I really love that story because when you started I thought oh you didn't say you built a recommendation engine but actually it was improving business processes so just speeding up the operations and also finding more effective ways for you to sell things as well so that sounds like it's had like an impact on many different levels yes yes it did and it was quite successful excellent uh just on the flip side though have you ever had any cases where you've had a customer that's been very excited about generative Ai and then after talking you've decided actually that's not the right technology for their problem yes and mostly in the in the past um I worked at cognitive systems which was you know come for the uh not shoot of Yale Department of artificial intelligence um and we were doing natural language processing back in the eight um and we created systems that would basically for example at a bank would receive the bills of Landing which are documents that the ships arrive with saying this is the cargo that we bring and these are the permissions you need to get and these are the charges and how to extract the cargo from um and but there were many different types of bills so we created a system to categorize them and put them in specific bins um to be um read and either accepted or rejected by specific people that had a particular knowledge of that okay um so that that was an interesting one um and it was challenging because for example we had to have several thousand documents and we use techniques that are they are called case-based reasons that that was at the time that's been superseded completely um other um aspects more recent is for example to to use word which is a type of neural network to do redaction okay um chatbot.gbd can also do it nowadays um and um a lot of people for example they they do redaction by just erasing words okay and um I discovered that instead of doing that if you use special characters surrounding a label the Social Security ID goes here blanking it out and then uh if you use the output that has those labels into a vector database or even chat Unity itself you can you could fine tune it uh then it starts looking at the patterns in terms of okay um Social Security ideas could go in this position for that position or that position not just blanks okay um so it would start seeing behaviors of where people tend to use your Social Security IDs for example in their correspondence okay and then from there um what was the use of that was this just to provide better redaction then or this month yes and then um you know to be able to predict when you are not sure that these uh sequence of numbers is some International ID or a social security number if you know the parents in which a social security number pops up you may be able to do a rotation okay this is not associated about you in Target 40. okay oh so it might be like a phone number instead but you need the context it was attached metadata when you see something and we know and for example if you can redacted then you cross it over but you know it was a social security now that would be useful later on if you do this day in and day out you'll recognize the path that's like a better level of knowledge that one team packing to catch you so as I talk a little bit about how generative AI um can um democratize access to data so I guess one of the big sort of potential benefits of this technology is that some tasks are usually performed by a data team can now be performed by and mom like even if they don't have a dative background so have you seen any examples of this well I've heard and read about so many samples but they are limited to particular contexts okay um is that everybody is trying to sell these tools that supposedly don't need any code they are fantastic if you apply them to use cases that everybody has saw okay because that's what the tools are for but if you have anything even slightly complicated then they need to program you need to add your own code into it to make it to what you wanted to do and not what somebody thought you would want to do always um go I have not seen anything that's impressive from this point of view now the the democratizing of the data there's several different layers because for example if you are talking about personal identifiable information like EI uh then you need to determine okay you can look at your own personal data but you cannot look at anybody else okay or if you have your higher um in the talking Bowl let's say at a bank you may be able to use somebody else's personal datability depends on your rank and your particular function within an organization all of those rub issues need to be set up um even before you can make the quick all right so what you get is what is maybe due to the security process you go through rather than the chatbot itself so the chatbot has no this it assumes okay you ask it a question you already have the right to ask okay um so by the time you get to it you have to go through all of security layers you know um now the democratization of data in terms of for example you could go and completely obliterate all the data that's pii no matter what and then you feed it or you put those wings okay like I was saying Social Security number personal phone number or cell number uh name of a company without mentioning the company and so on that's democratization anybody can read it you're not damaging anyone some people could guess who you are talking about okay from the context um but you know that's one aspect where human intelligence it still exceeds anything that chat GDP can do um so in that sense yes you can democratize data you can actually exchange information without giving away any Secret um so well related to this as you've been working on uh natural language interfaces uh for SQL queries and as that helped provide access to data for people that couldn't normally get access to those databases yes because they will need to be experts in SQL you know and have the access rights to go into the database to start and then know how to actually see and in some of these queries get very very complex um so yes now um as far as further democratization um I things that very soon will be able to um exchange information to companies between people in such a way that we are basically guaranteeing that no piis being of any time or for example that they fully they exchange fully complies with Deepa or some other um so I believe these things are going to be possible very short that's really interesting the idea that um you can safely exchange data from one company to the next uh and not have to worry about those data privacy issues um in general um do you think or where do you think organizations should make use of generative AI just to improve their data capabilities well one of the problems everybody has is having enough examples of data to be able to train the neural network specific tasks not related to catchy and so let's say for example how one of my questions is how many different ways to chat GPT can you ask about what was the most profitable month of the year for a specific how many different ways can you ask that and instead of me spending days or weeks you know writing down everything that occurs to me um I have charge APD generally that data and then I take all those 100 200 questions whatever they are all asking the same thing and now I go and use it in my chatbot make sure that it comes up with the same right SQL query every time and usually it doesn't so I need then to take care of um because language isn't most of the time is not specific enough it's indeed in that ambiguity and it depends on how you ask the question that'll be given to your script if you don't get rid of it you don't get the right word Ah that's interesting so if everyone said this correctly you're generating hundreds of different prompt variations in order to make sure that regardless whatever a user inputs into your chat bot you're going to get a consistent correct answer yes and if there are ways of asking that always get gets it wrong then one of the things that I do is I check let's say using a Pinecone Vector database um are you asking this type of question and if the answer is yes it comes back he said I'm going to charge it please say um would you be more precise okay okay and related to this do you have to um do any like testing of prompt quality like do you do a b testing on your prompt or anything like that yes yeah depending on the needs of the client yes so we prepare a whole test that we can apply but again you know the the labeling and the data generational that I I had a a client um an insurance company in Los Angeles uh that they want to be able to get the readings from dongles in The Limousines and taxis uh that they insured so that they could see first whether the drivers were behaving themselves while driving okay and secondary if there was um an accident that we would be able to detect it but after more than ten thousand messages we only found 13 of them that were two actions so every time I try to train the neural network you will say well it's just 13 of them let's assume they don't exist so it would never classify anything as uh accident so this was a huge problem and it's very hard to fake data from an action okay so I'm in your door uh in the car it's 4G right there and there it's the same as for example a bike bumping against the door um so how can you tell the difference and if you can't how can you produce more accidents um so I went to the website at the government website where they test the cars and bought their accident data transformed it into the right format as it became from a dunbook and then fit but those added a few dozen examples not enough um so in that case I was lucky because I used a mathematical technique called a fast Fourier transform which is used in physics and the Fast Food Network form changed it from uh you know acceleration versus time to acceleration versus frequency the frequency gave it a Telltale sign of the accident okay there were some Peaks a certain particular places the Roady occurred in accidents then I fed that result from the fast food that was formed into the neural network and you could easily tell one from here I was working because nowadays um we could use a generative AI to actually produce more examples of the accident that without having to be lucky and get a transformation that gave UPS absolutely so um fast Fourier transformed uh it's like detect accident data that just sound like quite quite an intuitive or not intuitive like a novel sort of leave for the imagination for how to solve that but yeah um uh so more generally it just seem like uh generative air is really good for creating synthetic data and I do like the idea that you can use that if you do have a problem with class imbalance where you've got you're trying to detect rare events okay well that that's one of the the big plus results um so does it change um the kind of skills that you need uh to work with data now the fact that you can do different things with AI yes um but if you are working at a different level you know in order to take full advantage of the engine let's say cat gbd you have to speak its language use it for what it was intended to be it's not like there's a lot of apis you can go through really how you set up your phone by context avoiding certain terminologies um and not making it too complicated um one other thing that I've noticed is for example if the database has too many tables then it even takes too long what it gets is wrong because there are a lot of joints okay and that takes time by itself if you take the query yourself but it takes time to set up correctly so catch it is not as good so um you would make the table simpler a data analytics type and for the relational database into Data workhouse type with with lots of attributes together the thickness in the table the tables as possible and and you get better results soon you have to make accommodations for the limitations of the chat okay I think that's a useful tip to know is that if you're trying to generate these SQL queries with lots of joins then it's not going to work so at least at the moment unique is lucky and it works if it has a very closely example in this memory but if it doesn't which far chances are not then easily get them wrong or more sensitive to changes in the wording the English wording behind English query you know ah yes okay so the the language used in the database matters perhaps because GPT in general is better with English rather than other languages for example uh if you're asking something about a specific record in the database then I found that using the word simple it's crucial you say similarly those is One Direction if you say something else or avoid the word syndrome it may give thumb of a particular cup because things are moving pretty fast in so many developments going on right now it's quite hard to keep track of everything so are there any generative AI projects that or tools that you're particularly excited about at the moment well as I said right now I'm into activity in Lion King but I'm also looking at other aspects I have a strong interest in as I said in xai um explain a little um and for example in the case of explainability the algorithms that work best in explaining themselves are things like decision trees force that type of thing but if you think about the complexity of certain problems um and we already have examples of those types of problems in math that's a math problems that have been solved with computers that no human understands we just take it for granted because people have checked specific portions of the code um but nobody will ever understand them because they involve thousands of steps um and the same occurs with a decision tree or a force um in uh that it may have thousands of decision points to give you an answer and the best that you can do is well if you find different variable between these range and that range and then you repeat that a hundred times who's going to understand you know if you have 100 there absolutely I can make them incredibly difficult to explain what's going on in particularly complicated models yeah and we may have to settle for more generic answers um if the only way that we can explain it even with the most explainable algorithms is by providing hundreds of thousands of decision points and so related to this I get the theme that um over the next few months there can be a lot of um sort of things that claim to be exciting AI but maybe aren't so do you have a way of deciding like or any heuristics for deciding like what is a good quality AI tool or company versus um what's uh just I don't know cashing in on the hype well usually I read the scientific papers um in arcade but not the Press I mean I also read the press to see but um I don't believe it until I see the background to the idea research and they have a certain level of confidence that this will work and then it's a question of reproducing it um and if the results are dive with the clients in in the Articles then I'll try okay and as I said we use this much open source as possible um open source has been fantastic for everyone but occasionally there are those gaps that need to have a client that's interesting things works um and as I said the The Avengers for them is the agreement Secret Ary okay uh yeah so I do find that interesting because that you mentioned having to read all these um these Journal papers because sometimes if you're not involved in Academia then you think well it's just something that happens in the background uh universities but actually this is a really important part of your like your research for building business products then yes and and also I read articles on other types of products and the level of confidence that they have um but I I do tend to read the literature um and um for example uh when I was at IBM DJ Watson labs they had all the papers there I I researched as much as there was before I started inventing things with my team um that's the way to do it you don't want to be then want to say step on shoulders of giants absolutely um all right so is there anything you're working on right now that you're excited about well as far as the explainability okay um it is a big concern of mine because it's going to come up very quickly with charity okay uh in general can explain when you ask explain it to me uh what the last answer is meant to do um but checking it is a different type of piece because you cannot use their gbt to check um many times you can make it hallucinate not as often okay um as it may otherwise by restraint and making sure that you design your queries so that it's well restained you for example there is one parameter called temperature that if you set it to zero will only give you answers that's 100 sure are correct in other words that answer is somewhere in one of the texts sorry okay that's still not a guarantee but yet we live with it a lot of scientific papers um are not of the best quote and that has been proven so we live with uncertainty uh and we may need to check multiple sources here and there um ppt4 can give you sources so you can go and check with and make sure that yes indeed that in subjects with what people already know um but in order to create guard bridge to keep a generative AI in general not just yeah TV but Valley and others from going off the rent you have you need two things you need um a set of rules okay of policy that keep you within about it it has it as an expert outside that reads the answer now this means cannot give you the answer word by word because it needs to take the whole answer before giving it to make sure this one this will add additional time okay so that's a consideration you don't get your answer you may want to go for coffee and come back all right uh but you have an independent check on the answer we are sure that he judge by all the guardrails that you've set up in this the second piece is a plan um what do I mean by a planner basically something that will say Okay um we have these facts and they want to know if we can meet these goals starting from those facts uh entering the plan and make sure every step of the plan advise by all the rules all right um let me give you a small example you know the cannibals and missionary problems I don't know oh Cannibal's in missionary I I think I might have heard with different people crossing the river but go on tell the story you know it never happened but uh it's uh three missionaries and three cannibals on one of the river's margins with the boat on that margin of the river and basically if you at any time had either in one of the margins or in the boat itself more cannibals than missionaries the cannibal the missionaries will be gone after a while um so you have to solve the problem by always keeping them in equal numbers both cannibals and missionaries or more missionaries right um and imagine that you were to draw all the different ways in which this can happen you start with three three and one margin but in other words the six people is one margin and with an empty vote on that March um one possibility is that the boat is in the other margins of gable they can't do anything um but other possibilities lead to situations where you have more animals than insurance so imagine you have all the possible you cross out every time the rule is violent okay so you prune the tree um and you use that tree to make the plan you are guaranteed that you'll always succeed because all of the Dead ends have been cut up all the time okay now this is a very simple problem the thought but what happens if the possibilities are basically ended and you do not know how it's going to progress like in a game of chess there are so many possibilities it's inconceivable that you could account for them all and prune all the ones are wrong so you need to set up a dynamic policy that calculates everything maybe two or three moves ahead from any point okay and then you proceed according to the answer by that policy you in other words you prone just a small part of the tree and then you prune again at the next step and so on and so on okay um we would need to do something like this in chat gbt to keep it from hallucinating and taking it in the wrong direction you also need a causal end or model that for example says okay certain things are impossible because of natural loss all right so that you can also use that as a policy criteria because you need to check for breaking natural laws when you do things of course it would be terrible if you are interpreting for example satisfaction novel or fantasy novel because it would violate but that you could have your own rules okay in an imaginary world saying this is valid and this is not valid and then have it interpreted accordingly okay that's one of the big advantages but if we were to implement the combination of these I believe that we could keep captivity within reasonable balance and it will not take you off in that one Chase okay um there was a lot to think in let me make sure I've understood this so um you're saying that if we use uh say uh gbt as uh or another large language model as part of a bigger um AI system where you would have some sort of like chess engine type thing where you're pruning decisions uh in order to limit the scope and also maybe have some kind of factual engine um yeah maybe something Based on data like uh or from alpha or even like just some sort of checking of uh like fact-checking thing then that would provide a better AI experience is that correct yes all right brilliant okay I I think that you've just solved uh the problem of AI That's brilliant um well no it's going to be quite a test this is a noise solution come up with it but it's worth uh um there have been attempts to creating sets of uh cause and effect rules of causal effect to interpret the work right um but it's too complex to be capturing food but you might be able to do it for a specific domains and it's worth it because then the feedback you get from an engine like chat GPT will be um purely reasonable impossible and it will obey the rules that you set up okay that would be good or he will tell you it's impossible it will explain why okay uh I I think I see how this leads back to chat Bots because in in a chatbot situation you really want it to be constrained until the answers it's giving you and so having less freedom is often better in a business situation than having a really broad AI that can say anything because acid resistancy of people even people working with uh chatbots say oh if they only throw more facts into the equation to train the of the engines like channel 2bd that would eventually uh you know another emerging property all of a sudden it will gain reason uh you want if you feed it crap it will learn it okay it will take it for granted and will say well something marvelous happened and then all of a sudden you were in a and you are in B where you wanted to because of this matter that makes no sense so depending on the problem you need to constrain it to the right domain and then come up with the rules the cause and effect rules and the planning engine that will allow you to go from A to B within that that context fantastic um before we wrap up Giovanni final advice for any organizations wanting to adopt generative AI um well that they they should um get as much experience up front as possible which advice as possible to avoid going in the wild case um the the researchers nowadays that they are well trained but they don't have that level of real life experience they haven't had you know multiple failures like we all have and learn from them more than successes and to know what to avoid and what to go for and tell you know their the companies they work for these may lead you nowhere it's not a guarantee um and be listened to as well so if you can tell a client no I've done that three times and against my advice it hasn't worked okay um not that I don't make mistakes I still do but I think different mistakes all right uh uh it's nice to know that everyone makes mistakes regardless of how much experience they have uh so uh thank you very much for coming on the show working uh I hope you enjoyed the experience thank you thank you\n"