Prompt Engineering with GPT & LangChain _ Get Better Outputs Instantly

**Approaching and Utilizing the Moderation API for Open AI**

The Moderation API is a powerful tool that can be utilized to analyze and filter out sensitive or inappropriate content from user-generated input. By leveraging this API, developers can create more robust and secure applications that protect users from harmful or unwanted content.

To get started with using the Moderation API, one must first understand its various approaches and how it can be applied in different contexts. For instance, when dealing with financial analysis, the company name is crucial to getting correct headlines and sentiments. Similarly, when working on projects involving open AI, moderation APIs are essential for ensuring that content is accurate and relevant.

One approach to utilizing the Moderation API is by analyzing Reddit comments on subreddits related to Wall Street bets or other financial topics. By reading in the text file of Reddit comments, developers can access and analyze the data to identify potential issues or areas for improvement. In this example, we read in the text file using the `data.read()` function and store the output as a variable named "comments". The comments are stored in Reddit's command format, which is optional by default.

To further analyze the comments, developers can use various techniques such as printing individual comments or selecting specific ones to test. In this example, we print the entire text file of comments using the `data.read()` function and store it in a variable named "comment". The comment is chosen to be offensive, but developers should exercise caution when working with sensitive content.

Once the comments are analyzed, developers can use the Moderation API to filter out unwanted or inappropriate content. To do this, we import the Open Eye package at the start of our notebook and create an instance of the moderation API using the `create()` function. Our input is simply the comment itself, which we pass to the API for analysis.

The output from the Moderation API provides valuable insights into the nature of the content being analyzed. In this example, the comment is flagged as "harassment", and we also receive a score indicating the level of confidence in our analysis. This score can be used to adjust the severity of the response based on the likelihood of the flagging.

The Moderation API provides various flags that indicate specific issues with the content being analyzed. These flags are categorized into different scores, which can be adjusted depending on the context and application requirements. In this example, we receive a single value called "flagged" which indicates whether the comment has been flagged or not. This value is true in this case, indicating that the comment has been flagged as harassment.

In an application, developers should only proceed with filtering out comments if they are flagged as unwanted content. To do this, we can simply check the "flagged" key and take action accordingly. In this example, we don't perform any further analysis but instead print the moderation output to demonstrate its capabilities.

**Conclusion**

In conclusion, the Moderation API is a powerful tool that provides valuable insights into user-generated content. By leveraging this API, developers can create more robust and secure applications that protect users from harmful or unwanted content. Understanding how to approach and utilize the Moderation API requires careful consideration of various factors such as content analysis, filtering, and flagging. With practice and experience, developers can master the art of using this API to create better applications for their users.

**Recommendations**

Based on our exploration of the Moderation API, we recommend that developers:

* Familiarize themselves with the API's various approaches and techniques.

* Practice analyzing and filtering out sensitive content.

* Use the API to analyze Reddit comments or other user-generated input.

* Develop robust applications that protect users from harmful or unwanted content.

By following these recommendations and continuing to learn about the Moderation API, developers can create more effective and secure applications for their users.

"WEBVTTKind: captionsLanguage: enwelcome to prompt engineering with GPT and Lang chain Lang chain is a very popular open- Source framework that helps us add extra functionalities um when integrating generative AI capabilities with uh applications or or data Platforms in this module we will get a gentle introduction to the framework and some prompt engineering techniques um and understand some of the basic concepts of of Lang chain in later modules we're going to look at some of the more advanced concepts such as retrieval augmented generation so in this in this module we're going to start off with calling the open IM models through the framework we're going to set up some prompt templates to write reusable and dynamic prompts these can then be versioned along with the rest of our code base um we're going to work with llm chains which is basically the combination of uh large language models and these uh forementioned prompt templates then we're going to look at the output parser in the Lang chain framework something very valuable um that can instantly generate certain objects like python lists dictionaries Json objects Etc that we can then um instantly use further in our code next we're going to work work with Lang chain agents and tools these will allow us to do many cool things like instantly executing code that's being generated by the model or um let's say do SQL lookups Etc and then lastly we're going to look at the openi moderation API this is especially useful when you're trying to let's say build your own API uh no build your own AI application you want to have uh something like this as a filter in between the user and your actual model endpoint now for this project we're going to use two small text samples um I wouldn't necessarily call them data sets it's basically just five to six line samples they're being kept very short so that every time we print our results we can just get a very simple overview of everything just keep in mind that all the techniques that are thought here um and all the code that is used here is very scalable we can just use it on any um size of of data set now in the file system here let's take a very quick look at these headlines so we have six Financial headlines on finished companies and whether their profits Rose or or uh decreased or other information such as uh company X has been awarded a contract for a certain project Etc now the goal is to do some Financial sentiment analysis here so for example using these GPT models we automatically want to generate um investment advice for example uh based on the current news so let's go back to our notebook oh yeah maybe a good thing to mention if you ever get stuck there's also this notebook solution here um so you will find the solution to all the code problems in this uh solution notebook before we get started one of the prerequisites is that we need an openi API key now for this openi API key this has been covered in the previous module um how to how to get one and how to add it to the environment variables if you don't know how to do this or you have not followed um the previous module you can always check out the getting started notebook in the file system with that being said let's start with our setup the first thing that we want to do is um do some installs so we're just going to install uh some packages uh you might get some warnings or sometimes even some some errors with these uh with these versions but just for this project um these are not breaking changes so don't be afraid um you can just continue the project with any um with the versions mentioned here now with those being done let's start with our first instructions the first thing that we have to do is some basic Imports um to basically fetch our environment variable and um um set it within the Open Eye package so let's first import the OS package and that's being done by just simple um import OS statement and we're going to do the same thing for the open ey package just import open ey then we can um set our API key so open ey. API key and we want it to be our environment variable that we can call with os. enin so I want to have the open AI API key just going to run this cell to set our key um and we should be good to go next we are going to start just doing some basic calls to the open ey models um from Lang chain and that's being done using the openi and the chat open classes the openi classes are used for completion models whereas the chat openi are used for the conversational models so maybe a quick um overview here comption completion models refers to typically the the older generation of models if we don't count the new instruct models um but basically they just give like a single completion to a text input whereas the chat models are made for conversational use so the models used in chat GPT which are GPT 3.5 turbo or GPT 4 are the chat uh conversational models so they will be loaded using the chat openi uh package uh no chat open ey Clause sorry but first let's let's start start importing these classes so these are also just very straightforward Imports so from Lang chain. llms we want to import openi and from Lang chain do chat models we want to import chat openi okay run the cell we have done all of our Imports now let's get started on task one we want to import our financial news headlines data so we have taken a look at the uh sample text that we have the financial headlines we just want to read in um this this text file and basically store it within a a simple python list let's say so what we're going to do is um we're going to open the text file in a read format we're going to use the read lines method and basically assign it to a new variable called headlines so let's do with open um then the name of our text file which is basically Financial headlines oh that's not do that which is basically Financial headlines. text and we want to open this in a read format let's open this as let's say data and then we can read this in by um doing headlines equals data. read lines to get a quick overview let's also print out this python list so here we basically have our um headlines loaded it into a list just a list of strings with every element being a single headline now we might not is that we have this uh new line uh character at the end of every line um let's just get rid of it um it doesn't really influence the large language model that much but just for like sanity let's let's get rid of these uh of these characters to get rid of these characters we can um simply use the do strip method um for the for string objects within python so to make it a little bit easier I'm going to use a list comprehension here so for headlines let's do x 4 X in headlines so I'm just going to Loop through headlines and instantly generate a new list and reassign this to headlines now the operation that I want to perform here is basically applying that dot strip method to every element of the headlines do strip basically removes any character at the end of the string um and I want to remove the slash n now let's print out headlines again and look at our result so now we basically have a bit of a cleaned up version of our headlines without these uh these new line characters everywhere for the second task we're going to learn how to set up prompt templates now promp Chrome templates as I've told you before um they're made to be like uh reusable and dynamic uh prompts that we can version within our database uh within our code base so basically we can save these with the rest of our code we can add it to our repositories and um basically reuse it in any way like this we can add input variables to these prom templates to make them Dynamic and within line chain we have built-in verification tools um that kind of check whether that input is always uh given when it's being called from anywhere else which makes our code more robust now the the format of these prom templates they are kind of structured like f strings in Python basically meaning we have a a typical string um and if we want to add any uh Dynamic variable within the string we're using the curly brackets here so for example if I want to have a prom template to give me suggestions for a trip to any kind of City I could set up a quick um template that says can you give me some suggestions for my trip to and then a city variable this allows for dynamic templating basically or dynamic prompting now we have two different types of prompt templates um and that is that has to do with like the two different types of models that we've spoken about earlier basically we have the prompt template and the chat prompt template one is for the completion models one is for the chat models the reason for that is because the completion models they basically just generate once they uh it's like single input single output while chat completion they can store um prompt histories Etc and these templates will also be able to store certain prompt histories and a system message as well so the system message um might have seen this in the in the previous module but this is some sort of meta prompt that we can use to do all sorts of things such as um putting the model into a certain role uh asking it to um to to use a certain tone of voice or specify um specific output formats Etc Now to create a prompt template we are going to use the prompt template class in Lang chain so for a for a very simple one we're just going to use the from template method um in in uh in the framework um and then we basically have to pass along the template itself uh the the text itself The Prompt itself so for that we're just going to use this prompt and we can already see see that we have one input variable here so we have headline which is basically a dynamic variable within our prompt so we can use the same prompt template against our entire data set of headlines then we're going to format this prompt which is basically um passing along a certain headline for that input variable and we're going to see what the end result looks like so first let's start by uh importing The Prompt template class and that is done by from Lang chain. prompts we want to import The Prompt template class next we want to um basically set up a new prompt template and we're just going to call it we're going to assign it to a variable called prompt template in snake case so we're going to do this by prompt template do front template and in here basically we just want to use our um our prompt that we have here so that's that's that um now we will see more advanced ways of of setting up these templates with some verification methods um further down the line but for now let's just create a very simple example of a prom template now let's say we want to format this one with the first um headline of um our data set what we're going to do is we are first of all we're going to add uh we're going to assign our result to a new variable called format at prompt and we're going to call this format method on our newly created prompt template so I'm just going to do prompt template. format and then it will expect um this input here so this headline input so let's say headline equals and I'm going to take the first element from our headlines data set so remember the headlines was our uh python list with all of our headlines stored in there as a string I'm just going to take the first element there by putting in index zero and let's take a look at our formatted prompt and what this looks like so here we see um basically are our instruction so analyze the following Financial headline for sentiment and then uh combined with the headline that we fetched from our data set now that we have set up a prompt template um let's continue towards a chat prompt template so here we can have some um some prompt history if we want but mainly we get access to this system message that's available within the gp4 model and within the GPT 3.5 turbo model so the models used in Chad GPT first what we want to do here is import the other class so chat prompt template um from the same package and we're going to use this system message now the system message if we're going to look at um how it's written uh so you might remember I told you like system message can be used to put the um to put the the model into a certain role and that's what I'm doing here with this first uh sentence so I'm telling the model what are you doing right now or or what kind of role do you do you perform so I'm telling you you are performing sentiment analysis on news headlines regarding financial analysis this sentiment is to be used for blah blah blah and that output the format of the output has to be consistent so the thing is I want the GPT model to only output either the words positive negative or neutral and nothing else I don't want it to to um to respond to me with like sentences whether telling me whether it's like slightly positive or kind of negative I just want to have the single word positive negative neutral that I can then later on um use for analytics so we're going to use this system message um and we will um yeah we will start by instantiating a new chat prompt template just as we did with the other template and instead of from template we're going to do from messages we're going to add a system message and a human message so a human message is basically like the user prompt um this will be put into a tuple um together with their respective contents now we're going to format that template in pretty much the same way as we did with um with the previous prompt template and just check out the results so first of all let's start by importing our prompt template so from Lang chain. prompts we want to import our chat prompt template next we we want to um we want to Define our system message so we will just copy this string up here and assign it to the system message variable like this next we are going to instantiate our chat prompt template and the name for this variable is is chat template so let's call it chat template we're going to set up a new chat prompt template from messages now this will take in a list of different messages so let's start up with a list and we are going to do a small list of a single system message message and then a human message but if you want to in these chat prompt templates you can also add like a bit of conversation that already happened before you are using this certain prompt so you could add like a human message um an AI generated message and then a human message as well uh to simulate like the start of a conversation already now for us we're basically using this just to get access to a system message so what we're going to do is we're going to pass a tuple and then first of all we're going to say this is the system message so here we can do system or human or stuff like this so let's start with system message and then the second element of our Tuple is the content itself so we've just stored the system message in the variable called system message so I'm just going to call that one here that's basically this long string here that we just find second going to make a second Tuple and that will be a our human message um and for our human message we basically just want to have like this same um the same one that we used here within our prom template so this will become our human message now again you can see that we have um one single input here so again we have this headline input that we want to uh format um our prom template on so let's quickly go back to which one yeah so we want to have it assigned to formatted chat template so we're going to Define formatted chat template equals our chat template. format and again just for that headline uh variable we want to pass along our first headline now let's see what this looks like once we start to print it so this is a bit more elaborate than the one before because here we first have this entire system message that we defined and then we have our human message this is basically what the whole chat GPT um application looks like as well behind the scenes um so behind the scenes you also have this system message integrated in there and your input prompts in ch GPT will basically be defined as this human message so now we have set up a prom template and a chat prompt template next up we're going to set up some basic llm chains now we're going we're not going to go too deep into this but the concept is quite simple an llm chain is basically the combination of a model and the templates that we just have set up and then we can we're able to chain multi mple of these objects um to each other by basically adding the output of a certain llm chain as the input for a next llm chain now again we have this for completion models and chat completion models um so we have two different chains um but first of all let's start with with um using it for our completion models so here we're also going to do the necessary Imports so that is from Lang chain. chains we want to import llm chain now first of all um yeah we're going to reuse the is open ey class that we um that we imported all the way at the start and basically calling or instantiating one of these open AI uh classes will give us access to a model to a GPT model now the openi instead of the chat openi as you might remember um by default that will or in the current version at least for Lang chain that will steer us towards the text Vinci model um and we can instantiate multiple of these classes to for example change parameters so I could instantiate one with a temperature set to zero or a higher temperature and um maybe to touch a little bit on that one uh a very simple and brief explanation of the temperature is basically the randomness that we put into the model like the higher the temperature the more random a model becomes which is is favorable for Creative use cases while if we have like um use cases that require a lot of precision like we want to have more factually correct answers um or for example when generating code we generally prefer prefer very low or even zero uh temperature to get the best result possible okay so let's start by um instantiating one of these llm chain um classes so we're going to assign it to a variable called completion chain so let's create that one first completion chain equals llm chain and here we want to have two different um parameters so we have llm and we have promt now for llm let's just um quickly instantiate a new class uh instantiate a new um model object here just a simple one um just using the default um parameters here and for the prompt let's use our prompt template that we've defined earlier so that is our prompt template now remember this is not our formatt prompt so this is where we have this um flexibility like every time we run this chain we can run it with different inputs so again let's try to run it on our first headline so we do our completion chain. run and as input for that headline oh for that headline input that we had in our chat prom template we're just going to pass along our first headline now we're going to automatically format our promt template and send over that prompt template to a completion model so let's see what we get as a result we get D n-n positive so it um it class CL ified the first headline as positive um but it still has these characters in front of it which we don't necessarily want but let's get we will get rid of them like um later in this this module next let's do the same thing but using our chat prom template so for this we're going to be creating an llm chain but just using the chat openi model um instead of the normal OPI model the concept is pretty much the same so let's let's say chat chain um also equals llm chain and again we have these same two parameters so we have llm and we have our prompt now instead of the opening IM model let's create chat open ey model by default this will direct us to the GPD 3.5 turbo model um but we could also fill in like for example gp4 or anything like this and for a prom template we have our chat template that we've created earlier now again keep in mind that this is not the formatted template so we have that flexibility to format it while we or when we run the chain so for that I'm just going to do chat chain. run and with this template we had two um or we have two um input variables because we also now have that system message that we can uh tune so so for the headline I'm just going to fetch our first headlines again oh uh yeah like this and then our system message because we can change this with every call to the model um we have stored this in a variable called system message as well if we run this we get positive so both the completion model and the chat completion model classify that first headline as positive which is a good thing because if I remember correctly there was a headline that said like um that company's profits went up from X to Y so for task four we're going to make use of the output parser in Lang chain the output parser as said before um is very useful when we want to have like uh python objects straight away from our large language model output now it's good to know that the large language model itself it will always just output um strings it will always just output texts but the output parser um has some built-in tricks to automatically create like python lists date times Jason dictionaries Etc now how it does that is first of all it will parse the text output and basically create that uh specific python object from it but on the other hand it also adds format instructions to our prompt behind the scenes so it will actually add some extra text to our prompt to make sure that the text output of our large language model is in the necessary format that we asked for now in this specific case we're going to um we're going to extract the company name from every headline later we can then um basically set up a data frame that combines the company name with their respective sentiment um to basically have a very basic version of financial news analysis so what we want to do in terms of output parsing is when we pass along all of our headlines we want to get a python list which basically contains all of the uh company names now to do this first we're going to make the necessary Imports um so we're going to be using the comma separated list output parser um and we're we're going to fetch the format instructions from it this is basically what we're adding behind the scenes to our prompt so we'll have to pass this along within our new prompt template as well so with that being said we're going to set up a new prompt template we're not going to use this from template method because um we want some extra flexibility here um and this will allow us to um basically call the init function of this prompt template which has um or where we're going to use three parameters first template this is basically what we've used before so this is where we actually give our our text prompt as a string um and here we're going to ask it to list all the company names from the following headlines now one of the problems I've noticed while developing this notebook is there is one headline I think with two company names in there um so we just want to limit it to one um company name per headline for now just to have like the easy analysis so we can map them one to one to the sentiments that we will get later on and you can see that we will um we will insert all the headlines this time not just a single headline so we're just asking it to list all the headlines um to list all the company names from all these headlines and then afterwards it will automatically add like these format instructions like this we're going to specifically Define an extra parameter that's input variables and we do this to verify that these input parameters are um are are actually being inputed like if we if we try to call this prompt template without inputting certain headlines then we will get an error so that is just a best practice to get robust code um especially when using this in like larger applications as then as partial variables this is basically where we um where we pass along the format instructions that we get from our out put parser um to have it parsed in a list and then we're going to look at the result oh no we're not going to look at result we're just going to format the prom template uh in this case and then in the next one we're going to look at the result because we still need to run it through the model like with this with this code block we're just going to end up with a formatted prompt template that includes output parsing so first let's start with our necessary Imports so we need this comma separated list output parser from Lang chain. output parsers so from Lang chain output parsers import comma separated list output parser now we're going to instantiate a new one and we're going to assign it to a variable called output parser so just use output parser equals comma separate list output parser and we don't need to pass along any parameters here the default ones are more than fine next we're going to fetch those format instructions from that specific output parser and that is that is very easy that's just calling this method the get format instructions um and let's just store these under format instructions so we get get them from that comma separated list output parser by using output parser doget format instructions now we are going to set up our new prom template to get um to get the company names from all of our headlines so the name for the variable that we're going to be using here um I see that it's not listed here but let's just use company name template um so this will become a new prom template and we're just using the init method so we're just doing it like this and here we will have three different parameters so we have template we have have um input variables and we have partial variables for the template we have we have it here so we're just going to copy this one and add this string in here for our input variables um we basically pass along a list with our input variables um as a string so keep in mind that these format instructions they do not count as an input variable they are partial variables so the only input variable that we have here is headlines and then partial variables that's basically these format instructions and here um the code is expecting a dictionary so we want to have a key value pair that says um format instructions and that equals the format instructions that we've received here from our output parser so to get a quick look at our template let's just format that template um with our headlines so so let me see here Le is basically we're going to call it format company template to keep the same structure as before um and we want to basically do company name template. format much in the same way as we've done before and now as an input variable we have these headlines now what we want to assign here is basically that string that list of strings that we had before that we read from the text file and that is also the same name so headlines the headlines variable now let's print this and um yeah maybe let's just print this as well to have a look so basically what we see here is um our instruction plus our list with all of our headlines but within a um within a list that is being converted to a string and then what we see at the end here is basically those format instructions that we have added those for inst instructions are then um basically being uh or they put it in the right format to be parsed by um by the Lang chain framework so next um what we're going to do is we're basically just going to run um this template through a model but now instead of using an llm chain let's try a little bit of a different approach so you have like multiple so you know uh multiple approaches um to actually make use of models and make use of templates so in this case we're just going to set up a um open ey model so the opening eye model here and we want to set its temperature to zero the reason for that being we want to have it as precise as possible um we don't need Randomness at this point we want to be as precise as possible on extracting these company names from our headlines so the way we do this is we're basically just going to create this open ey um or create an instance of this open ey object Again by default by referring to the the text of Vinci model um and we're going to pass temperature equals zero so we have a model with zero temperature now we're going to run that model um basically by just calling that model variable um and we're going to store our output in underscore output now an underscore basically um yeah it indicates that that we're just using a temporary Val variable here and we will override it a couple of times as well um because we just uh want to have like this intermediary step in between now you could chain all of these methods but for this module just to keep it a bit more easy to uh to follow I'm adding this extra step here so for model uh to run this model I'm just going to do mod and ask uh and um insert our template here so that was our formatted company name template next we want to parse the output so we will need um our output parser um and that output parser has this do parse method um with with with which we can parse that output that we got from the model we're going to assign this to a new list that we've created that is company names so this will be this will instantly be a python list because of this output parser within within Lang chain that's the whole like power of it so output parser do pars and we parse the output that we've received from the model then let's quickly print the data type of this uh company names variable just to show you that we indeed have a python list object straight away so that is done by doing type and then our variable itself which is company names and lastly I'm also just going to print the company names themselves so that we can have a look at them might see this takes slightly longer because we're adding more and more functionalities um and later that might even increase but uh here we see um our class is indeed a list so we instantly have a python list here that we can use like first there down our code um and we've extracted the company names from our headlines now with that being said let's move on now to agents and tools to uh continue with these company names next up we are going to work with agents and tools in Lang chain now agents and tools an agent is basically um let's say a bit of a code executor um that can work with the the output of an llm and can also call different tools if we want to so an agent is kind of like the intermediary step um or the intermediary component that glues all of our components together if we have like user um or our web app front end on one end our GPT model on another and like extern external tools uh Etc the main component in in the middle that glues everything together is our agent now to really understand um some of the capabilities of the agents uh we first want to touch upon the concept of Merkle prompts so Merkle prompts are modular reasoning knowledge and language prompts and they come in different formats but a format that's very popular is basically um is basically looping through a certain set of steps so we ask a question in the very first Loop this is just going to be the user question and we tell the model um explicitly to think about what you should do as a next step like explicitly adding this like ordering it to think about what to do actually increases the reasoning capabilities of the model next it will perform an action and the definition of an action here is basically basically picking a tool from a list of tools that we give it access to so we can Define let's say a certain toolkit um let's say one tool is um a python code executor another tool might be um an SQL database another tool might be our calendar to look up availabilities another tool might be a calculator Etc so it will pick a certain tool from our from our tool list um uh with a certain input so that's the action input so for example when we are using a calculator that action input will be um will be the actual equation when we're doing an SQL lookup that action input will be an SQL query Etc then the observation is basically just the result from step three and four and then with that observation we're going to think is this the final answer that we uh expect from our question if no then we're just going to Loop through Loop through and continue this until we get that final answer so what are tools and agents I've already kind of touched upon this um so I will leave this to you um next up let's start by actually creating one of these agents so before we actually want to create an agent that's going to assist us in our whole financial um sentiment analysis we um are going to run a very little Quick Test um for this test we're just going to ask the GPT model to perform a or to to um solve a mathematical equation large language models are typically not that um good at performing mathematics like they are thought how to uh generate language and not necessarily equations like okay they might know some equations but that's because they have uh learned those from their from their training data set but if we're going to replace um certain numbers by very random numbers then it will get lost because it does not know the actual ual reasoning that it has to do it does not know it might know that for example 2 + 2 is 4 but it does not know that x + y equals the sum of X and Y so to say so to solve this we're just going to create um an agent and we're going to give it access to a python tool this will allow it to instantly execute the python code that the GPT model um generates now um we're going to to create this agent and we will need three parameters so the llm the tool and um I've also set for Bose to True here so that we can actually see what's going on in this intermediary steps now we're going to run that agent on a mathematical equation now I have been looking for a tool that requires the least amount of setup from your side um so that's why I've not chosen to do like SQL databases or something like this because then again you would need to go somewhere set up more API Keys Etc um so we're just going with a tool that uh can be used very very fast just keep in mind that if you want to use other tools it pretty much works in the same way so let's get started first we're going to do the necessary Imports again and that is from Lang chain. agent. agent toolkits we want to import our create python agent um function oh it just has to be agents sorry then from line chain do tools not python. tool we import python reppel tool cool so now we can start creating our agent and we're going to assign this to a variable Called Agent executor agent executor equals um create p an agent and we need three parameters here as stated above so we need llm we need tool and we set verbos to true just so we can see all the output so for the LM um we could be reusing some of the previous openi instances that we've created but we want to change some um we want to change one of the parameters a little bit so let's just create a new one here first of all we want to set temperature to zero because we want it to be as precise as possible and next a good practice when when using these agents is also specifically configuring our Max tokens because we want to sorry we want to um limit the size of our output since these tools um or agents can can perform certain loops and we don't want to run the risk of having like a massive output that could Skyrocket our our prediction costs so just safety measure here when we're working with agents we're going to configure fure some Max tokens and then finally as a tool we're just going to create an instance of this python reppel tool that we have uh imported so we're going to ask the agent to run a certain mathematical equation which is basically that agent. ex agent executor uh. run and we want to have this string here so we're going to ask it what is a square root of 250 and round it down to four decimals we're not using a calculator tool we're using the python tool so we expect it to uh generate some python code and instantly execute it so I was expect it to um use the square root function and then the round function on that result um it will probably chain it but let's see what happens so the intermediary steps should be showing up um in green normally and here we see like the structure of these Merkel proms so this is basically our thought like we asked it like think about what you need to do it says I need to calculate the square root of 250 okay sure uh um and it picks an action now for now we have only defined one tool so there's not much to choose from but it indeed chose for the python reppel tool and this input that will basically input um the python code that the large language model generated now the observation is basically the output of that whole task so that is equal to this um and then we're going to go into our loop again of our Merkel prompt until that thought says that we have the final answer now in this case we only needed one step because it now knows like okay I have the final answer so it's going to return the final answer and finish that whole chain and here we have it the answer okay now let's try to use that agent to help us with our financial news analysis for our financial news analysis um we're going to ask our agent to um do a bit of a elaborate task that could um that requires a bit of code as well so first of all we want to have the company name from every headline and we also want to have that Financial sentiment so we want to know is that headline positive neutral or negative then we don't just want to have these two lists but let's say as a end result we instantly want to have a CSV saved locally with all of that information in there so we're going to ask it load this data into a pandas data frame and I'm quickly describing here what the schema of the pandas where frame will look like so I want to have three columns um first one is the name of the company the second one is that Financial sentiment and then lastly I want to have like the raw data the headline itself in the in the third column then I want to just save this data frame to a CSV file and I pass along a um a file name that I wanted to use and just to be safe um because I know how like the panas to CSV method Works um I want to have that overwrite is true argument set to true so I just say um if that file already exists make sure it uh it should be overwritten then I'm just going to pass along the headlines itself now normally we would be doing this with a uh prompt template as well if you want to uh to keep this modular Etc but we've learned how that works and now we just um want to go fast through the other stuff um otherwise we just keep creating more and more and more of these prompt templates every time but just so you know in in scalable Solutions in large applications this is typically what you would set a prom template uh up for so what we're going to do is we're just going to run that agent so that in terms of code very simple we're just going to do the agent executor do run again going to copy this and basically paste that here and let's see what happens when we run this so basically um gets all of our headlines it creates two lists company names and sentiment uh it extracts the sentiment and um the company names and adds that to a data frame and writes the data frame um to a CSV file now from what I see here it actually does not uh include that parameter like override equals true um but yeah okay now I've added this here like do you see anything that could be improved from this output I'm going to give a quick hint here already um it is basically going to perform everything here using that python tool so for example when we are uh creating our sentiments it is going to create very simple rules uh to do that sentiment analysis same goes for our company names it's going to create a very simple rule for those company names um where actually we're not making use of the large language model anymore to do this so that's kind of a downside but there is a solution for this and we're coming back to this later in The Notebook but uh the cool thing is we have this data frame we've written it to a CSV so if I go and look at my file system here I now see that this CSV popped up um in in our file system so the next step we want to do now we that we have this CSV is to basically um open up the data frame and check uh what the results look like so this is going to be very straightforward um we're just going to open the data frame so from panas no we import panas under its usual Alias which is PD so we do import panas as PD then our data frame is basically uh DF so we do panas read CSV and we want to read um Financial anal analysis. CSV into a data frame and we're just going to print that data frame so one of the things um that you might notice here is first of all this company name is weird and these are as well the reason for that is as I said before um that python tool generated very uh simple um rules to perform like this company name extraction and to perform that sentiment analysis so we actually don't want to use the python tool for that one but we do want to use a python tool for stuff that we want to do with code such as creating a data frame writing it to a CSV file and and many other cumbersome tasks that we can add to this whole process now from um from the sentiment analysis point of view as well I think this one should not be negative so we see here um these uh quarterly revenues actually increased and it is um defined as negative now the reason for that as well is because if we look up here it just um um it said that if yeah basically just very simple rule sets if loss is somewhere within that headline it should be uh considered as negative now I'm not even sure oh yeah so up from A Loss in 2008 so this should be positive but it's considered negative because of these simple rule sets so we want to look at at how to improve this now before we're going to improve on what do we want to have uh performed with um with python and what which tasks do we want to have performed by the large language model we're going to touch upon One More Concept first and that is another way of improving our um our performance of these promts and that is by doing F shot learning so the next step is to add F shot learning to our prompts F shot learning is basically um just uh adding a couple of examples to our prompts so for fuchal learning um yeah yeah we have few shot learning and that goes together with stuff like uh single shot learning zero shot learning where zero shot learning is basically we don't uh add any examples single shot is we add a single example few shot is we add a couple of examples now those examples could be um something like explicitly saying this this and this is considered POS positive in my opinion this this and this is considered negative um it could also be just pasting in a couple of headlines with a label in there so specific observations with a grand truth that we add to uh to our proms that will actually increase the model because it will have an increased understanding of what we mean um with our with our PRS so for example when we're talking about um sentiment analysis there could be that ambiguity of do I mean like emotional sentiment analysis or financial sentiment analysis like I could say something like I hated that company access uh uh profits skyrocketed now from a emotional perspective that is a negative analysis I say I hate this um from a financial analysis that is considered a positive um sentiment like uh yeah their profit their profits skyrocketed so that's a very good thing from an investment point of view um now that kind of ambiguity we can solve that with fuse shot learning so we're explicitly saying um we're explicitly giving a couple of examples to show the model exactly what we mean but let's let's um quickly create two templates um one without fuse shot learning one with fuse shot learning and then we can compare the results uh to each other so first um we're going to create a new prom template without output parsing this is nothing much new um this is what we have done before um but this time instead of for extracting the company names we're going to extract that sentiment so I'm going to create a new um prompt template called sentiment template and this is a prompt template um which again takes these three variables because I want to add that output parser again to have a instant list of sentiments that correspond to all of our different headlines so first of all we have template and our template itself is listed here so that is one that we've seen before next um we have our input variables our input variables we have this headlines input variable so we have a list with just headlines in here and then lastly we have our partial variables which again takes in the dictionary of format instructions which equals to the format instructions that we had previously now we're going to format that template so we're basically just going to input something to our in that input variable so we do um format sentiment template sentiment template equals sentiment template. format and for those headlines parameters just as we've done before we're just going to pass in all of our headlines as well then we're going to run that model much in the same way as we've done before so we're just going to do output equals model and we're going to fill in our formated uh sentiment template and then we can output that um we can pars it output just as we did before as well and that output will be our final python list so that will be our sentiments list as we've seen here and this is just the same stuff of as we've done before so let's do output par. pars on the output of our model and let's take a look at the sentiments so now we see a slightly different um approach um we see positive here instead of that negative from the rule base approach uh um now it is um it it might be like fully correct so I'm not sure if we will see an increase in quality when we do this uh or when we add F shot learning in a second but just so you know um just in general this is a better approach and this will yield better results on larger data sets because we only have six instances now um our sample is not that big now to to visualize to make visualization a little bit easier we're just going to create a quick function that um that combines this label with their headlines and here I also wanted to teach um maybe if you're a python beginner um teach some extra stuff here such as a aert statement so what we want to do is we want to create a quick function which basically takes in the headlines um list of strings and the sentiment list of strings as input and we're basically going to display them in any way possible like you can choose how you do it it could be as simple as just having an F string that that deals with both but first what we want to do is we want to check if these lists have the same length and this is for example where an assert statement can come in an assert statement will check for a certain condition and if that condition is not true then it will throw an error so it can make sure that if our code runs it runs as we expected to so for example we're going going to assert that our headlines list and our sentiments list are of the same length because let's say if um if something unexpected might have happened in or with the um with the model output somewhere along the way we want to know so let's create a quick function called visualize sentiments and this will take in two inputs um so we need some headlines and we need some sentiments first of all I'm going to assert that the headlines list and the sentiments list are of equal length so I'm am just going to check if the length of headlines is equal to the length of sentiments next I am going to visualize them in anyway now I would um I would just because of the Simplicity I would just uh advise to use like python string like a python F string for example and a format that I prefer to use is let's say we put in our sentiments here so here we're going to have our specific sentiment and we're just going to have our headline here after but we want to do this for every um sentiment and for every headline so we're going to Loop through our list um but I want to have the index so I could do for I in range of the headline what is also a possibility is that I do for I an underscore because I don't use this one in enumerate and then I just choose one of the lists at random because I know that they're in the same length and this enumerate will basically make sure that we get two um variables here this will be the index this will be the actual value of that uh list now I am not using the actual value um so I am using an underscore here to uh show that I'm not using this this variable so what I want to do is um I want to say something like positive colon and then the headlines now maybe to make it a little bit more clear let's put um that positive or negative or something like this in in uppercase in all caps so we can do with Dot Upper and then we're just going to yeah call that function so let's do visualize sentiments and we pass along our headlines list and our sentiments list let's see so this is a bit easier to uh to evaluate our output so we see positive and this okay the profits row that looks correct uh negative for this one and we see the loss is widened so that looks also correct Etc okay now we want to add few shot learning to our prompts so as I as I kind of explained before like uh emotional sentiment and financial sentiment might not always mean the same thing so that is one example of why we should use Fus shot learning but just in general for for most tasks adding fuse shot learning in increases the actual performance the only downside to it is that it requires a bit more work from our end um on basically building that prompt so for our example for fuch shot learning I just added a couple of examples like what do I consider positive for examples when our profits or Revenue increased um when we're exceeding expectations when a contract is awarded Etc when losses are mounting up blah blah blah that's considered negative and if nothing is positive or negative then yeah we have a neutral sentiment so this is our Fus shot learning and we just want to have this added to our prom so let's include this in our code as these sentiment examples so we start by storing the few Shi examples in a variable so we do sentiment examples equals our examples here then we're going to instantiate a new prom template and we're going to call it let's see we're just going to overwrite the old one so we're going to do sentiment template is prompt template with the same with the same three um with the same three input variables so we have template um and for the template we want to include the the few shot examples um as an input so let's see here we basically start by getting our same um the same normal template here and let's see what what would be the best location to add our um our examples here so first I ask you to get the financial sentiment of the following headlines that makes sense to start off with that it makes sense that we add it somewhere like here so first we tell it get the financial sentiment then we we tell the model what do we consider to be correct Financial sentiment and then we're going to talk about the output so I'm just going to add a v input variable here called f sh examples next up we have our input variables so previously we just had the headlines here right now we have two input examples uh input variables so we have both headlines and we also have these few shot examples now for a real life case you typically wouldn't add few shot examples as an input variable most of the time um because you want to you want them to be in there uh for every prompt as long as you're doing the same use case it's only when you're using the same template for different use cases um which is more rare less occasional um then um then you might want to add different fuse shot examples based on what you're going to use that template actually for then lastly just like we did before we're going to add like partial example uh partial variables which takes in the format instructions and we're going to use the same format instructions as before this will also be pretty much the same so our formatted sentiment template equals our sentiment template. format and now we need to fill in two um inputs here so we need both our headlines uh which equals our headlines list and our input is called our input variable is called f shot examples but the input here is going to be the examples for the Cent sentiment analysis so here we pass along sentiment examples and now it's just like the same few steps as before so we're going to run the model so again using this output variable I'm going to run the model on that formatted sentiment template and parsing that output again and overwriting basically our old old uh sentiments list so I want to do output pars her pars output okay good to have a very quick overview of of what the the output now looks like we we have just created this function um to quickly visualize so let's start using it again uh so we do visualize sentiments um I believe headlines was the first one yeah and sentiments was the second one sentiments so let's see now in this case we do get the same results because the results were already looking pretty good but in general just expect on larger use cases on larger data sets expect to be uh for more answers to be correct once we start using the this fuch should learning at least if we're doing it the right way all right now that we have covered fusure learning we can address the problem that we had before the problem that we had before is the fact that our python rippel tool was basically executing tasks that we didn't want it to do so we did not not want uh to extract company names or to uh analyze sentiment using simple rule sets in in Python that's what we're using large language models for but the simpler tasks which are still I would say cumbersome as a as a developer like just setting up another data frame writing uh locally this is an simple example but we could replace with it much more complex tasks that is what we want to use code for so how do we um how do we deal with this basically an easy answer is multiple large language model calls so we make multiple requests and we've already done a few of them because we've done um calls to large language model without the usage of any tools to get that company names list and to get that sentiment list now let's just ask it to use those lists and uh perform the task that actually requires the python tool so for this we change your prompt a little bit now I am um I see a small mistake here create a data frame with two columns should be three columns let's see how it how it reacts to this um so again company name sentiment and headline um we fill that data frame with the following lists so we have the lists of all the data we're just asking it to generate the pandas code um very simple for now and just writing it to a local CSV this time we use with uh parsing as a file name so codewise again super simple we're just running the agent executor this is more about how to write prompts than writing code to be honest but just let's run this now I am curious what's going to happen with this so I think it will generate a data frame with three um columns yeah so it is smart enough to know that I pass along three column names so I actually want to have three columns um so yeah look at that I have a data frame with three columns and it is being written locally so if I look at my file system here I see that I have a new new file here financial analysis with parsing now again let's take a look at this data frame so we're just going to do a very quick look with p uh pan read CSV um and I want to read in this file and let's just take a look here so this is looking a lot better than what we did in task five where the sentiment was extracted uh using rule based approaches and the same goes for the company name now we get like the correct company names we get the correct sentiments we get all the headlines with it so this is exactly what we wanted good now this basically rounds it up what's uh for our financial analysis as a last task let's look at using the moderation API for open AI now for that moderation API um I have been on a subreddit called Wall Street bets um to kind of stay within that Financial topic but um I had to find some examples that could trigger the moderation API in some ways now there is a uh content warning here um the comments are stored in Reddit commands. Tex um and printing these comments is completely optional by default um and especially like in within this recording we're not going to actually do that but if you really want you could it's just um like as we say here these comments are chosen to be offensive um so if you do not wish to actually read these comments do not print them out but we're just going to take a look at the output good so first of all we're going to read in the text file the Reddit commments text file um much in the same way as we did with the financial headlines so again we're doing with open um Reddit commands. txt um as read format and we do this as data uh um commments equals data. read lines now optionally we can print these comments but we're not going to do this at this point we're going to pick a random comment so let's just do comment and we'll start with the first one um so we can mess around with them a little bit we can do we can try with index one one index 2 4 we have five commments in total so you can try any number between zero and four um and just check if you get like um let's say if you get flagging for different categories but basically what we want to do is we want to send this comment which is just a string we want to send it to the moderation API and see what the output gives us now from a generative AI perspect perspective or from a like more operational perspective how we're going to do this is basically when we're sending prompts to the model that prompt would be sent to this moderation API so that prompt is in in in the end just like a long string we're going to send that prompt to the moderation API check if everything is okay and then we're going to send it to the actual model um so let's see we want to Output we're going to assign the output to moderation output so that's what we're going to start with the moderation output is going to be we're just going to use open ey moderation. create we already imported the Open Eye package at the start of our notebook so this should be able to work straight away like this and then our input is simply going to be our Command and we just want to print out that moderation output so what we see here is um the output uh the comment itself is being flagged more specifically it's being flagged for harassment now here we see the the different Cate scores so if you want to have like different confidence levels Etc it's also in there and finally we have a single value flagged equals true and that is just to make it a bit easier if any of these categories turn to through um this flagged um key value pair the value here will be true as well so in an application we're just going to check like if flagged is true then we might check like okay what category is actually it's being flagged for or if we just want to keep it simple we're only going to look at this flagged like anything that's flagged we're just going to to block it we're going to filter it out of our stream and not send it over to the model right so with that um I I have to congratulate you on completing this module I hope you learned a lot um on like the basic concepts of Lang chain and also um like prompt engineering techniques um we've seen some um some prompt templates we've seen some prompt engineering tricks like the the F shot learning um we very briefly touched upon llm chains but I didn't want to go too deep into this because the topic is not very complicated output parsing and agents and tools are some very very powerful options in Lang chain um could be very valuable to go to go deeper into this um and to to basically yeah look up more and and um maybe build some projects on this and then lastly leveraging the moderation API to access a filter Now using that moderation API is super simple and gladly so you just need to know like when do you need to use it so basically for AI applications you mostly want to to put that as a filter for anything coming in now with that being said I wish you the best of luck in the following modules and uh good luck in all the other courseswelcome to prompt engineering with GPT and Lang chain Lang chain is a very popular open- Source framework that helps us add extra functionalities um when integrating generative AI capabilities with uh applications or or data Platforms in this module we will get a gentle introduction to the framework and some prompt engineering techniques um and understand some of the basic concepts of of Lang chain in later modules we're going to look at some of the more advanced concepts such as retrieval augmented generation so in this in this module we're going to start off with calling the open IM models through the framework we're going to set up some prompt templates to write reusable and dynamic prompts these can then be versioned along with the rest of our code base um we're going to work with llm chains which is basically the combination of uh large language models and these uh forementioned prompt templates then we're going to look at the output parser in the Lang chain framework something very valuable um that can instantly generate certain objects like python lists dictionaries Json objects Etc that we can then um instantly use further in our code next we're going to work work with Lang chain agents and tools these will allow us to do many cool things like instantly executing code that's being generated by the model or um let's say do SQL lookups Etc and then lastly we're going to look at the openi moderation API this is especially useful when you're trying to let's say build your own API uh no build your own AI application you want to have uh something like this as a filter in between the user and your actual model endpoint now for this project we're going to use two small text samples um I wouldn't necessarily call them data sets it's basically just five to six line samples they're being kept very short so that every time we print our results we can just get a very simple overview of everything just keep in mind that all the techniques that are thought here um and all the code that is used here is very scalable we can just use it on any um size of of data set now in the file system here let's take a very quick look at these headlines so we have six Financial headlines on finished companies and whether their profits Rose or or uh decreased or other information such as uh company X has been awarded a contract for a certain project Etc now the goal is to do some Financial sentiment analysis here so for example using these GPT models we automatically want to generate um investment advice for example uh based on the current news so let's go back to our notebook oh yeah maybe a good thing to mention if you ever get stuck there's also this notebook solution here um so you will find the solution to all the code problems in this uh solution notebook before we get started one of the prerequisites is that we need an openi API key now for this openi API key this has been covered in the previous module um how to how to get one and how to add it to the environment variables if you don't know how to do this or you have not followed um the previous module you can always check out the getting started notebook in the file system with that being said let's start with our setup the first thing that we want to do is um do some installs so we're just going to install uh some packages uh you might get some warnings or sometimes even some some errors with these uh with these versions but just for this project um these are not breaking changes so don't be afraid um you can just continue the project with any um with the versions mentioned here now with those being done let's start with our first instructions the first thing that we have to do is some basic Imports um to basically fetch our environment variable and um um set it within the Open Eye package so let's first import the OS package and that's being done by just simple um import OS statement and we're going to do the same thing for the open ey package just import open ey then we can um set our API key so open ey. API key and we want it to be our environment variable that we can call with os. enin so I want to have the open AI API key just going to run this cell to set our key um and we should be good to go next we are going to start just doing some basic calls to the open ey models um from Lang chain and that's being done using the openi and the chat open classes the openi classes are used for completion models whereas the chat openi are used for the conversational models so maybe a quick um overview here comption completion models refers to typically the the older generation of models if we don't count the new instruct models um but basically they just give like a single completion to a text input whereas the chat models are made for conversational use so the models used in chat GPT which are GPT 3.5 turbo or GPT 4 are the chat uh conversational models so they will be loaded using the chat openi uh package uh no chat open ey Clause sorry but first let's let's start start importing these classes so these are also just very straightforward Imports so from Lang chain. llms we want to import openi and from Lang chain do chat models we want to import chat openi okay run the cell we have done all of our Imports now let's get started on task one we want to import our financial news headlines data so we have taken a look at the uh sample text that we have the financial headlines we just want to read in um this this text file and basically store it within a a simple python list let's say so what we're going to do is um we're going to open the text file in a read format we're going to use the read lines method and basically assign it to a new variable called headlines so let's do with open um then the name of our text file which is basically Financial headlines oh that's not do that which is basically Financial headlines. text and we want to open this in a read format let's open this as let's say data and then we can read this in by um doing headlines equals data. read lines to get a quick overview let's also print out this python list so here we basically have our um headlines loaded it into a list just a list of strings with every element being a single headline now we might not is that we have this uh new line uh character at the end of every line um let's just get rid of it um it doesn't really influence the large language model that much but just for like sanity let's let's get rid of these uh of these characters to get rid of these characters we can um simply use the do strip method um for the for string objects within python so to make it a little bit easier I'm going to use a list comprehension here so for headlines let's do x 4 X in headlines so I'm just going to Loop through headlines and instantly generate a new list and reassign this to headlines now the operation that I want to perform here is basically applying that dot strip method to every element of the headlines do strip basically removes any character at the end of the string um and I want to remove the slash n now let's print out headlines again and look at our result so now we basically have a bit of a cleaned up version of our headlines without these uh these new line characters everywhere for the second task we're going to learn how to set up prompt templates now promp Chrome templates as I've told you before um they're made to be like uh reusable and dynamic uh prompts that we can version within our database uh within our code base so basically we can save these with the rest of our code we can add it to our repositories and um basically reuse it in any way like this we can add input variables to these prom templates to make them Dynamic and within line chain we have built-in verification tools um that kind of check whether that input is always uh given when it's being called from anywhere else which makes our code more robust now the the format of these prom templates they are kind of structured like f strings in Python basically meaning we have a a typical string um and if we want to add any uh Dynamic variable within the string we're using the curly brackets here so for example if I want to have a prom template to give me suggestions for a trip to any kind of City I could set up a quick um template that says can you give me some suggestions for my trip to and then a city variable this allows for dynamic templating basically or dynamic prompting now we have two different types of prompt templates um and that is that has to do with like the two different types of models that we've spoken about earlier basically we have the prompt template and the chat prompt template one is for the completion models one is for the chat models the reason for that is because the completion models they basically just generate once they uh it's like single input single output while chat completion they can store um prompt histories Etc and these templates will also be able to store certain prompt histories and a system message as well so the system message um might have seen this in the in the previous module but this is some sort of meta prompt that we can use to do all sorts of things such as um putting the model into a certain role uh asking it to um to to use a certain tone of voice or specify um specific output formats Etc Now to create a prompt template we are going to use the prompt template class in Lang chain so for a for a very simple one we're just going to use the from template method um in in uh in the framework um and then we basically have to pass along the template itself uh the the text itself The Prompt itself so for that we're just going to use this prompt and we can already see see that we have one input variable here so we have headline which is basically a dynamic variable within our prompt so we can use the same prompt template against our entire data set of headlines then we're going to format this prompt which is basically um passing along a certain headline for that input variable and we're going to see what the end result looks like so first let's start by uh importing The Prompt template class and that is done by from Lang chain. prompts we want to import The Prompt template class next we want to um basically set up a new prompt template and we're just going to call it we're going to assign it to a variable called prompt template in snake case so we're going to do this by prompt template do front template and in here basically we just want to use our um our prompt that we have here so that's that's that um now we will see more advanced ways of of setting up these templates with some verification methods um further down the line but for now let's just create a very simple example of a prom template now let's say we want to format this one with the first um headline of um our data set what we're going to do is we are first of all we're going to add uh we're going to assign our result to a new variable called format at prompt and we're going to call this format method on our newly created prompt template so I'm just going to do prompt template. format and then it will expect um this input here so this headline input so let's say headline equals and I'm going to take the first element from our headlines data set so remember the headlines was our uh python list with all of our headlines stored in there as a string I'm just going to take the first element there by putting in index zero and let's take a look at our formatted prompt and what this looks like so here we see um basically are our instruction so analyze the following Financial headline for sentiment and then uh combined with the headline that we fetched from our data set now that we have set up a prompt template um let's continue towards a chat prompt template so here we can have some um some prompt history if we want but mainly we get access to this system message that's available within the gp4 model and within the GPT 3.5 turbo model so the models used in Chad GPT first what we want to do here is import the other class so chat prompt template um from the same package and we're going to use this system message now the system message if we're going to look at um how it's written uh so you might remember I told you like system message can be used to put the um to put the the model into a certain role and that's what I'm doing here with this first uh sentence so I'm telling the model what are you doing right now or or what kind of role do you do you perform so I'm telling you you are performing sentiment analysis on news headlines regarding financial analysis this sentiment is to be used for blah blah blah and that output the format of the output has to be consistent so the thing is I want the GPT model to only output either the words positive negative or neutral and nothing else I don't want it to to um to respond to me with like sentences whether telling me whether it's like slightly positive or kind of negative I just want to have the single word positive negative neutral that I can then later on um use for analytics so we're going to use this system message um and we will um yeah we will start by instantiating a new chat prompt template just as we did with the other template and instead of from template we're going to do from messages we're going to add a system message and a human message so a human message is basically like the user prompt um this will be put into a tuple um together with their respective contents now we're going to format that template in pretty much the same way as we did with um with the previous prompt template and just check out the results so first of all let's start by importing our prompt template so from Lang chain. prompts we want to import our chat prompt template next we we want to um we want to Define our system message so we will just copy this string up here and assign it to the system message variable like this next we are going to instantiate our chat prompt template and the name for this variable is is chat template so let's call it chat template we're going to set up a new chat prompt template from messages now this will take in a list of different messages so let's start up with a list and we are going to do a small list of a single system message message and then a human message but if you want to in these chat prompt templates you can also add like a bit of conversation that already happened before you are using this certain prompt so you could add like a human message um an AI generated message and then a human message as well uh to simulate like the start of a conversation already now for us we're basically using this just to get access to a system message so what we're going to do is we're going to pass a tuple and then first of all we're going to say this is the system message so here we can do system or human or stuff like this so let's start with system message and then the second element of our Tuple is the content itself so we've just stored the system message in the variable called system message so I'm just going to call that one here that's basically this long string here that we just find second going to make a second Tuple and that will be a our human message um and for our human message we basically just want to have like this same um the same one that we used here within our prom template so this will become our human message now again you can see that we have um one single input here so again we have this headline input that we want to uh format um our prom template on so let's quickly go back to which one yeah so we want to have it assigned to formatted chat template so we're going to Define formatted chat template equals our chat template. format and again just for that headline uh variable we want to pass along our first headline now let's see what this looks like once we start to print it so this is a bit more elaborate than the one before because here we first have this entire system message that we defined and then we have our human message this is basically what the whole chat GPT um application looks like as well behind the scenes um so behind the scenes you also have this system message integrated in there and your input prompts in ch GPT will basically be defined as this human message so now we have set up a prom template and a chat prompt template next up we're going to set up some basic llm chains now we're going we're not going to go too deep into this but the concept is quite simple an llm chain is basically the combination of a model and the templates that we just have set up and then we can we're able to chain multi mple of these objects um to each other by basically adding the output of a certain llm chain as the input for a next llm chain now again we have this for completion models and chat completion models um so we have two different chains um but first of all let's start with with um using it for our completion models so here we're also going to do the necessary Imports so that is from Lang chain. chains we want to import llm chain now first of all um yeah we're going to reuse the is open ey class that we um that we imported all the way at the start and basically calling or instantiating one of these open AI uh classes will give us access to a model to a GPT model now the openi instead of the chat openi as you might remember um by default that will or in the current version at least for Lang chain that will steer us towards the text Vinci model um and we can instantiate multiple of these classes to for example change parameters so I could instantiate one with a temperature set to zero or a higher temperature and um maybe to touch a little bit on that one uh a very simple and brief explanation of the temperature is basically the randomness that we put into the model like the higher the temperature the more random a model becomes which is is favorable for Creative use cases while if we have like um use cases that require a lot of precision like we want to have more factually correct answers um or for example when generating code we generally prefer prefer very low or even zero uh temperature to get the best result possible okay so let's start by um instantiating one of these llm chain um classes so we're going to assign it to a variable called completion chain so let's create that one first completion chain equals llm chain and here we want to have two different um parameters so we have llm and we have promt now for llm let's just um quickly instantiate a new class uh instantiate a new um model object here just a simple one um just using the default um parameters here and for the prompt let's use our prompt template that we've defined earlier so that is our prompt template now remember this is not our formatt prompt so this is where we have this um flexibility like every time we run this chain we can run it with different inputs so again let's try to run it on our first headline so we do our completion chain. run and as input for that headline oh for that headline input that we had in our chat prom template we're just going to pass along our first headline now we're going to automatically format our promt template and send over that prompt template to a completion model so let's see what we get as a result we get D n-n positive so it um it class CL ified the first headline as positive um but it still has these characters in front of it which we don't necessarily want but let's get we will get rid of them like um later in this this module next let's do the same thing but using our chat prom template so for this we're going to be creating an llm chain but just using the chat openi model um instead of the normal OPI model the concept is pretty much the same so let's let's say chat chain um also equals llm chain and again we have these same two parameters so we have llm and we have our prompt now instead of the opening IM model let's create chat open ey model by default this will direct us to the GPD 3.5 turbo model um but we could also fill in like for example gp4 or anything like this and for a prom template we have our chat template that we've created earlier now again keep in mind that this is not the formatted template so we have that flexibility to format it while we or when we run the chain so for that I'm just going to do chat chain. run and with this template we had two um or we have two um input variables because we also now have that system message that we can uh tune so so for the headline I'm just going to fetch our first headlines again oh uh yeah like this and then our system message because we can change this with every call to the model um we have stored this in a variable called system message as well if we run this we get positive so both the completion model and the chat completion model classify that first headline as positive which is a good thing because if I remember correctly there was a headline that said like um that company's profits went up from X to Y so for task four we're going to make use of the output parser in Lang chain the output parser as said before um is very useful when we want to have like uh python objects straight away from our large language model output now it's good to know that the large language model itself it will always just output um strings it will always just output texts but the output parser um has some built-in tricks to automatically create like python lists date times Jason dictionaries Etc now how it does that is first of all it will parse the text output and basically create that uh specific python object from it but on the other hand it also adds format instructions to our prompt behind the scenes so it will actually add some extra text to our prompt to make sure that the text output of our large language model is in the necessary format that we asked for now in this specific case we're going to um we're going to extract the company name from every headline later we can then um basically set up a data frame that combines the company name with their respective sentiment um to basically have a very basic version of financial news analysis so what we want to do in terms of output parsing is when we pass along all of our headlines we want to get a python list which basically contains all of the uh company names now to do this first we're going to make the necessary Imports um so we're going to be using the comma separated list output parser um and we're we're going to fetch the format instructions from it this is basically what we're adding behind the scenes to our prompt so we'll have to pass this along within our new prompt template as well so with that being said we're going to set up a new prompt template we're not going to use this from template method because um we want some extra flexibility here um and this will allow us to um basically call the init function of this prompt template which has um or where we're going to use three parameters first template this is basically what we've used before so this is where we actually give our our text prompt as a string um and here we're going to ask it to list all the company names from the following headlines now one of the problems I've noticed while developing this notebook is there is one headline I think with two company names in there um so we just want to limit it to one um company name per headline for now just to have like the easy analysis so we can map them one to one to the sentiments that we will get later on and you can see that we will um we will insert all the headlines this time not just a single headline so we're just asking it to list all the headlines um to list all the company names from all these headlines and then afterwards it will automatically add like these format instructions like this we're going to specifically Define an extra parameter that's input variables and we do this to verify that these input parameters are um are are actually being inputed like if we if we try to call this prompt template without inputting certain headlines then we will get an error so that is just a best practice to get robust code um especially when using this in like larger applications as then as partial variables this is basically where we um where we pass along the format instructions that we get from our out put parser um to have it parsed in a list and then we're going to look at the result oh no we're not going to look at result we're just going to format the prom template uh in this case and then in the next one we're going to look at the result because we still need to run it through the model like with this with this code block we're just going to end up with a formatted prompt template that includes output parsing so first let's start with our necessary Imports so we need this comma separated list output parser from Lang chain. output parsers so from Lang chain output parsers import comma separated list output parser now we're going to instantiate a new one and we're going to assign it to a variable called output parser so just use output parser equals comma separate list output parser and we don't need to pass along any parameters here the default ones are more than fine next we're going to fetch those format instructions from that specific output parser and that is that is very easy that's just calling this method the get format instructions um and let's just store these under format instructions so we get get them from that comma separated list output parser by using output parser doget format instructions now we are going to set up our new prom template to get um to get the company names from all of our headlines so the name for the variable that we're going to be using here um I see that it's not listed here but let's just use company name template um so this will become a new prom template and we're just using the init method so we're just doing it like this and here we will have three different parameters so we have template we have have um input variables and we have partial variables for the template we have we have it here so we're just going to copy this one and add this string in here for our input variables um we basically pass along a list with our input variables um as a string so keep in mind that these format instructions they do not count as an input variable they are partial variables so the only input variable that we have here is headlines and then partial variables that's basically these format instructions and here um the code is expecting a dictionary so we want to have a key value pair that says um format instructions and that equals the format instructions that we've received here from our output parser so to get a quick look at our template let's just format that template um with our headlines so so let me see here Le is basically we're going to call it format company template to keep the same structure as before um and we want to basically do company name template. format much in the same way as we've done before and now as an input variable we have these headlines now what we want to assign here is basically that string that list of strings that we had before that we read from the text file and that is also the same name so headlines the headlines variable now let's print this and um yeah maybe let's just print this as well to have a look so basically what we see here is um our instruction plus our list with all of our headlines but within a um within a list that is being converted to a string and then what we see at the end here is basically those format instructions that we have added those for inst instructions are then um basically being uh or they put it in the right format to be parsed by um by the Lang chain framework so next um what we're going to do is we're basically just going to run um this template through a model but now instead of using an llm chain let's try a little bit of a different approach so you have like multiple so you know uh multiple approaches um to actually make use of models and make use of templates so in this case we're just going to set up a um open ey model so the opening eye model here and we want to set its temperature to zero the reason for that being we want to have it as precise as possible um we don't need Randomness at this point we want to be as precise as possible on extracting these company names from our headlines so the way we do this is we're basically just going to create this open ey um or create an instance of this open ey object Again by default by referring to the the text of Vinci model um and we're going to pass temperature equals zero so we have a model with zero temperature now we're going to run that model um basically by just calling that model variable um and we're going to store our output in underscore output now an underscore basically um yeah it indicates that that we're just using a temporary Val variable here and we will override it a couple of times as well um because we just uh want to have like this intermediary step in between now you could chain all of these methods but for this module just to keep it a bit more easy to uh to follow I'm adding this extra step here so for model uh to run this model I'm just going to do mod and ask uh and um insert our template here so that was our formatted company name template next we want to parse the output so we will need um our output parser um and that output parser has this do parse method um with with with which we can parse that output that we got from the model we're going to assign this to a new list that we've created that is company names so this will be this will instantly be a python list because of this output parser within within Lang chain that's the whole like power of it so output parser do pars and we parse the output that we've received from the model then let's quickly print the data type of this uh company names variable just to show you that we indeed have a python list object straight away so that is done by doing type and then our variable itself which is company names and lastly I'm also just going to print the company names themselves so that we can have a look at them might see this takes slightly longer because we're adding more and more functionalities um and later that might even increase but uh here we see um our class is indeed a list so we instantly have a python list here that we can use like first there down our code um and we've extracted the company names from our headlines now with that being said let's move on now to agents and tools to uh continue with these company names next up we are going to work with agents and tools in Lang chain now agents and tools an agent is basically um let's say a bit of a code executor um that can work with the the output of an llm and can also call different tools if we want to so an agent is kind of like the intermediary step um or the intermediary component that glues all of our components together if we have like user um or our web app front end on one end our GPT model on another and like extern external tools uh Etc the main component in in the middle that glues everything together is our agent now to really understand um some of the capabilities of the agents uh we first want to touch upon the concept of Merkle prompts so Merkle prompts are modular reasoning knowledge and language prompts and they come in different formats but a format that's very popular is basically um is basically looping through a certain set of steps so we ask a question in the very first Loop this is just going to be the user question and we tell the model um explicitly to think about what you should do as a next step like explicitly adding this like ordering it to think about what to do actually increases the reasoning capabilities of the model next it will perform an action and the definition of an action here is basically basically picking a tool from a list of tools that we give it access to so we can Define let's say a certain toolkit um let's say one tool is um a python code executor another tool might be um an SQL database another tool might be our calendar to look up availabilities another tool might be a calculator Etc so it will pick a certain tool from our from our tool list um uh with a certain input so that's the action input so for example when we are using a calculator that action input will be um will be the actual equation when we're doing an SQL lookup that action input will be an SQL query Etc then the observation is basically just the result from step three and four and then with that observation we're going to think is this the final answer that we uh expect from our question if no then we're just going to Loop through Loop through and continue this until we get that final answer so what are tools and agents I've already kind of touched upon this um so I will leave this to you um next up let's start by actually creating one of these agents so before we actually want to create an agent that's going to assist us in our whole financial um sentiment analysis we um are going to run a very little Quick Test um for this test we're just going to ask the GPT model to perform a or to to um solve a mathematical equation large language models are typically not that um good at performing mathematics like they are thought how to uh generate language and not necessarily equations like okay they might know some equations but that's because they have uh learned those from their from their training data set but if we're going to replace um certain numbers by very random numbers then it will get lost because it does not know the actual ual reasoning that it has to do it does not know it might know that for example 2 + 2 is 4 but it does not know that x + y equals the sum of X and Y so to say so to solve this we're just going to create um an agent and we're going to give it access to a python tool this will allow it to instantly execute the python code that the GPT model um generates now um we're going to to create this agent and we will need three parameters so the llm the tool and um I've also set for Bose to True here so that we can actually see what's going on in this intermediary steps now we're going to run that agent on a mathematical equation now I have been looking for a tool that requires the least amount of setup from your side um so that's why I've not chosen to do like SQL databases or something like this because then again you would need to go somewhere set up more API Keys Etc um so we're just going with a tool that uh can be used very very fast just keep in mind that if you want to use other tools it pretty much works in the same way so let's get started first we're going to do the necessary Imports again and that is from Lang chain. agent. agent toolkits we want to import our create python agent um function oh it just has to be agents sorry then from line chain do tools not python. tool we import python reppel tool cool so now we can start creating our agent and we're going to assign this to a variable Called Agent executor agent executor equals um create p an agent and we need three parameters here as stated above so we need llm we need tool and we set verbos to true just so we can see all the output so for the LM um we could be reusing some of the previous openi instances that we've created but we want to change some um we want to change one of the parameters a little bit so let's just create a new one here first of all we want to set temperature to zero because we want it to be as precise as possible and next a good practice when when using these agents is also specifically configuring our Max tokens because we want to sorry we want to um limit the size of our output since these tools um or agents can can perform certain loops and we don't want to run the risk of having like a massive output that could Skyrocket our our prediction costs so just safety measure here when we're working with agents we're going to configure fure some Max tokens and then finally as a tool we're just going to create an instance of this python reppel tool that we have uh imported so we're going to ask the agent to run a certain mathematical equation which is basically that agent. ex agent executor uh. run and we want to have this string here so we're going to ask it what is a square root of 250 and round it down to four decimals we're not using a calculator tool we're using the python tool so we expect it to uh generate some python code and instantly execute it so I was expect it to um use the square root function and then the round function on that result um it will probably chain it but let's see what happens so the intermediary steps should be showing up um in green normally and here we see like the structure of these Merkel proms so this is basically our thought like we asked it like think about what you need to do it says I need to calculate the square root of 250 okay sure uh um and it picks an action now for now we have only defined one tool so there's not much to choose from but it indeed chose for the python reppel tool and this input that will basically input um the python code that the large language model generated now the observation is basically the output of that whole task so that is equal to this um and then we're going to go into our loop again of our Merkel prompt until that thought says that we have the final answer now in this case we only needed one step because it now knows like okay I have the final answer so it's going to return the final answer and finish that whole chain and here we have it the answer okay now let's try to use that agent to help us with our financial news analysis for our financial news analysis um we're going to ask our agent to um do a bit of a elaborate task that could um that requires a bit of code as well so first of all we want to have the company name from every headline and we also want to have that Financial sentiment so we want to know is that headline positive neutral or negative then we don't just want to have these two lists but let's say as a end result we instantly want to have a CSV saved locally with all of that information in there so we're going to ask it load this data into a pandas data frame and I'm quickly describing here what the schema of the pandas where frame will look like so I want to have three columns um first one is the name of the company the second one is that Financial sentiment and then lastly I want to have like the raw data the headline itself in the in the third column then I want to just save this data frame to a CSV file and I pass along a um a file name that I wanted to use and just to be safe um because I know how like the panas to CSV method Works um I want to have that overwrite is true argument set to true so I just say um if that file already exists make sure it uh it should be overwritten then I'm just going to pass along the headlines itself now normally we would be doing this with a uh prompt template as well if you want to uh to keep this modular Etc but we've learned how that works and now we just um want to go fast through the other stuff um otherwise we just keep creating more and more and more of these prompt templates every time but just so you know in in scalable Solutions in large applications this is typically what you would set a prom template uh up for so what we're going to do is we're just going to run that agent so that in terms of code very simple we're just going to do the agent executor do run again going to copy this and basically paste that here and let's see what happens when we run this so basically um gets all of our headlines it creates two lists company names and sentiment uh it extracts the sentiment and um the company names and adds that to a data frame and writes the data frame um to a CSV file now from what I see here it actually does not uh include that parameter like override equals true um but yeah okay now I've added this here like do you see anything that could be improved from this output I'm going to give a quick hint here already um it is basically going to perform everything here using that python tool so for example when we are uh creating our sentiments it is going to create very simple rules uh to do that sentiment analysis same goes for our company names it's going to create a very simple rule for those company names um where actually we're not making use of the large language model anymore to do this so that's kind of a downside but there is a solution for this and we're coming back to this later in The Notebook but uh the cool thing is we have this data frame we've written it to a CSV so if I go and look at my file system here I now see that this CSV popped up um in in our file system so the next step we want to do now we that we have this CSV is to basically um open up the data frame and check uh what the results look like so this is going to be very straightforward um we're just going to open the data frame so from panas no we import panas under its usual Alias which is PD so we do import panas as PD then our data frame is basically uh DF so we do panas read CSV and we want to read um Financial anal analysis. CSV into a data frame and we're just going to print that data frame so one of the things um that you might notice here is first of all this company name is weird and these are as well the reason for that is as I said before um that python tool generated very uh simple um rules to perform like this company name extraction and to perform that sentiment analysis so we actually don't want to use the python tool for that one but we do want to use a python tool for stuff that we want to do with code such as creating a data frame writing it to a CSV file and and many other cumbersome tasks that we can add to this whole process now from um from the sentiment analysis point of view as well I think this one should not be negative so we see here um these uh quarterly revenues actually increased and it is um defined as negative now the reason for that as well is because if we look up here it just um um it said that if yeah basically just very simple rule sets if loss is somewhere within that headline it should be uh considered as negative now I'm not even sure oh yeah so up from A Loss in 2008 so this should be positive but it's considered negative because of these simple rule sets so we want to look at at how to improve this now before we're going to improve on what do we want to have uh performed with um with python and what which tasks do we want to have performed by the large language model we're going to touch upon One More Concept first and that is another way of improving our um our performance of these promts and that is by doing F shot learning so the next step is to add F shot learning to our prompts F shot learning is basically um just uh adding a couple of examples to our prompts so for fuchal learning um yeah yeah we have few shot learning and that goes together with stuff like uh single shot learning zero shot learning where zero shot learning is basically we don't uh add any examples single shot is we add a single example few shot is we add a couple of examples now those examples could be um something like explicitly saying this this and this is considered POS positive in my opinion this this and this is considered negative um it could also be just pasting in a couple of headlines with a label in there so specific observations with a grand truth that we add to uh to our proms that will actually increase the model because it will have an increased understanding of what we mean um with our with our PRS so for example when we're talking about um sentiment analysis there could be that ambiguity of do I mean like emotional sentiment analysis or financial sentiment analysis like I could say something like I hated that company access uh uh profits skyrocketed now from a emotional perspective that is a negative analysis I say I hate this um from a financial analysis that is considered a positive um sentiment like uh yeah their profit their profits skyrocketed so that's a very good thing from an investment point of view um now that kind of ambiguity we can solve that with fuse shot learning so we're explicitly saying um we're explicitly giving a couple of examples to show the model exactly what we mean but let's let's um quickly create two templates um one without fuse shot learning one with fuse shot learning and then we can compare the results uh to each other so first um we're going to create a new prom template without output parsing this is nothing much new um this is what we have done before um but this time instead of for extracting the company names we're going to extract that sentiment so I'm going to create a new um prompt template called sentiment template and this is a prompt template um which again takes these three variables because I want to add that output parser again to have a instant list of sentiments that correspond to all of our different headlines so first of all we have template and our template itself is listed here so that is one that we've seen before next um we have our input variables our input variables we have this headlines input variable so we have a list with just headlines in here and then lastly we have our partial variables which again takes in the dictionary of format instructions which equals to the format instructions that we had previously now we're going to format that template so we're basically just going to input something to our in that input variable so we do um format sentiment template sentiment template equals sentiment template. format and for those headlines parameters just as we've done before we're just going to pass in all of our headlines as well then we're going to run that model much in the same way as we've done before so we're just going to do output equals model and we're going to fill in our formated uh sentiment template and then we can output that um we can pars it output just as we did before as well and that output will be our final python list so that will be our sentiments list as we've seen here and this is just the same stuff of as we've done before so let's do output par. pars on the output of our model and let's take a look at the sentiments so now we see a slightly different um approach um we see positive here instead of that negative from the rule base approach uh um now it is um it it might be like fully correct so I'm not sure if we will see an increase in quality when we do this uh or when we add F shot learning in a second but just so you know um just in general this is a better approach and this will yield better results on larger data sets because we only have six instances now um our sample is not that big now to to visualize to make visualization a little bit easier we're just going to create a quick function that um that combines this label with their headlines and here I also wanted to teach um maybe if you're a python beginner um teach some extra stuff here such as a aert statement so what we want to do is we want to create a quick function which basically takes in the headlines um list of strings and the sentiment list of strings as input and we're basically going to display them in any way possible like you can choose how you do it it could be as simple as just having an F string that that deals with both but first what we want to do is we want to check if these lists have the same length and this is for example where an assert statement can come in an assert statement will check for a certain condition and if that condition is not true then it will throw an error so it can make sure that if our code runs it runs as we expected to so for example we're going going to assert that our headlines list and our sentiments list are of the same length because let's say if um if something unexpected might have happened in or with the um with the model output somewhere along the way we want to know so let's create a quick function called visualize sentiments and this will take in two inputs um so we need some headlines and we need some sentiments first of all I'm going to assert that the headlines list and the sentiments list are of equal length so I'm am just going to check if the length of headlines is equal to the length of sentiments next I am going to visualize them in anyway now I would um I would just because of the Simplicity I would just uh advise to use like python string like a python F string for example and a format that I prefer to use is let's say we put in our sentiments here so here we're going to have our specific sentiment and we're just going to have our headline here after but we want to do this for every um sentiment and for every headline so we're going to Loop through our list um but I want to have the index so I could do for I in range of the headline what is also a possibility is that I do for I an underscore because I don't use this one in enumerate and then I just choose one of the lists at random because I know that they're in the same length and this enumerate will basically make sure that we get two um variables here this will be the index this will be the actual value of that uh list now I am not using the actual value um so I am using an underscore here to uh show that I'm not using this this variable so what I want to do is um I want to say something like positive colon and then the headlines now maybe to make it a little bit more clear let's put um that positive or negative or something like this in in uppercase in all caps so we can do with Dot Upper and then we're just going to yeah call that function so let's do visualize sentiments and we pass along our headlines list and our sentiments list let's see so this is a bit easier to uh to evaluate our output so we see positive and this okay the profits row that looks correct uh negative for this one and we see the loss is widened so that looks also correct Etc okay now we want to add few shot learning to our prompts so as I as I kind of explained before like uh emotional sentiment and financial sentiment might not always mean the same thing so that is one example of why we should use Fus shot learning but just in general for for most tasks adding fuse shot learning in increases the actual performance the only downside to it is that it requires a bit more work from our end um on basically building that prompt so for our example for fuch shot learning I just added a couple of examples like what do I consider positive for examples when our profits or Revenue increased um when we're exceeding expectations when a contract is awarded Etc when losses are mounting up blah blah blah that's considered negative and if nothing is positive or negative then yeah we have a neutral sentiment so this is our Fus shot learning and we just want to have this added to our prom so let's include this in our code as these sentiment examples so we start by storing the few Shi examples in a variable so we do sentiment examples equals our examples here then we're going to instantiate a new prom template and we're going to call it let's see we're just going to overwrite the old one so we're going to do sentiment template is prompt template with the same with the same three um with the same three input variables so we have template um and for the template we want to include the the few shot examples um as an input so let's see here we basically start by getting our same um the same normal template here and let's see what what would be the best location to add our um our examples here so first I ask you to get the financial sentiment of the following headlines that makes sense to start off with that it makes sense that we add it somewhere like here so first we tell it get the financial sentiment then we we tell the model what do we consider to be correct Financial sentiment and then we're going to talk about the output so I'm just going to add a v input variable here called f sh examples next up we have our input variables so previously we just had the headlines here right now we have two input examples uh input variables so we have both headlines and we also have these few shot examples now for a real life case you typically wouldn't add few shot examples as an input variable most of the time um because you want to you want them to be in there uh for every prompt as long as you're doing the same use case it's only when you're using the same template for different use cases um which is more rare less occasional um then um then you might want to add different fuse shot examples based on what you're going to use that template actually for then lastly just like we did before we're going to add like partial example uh partial variables which takes in the format instructions and we're going to use the same format instructions as before this will also be pretty much the same so our formatted sentiment template equals our sentiment template. format and now we need to fill in two um inputs here so we need both our headlines uh which equals our headlines list and our input is called our input variable is called f shot examples but the input here is going to be the examples for the Cent sentiment analysis so here we pass along sentiment examples and now it's just like the same few steps as before so we're going to run the model so again using this output variable I'm going to run the model on that formatted sentiment template and parsing that output again and overwriting basically our old old uh sentiments list so I want to do output pars her pars output okay good to have a very quick overview of of what the the output now looks like we we have just created this function um to quickly visualize so let's start using it again uh so we do visualize sentiments um I believe headlines was the first one yeah and sentiments was the second one sentiments so let's see now in this case we do get the same results because the results were already looking pretty good but in general just expect on larger use cases on larger data sets expect to be uh for more answers to be correct once we start using the this fuch should learning at least if we're doing it the right way all right now that we have covered fusure learning we can address the problem that we had before the problem that we had before is the fact that our python rippel tool was basically executing tasks that we didn't want it to do so we did not not want uh to extract company names or to uh analyze sentiment using simple rule sets in in Python that's what we're using large language models for but the simpler tasks which are still I would say cumbersome as a as a developer like just setting up another data frame writing uh locally this is an simple example but we could replace with it much more complex tasks that is what we want to use code for so how do we um how do we deal with this basically an easy answer is multiple large language model calls so we make multiple requests and we've already done a few of them because we've done um calls to large language model without the usage of any tools to get that company names list and to get that sentiment list now let's just ask it to use those lists and uh perform the task that actually requires the python tool so for this we change your prompt a little bit now I am um I see a small mistake here create a data frame with two columns should be three columns let's see how it how it reacts to this um so again company name sentiment and headline um we fill that data frame with the following lists so we have the lists of all the data we're just asking it to generate the pandas code um very simple for now and just writing it to a local CSV this time we use with uh parsing as a file name so codewise again super simple we're just running the agent executor this is more about how to write prompts than writing code to be honest but just let's run this now I am curious what's going to happen with this so I think it will generate a data frame with three um columns yeah so it is smart enough to know that I pass along three column names so I actually want to have three columns um so yeah look at that I have a data frame with three columns and it is being written locally so if I look at my file system here I see that I have a new new file here financial analysis with parsing now again let's take a look at this data frame so we're just going to do a very quick look with p uh pan read CSV um and I want to read in this file and let's just take a look here so this is looking a lot better than what we did in task five where the sentiment was extracted uh using rule based approaches and the same goes for the company name now we get like the correct company names we get the correct sentiments we get all the headlines with it so this is exactly what we wanted good now this basically rounds it up what's uh for our financial analysis as a last task let's look at using the moderation API for open AI now for that moderation API um I have been on a subreddit called Wall Street bets um to kind of stay within that Financial topic but um I had to find some examples that could trigger the moderation API in some ways now there is a uh content warning here um the comments are stored in Reddit commands. Tex um and printing these comments is completely optional by default um and especially like in within this recording we're not going to actually do that but if you really want you could it's just um like as we say here these comments are chosen to be offensive um so if you do not wish to actually read these comments do not print them out but we're just going to take a look at the output good so first of all we're going to read in the text file the Reddit commments text file um much in the same way as we did with the financial headlines so again we're doing with open um Reddit commands. txt um as read format and we do this as data uh um commments equals data. read lines now optionally we can print these comments but we're not going to do this at this point we're going to pick a random comment so let's just do comment and we'll start with the first one um so we can mess around with them a little bit we can do we can try with index one one index 2 4 we have five commments in total so you can try any number between zero and four um and just check if you get like um let's say if you get flagging for different categories but basically what we want to do is we want to send this comment which is just a string we want to send it to the moderation API and see what the output gives us now from a generative AI perspect perspective or from a like more operational perspective how we're going to do this is basically when we're sending prompts to the model that prompt would be sent to this moderation API so that prompt is in in in the end just like a long string we're going to send that prompt to the moderation API check if everything is okay and then we're going to send it to the actual model um so let's see we want to Output we're going to assign the output to moderation output so that's what we're going to start with the moderation output is going to be we're just going to use open ey moderation. create we already imported the Open Eye package at the start of our notebook so this should be able to work straight away like this and then our input is simply going to be our Command and we just want to print out that moderation output so what we see here is um the output uh the comment itself is being flagged more specifically it's being flagged for harassment now here we see the the different Cate scores so if you want to have like different confidence levels Etc it's also in there and finally we have a single value flagged equals true and that is just to make it a bit easier if any of these categories turn to through um this flagged um key value pair the value here will be true as well so in an application we're just going to check like if flagged is true then we might check like okay what category is actually it's being flagged for or if we just want to keep it simple we're only going to look at this flagged like anything that's flagged we're just going to to block it we're going to filter it out of our stream and not send it over to the model right so with that um I I have to congratulate you on completing this module I hope you learned a lot um on like the basic concepts of Lang chain and also um like prompt engineering techniques um we've seen some um some prompt templates we've seen some prompt engineering tricks like the the F shot learning um we very briefly touched upon llm chains but I didn't want to go too deep into this because the topic is not very complicated output parsing and agents and tools are some very very powerful options in Lang chain um could be very valuable to go to go deeper into this um and to to basically yeah look up more and and um maybe build some projects on this and then lastly leveraging the moderation API to access a filter Now using that moderation API is super simple and gladly so you just need to know like when do you need to use it so basically for AI applications you mostly want to to put that as a filter for anything coming in now with that being said I wish you the best of luck in the following modules and uh good luck in all the other courses\n"