The World of Vector Databases and AI: A New Frontier

As we continue to navigate the rapidly evolving landscape of artificial intelligence, one area that is gaining significant attention is vector databases. These innovative tools are poised to revolutionize the way we store and utilize data, particularly in the realm of machine learning. According to experts, only about 10% of the world's data is currently stored in a database, with the remaining 90% consisting of unstructured data such as images, videos, and emails.

The Challenges of Unstructured Data

Unstructured data presents several challenges for organizations looking to harness its potential. One major issue is that it is difficult to query or search over using traditional SQL databases. This limitation can lead to significant costs associated with manual data processing and analysis, as well as decreased productivity due to the time spent searching through large volumes of unorganized data.

Enter Vector Databases

Vector databases are designed specifically to address these challenges by providing a powerful tool for storing and querying unstructured data. These innovative tools utilize machine learning algorithms to create complex relationships between different pieces of data, allowing for more accurate and efficient searches.

The Benefits of Vector Databases

One of the primary benefits of vector databases is their ability to enable organizations to harness the full potential of unstructured data. By providing a platform for searching and analyzing vast amounts of images, videos, and other types of content, these tools can help companies reduce costs associated with manual processing and analysis.

Another significant advantage of vector databases is their potential to drive innovation in various industries. For example, tech support chatbots have the ability to provide high-quality answers immediately, reducing wait times for customers and improving overall customer satisfaction. Additionally, applying machine learning capabilities to internal knowledge bases can allow employees to quickly access relevant information and make more informed decisions.

Getting Started with Vector Databases

So how can organizations get started with vector databases? According to experts, the first step is often simply to start building and experimenting with different tools and techniques. This can involve downloading open-source frameworks such as Pinecone and experimenting with various machine learning algorithms. Companies like Apache have also developed courses and training programs to help individuals learn more about vector databases.

One such resource is a course offered by Corsera, which covers the basics of vector databases and their applications in industry. Additionally, companies like Pinecone offer free tiers of their services, allowing developers to test out their tools and techniques before investing in full-scale deployments.

Industry Applications

So what kinds of industries can benefit from vector databases? The answer is a wide range of fields, from tech support chatbots to internal knowledge bases. One example mentioned by the expert is applying machine learning capabilities to company-owned internal data sources. For instance, allowing employees to quickly search and access relevant information within their own knowledge base could greatly improve productivity and decision-making.

Another significant opportunity lies in the realm of tech support. Chatbots have the potential to revolutionize customer service by providing high-quality answers immediately, reducing wait times for customers and improving overall customer satisfaction.

The Future of Vector Databases

As we look to the future, it is clear that vector databases are poised to play a major role in shaping the next generation of AI applications. With only about 10% of the world's data currently stored in traditional SQL databases, there is a huge opportunity for innovation and growth.

According to experts, one area that will be particularly exciting to watch in the coming years is the use of machine learning with large amounts of unstructured data. As the ability to search and analyze vast amounts of images, videos, and other types of content becomes more sophisticated, we can expect to see significant advancements in fields such as tech support chatbots and internal knowledge bases.

In conclusion, vector databases represent a major leap forward in the world of AI, with the potential to transform the way we store and utilize data. By providing a platform for searching and analyzing unstructured data, these tools have the ability to drive innovation and growth across a wide range of industries.

"WEBVTTKind: captionsLanguage: enwhat companies like pine cone and uh and the the foundation model companies like open AI are doing are the sort of big uh building blocks that enable um uh Innovation the enable using machine learning um in the real world and in conjunction with um huge uh data sets of existing knowledge or you know new information new knowledge like you know applying it to you know video streams or uh or audio or or whatever so uh what we're doing is putting in place the infrastructure and putting in place INF infrastructure is hard requires you know specialized uh uh skill sets um but taking these building blocks uh and and building um very powerful and uh very useful applications for end users um is is a very different thing and um and I think there's huge opportunity for that hi alen great to have you on the show hey uh nice to meet you and thanks for having me I'd like to start with a problem actually so one of the big criticisms of large language models is that they sometimes make things up or hallucinate so when might you need to worry about this problem so one way to think about it is that uh a large language model um is a combination of a reasoning engine like a brain and a bunch of data a bunch of knowledge that it's uh that it was trained on and is sort of stored in its uh you know in the model and um the problem runs into you run into problems when you try and ask it a question that was not in the the data that it was trained on and in that case it's it's not going to tell you hey I don't know the answer uh because the model would was uh was trained the weights are set it's perfectly happy to answer the question and it'll perfectly confidently um generate an answer and uh and that's that's what we call hallucinations because it's uh these answers are generally totally made up they sound they sound very uh realistic and and it's it says the answer very confidently so uh it looks true uh but in many cases it's uh yeah it's totally made up okay and Vector databases like Pine con have been touted as one of the solutions to this problem of hallucinations so can you tell me what a vector database is and how it's different from a standard SQL database so uh a vector database um is a database that's designed to work with Vector embeddings uh Vector embeddings are basically uh think of it as a a long list of numbers um that's a vector and these numbers are generated typically by by Machine learning models um and these numbers represent the semantic meaning of whatever that object is that you fed into the model so it could be a bit of text U and the model will convert that to a vector a different model might take an image uh or a video frame and do the same thing so uh so basically you end up with a whole bunch of these vectors and you need a way to uh search over them and uh a typical uh database regular SQL database is not designed uh to do that um so that's where Vector databases come in okay so once you've converted all the text or whatever into numbers then it's just um it's something that uh machine learning can deal with or it's uh it's just math to uh work with it rather than uh having to deal with text I suppose okay so uh can you talk me through what the most common use cases of vector databases are yeah there's there's many uh Vector databases are um you can think of it as the you know in this new machine learning uh era um essentially all information at some point is going to be represented as uh as these vectors because that's how you apply machine learning to uh to data um and uh uh if in the past you had databases with lots of tables with text and things like that in them in the future um databases will all have vectors not necessarily replacing the regular SQL databases but in many cases uh um working alongside it but for many use cases that existing databases simply aren't designed for um Vector databases will be used by themselves okay um and are there particular business applications that um are particularly popular at the moment so yeah again there's many um so obviously in the last year since chat GPT was launched many people as you as you mentioned are using it for what's called retrieval augmented uh generation which is where you um augment the knowledge in the large language model with knowledge that's retrieved from the vector database so that's a very popular one people are using uh Vector databases to create um uh things like chatbots that are uh that that know how to answer questions based on a proprietary data sets say a companies uh support information or internal knowledge that's only exists within a company for example that uh chat GPT or other models didn't have access to when they were being trained um image recognition is another one uh image search recommendation systems um tons and tons of things so there's uh you can think of vector databases as essentially a very um it's going to be a low-level primitive that essentially every application in this sort of machine learning age will will be using in the near future these all seem like incredibly important um use cases so uh things like chat Bots is something um more every company needs um I'd like to go into a bit more detail on some of these use cases so maybe we'll start with the simplest thing that you mentioned which is search now we've had search capabilities for documents and websites and the whole internet for decades so how is semantic search different yeah it's a great question so um search engines started by uh essentially um searching over uh text strings so you type in say you know uh few characters like uh Peter and then it'll search through uh all the texts in its database to find that exact sequence of strings that was sort of the initial type of search um obviously it's uh it's uh problematic say you've misspelled the name um or say you're talking about a concept say you're talking about uh a dog and the article that you're searching over refers to K9 or something like that you won't find any of that um so uh that's where we come into something called semantic search which is where the search engine knows the the meaning of what you're searching for and it knows how to do things like uh you know uh handle misspellings and things like that so those are much more sophisticated uh search engines and companies like Google spent thousands and thousands of um programmer years uh essentially um making a uh making their search engine um be aware of the semantic uh information uh in the document to make a high quality search so um uh the great thing about uh using vector embeddings and using um large language models for search is that you essentially can get to the Quality almost to the quality of Google search uh just by using a model and feeding the text through it using vector embeddings and doing Vector search you don't need um you know thousands and thousands of uh of programmer years of uh of work to build uh to build your own search engine yeah so seems incredibly timec consuming trying to work out which words are synonymous or nearly synonymous with other words and just having that done for you automatically um has got to be a huge productivity boost exactly and it's uh it's it's very very complex and um again like companies like Google and Microsoft have thousands of Engineers like making that work really well and in a sense you can think of these large language models as sort of you know in one Fell Swoop allowing people to um entally reproduce that level of quality if not better in many cases that's pretty fantastic um and you also mentioned image search before so that seems like a a very cool idea but I'm not quite sure when you would use it so um what are the sort of business applications of image search yeah good question so there's different types of use cases there's um uh there's two kind it's a image search is a we call it a classification problem so we have uh classification then we have extreme classification so uh classification would be like uh hey this thing here is a hamburger uh this this is a picture of uh you know whatever say You're Building A a website uh where you can search over food or things like that and people are uploading pictures of their dinner so you can you can sort of figure out what uh what that dish is Extreme classification on the other hand is where you're actually trying to find a an individual so for example if it's people you can you can do um like face recognition so if you have like millions of of Faces in your database and um you know you're building a say a system to verify your identity or some sort of security system and you take a picture of yourself um that it can actually find an exact match of that face it classification would say hey this is a face or this is a person extreme classification would say oh this is Peter this is Elon this is uh this is John so both of those can be supported by U by uh these techniques and by Vector databases and uh again um many many different use cases yeah I can certainly see how there's different levels of difficulty there so just saying um is this uh a person in the video is much easier than is to see in the video exactly so for example like uh people have used Pine con for um for security systems for like um if you think of um a company that has security cameras and they have thousands of cameras with feeds of video and you want to find uh anomalies like situations that are uh unexpected like oh a person is now going through that doorway in my data center which people don't usually go through like I'd like to get alert so that's the kind of thing that you could build using a vector database so we talked about text and we talked about images are other different types of data that you can also include in your vector database yeah totally I mean again anything that you can represent as a as a vector embedding is uh is fair game um one example um um there's there's systems uh like YouTube and other companies that um that deal with copyrighted cont so where you upload a video say somebody's uh taking a video that they shot at home um these systems know how to detect um the background music and detect whether it's uh matches uh a piece of copyrighted music from a library of copyrighted songs and then it can decide what to do with it so that's an example of you know representing the the music uh as a vector and then uh and then matching it with uh with the database of uh vectors representing different types of music okay so um it it seems like more as any data type you can have there so um audio seems to be an important thing as well exactly now you mentioned um a technique called retrieve loged generation before can you just tell me a little bit more about what that involves when you when you query uh or or when you work with a large language model um to build something like a chatbot um you typically uh uh generate your query by creating um creating a providing it with some context about your question um so you can say um you know what's an example like how do I how do I use uh Pine how do I upsert data into Pine so one way of doing it is to just go to chat GPT and and entering that as a query and uh chat GPT may or may not give you the right answer it may provide a a hallucinated answer that it just made up um so what you do is uh to get rid of this hallucination you use something called retrieval augmented generation which means um you use the generative capabilities of the model but you augment it with information that you get via retrieval so retrieval is the process of querying some kind of information retrieval system in this case it would be a vector database so you would query a database say we had a database full of pine cones um support uh support information um we say how do I upsert data into pine cone we would we create Pine con's Vector database and we'd find the appropriate uh support documents we'd retrieve the text from them and then we would generate uh um context for the query so you have that same query that you're asking chat GPT how do I upload data into pine cone but then you add to it um please use the information um uh that follows and then then you paste in the text that we've retrieved uh via quering the vector database so you basically have your query and then you have the context which follows it and this whole package you upload into uh chat GPT or your large language model of choice and it will know how to um uh understand the content and the context and the query and provide a uh really well-written um response that is now accurate because it's based on the knowledge that you've just fed it so this whole process is called retrieval augmented generation okay so just to make sure I've understood this it sounds like um the case you talking about before with search is just going to directly return entries that are in your vector database and in this case with retrieval augmented generation it's then combining those results with some kind of prompt to a large language model to have a more natural language interface to um to get results so so the the model will be able to understand the text that you've given it it understands your question and it finds the the knowledge from the uh context you provided and then it constructs a reason well- reasoned response so in many cases the response you give it could be like two or three pages of of text which may include random gibberish so it actually goes through and finds the precise um piece of information and then reformats it reformulates it into a response and then provides the response it's quite remarkable that it works but it works very very well it's such an incredible technique and it sounds like this is leading towards um being able to build better chat bots so can you maybe talk me through how retrieval augmented Generation Um works with um with chatbots so many chat Bots today like I think some of the first use cases have been um uh um customer support um so it's helpful because instead of waiting for a support person uh you can get an instant uh answer for uh from um from the chatbot um and in many cases companies are also hesitant to uh let customers talk directly to a person because it's expensive increases their support cost etc etc with a highquality uh retrieval augmented generation based chatbot you can actually uh provide um instant and and very high quality responses for many cases many companies are finding that um that uh it works quite well and and customers are quite happy and um uh reducing their costs and things like that there's many other use cases for this by the way like uh llms are amazing at um uh reviewing legal documents and things like that so there's many there's many use cases that um that companies are using this for um I'd love to hear more about those um in particular have you heard any success stories from uh companies using this technique yeah um there's many uh I can't talk about all of them I don't have approval to mention every them mention many of our customers but um for example notion is a is a customer that uses pine cone for um allowing people to ask questions about all the material that they've upload uploaded into your North notion workspace so for example here at Pine Cone we use North notion for all our Internal Documentation including our benefits information and healthc care and uh as well as just general internal corporate documentation so you can go there and you can say hey um when are the corporate holidays for Pine con and it'll find the appropriate page pull out the information and respond um you know like chat TPT does so it's it's super super useful that's seem incredibly important I'm always amazed at um how hard it is to find information about what's going on in your own company sometimes so uh solving that uh would be an amazing way for a lot of places um are there any other particular use cases you mentioned the legal case I'd love to hear a bit more about that um what happens there so actually legal and sort of business use cases in general people are finding that using uh Vector datab bases for the semantic search and then using uh the reasoning um strengths of a large language model are you know very very powerful uh combination so we're seeing some of our biggest customers are from the legal Tech space um and uh for example uh when you have a case you have thousands of pages of uh documentation that um that you have to um wave through and find the you know the exact uh right um you know paragraph that's relevant to your uh to your case so you know they're using it for things like that so you can essentially ask questions about all the documentation that's uh that's been provided and it's I'm not a lawyer but apparently it's uh it works quite well for this use case Okay uh so we've established that um for a lot of these um generative AI use cases you need a large language model and a vect database um is there anything else in the Tex stack so clearly there's many pieces but an important piece is getting this information from the raw data which could be a bunch of PDF files or could be web pages or you know could even be in a existing database um the getting that data into the vector database to be used in this way um can be uh can be you know a bit of work so um different uh um uh Solutions have sprung up uh Frameworks uh of different kinds are available open source and uh non-open Source but the problem is uh basically the same you have a bunch of U documentation you need to extract the text from it you then have to go through a process called chunking the chunking is basically um you know have a whole bunch of text document might be 100 pages long um uh and typically uh one vector uh uh can represent again depending on the model that you use can represent a certain amount of uh text um in a in a way that you can retrieve from it well so for example um for this legal documentation you typically want to chunk it up in terms of you know a number of paragraphs you know maybe a thousand uh uh words you know plus or minus something like that and then uh and then create a vector that represents that amount of text so a large document could have you know several tens or or hundreds of vectors that represent parts of that document so anyway so this's a process of extracting all that text chunking it up creating the vector embeddings creating the embeddings by the way we should discuss for a second um you have to use a uh something called an embedding model so it's similar to a large language model but a bit different because it's trained to Output uh the embedding which is uh again the stream of numbers instead of responding in natural language so it takes his input uh chunk of text outputs the vector and then you store those vectors uh in the database and once you've done all of this then you have uh then you have your index and you're and you're ready to go uh um The Next Step Above That is to build um an application that knows how to connect uh the vector database with the large language model so it's uh it's essentially uh it does some orchestration where it gets a query from the user um it uh um it analyzes the query and potentially even uses a large language model to break up the query and say you've asked a relatively complex question it might decide to subdivide that question into several sub questions and then for each sub question it'll then query the vector database get the responses and then construct this uh this context which it then feeds into the model for uh for the uh for the for the response so um that's at a very very high level that's that's how this uh retrieval augmented generation uh framework uh looks like okay so uh it sounds like it can get quite sophisticated and maybe a little bit fiddly in places and that certainly that's my experience of sort of playing around trying to use these Technologies is that once you start trying to split your documents up into chunks you're not quite sure how big the chunk should be and then it can get uh I don't want to say tedious but it can be quite um difficult to get the right answer I'm wondering um are there any ways of or there any Technologies to help make it easier to build chatbots using these Technologies to totally and in fact um when we first tried to do it internally we found that it was quite challenging to get a high quality uh answer so it turns out that you need to um figure out exactly the the right size of junks you have to um understand exactly how to generate the the the query and the prompts and um and uh and yeah there there's a bunch of you know um fiddly aspects as you uh as you said uh so what we did here is we also built an eval uh system which uh allows us to evaluate the quality of the results um and try it out with different uh types of data uh and that would allow us to tune these different parameters um so that's what we built and we actually released an open source package that uh that does this it's called canopy um it's available on GitHub and uh and that um provides some tooling which makes all of this a lot simpler and of course there's many companies that have uh that have built built um um solutions for this some paid Solutions and other uh open source Frameworks like llama index and L chain that are again open source and and quite powerful so there's a there's a lot of uh excitement a lot of innovation happen happening in this space right now it does seem like um that's going to make it a lot easier to build these things quickly and just get much more reliable systems and um this idea of tuning parameters it reminds me of the concept from machine learning called hyper parameter tuning where you don't have to worry about setting some of some of the well the hyper parameters you just get it to run automatically try a few different values and it's automatically going to pick the best ones so that's excellent um okay so the other tricky thing when you're building with this uh is cost so already I've I've been speaking to a few um Chief data officers and like it's gone from like last year where everyone's excited to like build prototypes and now they're like well actually you know once we put things in production I watch inly expensive so um I guess at what point should organizations start worrying about the cost of generative AI there's probably different uh aspects to cost here one is um um based on simply the size of the data set you're working with like do you have a you know really huge data set with hundreds of millions or billions of documents uh that then get converted into um you know hundreds of millions or billions or tens of billions of uh of vectors so that can get expensive to uh generate the vectors and then post them uh in a vector database uh in query them so that that's one element the other element is um um and by the way generating the embeddings can also be expensive um um because you're paying open AI or you know some other company typically uh to generate the embeddings for you so so again depending on the size of your data set there's one set of costs and then on the um uh query side if you have a a use case that's a um High uh High QPS uh basically you know you have millions of people that are querying uh your system constantly so then you have to stand up a system that can support High QPS and and you're also paying um uh for the um the inference side using the using chat GPT to generate the generate the responses um uh um and paying an embedding model to generate the query and all that so so there's uh again costs are either based on performance your performance needs number of queries per second uh and on the size of your data set um so there two two aspects that I level okay so it sounds like there's a lot of different um areas where you're going to be paying things in this case so um do you have any advice for organizations who want to reduce the cost of working with genni so there's different techniques here so uh on the large amount of data side um Pine con has made uh a uh uh We've launched a new system called Pine con serverless which is designed specifically for companies that have very very large data sets uh and we can store it uh store the vectors in a uh uh using um object storage using like blob storage like Amazon S3 which uh make it very very cheap to to store the data set and if like most organizations their uh workload pattern is such that they have a large data set but relatively small um compute requirements like relatively low QPS so so essentially what we've done with pine conone serverless is separated storage from compute you can essentially buy the amount of storage you need and we're providing very very low cost storage and then you get the amount of compute that you need specifically for your use case so we found that uh most uh organizations that have that um that uh you know workload pattern of uh large data small QPS will'll find a huge you know up to 50x reduction in cost compared to existing uh Vector databases um some uh customers have a very high performance use case like you're building a recommendation system you may have a very very small um data set you know hundreds of thousands or maybe small number of millions of of uh of items of of vectors but they have a very very high QPS so that that requires uh a different solution which um which we have uh in our in our existing pod-based platform um the other way you can reduce cost is by um um looking at which uh model you're using both for the embedding and both for the for the um um for the large language model to answer essentially answer your uh your questions so uh we found that there are open source models that are um uh very high performance uh in terms of quality and can be much cheaper to run than calling open AI for example uh and in fact we found that the combination of a uh slightly lower quality model um using retrieval augmented generation can provide a very high quality result um so that's that's another interesting way that people can uh can reduce costs if needed okay so your first point about um some customers having very low Compu requirements compared to their um document size I can certainly see how something like uh trying to search a corporate internet might be like that so it's got lots and lots of documents but you know people aren't trying to search for all the time but it's a recommendation engine where everyone's trying to like find out which product should I buy that's going to have much higher commute requirements um okay so uh the other point you made was about um using open source models now I've had a look on um like the hugging Place hugging face platform and they' just got hundreds and hundreds of different models so are there any open source models that you think are particularly popular with your customers um I think the Llama model are people uh people use the Falcon models um they they actually publish a uh there's a table where you can see the kind of like a leaderboard of uh of the newest models and and their quality rankings um so honestly that's what I would that's what I would look at okay instead of just yeah instead of just naming uh naming some off the top of my head all right I'm I'm sure yeah every quarter there's going to be a new leader and uh totally this changes all the time there's so much innovation in this in this space people are creating um there's an interesting company called Voyager that's creating um uh vertical specific models like a model embedding model just for you know legal or finance things like that which Super interesting as well just on that note um do you have any advice on how to keep up because there are new models coming out all the time there are new techniques coming out all the time there new Frameworks it seems like everything changes every few months so how do you deal with that of constant change oh gosh uh it's a good question uh I think that's you know the challenge for everybody working in this space I think uh first of all as a as a company you need to have a long-term vision for what you're trying to achieve and keep focused on that um a lot of new things pop up many of them are you know some are interesting many of them are just not that relevant uh and the most important thing is that you can you focus on what you're building do a really good job with that and in many cases you could swap out the model you know for the latest greatest thing at a later stage so in many you know many cases you're not building your core product around the specifics of one model to you know you can keep updating them as as they get better and better okay so maybe you don't need to jump on every new technology that appears and start swapping things out every few weeks but just you know uh make sure that you're building stuff where you can swap out the model or whatever other component later all right so uh I think this lead I started talking about skills so um I think a lot of these AI applications are kind of odd because they require both software skills and AI skills so are you seeing any new roles appear for the creation of the a these AI applications it's it's hard to tell so right now so we're a database company so we uh we build uh infrastructure so we hire you know Engineers that are familiar with you know very complex distributed systems know how to design for high performance uh and things like that we're not our customers are the ones building uh the chat Bots and things like that on top of us uh we build Frameworks in between us in that level um so as far as I can tell now it's a combination of you know real just deep software engineering and uh understanding of machine learning um I understand that you know there's uh new roles like prompt engineer and stuff like that that are popping up but that's uh that's uh you know we operate at a lower level than all of that okay so it seems like the might be a sort of Boom in these uh lower in infrastructure level jobs and then also maybe the machine learning level as well and so for everyone else who isn't a developer they're not into infrastructure uh whatever um are there any other opportunities in this area that you're seeing 100% I think the um what companies like Pine con and uh and the the foundation model companies like open AI are doing are the sort of big uh building blocks that enable um uh Innovation the enable using machine learning um in the real world and in conjunction with um huge uh data sets of existing knowledge or you know new information new knowledge like you know applying it to you know video streams or uh or audio or whatever so uh what we're doing is putting in place the infrastructure and putting in place in instructor is hard requires you know specialized uh uh skill sets um but taking these building blocks uh and and building um very powerful and uh very useful applications for end users um is is a very different thing and um and I think there's huge opportun for that essentially every um vertical out there when I say vertical I mean things like you know the automotive industry the accounting industry the legal industry Finance um uh Pharmaceuticals etc etc all of those are going to have uh thousands and thousands of use cases that are all built using the same uh infrastructure so if I were somebody looking to get into the machine learning field or at least the you know um um you know using machine learning to innovate in uh you know in traditional Industries I would learn how to use on the one hand uh these core building blocks that are built by companies like again like Pine and open Ai and go here and others um but then also deeply understand a uh a a specific uh use case or specific industry um and once you understand that industry you'll you'll be able to figure out areas where uh you can innovate and optimize um um some process or um um you know allow people to do things that they simply couldn't do before because of the scale of the data or the complexity of the task or or things like that so there's um you know I would say almost an in inum innumerable um list of uh of opportunities out there um so again it's learning how these Technologies work and uh it's a lot easier to use pine cone and to use open AI than it is to actually build those things um but once you have these buildings blocks it's fairly manageable and straightforward even for small themes of you know one or two people to build very very powerful solutions that actually can make a substantial impact on a company or on an industry I do love that um this has become much more accessible like you mentioned um AI has just got a lot easier to use in the last couple of years and I think it's continuing that way so um because of this it feels like there are a lot of people without a technical background who suddenly become interested in um AI so what do you think are the most important things that everybody needs to know about Ai and about Vector databases I think uh understanding at a conceptual level what they do how they work how they interact with data how you Fair data to be used in the models how to how to wire them together and again wiring things together is you know much easier uh than it was in the past there's many many solutions and products that help you do that um again I think it's really down to understanding the problem that you're trying to solve um you know and we see these you know uh every day like uh uh somebody's trying to um help an insurance company take images of accidents and and uh you know figure out the um the uh you know the damage that was caused like what was you know what with the specific types of damage and how much does it cost to fix it you know um that's just one example or you're an oil company that uh is Drilling and as you're drilling you're uncovering all kinds of sea shells so you're building a a database of like literally millions hundreds of millions of these samples of things that you found in the seabed and you want to be able to you know search over them to figure out um you know to gain insights about what you're Drilling in um again these are use cases I never would have imagined but there's you know as soon as you start digging in to any industry you'll find um you know a huge number of these and these are again huge data sets uh so it's definitely big data problem and they need machine learning and they need uh techniques like vector embeddings to solve I have to say I've also never thought about the use case having a giant database of seashells uh but yeah you're right that um once you start think about all the different industry use cases there's so many options and and and thing is as like uh if you're entrepreneur or just a data scientist or a software engineer you'll never uncover these just by you know reading or you know browsing the internet or whatever you have to actually get out there and interact with u with these businesses and uh and talk to them and say hey what are what are the types of things that you're trying to achieve like what are the you know if you had this Magic Machine learning you know thing what would you like to do with it um and they'll probably come up with ideas and you can help them um help them achieve that because I think you know they're they're struggling from also lack of uh uh machine learning expertise um again this stuff is all new and um it's also new to them it is a tricky thing where there are so many possible things that you could do with this it's sometimes hard to know where to begin I was wondering whether you've seen any common patterns uh across your customers or just from talking to people like what's a a good sort of simple first project the the text based uh ragged use cases I think are the simplest and most common um and um example projects could be what I mentioned like um applying that to uh to your internal knowledge uh in your company to allow people to essentially uh um chat with their knowledge base and ask questions like hey you know um how does health insurance work in uh you know in my company or things like that so that's uh relatively you know conceptually straightforward it's not entirely trivial to put in place but conceptually straightforward and there's uh patterns and Frameworks but if you think about it literally every company in the world will implement this in the next uh five or 10 years like literally every single one so you know and probably 1% of them have done that now so it's a huge huge opportunity um and um similarly for things like tech support as I mentioned um any company that has customers would love to reduce their tech support costs and allow their customers to to get a high quality answer immediately instead of having to wait you know over the weekend um you know those are just two examples that um you know if you want to get started with the space you know easy to start experimenting with and I'm sure you'll find customers who will need it immediately yeah certainly support chatbot seems to be a huge thing I can certainly see how having a bot where it's programmed to be friendly all the time is is going to be uh a good thing um okay so um do you have any other advice on how to get started just start building start hacking on stuff download some of these open source Frameworks uh use pine con we have a free tier just start experiment menting we have a a corsera course with Andrew Ang you can they can learn about Vector databases uh with um and uh and then start learning about um uh an you know an industry find some uh interesting use case and uh to start building and uh you'll probably surprise yourself by how quickly you can get something usable something useful um that's great I love the idea that you should uh just start building um actually I would be remiss not to mention that you can also learn about um pine cone on data Camp as well uh from one of uh Pine con's own uh developer Advocates James Briggs uh so yeah also POS beautiful yes love J Love James and I'm so pleased that he's uh that he's uh work with you yes excellent all right so just to wrap up um what are you most excited about in the world of vector databases and AI I I think uh so the way the way we look at it is um if you look at all the data in the world today only about 10% of uh of that data I'm talking about like Digital Data is uh currently stored in a database um and like 90% of the data out there all the unstructured data all the videos and uh and uh you know even emails and things like that th those are all currently not uh in a database and are exceedingly hard to to search over and uh and use so this is where um Vector databases come in um the reason they're not in a database is because you can't really query over images um or uh emails in a useful way just you know in a SQL database that's why you need search engines um the search engines today aren't great at these things so that's why Vector databases are sort of another leap forward um so we're excited about the fact that there's a huge opportunity here to uh use machine learning with um huge amounts of data that are currently not essentially not accessible not usable so um so we think there's going to be tons of uh interesting things um to build and to and to um you know innovate with it's fantastic stuff I love the idea that absolutely everything is data now and you can actually yeah work with it search it and uh calculate on it excellent all right uh thank you so much time elen thank you Richie it was great talking to youwhat companies like pine cone and uh and the the foundation model companies like open AI are doing are the sort of big uh building blocks that enable um uh Innovation the enable using machine learning um in the real world and in conjunction with um huge uh data sets of existing knowledge or you know new information new knowledge like you know applying it to you know video streams or uh or audio or or whatever so uh what we're doing is putting in place the infrastructure and putting in place INF infrastructure is hard requires you know specialized uh uh skill sets um but taking these building blocks uh and and building um very powerful and uh very useful applications for end users um is is a very different thing and um and I think there's huge opportunity for that hi alen great to have you on the show hey uh nice to meet you and thanks for having me I'd like to start with a problem actually so one of the big criticisms of large language models is that they sometimes make things up or hallucinate so when might you need to worry about this problem so one way to think about it is that uh a large language model um is a combination of a reasoning engine like a brain and a bunch of data a bunch of knowledge that it's uh that it was trained on and is sort of stored in its uh you know in the model and um the problem runs into you run into problems when you try and ask it a question that was not in the the data that it was trained on and in that case it's it's not going to tell you hey I don't know the answer uh because the model would was uh was trained the weights are set it's perfectly happy to answer the question and it'll perfectly confidently um generate an answer and uh and that's that's what we call hallucinations because it's uh these answers are generally totally made up they sound they sound very uh realistic and and it's it says the answer very confidently so uh it looks true uh but in many cases it's uh yeah it's totally made up okay and Vector databases like Pine con have been touted as one of the solutions to this problem of hallucinations so can you tell me what a vector database is and how it's different from a standard SQL database so uh a vector database um is a database that's designed to work with Vector embeddings uh Vector embeddings are basically uh think of it as a a long list of numbers um that's a vector and these numbers are generated typically by by Machine learning models um and these numbers represent the semantic meaning of whatever that object is that you fed into the model so it could be a bit of text U and the model will convert that to a vector a different model might take an image uh or a video frame and do the same thing so uh so basically you end up with a whole bunch of these vectors and you need a way to uh search over them and uh a typical uh database regular SQL database is not designed uh to do that um so that's where Vector databases come in okay so once you've converted all the text or whatever into numbers then it's just um it's something that uh machine learning can deal with or it's uh it's just math to uh work with it rather than uh having to deal with text I suppose okay so uh can you talk me through what the most common use cases of vector databases are yeah there's there's many uh Vector databases are um you can think of it as the you know in this new machine learning uh era um essentially all information at some point is going to be represented as uh as these vectors because that's how you apply machine learning to uh to data um and uh uh if in the past you had databases with lots of tables with text and things like that in them in the future um databases will all have vectors not necessarily replacing the regular SQL databases but in many cases uh um working alongside it but for many use cases that existing databases simply aren't designed for um Vector databases will be used by themselves okay um and are there particular business applications that um are particularly popular at the moment so yeah again there's many um so obviously in the last year since chat GPT was launched many people as you as you mentioned are using it for what's called retrieval augmented uh generation which is where you um augment the knowledge in the large language model with knowledge that's retrieved from the vector database so that's a very popular one people are using uh Vector databases to create um uh things like chatbots that are uh that that know how to answer questions based on a proprietary data sets say a companies uh support information or internal knowledge that's only exists within a company for example that uh chat GPT or other models didn't have access to when they were being trained um image recognition is another one uh image search recommendation systems um tons and tons of things so there's uh you can think of vector databases as essentially a very um it's going to be a low-level primitive that essentially every application in this sort of machine learning age will will be using in the near future these all seem like incredibly important um use cases so uh things like chat Bots is something um more every company needs um I'd like to go into a bit more detail on some of these use cases so maybe we'll start with the simplest thing that you mentioned which is search now we've had search capabilities for documents and websites and the whole internet for decades so how is semantic search different yeah it's a great question so um search engines started by uh essentially um searching over uh text strings so you type in say you know uh few characters like uh Peter and then it'll search through uh all the texts in its database to find that exact sequence of strings that was sort of the initial type of search um obviously it's uh it's uh problematic say you've misspelled the name um or say you're talking about a concept say you're talking about uh a dog and the article that you're searching over refers to K9 or something like that you won't find any of that um so uh that's where we come into something called semantic search which is where the search engine knows the the meaning of what you're searching for and it knows how to do things like uh you know uh handle misspellings and things like that so those are much more sophisticated uh search engines and companies like Google spent thousands and thousands of um programmer years uh essentially um making a uh making their search engine um be aware of the semantic uh information uh in the document to make a high quality search so um uh the great thing about uh using vector embeddings and using um large language models for search is that you essentially can get to the Quality almost to the quality of Google search uh just by using a model and feeding the text through it using vector embeddings and doing Vector search you don't need um you know thousands and thousands of uh of programmer years of uh of work to build uh to build your own search engine yeah so seems incredibly timec consuming trying to work out which words are synonymous or nearly synonymous with other words and just having that done for you automatically um has got to be a huge productivity boost exactly and it's uh it's it's very very complex and um again like companies like Google and Microsoft have thousands of Engineers like making that work really well and in a sense you can think of these large language models as sort of you know in one Fell Swoop allowing people to um entally reproduce that level of quality if not better in many cases that's pretty fantastic um and you also mentioned image search before so that seems like a a very cool idea but I'm not quite sure when you would use it so um what are the sort of business applications of image search yeah good question so there's different types of use cases there's um uh there's two kind it's a image search is a we call it a classification problem so we have uh classification then we have extreme classification so uh classification would be like uh hey this thing here is a hamburger uh this this is a picture of uh you know whatever say You're Building A a website uh where you can search over food or things like that and people are uploading pictures of their dinner so you can you can sort of figure out what uh what that dish is Extreme classification on the other hand is where you're actually trying to find a an individual so for example if it's people you can you can do um like face recognition so if you have like millions of of Faces in your database and um you know you're building a say a system to verify your identity or some sort of security system and you take a picture of yourself um that it can actually find an exact match of that face it classification would say hey this is a face or this is a person extreme classification would say oh this is Peter this is Elon this is uh this is John so both of those can be supported by U by uh these techniques and by Vector databases and uh again um many many different use cases yeah I can certainly see how there's different levels of difficulty there so just saying um is this uh a person in the video is much easier than is to see in the video exactly so for example like uh people have used Pine con for um for security systems for like um if you think of um a company that has security cameras and they have thousands of cameras with feeds of video and you want to find uh anomalies like situations that are uh unexpected like oh a person is now going through that doorway in my data center which people don't usually go through like I'd like to get alert so that's the kind of thing that you could build using a vector database so we talked about text and we talked about images are other different types of data that you can also include in your vector database yeah totally I mean again anything that you can represent as a as a vector embedding is uh is fair game um one example um um there's there's systems uh like YouTube and other companies that um that deal with copyrighted cont so where you upload a video say somebody's uh taking a video that they shot at home um these systems know how to detect um the background music and detect whether it's uh matches uh a piece of copyrighted music from a library of copyrighted songs and then it can decide what to do with it so that's an example of you know representing the the music uh as a vector and then uh and then matching it with uh with the database of uh vectors representing different types of music okay so um it it seems like more as any data type you can have there so um audio seems to be an important thing as well exactly now you mentioned um a technique called retrieve loged generation before can you just tell me a little bit more about what that involves when you when you query uh or or when you work with a large language model um to build something like a chatbot um you typically uh uh generate your query by creating um creating a providing it with some context about your question um so you can say um you know what's an example like how do I how do I use uh Pine how do I upsert data into Pine so one way of doing it is to just go to chat GPT and and entering that as a query and uh chat GPT may or may not give you the right answer it may provide a a hallucinated answer that it just made up um so what you do is uh to get rid of this hallucination you use something called retrieval augmented generation which means um you use the generative capabilities of the model but you augment it with information that you get via retrieval so retrieval is the process of querying some kind of information retrieval system in this case it would be a vector database so you would query a database say we had a database full of pine cones um support uh support information um we say how do I upsert data into pine cone we would we create Pine con's Vector database and we'd find the appropriate uh support documents we'd retrieve the text from them and then we would generate uh um context for the query so you have that same query that you're asking chat GPT how do I upload data into pine cone but then you add to it um please use the information um uh that follows and then then you paste in the text that we've retrieved uh via quering the vector database so you basically have your query and then you have the context which follows it and this whole package you upload into uh chat GPT or your large language model of choice and it will know how to um uh understand the content and the context and the query and provide a uh really well-written um response that is now accurate because it's based on the knowledge that you've just fed it so this whole process is called retrieval augmented generation okay so just to make sure I've understood this it sounds like um the case you talking about before with search is just going to directly return entries that are in your vector database and in this case with retrieval augmented generation it's then combining those results with some kind of prompt to a large language model to have a more natural language interface to um to get results so so the the model will be able to understand the text that you've given it it understands your question and it finds the the knowledge from the uh context you provided and then it constructs a reason well- reasoned response so in many cases the response you give it could be like two or three pages of of text which may include random gibberish so it actually goes through and finds the precise um piece of information and then reformats it reformulates it into a response and then provides the response it's quite remarkable that it works but it works very very well it's such an incredible technique and it sounds like this is leading towards um being able to build better chat bots so can you maybe talk me through how retrieval augmented Generation Um works with um with chatbots so many chat Bots today like I think some of the first use cases have been um uh um customer support um so it's helpful because instead of waiting for a support person uh you can get an instant uh answer for uh from um from the chatbot um and in many cases companies are also hesitant to uh let customers talk directly to a person because it's expensive increases their support cost etc etc with a highquality uh retrieval augmented generation based chatbot you can actually uh provide um instant and and very high quality responses for many cases many companies are finding that um that uh it works quite well and and customers are quite happy and um uh reducing their costs and things like that there's many other use cases for this by the way like uh llms are amazing at um uh reviewing legal documents and things like that so there's many there's many use cases that um that companies are using this for um I'd love to hear more about those um in particular have you heard any success stories from uh companies using this technique yeah um there's many uh I can't talk about all of them I don't have approval to mention every them mention many of our customers but um for example notion is a is a customer that uses pine cone for um allowing people to ask questions about all the material that they've upload uploaded into your North notion workspace so for example here at Pine Cone we use North notion for all our Internal Documentation including our benefits information and healthc care and uh as well as just general internal corporate documentation so you can go there and you can say hey um when are the corporate holidays for Pine con and it'll find the appropriate page pull out the information and respond um you know like chat TPT does so it's it's super super useful that's seem incredibly important I'm always amazed at um how hard it is to find information about what's going on in your own company sometimes so uh solving that uh would be an amazing way for a lot of places um are there any other particular use cases you mentioned the legal case I'd love to hear a bit more about that um what happens there so actually legal and sort of business use cases in general people are finding that using uh Vector datab bases for the semantic search and then using uh the reasoning um strengths of a large language model are you know very very powerful uh combination so we're seeing some of our biggest customers are from the legal Tech space um and uh for example uh when you have a case you have thousands of pages of uh documentation that um that you have to um wave through and find the you know the exact uh right um you know paragraph that's relevant to your uh to your case so you know they're using it for things like that so you can essentially ask questions about all the documentation that's uh that's been provided and it's I'm not a lawyer but apparently it's uh it works quite well for this use case Okay uh so we've established that um for a lot of these um generative AI use cases you need a large language model and a vect database um is there anything else in the Tex stack so clearly there's many pieces but an important piece is getting this information from the raw data which could be a bunch of PDF files or could be web pages or you know could even be in a existing database um the getting that data into the vector database to be used in this way um can be uh can be you know a bit of work so um different uh um uh Solutions have sprung up uh Frameworks uh of different kinds are available open source and uh non-open Source but the problem is uh basically the same you have a bunch of U documentation you need to extract the text from it you then have to go through a process called chunking the chunking is basically um you know have a whole bunch of text document might be 100 pages long um uh and typically uh one vector uh uh can represent again depending on the model that you use can represent a certain amount of uh text um in a in a way that you can retrieve from it well so for example um for this legal documentation you typically want to chunk it up in terms of you know a number of paragraphs you know maybe a thousand uh uh words you know plus or minus something like that and then uh and then create a vector that represents that amount of text so a large document could have you know several tens or or hundreds of vectors that represent parts of that document so anyway so this's a process of extracting all that text chunking it up creating the vector embeddings creating the embeddings by the way we should discuss for a second um you have to use a uh something called an embedding model so it's similar to a large language model but a bit different because it's trained to Output uh the embedding which is uh again the stream of numbers instead of responding in natural language so it takes his input uh chunk of text outputs the vector and then you store those vectors uh in the database and once you've done all of this then you have uh then you have your index and you're and you're ready to go uh um The Next Step Above That is to build um an application that knows how to connect uh the vector database with the large language model so it's uh it's essentially uh it does some orchestration where it gets a query from the user um it uh um it analyzes the query and potentially even uses a large language model to break up the query and say you've asked a relatively complex question it might decide to subdivide that question into several sub questions and then for each sub question it'll then query the vector database get the responses and then construct this uh this context which it then feeds into the model for uh for the uh for the for the response so um that's at a very very high level that's that's how this uh retrieval augmented generation uh framework uh looks like okay so uh it sounds like it can get quite sophisticated and maybe a little bit fiddly in places and that certainly that's my experience of sort of playing around trying to use these Technologies is that once you start trying to split your documents up into chunks you're not quite sure how big the chunk should be and then it can get uh I don't want to say tedious but it can be quite um difficult to get the right answer I'm wondering um are there any ways of or there any Technologies to help make it easier to build chatbots using these Technologies to totally and in fact um when we first tried to do it internally we found that it was quite challenging to get a high quality uh answer so it turns out that you need to um figure out exactly the the right size of junks you have to um understand exactly how to generate the the the query and the prompts and um and uh and yeah there there's a bunch of you know um fiddly aspects as you uh as you said uh so what we did here is we also built an eval uh system which uh allows us to evaluate the quality of the results um and try it out with different uh types of data uh and that would allow us to tune these different parameters um so that's what we built and we actually released an open source package that uh that does this it's called canopy um it's available on GitHub and uh and that um provides some tooling which makes all of this a lot simpler and of course there's many companies that have uh that have built built um um solutions for this some paid Solutions and other uh open source Frameworks like llama index and L chain that are again open source and and quite powerful so there's a there's a lot of uh excitement a lot of innovation happen happening in this space right now it does seem like um that's going to make it a lot easier to build these things quickly and just get much more reliable systems and um this idea of tuning parameters it reminds me of the concept from machine learning called hyper parameter tuning where you don't have to worry about setting some of some of the well the hyper parameters you just get it to run automatically try a few different values and it's automatically going to pick the best ones so that's excellent um okay so the other tricky thing when you're building with this uh is cost so already I've I've been speaking to a few um Chief data officers and like it's gone from like last year where everyone's excited to like build prototypes and now they're like well actually you know once we put things in production I watch inly expensive so um I guess at what point should organizations start worrying about the cost of generative AI there's probably different uh aspects to cost here one is um um based on simply the size of the data set you're working with like do you have a you know really huge data set with hundreds of millions or billions of documents uh that then get converted into um you know hundreds of millions or billions or tens of billions of uh of vectors so that can get expensive to uh generate the vectors and then post them uh in a vector database uh in query them so that that's one element the other element is um um and by the way generating the embeddings can also be expensive um um because you're paying open AI or you know some other company typically uh to generate the embeddings for you so so again depending on the size of your data set there's one set of costs and then on the um uh query side if you have a a use case that's a um High uh High QPS uh basically you know you have millions of people that are querying uh your system constantly so then you have to stand up a system that can support High QPS and and you're also paying um uh for the um the inference side using the using chat GPT to generate the generate the responses um uh um and paying an embedding model to generate the query and all that so so there's uh again costs are either based on performance your performance needs number of queries per second uh and on the size of your data set um so there two two aspects that I level okay so it sounds like there's a lot of different um areas where you're going to be paying things in this case so um do you have any advice for organizations who want to reduce the cost of working with genni so there's different techniques here so uh on the large amount of data side um Pine con has made uh a uh uh We've launched a new system called Pine con serverless which is designed specifically for companies that have very very large data sets uh and we can store it uh store the vectors in a uh uh using um object storage using like blob storage like Amazon S3 which uh make it very very cheap to to store the data set and if like most organizations their uh workload pattern is such that they have a large data set but relatively small um compute requirements like relatively low QPS so so essentially what we've done with pine conone serverless is separated storage from compute you can essentially buy the amount of storage you need and we're providing very very low cost storage and then you get the amount of compute that you need specifically for your use case so we found that uh most uh organizations that have that um that uh you know workload pattern of uh large data small QPS will'll find a huge you know up to 50x reduction in cost compared to existing uh Vector databases um some uh customers have a very high performance use case like you're building a recommendation system you may have a very very small um data set you know hundreds of thousands or maybe small number of millions of of uh of items of of vectors but they have a very very high QPS so that that requires uh a different solution which um which we have uh in our in our existing pod-based platform um the other way you can reduce cost is by um um looking at which uh model you're using both for the embedding and both for the for the um um for the large language model to answer essentially answer your uh your questions so uh we found that there are open source models that are um uh very high performance uh in terms of quality and can be much cheaper to run than calling open AI for example uh and in fact we found that the combination of a uh slightly lower quality model um using retrieval augmented generation can provide a very high quality result um so that's that's another interesting way that people can uh can reduce costs if needed okay so your first point about um some customers having very low Compu requirements compared to their um document size I can certainly see how something like uh trying to search a corporate internet might be like that so it's got lots and lots of documents but you know people aren't trying to search for all the time but it's a recommendation engine where everyone's trying to like find out which product should I buy that's going to have much higher commute requirements um okay so uh the other point you made was about um using open source models now I've had a look on um like the hugging Place hugging face platform and they' just got hundreds and hundreds of different models so are there any open source models that you think are particularly popular with your customers um I think the Llama model are people uh people use the Falcon models um they they actually publish a uh there's a table where you can see the kind of like a leaderboard of uh of the newest models and and their quality rankings um so honestly that's what I would that's what I would look at okay instead of just yeah instead of just naming uh naming some off the top of my head all right I'm I'm sure yeah every quarter there's going to be a new leader and uh totally this changes all the time there's so much innovation in this in this space people are creating um there's an interesting company called Voyager that's creating um uh vertical specific models like a model embedding model just for you know legal or finance things like that which Super interesting as well just on that note um do you have any advice on how to keep up because there are new models coming out all the time there are new techniques coming out all the time there new Frameworks it seems like everything changes every few months so how do you deal with that of constant change oh gosh uh it's a good question uh I think that's you know the challenge for everybody working in this space I think uh first of all as a as a company you need to have a long-term vision for what you're trying to achieve and keep focused on that um a lot of new things pop up many of them are you know some are interesting many of them are just not that relevant uh and the most important thing is that you can you focus on what you're building do a really good job with that and in many cases you could swap out the model you know for the latest greatest thing at a later stage so in many you know many cases you're not building your core product around the specifics of one model to you know you can keep updating them as as they get better and better okay so maybe you don't need to jump on every new technology that appears and start swapping things out every few weeks but just you know uh make sure that you're building stuff where you can swap out the model or whatever other component later all right so uh I think this lead I started talking about skills so um I think a lot of these AI applications are kind of odd because they require both software skills and AI skills so are you seeing any new roles appear for the creation of the a these AI applications it's it's hard to tell so right now so we're a database company so we uh we build uh infrastructure so we hire you know Engineers that are familiar with you know very complex distributed systems know how to design for high performance uh and things like that we're not our customers are the ones building uh the chat Bots and things like that on top of us uh we build Frameworks in between us in that level um so as far as I can tell now it's a combination of you know real just deep software engineering and uh understanding of machine learning um I understand that you know there's uh new roles like prompt engineer and stuff like that that are popping up but that's uh that's uh you know we operate at a lower level than all of that okay so it seems like the might be a sort of Boom in these uh lower in infrastructure level jobs and then also maybe the machine learning level as well and so for everyone else who isn't a developer they're not into infrastructure uh whatever um are there any other opportunities in this area that you're seeing 100% I think the um what companies like Pine con and uh and the the foundation model companies like open AI are doing are the sort of big uh building blocks that enable um uh Innovation the enable using machine learning um in the real world and in conjunction with um huge uh data sets of existing knowledge or you know new information new knowledge like you know applying it to you know video streams or uh or audio or whatever so uh what we're doing is putting in place the infrastructure and putting in place in instructor is hard requires you know specialized uh uh skill sets um but taking these building blocks uh and and building um very powerful and uh very useful applications for end users um is is a very different thing and um and I think there's huge opportun for that essentially every um vertical out there when I say vertical I mean things like you know the automotive industry the accounting industry the legal industry Finance um uh Pharmaceuticals etc etc all of those are going to have uh thousands and thousands of use cases that are all built using the same uh infrastructure so if I were somebody looking to get into the machine learning field or at least the you know um um you know using machine learning to innovate in uh you know in traditional Industries I would learn how to use on the one hand uh these core building blocks that are built by companies like again like Pine and open Ai and go here and others um but then also deeply understand a uh a a specific uh use case or specific industry um and once you understand that industry you'll you'll be able to figure out areas where uh you can innovate and optimize um um some process or um um you know allow people to do things that they simply couldn't do before because of the scale of the data or the complexity of the task or or things like that so there's um you know I would say almost an in inum innumerable um list of uh of opportunities out there um so again it's learning how these Technologies work and uh it's a lot easier to use pine cone and to use open AI than it is to actually build those things um but once you have these buildings blocks it's fairly manageable and straightforward even for small themes of you know one or two people to build very very powerful solutions that actually can make a substantial impact on a company or on an industry I do love that um this has become much more accessible like you mentioned um AI has just got a lot easier to use in the last couple of years and I think it's continuing that way so um because of this it feels like there are a lot of people without a technical background who suddenly become interested in um AI so what do you think are the most important things that everybody needs to know about Ai and about Vector databases I think uh understanding at a conceptual level what they do how they work how they interact with data how you Fair data to be used in the models how to how to wire them together and again wiring things together is you know much easier uh than it was in the past there's many many solutions and products that help you do that um again I think it's really down to understanding the problem that you're trying to solve um you know and we see these you know uh every day like uh uh somebody's trying to um help an insurance company take images of accidents and and uh you know figure out the um the uh you know the damage that was caused like what was you know what with the specific types of damage and how much does it cost to fix it you know um that's just one example or you're an oil company that uh is Drilling and as you're drilling you're uncovering all kinds of sea shells so you're building a a database of like literally millions hundreds of millions of these samples of things that you found in the seabed and you want to be able to you know search over them to figure out um you know to gain insights about what you're Drilling in um again these are use cases I never would have imagined but there's you know as soon as you start digging in to any industry you'll find um you know a huge number of these and these are again huge data sets uh so it's definitely big data problem and they need machine learning and they need uh techniques like vector embeddings to solve I have to say I've also never thought about the use case having a giant database of seashells uh but yeah you're right that um once you start think about all the different industry use cases there's so many options and and and thing is as like uh if you're entrepreneur or just a data scientist or a software engineer you'll never uncover these just by you know reading or you know browsing the internet or whatever you have to actually get out there and interact with u with these businesses and uh and talk to them and say hey what are what are the types of things that you're trying to achieve like what are the you know if you had this Magic Machine learning you know thing what would you like to do with it um and they'll probably come up with ideas and you can help them um help them achieve that because I think you know they're they're struggling from also lack of uh uh machine learning expertise um again this stuff is all new and um it's also new to them it is a tricky thing where there are so many possible things that you could do with this it's sometimes hard to know where to begin I was wondering whether you've seen any common patterns uh across your customers or just from talking to people like what's a a good sort of simple first project the the text based uh ragged use cases I think are the simplest and most common um and um example projects could be what I mentioned like um applying that to uh to your internal knowledge uh in your company to allow people to essentially uh um chat with their knowledge base and ask questions like hey you know um how does health insurance work in uh you know in my company or things like that so that's uh relatively you know conceptually straightforward it's not entirely trivial to put in place but conceptually straightforward and there's uh patterns and Frameworks but if you think about it literally every company in the world will implement this in the next uh five or 10 years like literally every single one so you know and probably 1% of them have done that now so it's a huge huge opportunity um and um similarly for things like tech support as I mentioned um any company that has customers would love to reduce their tech support costs and allow their customers to to get a high quality answer immediately instead of having to wait you know over the weekend um you know those are just two examples that um you know if you want to get started with the space you know easy to start experimenting with and I'm sure you'll find customers who will need it immediately yeah certainly support chatbot seems to be a huge thing I can certainly see how having a bot where it's programmed to be friendly all the time is is going to be uh a good thing um okay so um do you have any other advice on how to get started just start building start hacking on stuff download some of these open source Frameworks uh use pine con we have a free tier just start experiment menting we have a a corsera course with Andrew Ang you can they can learn about Vector databases uh with um and uh and then start learning about um uh an you know an industry find some uh interesting use case and uh to start building and uh you'll probably surprise yourself by how quickly you can get something usable something useful um that's great I love the idea that you should uh just start building um actually I would be remiss not to mention that you can also learn about um pine cone on data Camp as well uh from one of uh Pine con's own uh developer Advocates James Briggs uh so yeah also POS beautiful yes love J Love James and I'm so pleased that he's uh that he's uh work with you yes excellent all right so just to wrap up um what are you most excited about in the world of vector databases and AI I I think uh so the way the way we look at it is um if you look at all the data in the world today only about 10% of uh of that data I'm talking about like Digital Data is uh currently stored in a database um and like 90% of the data out there all the unstructured data all the videos and uh and uh you know even emails and things like that th those are all currently not uh in a database and are exceedingly hard to to search over and uh and use so this is where um Vector databases come in um the reason they're not in a database is because you can't really query over images um or uh emails in a useful way just you know in a SQL database that's why you need search engines um the search engines today aren't great at these things so that's why Vector databases are sort of another leap forward um so we're excited about the fact that there's a huge opportunity here to uh use machine learning with um huge amounts of data that are currently not essentially not accessible not usable so um so we think there's going to be tons of uh interesting things um to build and to and to um you know innovate with it's fantastic stuff I love the idea that absolutely everything is data now and you can actually yeah work with it search it and uh calculate on it excellent all right uh thank you so much time elen thank you Richie it was great talking to you\n"