The Future of Agentic Systems: A Discussion with Thomas and Nick

In a recent discussion on the future of agentic systems, two experts shared their insights on the potential applications and limitations of these powerful tools. The conversation highlighted the importance of understanding the strengths and weaknesses of agentic systems in order to harness their full potential.

Thomas began by emphasizing the significance of agentic systems as a tool for innovation and problem-solving. He stated that professionals who master these systems will be left behind, while those who fail to adapt will struggle to keep up with the pace of change. Thomas drew an analogy between the adoption of personal computers in the past and the current state of agentic systems. Just as computers revolutionized the way lawyers worked, agentic systems have the potential to transform various industries.

However, Nick cautioned that brute-force innovation may not be the most effective approach for agentic systems. He argued that while throwing more resources at a problem can lead to breakthroughs in certain areas, it is unlikely to solve all problems and may even exacerbate existing issues. Nick pointed out that the law of exponential return does not necessarily apply to agentic systems, as the complexity of these systems can quickly become overwhelming.

Instead, Nick suggested that humans have an essential role to play in innovation and problem-solving. He noted that researchers and experts can produce exponential amounts of compute in their own heads, even if it is not always apparent. Furthermore, Nick emphasized that human intuition and creativity are crucial for identifying novel solutions and navigating complex problems.

Thomas echoed this sentiment, stating that agentic systems will never be able to predict the effectiveness of their actions with absolute certainty. He argued that the compounding error rate across the various steps of the agentic workflow can lead to unpredictable outcomes. Thomas did suggest that agentic systems may eventually reach a level of proficiency where they can discover interesting insights and solutions, but this would depend on the quality of the input data and the expertise of those who design these systems.

Nick concluded the discussion by highlighting the importance of humility when working with agentic systems. He emphasized that while brute-force innovation may have its place in certain areas, it is essential to approach these systems with a critical and nuanced perspective. Nick also noted that the focus should be on creating systems that can learn from their mistakes and adapt over time.

In summary, the discussion between Thomas and Nick shed light on the potential applications and limitations of agentic systems. While brute-force innovation may have its place in certain areas, it is essential to approach these systems with a critical perspective and acknowledge the importance of human intuition and creativity. As agentic systems continue to evolve, it will be crucial to strike a balance between computational power and human expertise.

The Future of Agentic Systems: A Call for Caution

As the field of agentic systems continues to advance, there is a growing need for caution and responsible innovation. With the increasing complexity of these systems, it is essential to consider the potential risks and consequences of their widespread adoption. In this article, we will explore some of the key challenges and limitations of agentic systems and discuss the importance of human expertise in navigating these complex systems.

One of the primary concerns surrounding agentic systems is their ability to predict outcomes with absolute certainty. While they have shown impressive capabilities in various domains, there are still many areas where they struggle to provide reliable results. For instance, Thomas noted that classical machine learning approaches have had a difficult time predicting stable and repetitive patterns. He argued that this limitation will remain a challenge for agentic systems unless significant advances are made.

Furthermore, Nick highlighted the importance of understanding the limitations of brute-force innovation. While throwing more resources at a problem can lead to breakthroughs in certain areas, it is unlikely to solve all problems and may even exacerbate existing issues. The law of exponential return does not necessarily apply to agentic systems, as the complexity of these systems can quickly become overwhelming.

Human Expertise: A Critical Component

In order to navigate the complexities of agentic systems, human expertise plays a critical role. Researchers, experts, and innovators must be aware of their own strengths and limitations in working with these systems. Nick emphasized that humans have an essential role to play in innovation and problem-solving, as they can produce exponential amounts of compute in their own heads.

Furthermore, Thomas highlighted the importance of human intuition and creativity when working with agentic systems. He argued that while these systems are capable of making predictions and discoveries, they will never be able to predict the effectiveness of their actions with absolute certainty. Human expertise is essential for identifying novel solutions and navigating complex problems.

The Role of Agentic Systems in Innovation

Agentic systems have the potential to revolutionize various industries by providing innovative solutions and improving efficiency. However, it is essential to approach these systems with a critical perspective and acknowledge their limitations. Thomas noted that agentic systems will never be able to predict the effectiveness of their actions with absolute certainty, but they can still provide valuable insights and discoveries.

Nick suggested that agentic systems may eventually reach a level of proficiency where they can discover interesting insights and solutions. However, this would depend on the quality of the input data and the expertise of those who design these systems. The focus should be on creating systems that can learn from their mistakes and adapt over time.

Conclusion

The future of agentic systems holds much promise for innovation and problem-solving. However, it is essential to approach these systems with caution and responsibility. By acknowledging the limitations and challenges of agentic systems, we can harness their full potential while avoiding unnecessary risks. Human expertise plays a critical role in navigating these complex systems, and it is essential to strike a balance between computational power and human ingenuity.

"WEBVTTKind: captionsLanguage: enhell hello everyone and Welcome to our pen ultimate session of the day at data Camp radar I am so excited for this one on what the future holds for jar of AI everyone do give us a lot of love in the Emojis we want to see it let us know where you're joining from in the chat everyone give Thomas Ido Nick a lot of love today because they're going to be sharing a lot of excellent insights I've already been enjoying the behind the scenes conversation quite a lot so we're in for a treat today you know there are various Ways by which we can look at the Jan of AI ecosystem today uh and its future um you know from how the technical landscape will evolve to how llms themselves will improve to the infrastructure to the adoption the future of work and you know how J of AI in general will really change society and Tech space as a whole in the future so luckily today's guests all come at the J of AI you know come from the J of AI ecosystem but from different angles and it will be awesome to pick their brains and see how they view the future of AI so first let me introduce Ido Liberty Edo Liberty is the founder and CEO of pine con the managed Vector database that helps build knowledgeable AI systems he is also the former head of Amazon AI labs and the research director at Yahoo he Ido is key in building Amazon Sage maker if we have any fans of Amazon Sage maker let us know in the chat and has computered to over a 100 academic papers propelling a AI research forward his company pine cone was founded in 2019 and is now valued at $750 million that's not bad at all Ido it's great to see you it's great to see you I would if that valuation is right but other than that it was good and next we have Thomas tongos he is the general partner at Theory Venture Thomas has been a venture capitalist for more than 15 years before founding theory he has worked with companies including looker Monte Carlo mother duck omniex arbitrum and MLPs quite a few well-known names in the data nii space and he writes at tush.com which I highly recommend that you check out his blog one of my favorite reads of the week Tom it's great to see you pleasure to be here thanks for having me on awesome and last but not least is Nick elrin CEO of domino data lab as co-founder and CEO seems like we lost nick uh so I think we're having some technical issues will be no worries power outage yeah so Nick is the CEO of domino data lab as the co-founder and CEO of domino Nick had a front row seat to the transformative impact that Dominus platform has Unleashed for its customers Nick started domino in 2014 after spending nearly a decade building platforms for data scientists at Bridgewater the world's largest hedge fund since then Nick has built Domino into an industry leader whose ongoing Innovations continuously reshaped the standard for da data science and AI platforms he holds a ba and a MERS in computer science from Harvard Nick it's great to have you on nice to be here thanks so just a few housekeeping notes there will be time for Q&A at the end so make sure to use the Q&A feature before asking before asking any questions if you have any other messages or want to engage in the conversation use the chat feature if you want a network add your fellow attendees on LinkedIn we highly recommend joining our LinkedIn group any LinkedIn posts will be Auto removed immediately just to make sure that the chat is high signal to noise ratio so let's talk about the state of a generative AI today you know we've been talking about this behind the scenes but I'm very excited to pick your thoughts here you know all of you approach J of AI from different angles you know Thomas you're investing in the space Ed do you work at pine pine you're literally building the infrastructure that is powering a lot of Jen of AI use cases today and Nick You're Building tooling that makes building and managing AI models much easier so maybe let's set the stage for this conversation how do you view the current state of generative AI today Tom I'll start with you well I think we're at a really exciting time you look at the model performance is rapidly improving where uh small medium and large language models are you know there's a there's a score called the mlu which is high school equivalency and uh all those models are performing pretty well the Llama model you have open source and closed source that are both thriving with open Ai and anthropic being closed and then mistol and then a llama family of models being open source there's a lot lot of innovation at the infrastructure layer you know even the chips right we're seeing pretty meaningful improvements Nvidia with h200 you have AMD launching chips that are 30% more powerful than the current h100s and then even startups that have architectures that profess to beat 10 times faster so there's a lot of innovation at the infrastructure layer and I think as that evolves and improves there's a parallel desire from businesses to really take advantage of this there's dominant feeling that this is a big wave for good reason and so we see a lot of innovation a lot of Desire like Enterprise search is a major use case asking questions about a knowledge base customer support is a major use case applications and legal accounting code completion those are probably the five most active areas of investment today and then in terms of the future where we're going is making sure that these models are accurate over multi-step processes and once we're able to unlock that then I think you'll actually see some massive productivity improvements because you we we will be able to delegate tasks that that take a lot of time and have a computer do it instead of us okay that's great and Ido You Know You're Building infrastructure to power a lot of ji tools let us know how you see the state of ji today um yeah I I shared Thomas's uh uh tases I don't know I'm not sure how I'm pronouncing your name in is uh in the right way too much am I is this the you are it's an sh at the end it's kind of you to ask thanks uh the uh yeah I mean the space of course is rapidly evolving and so on I think we see a few different Trends uh uh I think the biggest one is Enterprise adoption which sort of like started in you know fits and hiccups last year and I think this year people really started to uh invest deeply in that and for that they their models and their AI in general to be knowledgeable and trustworthy and dependable and governed and like they sort of like the excitement of the new technology kind of hits the the rocks of reality of what it means to be able to actually work in in an Enterprise uh I think we see a lot of that uh and at the same time I mean I share the the excitement I mean there's there's a ton of improvements I think agents and assistants are going to be very big um we're going to improve a lot of them with with more knowledge from Vector the bases and other uh you know improved models and better mechanisms and so on um so yeah there's a ton coming and there a lot of energy and uh desire to build on those tools so so we're definitely going to impact a lot of that and maybe Nick from your perspective walk us through how you view the state of J ofi yeah um well maybe I'll be a little more contrarian or or a little less sanguin than toas and um and Ido you know I amum look that the Technology Innovation is very exciting from from my perspective working with customers and most of our customers are are large large Global Enterprises you know I'd say we're my take is we're still kind of in the inflated expectations period of the hype cycle and and what I mean by that is there's a lot of prototyping there's a lot of experimentation um but most of the real production use not all of them but most of them we're seeing are still sort of internally facing like Tomas said um you know Enterprise search question and answer chat Bots and look like that's that's fine um but I don't you know that stuff is not um world changing and where I've seen companies really get excited about initial ideas for um products based on J and I that would change a business model that would really be customer facing they have exciting prototypes but they can't get across the Finish Line because there's still too many issues around accuracy around around around safety or around Roi frankly and so so um you know I think look I I think that maybe we're jumping jumping a little ahead but but I I think there's a huge opportunity for for Gen to impact um Services heavy businesses uh you know Consulting businesses um graphic design agencies um but but I think that for a lot of other businesses I I worry actually that the the infatuation with Gen has distracted Business Leaders from the benefits the potential of traditional machine predictive machine learning traditional AI um and I I think that I think that in some ways to sort of set us back on the potential for all AI to really transform businesses and have positive impact on the world so that yeah like I said maybe maybe a bit more provocative take but that's how I see things right now yeah let maybe Tom and Ido react to that like so how do you view kind of the adoption landscape today so I'll let maybe Ido first I'll let you take that uh sure I agree 100% that there is a ton of experimentation and most of the energy in this field is spent with understanding how to run these things uh how to make your applications first of all perform what you want them to perform be governed be Dependable be not not make silly and embarrassing mistakes and so on uh and that's hard and that's hard and I agree with Nick 100% that some people might have unrealistic expectations from AI uh that said um I do think that a very large set of new kinds of workloads is now possible that was not possible with traditional machine learning and or whatever I would call them like predictive models whose job is mainly to score and predict and rank and and and do stuff that we do very well today already and and I agree with NI 100% are incredibly valuable and people should invest in them but the ability to consume language and generate language in a in a meaningful way is incredibly powerful I think yes of course a lot of people don't know how to commercialize it yet within their business and they're experimenting but I think a lot of them already know what they want to do and already know that this is the answer now the question is how to build it and and how to get there and you know when can we launch this okay Tom I'll let you react as well yeah I agree I think we're still early really people are still trying to understand exactly what's going on what the Technologies are good at what they're not good at think there are security concerns around data loss uh which uh and there's this Dynamic where historically the Chief Information Security Officer the ciso has been the one that's been responsible for securing these uh companies and now the heads of data heads of engineering a bit thrust into this role so figuring out how does that work I mean even like the procurement processes for some of these Technologies are really tough legal teams are needing to create new rules about well you know how do we think about software and there's been 20 or 30 years you have a sock two compliant to this procurement process now all of a sudden there's this thing where can a vendor use my company's data to train a model is that valuable what do we want the data that's on and inside of EPC or on Prem is that something that's really important to us and so I think even there's like the technology diffusion curve like what does this technology do and that's slowing adoption and there's the who's responsible for securing it and then there's the how do we get this through procurement in a way where nobody's fired and so those three things those three Dynamics I think have um people are still companies organizations are still figuring it out out mhm okay that's really interesting and you know how do you see you know you mentioned here these challenges uh Tom um U maybe from your perspective you see technical challenges as well that are hindering adoption today in the Enterprise yeah um I I would say there are three main challenges and uh not surprisingly this there's are three big uh Focus areas for us as well uh one is the adoption of the technology itself you have a lot of Engineers and and systems uh systems and and application builders that uh don't haven't had the experience of data scientists or data engineerings or ml Engineers uh and they are now interested with building those things so they need to catch up and and learn very quickly and we're trying to do a very good job at at both educating and giving really good tooling and making everything work out of the gate the second thing is people need to actually make the applications more knowledgeable and perform better so it's not just you know if if you could build something by just you know hooking up to an llm great you're done but most applications don't work that way they need to do support they need to know about your documents they need to know about what a processes they need to connect to your own data somehow uh and so that means you have to somehow provide them with knowledge and that is very difficult today Vector databases in Pine con specifically make it a lot easier but it's not enough we have launched a syst today uh today I think or this week uh that uh allows people just load document and run basically uh text queries and and so on and answer questions and complete tasks with their own data uh but that's still a challenge for a lot of people who don't use pinec con and the last thing really you know underscores what toar said it's not enough you can build an application that will blow the C coo like socks off but you would still need to be socks compliant and Hippa and and gdpr and you know have jump through all the fiery hoops and a lot of them are actually complicated questions you know and so it's it's still going to be hard uh so you know this Enterprise Readiness is I think a big uh area of of uh friction for buyers and investment for companies who work in the space okay that's great and maybe I think it's time as well that we can discuss you know outside of like uh challenges in deployment let's maybe switch gears and discuss the technical trends that we see within J today um you know we've seen llms become multimodal uh and llm providers like improve their different product experiences you think gp40 with the voice mode and data analysis mode CLA 3.5 with artifacts these are great you know uxui Innovations as well um however we haven't seen necessarily like a leap in intelligence comparable to the transition from gpt2 to gpt3 gpt3 to GPT 3.5 and GPT 3.5 to gb4 right do you think that we'll see this other massive leap in intelligence soon if so what do you think will drive this sleep if not why not Nick I'll start with you here um well you know actually you know I was gonna I was GNA yield or or seed my time to answer to to maybe you know I I'm I'm not as I'm not I don't have the most expertise on sort of the LM architecture details and so he might have a better answer um but yoube before he jumps into that I'll just say my my prediction would be that regardless of how some of those next wave of leaps play out I think we're going to see a lot more um juice being squeezed from different ways of combining multiple models together so more agentic workflows I know that's that's very a lot of people are talking about that right now and I think that's also going to enable uh kind of harness better harnessing or extracting more value from um smaller more specialized models and I think we'll get more um more system AI systems that are using a lot of more Specialized or small models um potentially likely in in sort of agentic patterns um to to improve overall performance of these systems but um anyway to answer your question directly I I don't feel like I have the best expertise so let's switch gears then EO I'll let you take that on um I'll offer a contrarian view that I actually don't models are going just the current way we think about models are not going to improve dramatically in way will qualitatively change what can be done with them okay uh for multiple reasons the like physics kicks in at some point on how these things are trained there is the economics of doing that you know when model training a top-notch model was quote unquote only $10 million which was way out for any tiny company but a lot of medium and large companies have that capacity when you need to invest a billion dollars to train a model suddenly you this is this is really kept to a very very small set of players um and so and for reasons that you know I you know it's going to be take a long time to go through I I personally think we're reaching a level of saturation that uh it's going to be hard to qualitatively change and yes improvements but to me Product wise if some accuracy measures for goes from 80 to 85% it might be a tremendous engineering F feat and and it could be a big difference but it's not a qualitative change you didn't go from you know 15 to 80 and you didn't go from 80 to 100 yeah okay and so yes it's a better model but is it qualitatively different I I don't think not in terms of the applications and use cases what is going to change uh is that we are going to make these models a lot more Dynamic knowledgeable plugged into the right places and find ways to make them uh basically improve the ways they train and make them 10 to 100 times bigger uh and that's happening but it's not the models you know and the the way you think about a language model it's going to be completely different like so a qual a qualitative change has to happen for that curve to break basically in my opinion okay so models are going to become more useful but not necessarily raw intelligence wise they're going to have a simple a jump in no no they they will but uh well it's it's hard to explain I might I might have to stop here but I will say that I don't think the the models the way they are trained today uh are I think hitting an an asmt an ASM toote uh that we're not going to improve much on okay and Tom I'll let you also react to here how how do you view the next wave of lmms yeah I think there's a bifurcation you'll have large language models that will cost like a billion or 10 billion and you'll have a handful of companies that are training them and I suspect they'll probably be government subsidies for training some of these massive massive models because they're of national importance um and then you'll have smaller models that are purpose-built that are much more accurate that are running on your mobile phone that are much more and so you have this basically B foration and they'll be used for different use cases you know query will come in there'll be a classifier that says have I seen this kind of query before and if I have it probably goes to a small language model to optimize for a particular task and if it hasn't goes to a large language model it's basically a big consumer search index effectively and so I think that's where we're going we call these things constellations of models we're starting to see enterpris has actually developed some pretty sophisticated layers on top there's a caching layers in order to reduce cost because if let's say you were working on a translation software company saying you know the word for hot dog in Russian doesn't change and so you don't need to go back to the index every time you can just cash it uh and so I you know that's that's the way that I see it I think you know multimodal is kind of the I think the two big waves of the day is multimodal meaning use text to produce video or vice versa or learn from video and use that to improve text models and then a genic systems which Nick and Ido have talked about already which is basically chaining these things together to act behalf of humans the multimodal Technologies are new I think you can see it with clouds artifacts starting to work people creating three-dimensional models of uh partical physics with just a handful of sentences and then it's rendering it's pretty cool uh and then on the agentic stuff I think the compounding error rates through multi-step processes is really hard to manage today and there are a lots of different architectures there there are suggestor critiques there's adversarial networks that people are starting to use there's the use of a suggestion with a classic classifier in order to minimize the error and that's all research today it's uh it's very very early yeah and one thing I think that is kind of uh we discussed this behind the scenes right which is you know um are we reaching Peak saturation to training data for example as well like when it comes to kind of training the next generation of models so and you hinted at this Edo like saturation of training and like saturation of models today uh maybe walk us through the your your kind of thinking here a bit as well in more depth I'd love to learn more how you see the next wave training will happen what do what what needs to be true to get incremental improvements on models well I can't talk about it because we're working on it but so Tom maybe I'll let you in th ear as well yeah sorry I missed a question what was it yeah so do you think kind of we're going to reach saturation on training data when it comes for uh models today and so what is the way out for training models yeah I think well okay so llama 3 a billion parameter was really interesting there so let me take a step back the way to think about training one way of thinking about training is there's a cost optimization function here which is how much data do I need and how many compute hours do I need to train a model and there was a paper produced called chinchilla which created a heuristic an algorithm to determine exactly what that was and meta decided on the Llama 38 billion parameter to go Way Beyond it where they spent they trained it on 15 trillion tokens and the performance there was actually was very expensive to do but the performance was really good especially compared to larger larger models and so I think we're sort of figuring out how do we squeeze as much juice from the lemons that we have and exploring that surface area we're probably in a local maximum the the other dynamic in you know Ido and I were talking about it before that llama model was trained on about 15 trillion tokens R estim is SAR with like 20 roughly 20 trillion tokens on the internet and the question is okay well we want to train a model on like 100 trillion tokens or 500 trillion tokens where does that come from and I think that's that's TBD nobody knows right it could video it could be content that's yet to be created it could be synthetic data where computers are actually identifying where models a week and then producing data to supplement it so I don't have a great answer for you okay so I think we'll find out and then maybe if we kind of look at you know Nick you mentioned this earlier today earlier in the conversation that you know we're still in experimentation phase for a lot of tools and you know but all three of you hinted at kind of the uh deployment challenges with j AI Solutions today you know a lot of the infrastructure technology to uh enable fast deployment of LM think monitoring debugging retraining systems all of that uh still needs to be built um so how do view maybe the maturity of the llm middleware ecosystem today or like the jant of AI middleware ecosystem today so Nick maybe I'll start with you here as well yeah I think it's similarly early I mean and the other thing I would say is I don't I think um I think a mistake people are making is thinking it's um conceptually a completely different thing from like let's say the traditional mlops challenges we've all we've had over the last five ten years so I think you know I hear I hear some people think well llms and geni make it so we don't we don't have mlops problems anymore it's a different set of problems and rather I think it's more like the the geni use cases take all the same problems we had with traditional mlops but they just they just exacerbate them or now now it's like if we're playing a video game we've activated the hard mode um so you know you have you have a lot more um a lot more intensive compute resource requirements infrastructure requirements uh you it's gpus it's not just CPUs you need to scale these things that comes with a whole bunch more costs you need to deal with um I mean we have customers that have have gone from deploying again for Prototype use cases traditional predictive ml to now gen and all of a sudden their compute bills are going out of control and now they they before they didn't need to worry about things like elastic Auto scaling for inference now they do because the gpus cost a lot more um you you know you talk about talk about model governance model monitoring um uh that's all based on the the fundamental characteristic of any sort of predictive AI system that um that they're probabilistic and so their behavior can they can have unexpected failure modes their behaviors can change um you know or their their performance can change S the world around them changes well gen has a much wider surface area of of unexpected failure modes than say a traditional a predictive ml model and so now the set of things you need to do to check and guard rail and the set of um human human processes that you need to put these things through as you move them through a sort of a quality assurance process that's all much more complicated so yeah I mean look the the sum it all up it's like all the same problems we we had with mlops are now um incremented or exponentially harder for for Gen um and I would just add I think I think a lot of these challenges are not necess you ask about middleware I'd say a lot of these challenges are not um strictly technical they are they are business process challenges they are people and process orchestration challenges uh if if we're going to put a gen model or system into production in an in an Enterprise that might touch customers especially in a regulated environment a bank an insurance company um a life sciences company uh what what checks have been what checks have been executed to ensure that is compliant that is safe these are um everyone that we work with is still trying to figure that out okay great and then do from your perspective you're investing in this space right like you have a very elev you know bird's eye view over the space I'd love to kind of see how you see the kind of infrastructure space for llms evolving today well there's I mean lots of exciting things happening right you know's building a great business Vector databases are really important we're investors in a company called um superlink which is a called a vector computer which allows you to take structured and unstructured data and put it together uh in a unique way uh and so I think where're you know but it's really early in this ecosystem where as we talked about buyers are still trying to figure figure out exactly what it is that they want to build and until they know what that is it's hard to have a view on exactly what the ideal architecture is then you have you know open Ai and the other big companies also deciding strategically what areas of infrastructure do they want to play with right so open AI released an open source evaluations framework um to help the software Engineers understand exactly how well a model is performing before they release a little bit like a testing harness and that was a move for them into a category where there are a lot of startups already so it's you know I guess for startups I guess the way I'd put it is you're dancing with elephants they're these big companies who really care about these markets Microsoft has A5 billion do run rate business in 18 months open AI north of three billion and and you have others that are growing pretty fast and they're still deciding what parts of the stack they want to play with you look at um uh you know open ai's acquisition of Rocket last week I think and Y open questions about what did they do with that technology and that team so the chess the chess board is not set there's uh there's still a lot of moves to be played and uh and you have to navigate I think startups have to do what they do best which is be nibble and navigate the ecosystem as it's rapidly forming yeah brilliant and Ido you know you're building a company that's at the center of the infrastructure space and llms uh you know walks through how you see this space evolving over the next year and I don't want to I don't want to hear like product road map that's not the question but I would love to see kind of how you think the the ecosystem will evolve in the year to come um I agree with toas 100% I mean this is the it's a very Dynamic space uh and um I think that the um there's going to be one truism that's going to still uh Prevail here which is uh uh companies and Enterprises specifically are going to find AI more production ready and useful uh very soon uh we're working on it others are working on it the the elephants in the room are working on it it's it's obvious that this needs to happen uh and so exactly who does what part of the stack and who gets to own what piece of the pie I think is is anybody's guess we of course you know want to have the biggest piece that we can own out of that it's obvious it's like any other uh but um I you know I'll be intellectually honest and say that nobody nobody can really tell yeah and I think this Segway to my next question really well because everyone's working on this problem of deployment making sure that you know models are very productive you know uh you you know uh I think Nick mentioned earlier uh that we're at the peak of uh inflated expectations uh but we will reach the slope of Enlightenment at one point in time here right and once we reach critical mass of adoption uh if J of AI you know reaches critical mass in terms of adoption how do you think it will shape the nature of work or you know the what does that future look like so maybe Thomas I'll start with you sorry just going off mute I think um you know the way that we think about it is there's lots of toil within work there are lots of repetitive actions that we do uh and each each discipline has its own right within sales there's the role in sales development of understanding leads and qualifying leads and deciding which ones to pursue within the world of legal there's paralal work which is the administration of documents within the world of accounting uh data entry across different kinds of tax forms that um have different formats and uh within the world of software engineering it's remembering which arguments go in which order in a function call and I think the fut of work with AI is having computers solve a lot of those rote tasks and work alongside of that's these co-pilot systems that we have and today the productivity gains at least from some of the early companies suggests it's about a 50 to a 75% productivity gain maybe that's a little bit aggressive um computers will not love each other that's true Luis uh but I think if we look to the Future where work is actually automated and we can delegate tasks to computers the way that we delegate tasks to humans the best analogy is looking at automobile manufacturing lines where robots have replaced a lot of the the labor and there you see about a 275% Improvement in productivity where one robot takes about uh can produce about the output of 2.7 humans and I think I think we should be able to get close that's not based on anything except a rough guess but I think you know order of magnitude we should be able to get to a place where a lot of this toil this rot work that uh is unappealing is uh is automated right you 100 years ago in America there were four million human dishwashers people who wash dishes for restaurants wow okay and then the first robot introduced in most people's houses was a was a dishwasher and now we don't you know but the chores that we give to our children is loading the dishwasher instead of washing the dishes I think that's that's a very good analogy for future of work and yeah that's it's very very fascinating and Ido I'll let you here also react how do you see you know once pine cone takes the biggest part of the pie as much as possible how do you see the view uh the future of workare look I mean I think we're producing great tools um there are are um I think we're very far from any model or set of models or agents completely replacing somebody's job I think that's that's that's very unlikely unless your job is like so minial and and just whatever like except for extreme cases okay all the lawyers and doctors and and accountants and analysts and this and that they're not going away what they're are getting is fantastic new tools that make them uh work more efficiently more correctly you know uh offload the grunt work that maybe they they don't want to do um and that that's going to happen I mean I'm I'm I'm 100% sure of that and history shows that when you have something like this that actually improves overall prod productivity for society which ends up being good for everyone okay great and then with productivity you know when the Paradigm changes for productivity generating Technologies you mentioned this Thomas uh the the chores one used to give to their kids was washing the dishes now it's loading the dishwasher that skill is CH has changed right the skill from washing dishes to loading dishwasher what do you think the skills needed like what would your advice be for individuals today to build the skills needed to kind of uh adapt to an Era where J of AI is widely adopted so Nick I'll start with you on that question yeah well I think I mean building on what tamas and both said um if if gen plays out the way everyone's expectations hope it will it's going to be leveraged for people it's not going to replace people and so in any given industry or any given job someone has today there are a set of things that can be delegated away but the there's going to be a Core Essence of that job that is the Insight part or the creative part um or the architecture part that is going to be that is going to remain um UND delegatable to to one of these AI systems and so you know I think yes if you're a software engineer then you don't need to remember the order of the arguments to a function call but you do need to be able to understand a customer requirement and and based on that understanding design a software architecture that appropriately reflects the abstractions you need to to model the customer's problem domain and you know that so so the the the conceptual or more in in Insight based parts of any job are going to be increasingly important you know if you're a if you're a a graphic artist um you'll still need to be able to have a picture of something in your head that you want then you can you can interact with you can coach you can instruct um an image generation system to to create that but it'll be it'll be getting a computer to create what you are picturing in your mind's eye um so maybe the the technique of how do I use Photoshop to you know to to um to compose this thing I'm picturing maybe that will will go away um and and you know and just combining that thought with know your your question a minute ago about what this all means for the future work the interesting opport the interesting potential Dynamic I am thinking about is um what will this mean for a whole wave of disruption that flows across every Services industry so if you're a consulting firm if you're a graphic design firm if your if your business is fundamentally a Services business I think that you are at risk over the next you know five years of being disrupted by a much more efficient much more scalable much more High leverage model of executing that business okay that's great and Tom I'll let you also react what do you think the future skills to to adapt are well I remember when all the search engines came out right Google metaweb Yahoo alter Vista and you needed to figure out exactly how to use them well and the people who could use them well were a lot more productive at work um and so that's where we are so I think prompt engineering is probably the skill of the day today MH uh because if you can do it really well you you can produce a blog post you can produce marketing copy you can produce images you can produce code and those marginal benefits if you're 50% more productive than a cooworker or competitor you will be promoted and uh that's sort of the name of the game right I think all software is sold because it's a promotion in a different form somebody buys the software and champions it because they can be promoted so I think prompt engineering is the thing that or the skill that is the most broadly applicable there was this awesome study of I think BCG or Mackenzie one of the big Consulting companies studied the way that yeah it was called centors and cyborgs and one of the most interesting use cases there was asking the AI to simulate a potential software buyer and be the um be the foil in a conversation we're seeing this in education too where uh students are creating their own personalities talk about how to solve a calculus problem or a chemistry problem set so I I think that's that's yeah that's probably the broadest and most applicable use case today okay great and then EO I'll let you finish us off with you know how do you view the the state of skills in an era where adoption has has matured um it's very hard to it's very hard to to to know to be honest and I I might not have the the best uh like a very uh strongly held opinion on exactly what's that's going to look like I remember also so when search engines came out like people imagin those sorts of professions that ended up not existing and um and so on so I think prophecy was is on on this topic is is a bit hard I think it's it's obvious though that uh this is a like a set of very powerful tools are going to be produced and professionals who don't Master those set of tools are going to be left behind uh you know if you're you know if you're that lawyer that rejected the use of a PC and kept using your uh you know you know whatnot yeah you you didn't you didn't uh stick around for a long time I couldn't agree more and I think we have time for one question to take from the audience and I'm going to take this one because I think everyone here hinted at the um you know potential Improvement of agentic system in the future so maybe I'll ask it here just looking for it do you see here it is do you think that we might see agentic approaches increasing in effective in Effectiveness to the point that it reaches the level of predictive so for example can you reach a point where agentic systems are so good that you can predict with AC accurately that they will work well most of the time are we headed to that type of future because you know Thomas you mentioned the compounding error rate across the different steps of the agentic workflow I'd love to see where you think agentic systems are headed in the near future so let start with you maybe Tom yeah I think uh we okay so I think let's put a different way I think we'll get to a place where we will Brute Force innovation in other words like uh we'll kick off massively parallel jobs that are trying to figure out what is the next Generation chemical compound and it's not that the systems will predict it's that we will just throw so many computers at it that at some point in time they will s they will find a solution and so um and we'll just you know it'll discover huge solution space and one of the answers will be there I I don't believe computers can predict very effectively um and if you look at classical machine learning they've had a I mean they've had a really hard time they can predict things that are very stable and um and repetitive and cyclical and Google made a really beautiful business model based on very predictable patterns but for things like core Innovation or creation that is not just Rec combination is just not the right technology to tell that okay Nick I see you're nodding an agreement I'd love to see your thoughts here as well oh yeah I was just gonna agree with Tas about the um you know brute forcing Innovation there's a um a hedge fund we work with that's doing that exact thing using gen to to come up with candidate investment algorithms and then of course it's easy to or not you know you can you can test and check each one and their their point of view is look if we generate 10,000 and 900 or you know 9,999 or bad ideas but we get one good one that's worth it but I think I think the that Bruth Force Innovation you only get the ROI for these areas where the innovation has a big payoff and you can kind of justify the investment to to search a big space because yeah the vast majority of the ideas would be bad Ido I'll let you finish this off here how do you Fe how do you see agentic systems evolving um well I'm I'm both in support and violent disagreement uh with I mean I think brute forcing doesn't work uh exponents tend to be uh exponential uh and anybody who wrote like a five nested f for Loop knows that it gets pretty terrible pretty quickly um and so um yeah I I don't think this is solved by just throwing more Hardware at it for the love of God we've been throwing literally billions dollar billions of dollars worth of energy on this thing and it you know we didn't solve everything so I don't think that's the way to go I I do think that some Innovation comes from those system that then agenic systems can actually discover interesting things because humans also can produce exponential amount of compute in in our own heads and the people who do research are also you know energy bound and so you know whether we do this in Brute Force I don't think so whether it actually produces interesting insights in Innovation I think 100% okay I think this is a great place to end our chat I want to make sure everyone send as much love as possible to our speakers today thank you so much Thomas thank you so much Ido Nick especially Ido and Nick are both on vacation in Greece and Italy respectively so I really appreciate them making time out of their uh you know uh precious vacation times as As Leaders of their organization so I really really appreciate your times and thank you so much for everyone who attended and again huge round of applause for our speakers and see you on the last session on our closing session thank you all so much thank youhell hello everyone and Welcome to our pen ultimate session of the day at data Camp radar I am so excited for this one on what the future holds for jar of AI everyone do give us a lot of love in the Emojis we want to see it let us know where you're joining from in the chat everyone give Thomas Ido Nick a lot of love today because they're going to be sharing a lot of excellent insights I've already been enjoying the behind the scenes conversation quite a lot so we're in for a treat today you know there are various Ways by which we can look at the Jan of AI ecosystem today uh and its future um you know from how the technical landscape will evolve to how llms themselves will improve to the infrastructure to the adoption the future of work and you know how J of AI in general will really change society and Tech space as a whole in the future so luckily today's guests all come at the J of AI you know come from the J of AI ecosystem but from different angles and it will be awesome to pick their brains and see how they view the future of AI so first let me introduce Ido Liberty Edo Liberty is the founder and CEO of pine con the managed Vector database that helps build knowledgeable AI systems he is also the former head of Amazon AI labs and the research director at Yahoo he Ido is key in building Amazon Sage maker if we have any fans of Amazon Sage maker let us know in the chat and has computered to over a 100 academic papers propelling a AI research forward his company pine cone was founded in 2019 and is now valued at $750 million that's not bad at all Ido it's great to see you it's great to see you I would if that valuation is right but other than that it was good and next we have Thomas tongos he is the general partner at Theory Venture Thomas has been a venture capitalist for more than 15 years before founding theory he has worked with companies including looker Monte Carlo mother duck omniex arbitrum and MLPs quite a few well-known names in the data nii space and he writes at tush.com which I highly recommend that you check out his blog one of my favorite reads of the week Tom it's great to see you pleasure to be here thanks for having me on awesome and last but not least is Nick elrin CEO of domino data lab as co-founder and CEO seems like we lost nick uh so I think we're having some technical issues will be no worries power outage yeah so Nick is the CEO of domino data lab as the co-founder and CEO of domino Nick had a front row seat to the transformative impact that Dominus platform has Unleashed for its customers Nick started domino in 2014 after spending nearly a decade building platforms for data scientists at Bridgewater the world's largest hedge fund since then Nick has built Domino into an industry leader whose ongoing Innovations continuously reshaped the standard for da data science and AI platforms he holds a ba and a MERS in computer science from Harvard Nick it's great to have you on nice to be here thanks so just a few housekeeping notes there will be time for Q&A at the end so make sure to use the Q&A feature before asking before asking any questions if you have any other messages or want to engage in the conversation use the chat feature if you want a network add your fellow attendees on LinkedIn we highly recommend joining our LinkedIn group any LinkedIn posts will be Auto removed immediately just to make sure that the chat is high signal to noise ratio so let's talk about the state of a generative AI today you know we've been talking about this behind the scenes but I'm very excited to pick your thoughts here you know all of you approach J of AI from different angles you know Thomas you're investing in the space Ed do you work at pine pine you're literally building the infrastructure that is powering a lot of Jen of AI use cases today and Nick You're Building tooling that makes building and managing AI models much easier so maybe let's set the stage for this conversation how do you view the current state of generative AI today Tom I'll start with you well I think we're at a really exciting time you look at the model performance is rapidly improving where uh small medium and large language models are you know there's a there's a score called the mlu which is high school equivalency and uh all those models are performing pretty well the Llama model you have open source and closed source that are both thriving with open Ai and anthropic being closed and then mistol and then a llama family of models being open source there's a lot lot of innovation at the infrastructure layer you know even the chips right we're seeing pretty meaningful improvements Nvidia with h200 you have AMD launching chips that are 30% more powerful than the current h100s and then even startups that have architectures that profess to beat 10 times faster so there's a lot of innovation at the infrastructure layer and I think as that evolves and improves there's a parallel desire from businesses to really take advantage of this there's dominant feeling that this is a big wave for good reason and so we see a lot of innovation a lot of Desire like Enterprise search is a major use case asking questions about a knowledge base customer support is a major use case applications and legal accounting code completion those are probably the five most active areas of investment today and then in terms of the future where we're going is making sure that these models are accurate over multi-step processes and once we're able to unlock that then I think you'll actually see some massive productivity improvements because you we we will be able to delegate tasks that that take a lot of time and have a computer do it instead of us okay that's great and Ido You Know You're Building infrastructure to power a lot of ji tools let us know how you see the state of ji today um yeah I I shared Thomas's uh uh tases I don't know I'm not sure how I'm pronouncing your name in is uh in the right way too much am I is this the you are it's an sh at the end it's kind of you to ask thanks uh the uh yeah I mean the space of course is rapidly evolving and so on I think we see a few different Trends uh uh I think the biggest one is Enterprise adoption which sort of like started in you know fits and hiccups last year and I think this year people really started to uh invest deeply in that and for that they their models and their AI in general to be knowledgeable and trustworthy and dependable and governed and like they sort of like the excitement of the new technology kind of hits the the rocks of reality of what it means to be able to actually work in in an Enterprise uh I think we see a lot of that uh and at the same time I mean I share the the excitement I mean there's there's a ton of improvements I think agents and assistants are going to be very big um we're going to improve a lot of them with with more knowledge from Vector the bases and other uh you know improved models and better mechanisms and so on um so yeah there's a ton coming and there a lot of energy and uh desire to build on those tools so so we're definitely going to impact a lot of that and maybe Nick from your perspective walk us through how you view the state of J ofi yeah um well maybe I'll be a little more contrarian or or a little less sanguin than toas and um and Ido you know I amum look that the Technology Innovation is very exciting from from my perspective working with customers and most of our customers are are large large Global Enterprises you know I'd say we're my take is we're still kind of in the inflated expectations period of the hype cycle and and what I mean by that is there's a lot of prototyping there's a lot of experimentation um but most of the real production use not all of them but most of them we're seeing are still sort of internally facing like Tomas said um you know Enterprise search question and answer chat Bots and look like that's that's fine um but I don't you know that stuff is not um world changing and where I've seen companies really get excited about initial ideas for um products based on J and I that would change a business model that would really be customer facing they have exciting prototypes but they can't get across the Finish Line because there's still too many issues around accuracy around around around safety or around Roi frankly and so so um you know I think look I I think that maybe we're jumping jumping a little ahead but but I I think there's a huge opportunity for for Gen to impact um Services heavy businesses uh you know Consulting businesses um graphic design agencies um but but I think that for a lot of other businesses I I worry actually that the the infatuation with Gen has distracted Business Leaders from the benefits the potential of traditional machine predictive machine learning traditional AI um and I I think that I think that in some ways to sort of set us back on the potential for all AI to really transform businesses and have positive impact on the world so that yeah like I said maybe maybe a bit more provocative take but that's how I see things right now yeah let maybe Tom and Ido react to that like so how do you view kind of the adoption landscape today so I'll let maybe Ido first I'll let you take that uh sure I agree 100% that there is a ton of experimentation and most of the energy in this field is spent with understanding how to run these things uh how to make your applications first of all perform what you want them to perform be governed be Dependable be not not make silly and embarrassing mistakes and so on uh and that's hard and that's hard and I agree with Nick 100% that some people might have unrealistic expectations from AI uh that said um I do think that a very large set of new kinds of workloads is now possible that was not possible with traditional machine learning and or whatever I would call them like predictive models whose job is mainly to score and predict and rank and and and do stuff that we do very well today already and and I agree with NI 100% are incredibly valuable and people should invest in them but the ability to consume language and generate language in a in a meaningful way is incredibly powerful I think yes of course a lot of people don't know how to commercialize it yet within their business and they're experimenting but I think a lot of them already know what they want to do and already know that this is the answer now the question is how to build it and and how to get there and you know when can we launch this okay Tom I'll let you react as well yeah I agree I think we're still early really people are still trying to understand exactly what's going on what the Technologies are good at what they're not good at think there are security concerns around data loss uh which uh and there's this Dynamic where historically the Chief Information Security Officer the ciso has been the one that's been responsible for securing these uh companies and now the heads of data heads of engineering a bit thrust into this role so figuring out how does that work I mean even like the procurement processes for some of these Technologies are really tough legal teams are needing to create new rules about well you know how do we think about software and there's been 20 or 30 years you have a sock two compliant to this procurement process now all of a sudden there's this thing where can a vendor use my company's data to train a model is that valuable what do we want the data that's on and inside of EPC or on Prem is that something that's really important to us and so I think even there's like the technology diffusion curve like what does this technology do and that's slowing adoption and there's the who's responsible for securing it and then there's the how do we get this through procurement in a way where nobody's fired and so those three things those three Dynamics I think have um people are still companies organizations are still figuring it out out mhm okay that's really interesting and you know how do you see you know you mentioned here these challenges uh Tom um U maybe from your perspective you see technical challenges as well that are hindering adoption today in the Enterprise yeah um I I would say there are three main challenges and uh not surprisingly this there's are three big uh Focus areas for us as well uh one is the adoption of the technology itself you have a lot of Engineers and and systems uh systems and and application builders that uh don't haven't had the experience of data scientists or data engineerings or ml Engineers uh and they are now interested with building those things so they need to catch up and and learn very quickly and we're trying to do a very good job at at both educating and giving really good tooling and making everything work out of the gate the second thing is people need to actually make the applications more knowledgeable and perform better so it's not just you know if if you could build something by just you know hooking up to an llm great you're done but most applications don't work that way they need to do support they need to know about your documents they need to know about what a processes they need to connect to your own data somehow uh and so that means you have to somehow provide them with knowledge and that is very difficult today Vector databases in Pine con specifically make it a lot easier but it's not enough we have launched a syst today uh today I think or this week uh that uh allows people just load document and run basically uh text queries and and so on and answer questions and complete tasks with their own data uh but that's still a challenge for a lot of people who don't use pinec con and the last thing really you know underscores what toar said it's not enough you can build an application that will blow the C coo like socks off but you would still need to be socks compliant and Hippa and and gdpr and you know have jump through all the fiery hoops and a lot of them are actually complicated questions you know and so it's it's still going to be hard uh so you know this Enterprise Readiness is I think a big uh area of of uh friction for buyers and investment for companies who work in the space okay that's great and maybe I think it's time as well that we can discuss you know outside of like uh challenges in deployment let's maybe switch gears and discuss the technical trends that we see within J today um you know we've seen llms become multimodal uh and llm providers like improve their different product experiences you think gp40 with the voice mode and data analysis mode CLA 3.5 with artifacts these are great you know uxui Innovations as well um however we haven't seen necessarily like a leap in intelligence comparable to the transition from gpt2 to gpt3 gpt3 to GPT 3.5 and GPT 3.5 to gb4 right do you think that we'll see this other massive leap in intelligence soon if so what do you think will drive this sleep if not why not Nick I'll start with you here um well you know actually you know I was gonna I was GNA yield or or seed my time to answer to to maybe you know I I'm I'm not as I'm not I don't have the most expertise on sort of the LM architecture details and so he might have a better answer um but yoube before he jumps into that I'll just say my my prediction would be that regardless of how some of those next wave of leaps play out I think we're going to see a lot more um juice being squeezed from different ways of combining multiple models together so more agentic workflows I know that's that's very a lot of people are talking about that right now and I think that's also going to enable uh kind of harness better harnessing or extracting more value from um smaller more specialized models and I think we'll get more um more system AI systems that are using a lot of more Specialized or small models um potentially likely in in sort of agentic patterns um to to improve overall performance of these systems but um anyway to answer your question directly I I don't feel like I have the best expertise so let's switch gears then EO I'll let you take that on um I'll offer a contrarian view that I actually don't models are going just the current way we think about models are not going to improve dramatically in way will qualitatively change what can be done with them okay uh for multiple reasons the like physics kicks in at some point on how these things are trained there is the economics of doing that you know when model training a top-notch model was quote unquote only $10 million which was way out for any tiny company but a lot of medium and large companies have that capacity when you need to invest a billion dollars to train a model suddenly you this is this is really kept to a very very small set of players um and so and for reasons that you know I you know it's going to be take a long time to go through I I personally think we're reaching a level of saturation that uh it's going to be hard to qualitatively change and yes improvements but to me Product wise if some accuracy measures for goes from 80 to 85% it might be a tremendous engineering F feat and and it could be a big difference but it's not a qualitative change you didn't go from you know 15 to 80 and you didn't go from 80 to 100 yeah okay and so yes it's a better model but is it qualitatively different I I don't think not in terms of the applications and use cases what is going to change uh is that we are going to make these models a lot more Dynamic knowledgeable plugged into the right places and find ways to make them uh basically improve the ways they train and make them 10 to 100 times bigger uh and that's happening but it's not the models you know and the the way you think about a language model it's going to be completely different like so a qual a qualitative change has to happen for that curve to break basically in my opinion okay so models are going to become more useful but not necessarily raw intelligence wise they're going to have a simple a jump in no no they they will but uh well it's it's hard to explain I might I might have to stop here but I will say that I don't think the the models the way they are trained today uh are I think hitting an an asmt an ASM toote uh that we're not going to improve much on okay and Tom I'll let you also react to here how how do you view the next wave of lmms yeah I think there's a bifurcation you'll have large language models that will cost like a billion or 10 billion and you'll have a handful of companies that are training them and I suspect they'll probably be government subsidies for training some of these massive massive models because they're of national importance um and then you'll have smaller models that are purpose-built that are much more accurate that are running on your mobile phone that are much more and so you have this basically B foration and they'll be used for different use cases you know query will come in there'll be a classifier that says have I seen this kind of query before and if I have it probably goes to a small language model to optimize for a particular task and if it hasn't goes to a large language model it's basically a big consumer search index effectively and so I think that's where we're going we call these things constellations of models we're starting to see enterpris has actually developed some pretty sophisticated layers on top there's a caching layers in order to reduce cost because if let's say you were working on a translation software company saying you know the word for hot dog in Russian doesn't change and so you don't need to go back to the index every time you can just cash it uh and so I you know that's that's the way that I see it I think you know multimodal is kind of the I think the two big waves of the day is multimodal meaning use text to produce video or vice versa or learn from video and use that to improve text models and then a genic systems which Nick and Ido have talked about already which is basically chaining these things together to act behalf of humans the multimodal Technologies are new I think you can see it with clouds artifacts starting to work people creating three-dimensional models of uh partical physics with just a handful of sentences and then it's rendering it's pretty cool uh and then on the agentic stuff I think the compounding error rates through multi-step processes is really hard to manage today and there are a lots of different architectures there there are suggestor critiques there's adversarial networks that people are starting to use there's the use of a suggestion with a classic classifier in order to minimize the error and that's all research today it's uh it's very very early yeah and one thing I think that is kind of uh we discussed this behind the scenes right which is you know um are we reaching Peak saturation to training data for example as well like when it comes to kind of training the next generation of models so and you hinted at this Edo like saturation of training and like saturation of models today uh maybe walk us through the your your kind of thinking here a bit as well in more depth I'd love to learn more how you see the next wave training will happen what do what what needs to be true to get incremental improvements on models well I can't talk about it because we're working on it but so Tom maybe I'll let you in th ear as well yeah sorry I missed a question what was it yeah so do you think kind of we're going to reach saturation on training data when it comes for uh models today and so what is the way out for training models yeah I think well okay so llama 3 a billion parameter was really interesting there so let me take a step back the way to think about training one way of thinking about training is there's a cost optimization function here which is how much data do I need and how many compute hours do I need to train a model and there was a paper produced called chinchilla which created a heuristic an algorithm to determine exactly what that was and meta decided on the Llama 38 billion parameter to go Way Beyond it where they spent they trained it on 15 trillion tokens and the performance there was actually was very expensive to do but the performance was really good especially compared to larger larger models and so I think we're sort of figuring out how do we squeeze as much juice from the lemons that we have and exploring that surface area we're probably in a local maximum the the other dynamic in you know Ido and I were talking about it before that llama model was trained on about 15 trillion tokens R estim is SAR with like 20 roughly 20 trillion tokens on the internet and the question is okay well we want to train a model on like 100 trillion tokens or 500 trillion tokens where does that come from and I think that's that's TBD nobody knows right it could video it could be content that's yet to be created it could be synthetic data where computers are actually identifying where models a week and then producing data to supplement it so I don't have a great answer for you okay so I think we'll find out and then maybe if we kind of look at you know Nick you mentioned this earlier today earlier in the conversation that you know we're still in experimentation phase for a lot of tools and you know but all three of you hinted at kind of the uh deployment challenges with j AI Solutions today you know a lot of the infrastructure technology to uh enable fast deployment of LM think monitoring debugging retraining systems all of that uh still needs to be built um so how do view maybe the maturity of the llm middleware ecosystem today or like the jant of AI middleware ecosystem today so Nick maybe I'll start with you here as well yeah I think it's similarly early I mean and the other thing I would say is I don't I think um I think a mistake people are making is thinking it's um conceptually a completely different thing from like let's say the traditional mlops challenges we've all we've had over the last five ten years so I think you know I hear I hear some people think well llms and geni make it so we don't we don't have mlops problems anymore it's a different set of problems and rather I think it's more like the the geni use cases take all the same problems we had with traditional mlops but they just they just exacerbate them or now now it's like if we're playing a video game we've activated the hard mode um so you know you have you have a lot more um a lot more intensive compute resource requirements infrastructure requirements uh you it's gpus it's not just CPUs you need to scale these things that comes with a whole bunch more costs you need to deal with um I mean we have customers that have have gone from deploying again for Prototype use cases traditional predictive ml to now gen and all of a sudden their compute bills are going out of control and now they they before they didn't need to worry about things like elastic Auto scaling for inference now they do because the gpus cost a lot more um you you know you talk about talk about model governance model monitoring um uh that's all based on the the fundamental characteristic of any sort of predictive AI system that um that they're probabilistic and so their behavior can they can have unexpected failure modes their behaviors can change um you know or their their performance can change S the world around them changes well gen has a much wider surface area of of unexpected failure modes than say a traditional a predictive ml model and so now the set of things you need to do to check and guard rail and the set of um human human processes that you need to put these things through as you move them through a sort of a quality assurance process that's all much more complicated so yeah I mean look the the sum it all up it's like all the same problems we we had with mlops are now um incremented or exponentially harder for for Gen um and I would just add I think I think a lot of these challenges are not necess you ask about middleware I'd say a lot of these challenges are not um strictly technical they are they are business process challenges they are people and process orchestration challenges uh if if we're going to put a gen model or system into production in an in an Enterprise that might touch customers especially in a regulated environment a bank an insurance company um a life sciences company uh what what checks have been what checks have been executed to ensure that is compliant that is safe these are um everyone that we work with is still trying to figure that out okay great and then do from your perspective you're investing in this space right like you have a very elev you know bird's eye view over the space I'd love to kind of see how you see the kind of infrastructure space for llms evolving today well there's I mean lots of exciting things happening right you know's building a great business Vector databases are really important we're investors in a company called um superlink which is a called a vector computer which allows you to take structured and unstructured data and put it together uh in a unique way uh and so I think where're you know but it's really early in this ecosystem where as we talked about buyers are still trying to figure figure out exactly what it is that they want to build and until they know what that is it's hard to have a view on exactly what the ideal architecture is then you have you know open Ai and the other big companies also deciding strategically what areas of infrastructure do they want to play with right so open AI released an open source evaluations framework um to help the software Engineers understand exactly how well a model is performing before they release a little bit like a testing harness and that was a move for them into a category where there are a lot of startups already so it's you know I guess for startups I guess the way I'd put it is you're dancing with elephants they're these big companies who really care about these markets Microsoft has A5 billion do run rate business in 18 months open AI north of three billion and and you have others that are growing pretty fast and they're still deciding what parts of the stack they want to play with you look at um uh you know open ai's acquisition of Rocket last week I think and Y open questions about what did they do with that technology and that team so the chess the chess board is not set there's uh there's still a lot of moves to be played and uh and you have to navigate I think startups have to do what they do best which is be nibble and navigate the ecosystem as it's rapidly forming yeah brilliant and Ido you know you're building a company that's at the center of the infrastructure space and llms uh you know walks through how you see this space evolving over the next year and I don't want to I don't want to hear like product road map that's not the question but I would love to see kind of how you think the the ecosystem will evolve in the year to come um I agree with toas 100% I mean this is the it's a very Dynamic space uh and um I think that the um there's going to be one truism that's going to still uh Prevail here which is uh uh companies and Enterprises specifically are going to find AI more production ready and useful uh very soon uh we're working on it others are working on it the the elephants in the room are working on it it's it's obvious that this needs to happen uh and so exactly who does what part of the stack and who gets to own what piece of the pie I think is is anybody's guess we of course you know want to have the biggest piece that we can own out of that it's obvious it's like any other uh but um I you know I'll be intellectually honest and say that nobody nobody can really tell yeah and I think this Segway to my next question really well because everyone's working on this problem of deployment making sure that you know models are very productive you know uh you you know uh I think Nick mentioned earlier uh that we're at the peak of uh inflated expectations uh but we will reach the slope of Enlightenment at one point in time here right and once we reach critical mass of adoption uh if J of AI you know reaches critical mass in terms of adoption how do you think it will shape the nature of work or you know the what does that future look like so maybe Thomas I'll start with you sorry just going off mute I think um you know the way that we think about it is there's lots of toil within work there are lots of repetitive actions that we do uh and each each discipline has its own right within sales there's the role in sales development of understanding leads and qualifying leads and deciding which ones to pursue within the world of legal there's paralal work which is the administration of documents within the world of accounting uh data entry across different kinds of tax forms that um have different formats and uh within the world of software engineering it's remembering which arguments go in which order in a function call and I think the fut of work with AI is having computers solve a lot of those rote tasks and work alongside of that's these co-pilot systems that we have and today the productivity gains at least from some of the early companies suggests it's about a 50 to a 75% productivity gain maybe that's a little bit aggressive um computers will not love each other that's true Luis uh but I think if we look to the Future where work is actually automated and we can delegate tasks to computers the way that we delegate tasks to humans the best analogy is looking at automobile manufacturing lines where robots have replaced a lot of the the labor and there you see about a 275% Improvement in productivity where one robot takes about uh can produce about the output of 2.7 humans and I think I think we should be able to get close that's not based on anything except a rough guess but I think you know order of magnitude we should be able to get to a place where a lot of this toil this rot work that uh is unappealing is uh is automated right you 100 years ago in America there were four million human dishwashers people who wash dishes for restaurants wow okay and then the first robot introduced in most people's houses was a was a dishwasher and now we don't you know but the chores that we give to our children is loading the dishwasher instead of washing the dishes I think that's that's a very good analogy for future of work and yeah that's it's very very fascinating and Ido I'll let you here also react how do you see you know once pine cone takes the biggest part of the pie as much as possible how do you see the view uh the future of workare look I mean I think we're producing great tools um there are are um I think we're very far from any model or set of models or agents completely replacing somebody's job I think that's that's that's very unlikely unless your job is like so minial and and just whatever like except for extreme cases okay all the lawyers and doctors and and accountants and analysts and this and that they're not going away what they're are getting is fantastic new tools that make them uh work more efficiently more correctly you know uh offload the grunt work that maybe they they don't want to do um and that that's going to happen I mean I'm I'm I'm 100% sure of that and history shows that when you have something like this that actually improves overall prod productivity for society which ends up being good for everyone okay great and then with productivity you know when the Paradigm changes for productivity generating Technologies you mentioned this Thomas uh the the chores one used to give to their kids was washing the dishes now it's loading the dishwasher that skill is CH has changed right the skill from washing dishes to loading dishwasher what do you think the skills needed like what would your advice be for individuals today to build the skills needed to kind of uh adapt to an Era where J of AI is widely adopted so Nick I'll start with you on that question yeah well I think I mean building on what tamas and both said um if if gen plays out the way everyone's expectations hope it will it's going to be leveraged for people it's not going to replace people and so in any given industry or any given job someone has today there are a set of things that can be delegated away but the there's going to be a Core Essence of that job that is the Insight part or the creative part um or the architecture part that is going to be that is going to remain um UND delegatable to to one of these AI systems and so you know I think yes if you're a software engineer then you don't need to remember the order of the arguments to a function call but you do need to be able to understand a customer requirement and and based on that understanding design a software architecture that appropriately reflects the abstractions you need to to model the customer's problem domain and you know that so so the the the conceptual or more in in Insight based parts of any job are going to be increasingly important you know if you're a if you're a a graphic artist um you'll still need to be able to have a picture of something in your head that you want then you can you can interact with you can coach you can instruct um an image generation system to to create that but it'll be it'll be getting a computer to create what you are picturing in your mind's eye um so maybe the the technique of how do I use Photoshop to you know to to um to compose this thing I'm picturing maybe that will will go away um and and you know and just combining that thought with know your your question a minute ago about what this all means for the future work the interesting opport the interesting potential Dynamic I am thinking about is um what will this mean for a whole wave of disruption that flows across every Services industry so if you're a consulting firm if you're a graphic design firm if your if your business is fundamentally a Services business I think that you are at risk over the next you know five years of being disrupted by a much more efficient much more scalable much more High leverage model of executing that business okay that's great and Tom I'll let you also react what do you think the future skills to to adapt are well I remember when all the search engines came out right Google metaweb Yahoo alter Vista and you needed to figure out exactly how to use them well and the people who could use them well were a lot more productive at work um and so that's where we are so I think prompt engineering is probably the skill of the day today MH uh because if you can do it really well you you can produce a blog post you can produce marketing copy you can produce images you can produce code and those marginal benefits if you're 50% more productive than a cooworker or competitor you will be promoted and uh that's sort of the name of the game right I think all software is sold because it's a promotion in a different form somebody buys the software and champions it because they can be promoted so I think prompt engineering is the thing that or the skill that is the most broadly applicable there was this awesome study of I think BCG or Mackenzie one of the big Consulting companies studied the way that yeah it was called centors and cyborgs and one of the most interesting use cases there was asking the AI to simulate a potential software buyer and be the um be the foil in a conversation we're seeing this in education too where uh students are creating their own personalities talk about how to solve a calculus problem or a chemistry problem set so I I think that's that's yeah that's probably the broadest and most applicable use case today okay great and then EO I'll let you finish us off with you know how do you view the the state of skills in an era where adoption has has matured um it's very hard to it's very hard to to to know to be honest and I I might not have the the best uh like a very uh strongly held opinion on exactly what's that's going to look like I remember also so when search engines came out like people imagin those sorts of professions that ended up not existing and um and so on so I think prophecy was is on on this topic is is a bit hard I think it's it's obvious though that uh this is a like a set of very powerful tools are going to be produced and professionals who don't Master those set of tools are going to be left behind uh you know if you're you know if you're that lawyer that rejected the use of a PC and kept using your uh you know you know whatnot yeah you you didn't you didn't uh stick around for a long time I couldn't agree more and I think we have time for one question to take from the audience and I'm going to take this one because I think everyone here hinted at the um you know potential Improvement of agentic system in the future so maybe I'll ask it here just looking for it do you see here it is do you think that we might see agentic approaches increasing in effective in Effectiveness to the point that it reaches the level of predictive so for example can you reach a point where agentic systems are so good that you can predict with AC accurately that they will work well most of the time are we headed to that type of future because you know Thomas you mentioned the compounding error rate across the different steps of the agentic workflow I'd love to see where you think agentic systems are headed in the near future so let start with you maybe Tom yeah I think uh we okay so I think let's put a different way I think we'll get to a place where we will Brute Force innovation in other words like uh we'll kick off massively parallel jobs that are trying to figure out what is the next Generation chemical compound and it's not that the systems will predict it's that we will just throw so many computers at it that at some point in time they will s they will find a solution and so um and we'll just you know it'll discover huge solution space and one of the answers will be there I I don't believe computers can predict very effectively um and if you look at classical machine learning they've had a I mean they've had a really hard time they can predict things that are very stable and um and repetitive and cyclical and Google made a really beautiful business model based on very predictable patterns but for things like core Innovation or creation that is not just Rec combination is just not the right technology to tell that okay Nick I see you're nodding an agreement I'd love to see your thoughts here as well oh yeah I was just gonna agree with Tas about the um you know brute forcing Innovation there's a um a hedge fund we work with that's doing that exact thing using gen to to come up with candidate investment algorithms and then of course it's easy to or not you know you can you can test and check each one and their their point of view is look if we generate 10,000 and 900 or you know 9,999 or bad ideas but we get one good one that's worth it but I think I think the that Bruth Force Innovation you only get the ROI for these areas where the innovation has a big payoff and you can kind of justify the investment to to search a big space because yeah the vast majority of the ideas would be bad Ido I'll let you finish this off here how do you Fe how do you see agentic systems evolving um well I'm I'm both in support and violent disagreement uh with I mean I think brute forcing doesn't work uh exponents tend to be uh exponential uh and anybody who wrote like a five nested f for Loop knows that it gets pretty terrible pretty quickly um and so um yeah I I don't think this is solved by just throwing more Hardware at it for the love of God we've been throwing literally billions dollar billions of dollars worth of energy on this thing and it you know we didn't solve everything so I don't think that's the way to go I I do think that some Innovation comes from those system that then agenic systems can actually discover interesting things because humans also can produce exponential amount of compute in in our own heads and the people who do research are also you know energy bound and so you know whether we do this in Brute Force I don't think so whether it actually produces interesting insights in Innovation I think 100% okay I think this is a great place to end our chat I want to make sure everyone send as much love as possible to our speakers today thank you so much Thomas thank you so much Ido Nick especially Ido and Nick are both on vacation in Greece and Italy respectively so I really appreciate them making time out of their uh you know uh precious vacation times as As Leaders of their organization so I really really appreciate your times and thank you so much for everyone who attended and again huge round of applause for our speakers and see you on the last session on our closing session thank you all so much thank you\n"