The Challenges and Opportunities of Building Intelligent Models in Finance

Attribution issues can be a significant challenge when it comes to building intelligent models in finance. When a model makes an incorrect prediction, it can be difficult to determine where exactly the mistake occurred. As one expert noted, "You actually don't know kind of what is driving the model's decision to produce the outcome that it produces." This lack of transparency can make it difficult to understand why a particular model performed poorly.

To address this challenge, many firms have turned to using ensemble methods, which combine the predictions of multiple models to produce a single prediction. However, even with ensembles, it can be difficult to understand what's driving the model's decision. One expert explained that "the models that we generally rely on tend to have a way to sort of reverse engineer why why it made the decision it made." This is in contrast to deep neural networks, which don't provide the same level of transparency.

To overcome these challenges, some firms are turning to more interpretable models, such as decision trees. These models can provide a clear understanding of what's driving the prediction, and can be used to identify areas where the model may have gone wrong. One firm is using this approach to build hybrid models that combine the strengths of multiple different techniques.

Another key challenge in building intelligent models in finance is overfitting. When a model is too complex, it can fit the noise in the data rather than the underlying patterns, leading to poor performance on new, unseen data. To avoid this, many firms are using techniques such as regularization and early stopping to prevent overfitting.

One firm has developed a hybrid approach that uses ensemble methods to combine the predictions of multiple models, while also incorporating more interpretable models to provide transparency into the decision-making process. This approach allows the firm to build models that are both accurate and interpretable, which is essential for making informed investment decisions.

The firm's ultimate goal is to develop an intelligent agent that can understand human intent and context, and can adapt its response accordingly. For example, a machine learning model may be able to provide investment advice for a sophisticated investor, but not for someone who is less experienced. By developing models that can adapt to different contexts, the firm hopes to create a more inclusive and user-friendly platform.

For those interested in learning more about this approach, there are several resources available. The firm's website is a great place to start, as it provides detailed information on their methodology and approaches. Additionally, individual investors or those interested in using these signals can access them for free on Quantopian, with no restrictions until they wish to incorporate them into a production strategy.

The company behind this technology is Alphavantage Vertex, which has developed a range of tools and resources for building intelligent models in finance. The firm's CEO was happy to share their approach and insights with our audience, and we were grateful for the opportunity to learn from them. By exploring the challenges and opportunities of building intelligent models in finance, we can gain a deeper understanding of how these technologies can be used to inform investment decisions and drive business success.

"WEBVTTKind: captionsLanguage: enall right so once again I am at uh NYU future Labs with the AI Nexus companies and this time I'm with mutisia inunda uh the co-founder and CEO of alpha vertex say hi hi uh thanks uh thanks for having me uh uh so why don't we get started by having you uh tell us a little Vex and uh what you guys are up to sure um so Alpha vertex is an Innovative uh Financial technology company that is using the next generation of AI tools to help support investing uh around the world uh practically what that means is that we develop predictive models that try to forecast the returns of stocks over multiple times Horizons okay and at present uh we cover all major markets around the world so we cover 30,000 stocks globally which represent about 95% of the investable universe okay and so presumably you came out of financial services prior to this yeah so my background is uh in finance I've spent over 17 years I started out in Investment Banking did uh private equity and uh ended up as the chief of staff of one of the largest propri retary Market making businesses in the world at a company called cesana International Group uh and then after leaving that I spent 5 years at Bloomberg as the head of strategy and business development for Bloomberg Enterprise Solutions okay so I've kind of done my Tour of Duty around Wall Street and around Wall Street Service you've got quite a few punches on your ticket um you know it strikes me that of all the companies that I've talked to uh here at uh axus today the um you know the trying to help people make better trades with you know financial data is you know there's nothing new about right that is a that is a classical idea and lots of people you know have tried to apply you know we've been trying to apply analytics to this for many many years what is new and different about your approach to it so um I'll first just start by sort of saying you're absolutely right that it is not a new idea because it's not a new problem but U the present moment there's about 98% of all funds underperform their Benchmark over 10 years and the simple reason is that they rely on kind of traditional portfolio managers to act as stock Pickers and uh what happens there is that you have maybe 1% of all fund managers are kind of really good at making money but then the 99% of the population uh ends up underperforming and so you're subject to kind of the talent that you bring in uh as financial markets get more and more complicated uh you know there's about 2.5 billion bytes of data nowadays that people kind of have to keep track of it's increasingly difficult for a human being to sort of incorporate all of this information in a useful way into the investment strategy so with the onset of you know Innovations in machine learning and uh AI in general uh you know we're able to build models uh that can capture all of these pieces of information much similar to the but butterfly effect so if you think about uh we have ai models that sort of will see things that are happening in Asia and then model how they Ripple through Global markets to affect a stock like Apple or Google in the US uh it's really really challenging to be able to do that as a human being or to build a model that can can can capture that complexity uh so that's sort of what we're doing that's completely different is we have these very sophisticated models that can kind of look out to 10 or you know 20 degrees of separation away and analyze how things that are going on very in very distant places can kind of ultimately come back to to impact stocks uh that people so the the I mean certainly the hedge funds have been trying to apply this type of stuff for a very long time as part of what you're doing bringing the you know the Quant type of Technology the hedge funds have been doing to the traditional uh investors the the institutional side yeah so we you know interestingly enough the earliest adopters of our technology are obviously the quantitative of systematic hedge funds they kind of understand how powerful this technology is and you know they employ datadriven Technology based investment strategies and so this to them is a high value signal uh that is additive to maybe other things that they do in a on a proprietary basis uh ultimately over the long run what we want to do is make these tools kind of more broadly accessible to the 99% of the investment managers out there that don't have you know this technology driven investment platforms so we're working on things like having you know kind of machine learning driven institutional class research and analytics facilities that you know can be accessed via natural language so you don't have to have the technical capability of a data scientist or a statistician or a PhD person to be able to do the types of analysis uh that we're we're going to be developing um so I think you know in the in the near term the earli adopters are the more Quant driven hedge funds but over the long run uh we want to go after everybody in the space okay so you guys are um you it's it's not so much arming the folks that can't keep up with the Quant it's you guys have develop some really cool stuff that the quants need too absolutely okay and so um um thinking about how one might try to build uh machine learning AI fly effect that sounds super complex like how do you what's the what's the approach there so it is actually a pretty huge problem um the first thing is you have to try to represent all the world's financial information in a way that makes sense oh well that's easy uh yeah so that's um so the approach that we've taken is we've modeled everything as a computational graph and for people who aren't familiar with that it's literally this multi-dimensional relationship map of how everything is interconnected to everything else so you can describe things uh like apple competes with Samsung apple is a customer of Samsungs and apple is suing Samsung and all of these are relationships between these two nodes or these two entities uh in a traditional model you can't represent these multi dimensional relationships across companies uh and in our case like in the case of where apple is suing Samsung Apple the relationship has a Direction so apple is instigating the this the legal lawsuit on Samsung and we can encode the value of the law suit how long it's been going on and then you can actually do computations off of all of these uh connections between companies uh but to do this you have to monitor all the world's information not just financial information uh so to be able to just do that we monitor over a million pieces of information in a given day and these are across all types of sources so we look at structured data sets like Financial Market data we look at uh unstructured data sets such as uh regulatory filings would say the SEC the FDA whatever uh what companies are saying in press releases news articles we read about 300,000 sources of news in a given day and and uh one of the things we had to build early on was this ability for the machine to understand the news on its own extract relationships that it thought were meaningful and then also weigh the quality of that source of information so not all things are Rel are created equal if I read something on a Blog in Asia uh it might be actually a really high quality Source or it may not be and so you know you have to make these autonomous machines have the capability to sort of wear the quality of the source of that information uh and once it's extracted you know and you're putting it into this computational graph you know it has to say I think this is actually a really good fact uh or if it's not then it says it's not a high quality fact and I need to wait on confirming or additional sources of you know to confirm this relationship that I just found uh and that's just the starting point right so we cover uh I guess 880,000 publicly traded companies around the world we monitor 15 million private companies and we track about 200 million people that work at these companies as a starting point and then from there we try to figure out all their products who they do business with whether they Contracting with other companies or with the government uh any kind of litigation that may be going on who their investors are and what their investors are doing so you know it's a spiderweb of knowledge that we have to build uh before we can actually do computation right right to your uh there's there's so much in there like even just the the issue of trying to identify the um the newsworthiness the trust level associated with a given site I mean that's Google right it's page rank right are you using something similar to page rank in um yes but no so um I mean Google's an amazing uh piece of technology but you know again with learning and a lot of things that we do the domain knowledge is extremely important so Google doesn't have page rank for finance right right so we kind of have to build that ourselves uh the good thing is I guess Google tries to index the world we just need to index the financial world so it's a subset of what uh Google does uh but we can take a lot of the same Concepts from what Google does and incorporate that into our own algorithms uh one of the things that we've done is we we apply like this machine learning approach to teaching things whether or not this is a high quality source so we can give the AI good examples of what high high quality you know like this is the Wall Street Journal this is a high quality Source these are the types of relationships or pieces of information that come from this Source versus here's a you know like a no-name Blog and then the machine can kind of extrapolate from that so it's a it's a supervised learning problem as opposed to go out and index all of these Financial sites and figure out which ones are high quality the way Google yeah cuz yeah Google basically has boiled the ocean ahead of time uh and we kind of when we come across new stuff we make a determination at that point in time right right um that's super interesting tell us about some of the other interesting kind of machine learning problems that you yeah um so yeah so the the other thing that is actually really really hard to do is kind of twofold the first one is entity resolution so you're reading all of this information about people places companies whatever uh the way things are written is not necessarily always clear and so you might be reading something about Apple but it's the fruit and not the company and you have to be able to disambiguate that uh so we do a lot of really cool stuff in the entity resolution space and then we've started to do some really exciting things that were in the domain of intelligence services but I think kind of the technology is leaking out now in the sense of something called record linkage so if you have these giant data sets that you can onboard or acquire or you know create yourself uh what typically happens is that something may call Apple Apple in one data set it may call it Apple Inc in a different one or it might just be a reference to the CEO of Apple you know so you need to be able to link all of these data sets together uh and basically you don't have a clear key on which to join everything so you have to make these calculated sort of uh you know similar you us to basically say this thing is likely this other disambiguating correct and connecting things uh based off of you know you say is this in the same location or you know does it is it referencing a product of this company so you know and then basically be able to say well actually it is talking about Apple but it was actually talking about the iPhone you know so basically merging like say the iPhone to the to the Apple company uh without ever having a NE necessarily mention of Apple in both data sets right so record linkage is actually really really really hard uh We've started to do a lot of that and it's it's a really fascinating kind of area in the Big Data world how do you attack that what technology approaches are you applying to so um obviously the first one is we have to be able to within the data extract kind of attributes or features that we think are important for the links that we're trying to establish so we might say what are we trying to link in the first place data you know so if we go and grab every patent in the US for companies and that's just one giant data dump uh linking that back to well all these patents are actually owned by these companies uh you know the US government patent office might not have the same names as you know the the publicly traded companies themselves uh so you have to look at things like is the address the same the people mentioned as the patent holders employees of these companies so you looking for Bodies of Evidence that suggest that these two things are kind of mapped together uh and you have different algorithms that can kind of give you a score and a confidence measure as to how closely two records are to each other um so that's one and then another one is uh I think called relationship extraction so if you read uh a piece of news as a human being it's very easy for you to sort of boil it down to what it means right so you know this company is buying another company right uh but in the real world that might be written out over three paragraphs and so for a computer uh to sort of establish that entity a is acquiring entity B after reading a paragraph or two is actually a pretty challenging problem um and so yeah there's a lot of really interesting stuff that we use there but most of it is in the Deep learning uh you know kind of kind of space so we're using a lot of tens oflow models that you know to try to understand their representation of language extract kind of entities and then the relationships that we're we're looking for so do these end up looking like um you know uh single or individual like super you know very deep uh neural networks or are they many neural networks that are each um serving their own purposes question yeah so we what we've realized is that uh we've had better results with building specialized machine learning models a neural networks uh as opposed to one kind of generalized AI or generalized Network and so even with our relationship extraction what we do is we have thousands of models that kind of work in unison and each one is highly specialized in in in one thing and so if you think about having a very wide funnel at the beginning where you say this is a raw document uh the first sort of gen set of models will basically say this document is actually talking about this concept and then that gets filtered into to okay so with this within this this concept these are the types of relationships I should be looking for and then have more sophisticated models kind of take that information and then say okay fine so this is about a corporation and corporations can have m&a as a relationship and so I kind of I'm seeing a very strong signal that this document in general is talking about m&a Let me give it to the most sophisticated guy to go and extract that very specific relationship and by sophisticated guy we talking about a person or another model another model sorry yeah um and so you we have this kind of pipeline of AI models that kind of work together uh that start really Broad and then end up uh kind of very spe hypers specialized at the end and are you doing any human in the loop anywhere or is it all we do but it's um there's a bit of reinforcement learning that goes on especially with the record linkage stuff um and then we obviously retrain our models fairly frequently and so we'll have the human in the loop on the retraining okay of the models but on a g on a given day everything's kind of running in an autonomous R that's super interesting so and when you talked about the identifying the relationships and how you have the you know you can have the three paragraphs to get to um you know that there was an acquisition that makes me think of like you know CFO speak right do you have a model that can train that is trying to like decipher when a c when a CFO is saying that they're going to miss say actually it's an area of interest to us we haven't yet done any kind of real hardcore development there but there is actually a huge community of users in the kind of you know hedge font space that want to understand tone not even just the text but like the tone uh if there was video of the guy making the statement or he darting around uh so wow uh there is a lot of of kind of interest in that we just haven't yet had a chance to spend their time in that space But it's clearly something in know okay and so where are you guys in the process um so we've launched uh a product about I guess less than two months now called precog uh which is our like it's a predictive service that basically tries to measure the butterfly effect and then produce uh short-term for forecasts for the returns of these 30,000 stocks around the world uh that is uh currently in production and we have a number of clients that we Service uh so meaning you give it uh you know a company and you probably have some unique company thing will just go into its database and tell you you know you you give it maybe say 30 days and it'll tell you a projected stock price in 30 days is it yes that yeah it's along those lines so what we do today is we produce predictions on specific time Horizons so it would be like a onee forecast two we forecast or one month forecast okay uh but it would be kind of a rolling day forecast for those Horizons so today being Monday will'll give you a forecast for the return of the stock not the price but the return of the stock by4 in 30 days okay um and that you know again if you do this systematically over a broad Universe of stocks and you sort of you know you're never going to get 100% accurac which is impossible uh but you you have this winning prediction you know like right now we think we get anywhere from 60 to almost 70% accurate uh in depending uh on the horizon and and and the stocks themselves but uh 60 to 70 is kind of the average but across 30,000 names that is like Vegas odds right so if you're the casino you may lose a hand to one player uh you know one table but across all players you're systematically making money and so uh the customers that are using our predictions want want them on a very large Universe of stocks and then they obviously not betting the farm on one prediction on one day they they're making this is one signal that they're using out of a portfolio of signals uh So you you're obviously gaining you're able to make a prediction at day Zero and then you know at days 17 14 and 30 you're getting additional signals as to you know how accurately your model is performing how do you uh then use that signal to retrain and then how do you deal with um like attribution issues you know you were off by you know 50% or even plus minus like how where in your model did you go wrong have you started to figure look at yeah we do um one of the things that we've tried to stay away shy away from just kind of with our models is that we've tried to avoid using deep neural Nets and all of these types of things when we're doing doing our forecasting and the reason being uh with these deep neural networks you actually don't know kind of what is driving the model's decision to produce the outcome that it produces uh so the models that we generally rely on tend to have a way to sort of reverse engineer why why it made the decision it made so more like decision trees or something else uh it's it's never one thing so we use kind of a hybrid approach where we will use an ensemble of Meth methods uh to come up with a single prediction okay but most of them uh have an ability where we can actually query the model and ask it what was the most important thing in the decision that you made okay uh and then those those those weights or those factors are things that we can kind of just you know sanity check in the markets to see if uh if it is what it was so you know we make say for financial stocks we think uh you know yields spreads and things of that nature should be important to financials but there may be a group of financials that are aren't moving with respect to interest rates uh so the model might rely on that but then you know in a month's time we could go back and say well why were you way off and they'll say well I overweighted you know this move in interest rate and we will try to retrain the model uh to make sure it doesn't do that uh the thing we try to do is we don't want to superfluously or just over engineer the models cuz you can end up with overfitting right so uh we're really careful about when we train the models and then what we give them to retrain you know and there's a lot of Arts and Science that goes into kind of being like you know 60% is good enough or not uh given everything that we know okay so it sounds like going back to your funnel analogy at the top of the funnel you're using a lot of deep learning to uh extract signal from various sources and then closer to the the to the end of the funnel you're using more um you're using models that have greater explainability absolutely that's cor okay interesting anything else that you'd like to share about what you guys are up to um I mean just kind of longterm um one of the things obviously we're trying to build like I said in the beginning was the capability for then you know non-technical users to be able to sort of surface some of these insights or even you know ask the discover new new facts that are kind of not obvious right within this knowledge base that we've built uh and then also be able to take advantage of some of these predictive models that we've building uh so what we're really trying to do here is develop you know like an ability for the machine to understand human intent uh and then also the context in which the intent is being asked so for example you sound fairly kind of familiar with the financial domain but like my grandmother may have a similar question to you but the response that the machine should give her versus you needs to be kind of adjusted for that so we're really looking to build an intelligent agent that would be able to work under different contexts right so if you're dealing with another professional investor kind of you know having a conversation at that level if you're dealing with somebody looking for financial advice who's not that sophisticated then you kind of want to boil down some of these things and just have you know them explained back to the user in a format that they can understand right so that ultimately is sort of where we're driving towards okay uh where can folks learn more about what you guys are doing um so on our website is a great place to start uh they can also check out uh at least for individual investors or people interested in sort of these these signals that we're producing they're available on quantopian uh so and they're free to use uh until you want to put them into a production strategy uh and they are available for 500 of the most liquid names in the United States okay and so the company website is alphav vertex.com alex. and uh we will be glad to uh you know reach out to them and just understand what their needs are awesome well thanks so much for being on the show I really uh learned a lot a lot about uh really learned a lot from what you guys are doing and it sounds really exciting yeah it was a pleasure thank you so much great thank you nall right so once again I am at uh NYU future Labs with the AI Nexus companies and this time I'm with mutisia inunda uh the co-founder and CEO of alpha vertex say hi hi uh thanks uh thanks for having me uh uh so why don't we get started by having you uh tell us a little Vex and uh what you guys are up to sure um so Alpha vertex is an Innovative uh Financial technology company that is using the next generation of AI tools to help support investing uh around the world uh practically what that means is that we develop predictive models that try to forecast the returns of stocks over multiple times Horizons okay and at present uh we cover all major markets around the world so we cover 30,000 stocks globally which represent about 95% of the investable universe okay and so presumably you came out of financial services prior to this yeah so my background is uh in finance I've spent over 17 years I started out in Investment Banking did uh private equity and uh ended up as the chief of staff of one of the largest propri retary Market making businesses in the world at a company called cesana International Group uh and then after leaving that I spent 5 years at Bloomberg as the head of strategy and business development for Bloomberg Enterprise Solutions okay so I've kind of done my Tour of Duty around Wall Street and around Wall Street Service you've got quite a few punches on your ticket um you know it strikes me that of all the companies that I've talked to uh here at uh axus today the um you know the trying to help people make better trades with you know financial data is you know there's nothing new about right that is a that is a classical idea and lots of people you know have tried to apply you know we've been trying to apply analytics to this for many many years what is new and different about your approach to it so um I'll first just start by sort of saying you're absolutely right that it is not a new idea because it's not a new problem but U the present moment there's about 98% of all funds underperform their Benchmark over 10 years and the simple reason is that they rely on kind of traditional portfolio managers to act as stock Pickers and uh what happens there is that you have maybe 1% of all fund managers are kind of really good at making money but then the 99% of the population uh ends up underperforming and so you're subject to kind of the talent that you bring in uh as financial markets get more and more complicated uh you know there's about 2.5 billion bytes of data nowadays that people kind of have to keep track of it's increasingly difficult for a human being to sort of incorporate all of this information in a useful way into the investment strategy so with the onset of you know Innovations in machine learning and uh AI in general uh you know we're able to build models uh that can capture all of these pieces of information much similar to the but butterfly effect so if you think about uh we have ai models that sort of will see things that are happening in Asia and then model how they Ripple through Global markets to affect a stock like Apple or Google in the US uh it's really really challenging to be able to do that as a human being or to build a model that can can can capture that complexity uh so that's sort of what we're doing that's completely different is we have these very sophisticated models that can kind of look out to 10 or you know 20 degrees of separation away and analyze how things that are going on very in very distant places can kind of ultimately come back to to impact stocks uh that people so the the I mean certainly the hedge funds have been trying to apply this type of stuff for a very long time as part of what you're doing bringing the you know the Quant type of Technology the hedge funds have been doing to the traditional uh investors the the institutional side yeah so we you know interestingly enough the earliest adopters of our technology are obviously the quantitative of systematic hedge funds they kind of understand how powerful this technology is and you know they employ datadriven Technology based investment strategies and so this to them is a high value signal uh that is additive to maybe other things that they do in a on a proprietary basis uh ultimately over the long run what we want to do is make these tools kind of more broadly accessible to the 99% of the investment managers out there that don't have you know this technology driven investment platforms so we're working on things like having you know kind of machine learning driven institutional class research and analytics facilities that you know can be accessed via natural language so you don't have to have the technical capability of a data scientist or a statistician or a PhD person to be able to do the types of analysis uh that we're we're going to be developing um so I think you know in the in the near term the earli adopters are the more Quant driven hedge funds but over the long run uh we want to go after everybody in the space okay so you guys are um you it's it's not so much arming the folks that can't keep up with the Quant it's you guys have develop some really cool stuff that the quants need too absolutely okay and so um um thinking about how one might try to build uh machine learning AI fly effect that sounds super complex like how do you what's the what's the approach there so it is actually a pretty huge problem um the first thing is you have to try to represent all the world's financial information in a way that makes sense oh well that's easy uh yeah so that's um so the approach that we've taken is we've modeled everything as a computational graph and for people who aren't familiar with that it's literally this multi-dimensional relationship map of how everything is interconnected to everything else so you can describe things uh like apple competes with Samsung apple is a customer of Samsungs and apple is suing Samsung and all of these are relationships between these two nodes or these two entities uh in a traditional model you can't represent these multi dimensional relationships across companies uh and in our case like in the case of where apple is suing Samsung Apple the relationship has a Direction so apple is instigating the this the legal lawsuit on Samsung and we can encode the value of the law suit how long it's been going on and then you can actually do computations off of all of these uh connections between companies uh but to do this you have to monitor all the world's information not just financial information uh so to be able to just do that we monitor over a million pieces of information in a given day and these are across all types of sources so we look at structured data sets like Financial Market data we look at uh unstructured data sets such as uh regulatory filings would say the SEC the FDA whatever uh what companies are saying in press releases news articles we read about 300,000 sources of news in a given day and and uh one of the things we had to build early on was this ability for the machine to understand the news on its own extract relationships that it thought were meaningful and then also weigh the quality of that source of information so not all things are Rel are created equal if I read something on a Blog in Asia uh it might be actually a really high quality Source or it may not be and so you know you have to make these autonomous machines have the capability to sort of wear the quality of the source of that information uh and once it's extracted you know and you're putting it into this computational graph you know it has to say I think this is actually a really good fact uh or if it's not then it says it's not a high quality fact and I need to wait on confirming or additional sources of you know to confirm this relationship that I just found uh and that's just the starting point right so we cover uh I guess 880,000 publicly traded companies around the world we monitor 15 million private companies and we track about 200 million people that work at these companies as a starting point and then from there we try to figure out all their products who they do business with whether they Contracting with other companies or with the government uh any kind of litigation that may be going on who their investors are and what their investors are doing so you know it's a spiderweb of knowledge that we have to build uh before we can actually do computation right right to your uh there's there's so much in there like even just the the issue of trying to identify the um the newsworthiness the trust level associated with a given site I mean that's Google right it's page rank right are you using something similar to page rank in um yes but no so um I mean Google's an amazing uh piece of technology but you know again with learning and a lot of things that we do the domain knowledge is extremely important so Google doesn't have page rank for finance right right so we kind of have to build that ourselves uh the good thing is I guess Google tries to index the world we just need to index the financial world so it's a subset of what uh Google does uh but we can take a lot of the same Concepts from what Google does and incorporate that into our own algorithms uh one of the things that we've done is we we apply like this machine learning approach to teaching things whether or not this is a high quality source so we can give the AI good examples of what high high quality you know like this is the Wall Street Journal this is a high quality Source these are the types of relationships or pieces of information that come from this Source versus here's a you know like a no-name Blog and then the machine can kind of extrapolate from that so it's a it's a supervised learning problem as opposed to go out and index all of these Financial sites and figure out which ones are high quality the way Google yeah cuz yeah Google basically has boiled the ocean ahead of time uh and we kind of when we come across new stuff we make a determination at that point in time right right um that's super interesting tell us about some of the other interesting kind of machine learning problems that you yeah um so yeah so the the other thing that is actually really really hard to do is kind of twofold the first one is entity resolution so you're reading all of this information about people places companies whatever uh the way things are written is not necessarily always clear and so you might be reading something about Apple but it's the fruit and not the company and you have to be able to disambiguate that uh so we do a lot of really cool stuff in the entity resolution space and then we've started to do some really exciting things that were in the domain of intelligence services but I think kind of the technology is leaking out now in the sense of something called record linkage so if you have these giant data sets that you can onboard or acquire or you know create yourself uh what typically happens is that something may call Apple Apple in one data set it may call it Apple Inc in a different one or it might just be a reference to the CEO of Apple you know so you need to be able to link all of these data sets together uh and basically you don't have a clear key on which to join everything so you have to make these calculated sort of uh you know similar you us to basically say this thing is likely this other disambiguating correct and connecting things uh based off of you know you say is this in the same location or you know does it is it referencing a product of this company so you know and then basically be able to say well actually it is talking about Apple but it was actually talking about the iPhone you know so basically merging like say the iPhone to the to the Apple company uh without ever having a NE necessarily mention of Apple in both data sets right so record linkage is actually really really really hard uh We've started to do a lot of that and it's it's a really fascinating kind of area in the Big Data world how do you attack that what technology approaches are you applying to so um obviously the first one is we have to be able to within the data extract kind of attributes or features that we think are important for the links that we're trying to establish so we might say what are we trying to link in the first place data you know so if we go and grab every patent in the US for companies and that's just one giant data dump uh linking that back to well all these patents are actually owned by these companies uh you know the US government patent office might not have the same names as you know the the publicly traded companies themselves uh so you have to look at things like is the address the same the people mentioned as the patent holders employees of these companies so you looking for Bodies of Evidence that suggest that these two things are kind of mapped together uh and you have different algorithms that can kind of give you a score and a confidence measure as to how closely two records are to each other um so that's one and then another one is uh I think called relationship extraction so if you read uh a piece of news as a human being it's very easy for you to sort of boil it down to what it means right so you know this company is buying another company right uh but in the real world that might be written out over three paragraphs and so for a computer uh to sort of establish that entity a is acquiring entity B after reading a paragraph or two is actually a pretty challenging problem um and so yeah there's a lot of really interesting stuff that we use there but most of it is in the Deep learning uh you know kind of kind of space so we're using a lot of tens oflow models that you know to try to understand their representation of language extract kind of entities and then the relationships that we're we're looking for so do these end up looking like um you know uh single or individual like super you know very deep uh neural networks or are they many neural networks that are each um serving their own purposes question yeah so we what we've realized is that uh we've had better results with building specialized machine learning models a neural networks uh as opposed to one kind of generalized AI or generalized Network and so even with our relationship extraction what we do is we have thousands of models that kind of work in unison and each one is highly specialized in in in one thing and so if you think about having a very wide funnel at the beginning where you say this is a raw document uh the first sort of gen set of models will basically say this document is actually talking about this concept and then that gets filtered into to okay so with this within this this concept these are the types of relationships I should be looking for and then have more sophisticated models kind of take that information and then say okay fine so this is about a corporation and corporations can have m&a as a relationship and so I kind of I'm seeing a very strong signal that this document in general is talking about m&a Let me give it to the most sophisticated guy to go and extract that very specific relationship and by sophisticated guy we talking about a person or another model another model sorry yeah um and so you we have this kind of pipeline of AI models that kind of work together uh that start really Broad and then end up uh kind of very spe hypers specialized at the end and are you doing any human in the loop anywhere or is it all we do but it's um there's a bit of reinforcement learning that goes on especially with the record linkage stuff um and then we obviously retrain our models fairly frequently and so we'll have the human in the loop on the retraining okay of the models but on a g on a given day everything's kind of running in an autonomous R that's super interesting so and when you talked about the identifying the relationships and how you have the you know you can have the three paragraphs to get to um you know that there was an acquisition that makes me think of like you know CFO speak right do you have a model that can train that is trying to like decipher when a c when a CFO is saying that they're going to miss say actually it's an area of interest to us we haven't yet done any kind of real hardcore development there but there is actually a huge community of users in the kind of you know hedge font space that want to understand tone not even just the text but like the tone uh if there was video of the guy making the statement or he darting around uh so wow uh there is a lot of of kind of interest in that we just haven't yet had a chance to spend their time in that space But it's clearly something in know okay and so where are you guys in the process um so we've launched uh a product about I guess less than two months now called precog uh which is our like it's a predictive service that basically tries to measure the butterfly effect and then produce uh short-term for forecasts for the returns of these 30,000 stocks around the world uh that is uh currently in production and we have a number of clients that we Service uh so meaning you give it uh you know a company and you probably have some unique company thing will just go into its database and tell you you know you you give it maybe say 30 days and it'll tell you a projected stock price in 30 days is it yes that yeah it's along those lines so what we do today is we produce predictions on specific time Horizons so it would be like a onee forecast two we forecast or one month forecast okay uh but it would be kind of a rolling day forecast for those Horizons so today being Monday will'll give you a forecast for the return of the stock not the price but the return of the stock by4 in 30 days okay um and that you know again if you do this systematically over a broad Universe of stocks and you sort of you know you're never going to get 100% accurac which is impossible uh but you you have this winning prediction you know like right now we think we get anywhere from 60 to almost 70% accurate uh in depending uh on the horizon and and and the stocks themselves but uh 60 to 70 is kind of the average but across 30,000 names that is like Vegas odds right so if you're the casino you may lose a hand to one player uh you know one table but across all players you're systematically making money and so uh the customers that are using our predictions want want them on a very large Universe of stocks and then they obviously not betting the farm on one prediction on one day they they're making this is one signal that they're using out of a portfolio of signals uh So you you're obviously gaining you're able to make a prediction at day Zero and then you know at days 17 14 and 30 you're getting additional signals as to you know how accurately your model is performing how do you uh then use that signal to retrain and then how do you deal with um like attribution issues you know you were off by you know 50% or even plus minus like how where in your model did you go wrong have you started to figure look at yeah we do um one of the things that we've tried to stay away shy away from just kind of with our models is that we've tried to avoid using deep neural Nets and all of these types of things when we're doing doing our forecasting and the reason being uh with these deep neural networks you actually don't know kind of what is driving the model's decision to produce the outcome that it produces uh so the models that we generally rely on tend to have a way to sort of reverse engineer why why it made the decision it made so more like decision trees or something else uh it's it's never one thing so we use kind of a hybrid approach where we will use an ensemble of Meth methods uh to come up with a single prediction okay but most of them uh have an ability where we can actually query the model and ask it what was the most important thing in the decision that you made okay uh and then those those those weights or those factors are things that we can kind of just you know sanity check in the markets to see if uh if it is what it was so you know we make say for financial stocks we think uh you know yields spreads and things of that nature should be important to financials but there may be a group of financials that are aren't moving with respect to interest rates uh so the model might rely on that but then you know in a month's time we could go back and say well why were you way off and they'll say well I overweighted you know this move in interest rate and we will try to retrain the model uh to make sure it doesn't do that uh the thing we try to do is we don't want to superfluously or just over engineer the models cuz you can end up with overfitting right so uh we're really careful about when we train the models and then what we give them to retrain you know and there's a lot of Arts and Science that goes into kind of being like you know 60% is good enough or not uh given everything that we know okay so it sounds like going back to your funnel analogy at the top of the funnel you're using a lot of deep learning to uh extract signal from various sources and then closer to the the to the end of the funnel you're using more um you're using models that have greater explainability absolutely that's cor okay interesting anything else that you'd like to share about what you guys are up to um I mean just kind of longterm um one of the things obviously we're trying to build like I said in the beginning was the capability for then you know non-technical users to be able to sort of surface some of these insights or even you know ask the discover new new facts that are kind of not obvious right within this knowledge base that we've built uh and then also be able to take advantage of some of these predictive models that we've building uh so what we're really trying to do here is develop you know like an ability for the machine to understand human intent uh and then also the context in which the intent is being asked so for example you sound fairly kind of familiar with the financial domain but like my grandmother may have a similar question to you but the response that the machine should give her versus you needs to be kind of adjusted for that so we're really looking to build an intelligent agent that would be able to work under different contexts right so if you're dealing with another professional investor kind of you know having a conversation at that level if you're dealing with somebody looking for financial advice who's not that sophisticated then you kind of want to boil down some of these things and just have you know them explained back to the user in a format that they can understand right so that ultimately is sort of where we're driving towards okay uh where can folks learn more about what you guys are doing um so on our website is a great place to start uh they can also check out uh at least for individual investors or people interested in sort of these these signals that we're producing they're available on quantopian uh so and they're free to use uh until you want to put them into a production strategy uh and they are available for 500 of the most liquid names in the United States okay and so the company website is alphav vertex.com alex. and uh we will be glad to uh you know reach out to them and just understand what their needs are awesome well thanks so much for being on the show I really uh learned a lot a lot about uh really learned a lot from what you guys are doing and it sounds really exciting yeah it was a pleasure thank you so much great thank you n\n"