**Unlocking High Performance Computing with Compression and Vectorization**

In this discussion, we're going to explore how compression and vectorization can lead to significant performance improvements in high-performance computing applications. Our conversation begins with an overview of scalable systems that can take advantage of multiple cores.

A hundred cores into the sector into the system, you can achieve extreme scalability so you can throw a hundred cores or you can throw a thousand cores, you can see those kinds of performance on your system and so that's using the avx-512 instructions. What were you seeing prior to that? Well, there's a combination of two techniques that goes into this type of performance. The first is how can you perform operations on compressed data. If like I said earlier, oftentimes it's the memory bandwidth that's the bottleneck of your computation so the better you compress but the better off you are in the final computation because at the end of the day you scan fewer bytes.

**Columnstore Compression and Encoding Similar Values**

The compression technique used is called columnstore compression. It's a combination of traditional binary compression, deduplication, and encoding similar values to tell the system which data points are most likely to occur together. For example, in a stock trading application, if you have 10,000 different stocks, each individual stock can be encoded using just a few bits, so that when performing computations, you never need to decode those bits into an integer value or even a string value of the stock. This reduces the computational cost significantly.

**Optimizing Computation for Branch Mispredictions and Cache Misses**

The second step is optimizing your computation for as few branch mispredictions and as few cache misses as possible. These are crucial because every time you have a branch misprediction, it flushes your CPU pipeline, slowing down your computation. Expressing your computation with no branches or minimal branches can prevent these mispredictions. There's some very cool innovation in this area that we've developed at NEM Sequel.

**Vector Operations and Avx-512**

The third boost comes from performing vector operations on top of the encoded values. This is where you use avx-512 instructions, which provide a significant performance improvement. The combination of these three techniques – compression, optimization for branch mispredictions and cache misses, and vectorization – can lead to two orders of magnitude improvement in performance.

**Maximizing Performance with Advanced Hardware**

Fancy hardware is cool, but there's also potential in the hardware you have at hand. By taking full advantage of it, you can achieve remarkable results. The point here is that advanced hardware is not just about using exotic or expensive processors; it's about understanding how to leverage your existing systems to unlock their full potential.

**Real-Time Machine Learning and AI Applications**

Our final piece of advice is don't settle for suboptimal performance in machine learning and AI applications. Systems like Mint equal can take end-to-end machine learning and application development pipelines and make them completely real-time. This means whatever you do in batch can be done in real-time, without waiting for the results. As we move forward, compute and storage will become utilities that provide on-demand resources, and the only limit will be the amount of money you're willing to pay for those resources.

**Conclusion**

In conclusion, high-performance computing applications can benefit significantly from compression, optimization techniques, and vectorization. By leveraging these strategies, developers can unlock exceptional performance improvements in their systems. As we move forward into a future where compute and storage are utilities, it's essential to take advantage of the available hardware to achieve real-time results and maximum efficiency.

"WEBVTTKind: captionsLanguage: enhello and welcome to another episode of we'll talk the podcast where I interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam Charrington this week on the podcast were featuring a series of conversations from the AWS reinvent conference in Las Vegas I had a great time at this event getting caught up on the new machine learning and AI products and services offered by AWS and its partners if you missed the news coming out of reinvent and want to know more about what one of the biggest AI platform providers is up to make sure you check out Monday's show - animal talk number 83 which was a roundtable discussion I held with Dave McCrory and Lawrence Chung we cover all of AWS s most important news including the new sage maker deep lens recognition video transcription service Alexa for business Greengrass MLM friends and more in this episode i'll be speaking with nikita semyonov co-founder and CEO of MEMS sequel a company offering a distributed memory optimized data warehouse of the same name Nikita and I take a deep dive into some of the features of their recently released vi auto version which supports built in vector operations to enable machine learning use cases like real-time image recognition visual search and predictive analytics for IOT we also discuss how to architect enterprise machine learning solutions around the data warehouse by including components like data lakes and spark finally we touch on some of the performance advantages that mem sequel is seen by implementing vector operations using Intel's latest avx2 and avx-512 instruction sets and speaking of Intel I'd like to thank our good friends over at Intel Nirvana for their sponsorship of this podcast and our reinvents series one of the big announcements from reinvent this year was the release of Amazon deep lens a fully programmable deep learning enabled wireless video camera designed to help developers learn and experiment with AI in the cloud and at the edge deep lens is powered by an Intel Atom x5 processor which delivers up to 100 gigaflops of processing power to onboard applications to learn more about deep lens and other interesting things Intel has been up to in the AI space make sure to check out Intel nirvana calm ok just a couple more quick announcements you may have heard me mention last time that over the weekend we hit a very exciting milestone for the podcast 1 million listens what an amazing way to close out an amazing year for the show it occurred to us that we'd hate to miss an opportunity to show you some love so we're launching a listener appreciation contest to celebrate the occasion to enter just tweet to us using the hashtag twill one mill every entry gets a fly twill one mill sticker plus a chance to win one of 10 limited run t-shirts commemorating the occasion we'll be giving away some other mystery prizes as well from the magic twill swag bag so you should definitely enter if you're not on Twitter or want more ways to enter visit twimble a i.com slash 1200 1 mil for the full rundown last but not least we are quickly approaching our final twill online meetup of the year which will be held on Wednesday December 13th at 3:00 p.m. Pacific time we'll start out by discussing the top ml and AI stories of 2017 and then for our main presentation Bruno Gonzales will be discussing the paper understanding deep learning requires rethinking generalization by Siwon Jiang from MIT and Google brain and others this will be a fun Meetup and one you don't want to miss so be sure to register at toiley Icom slash meetup if you haven't already done so and now onto the show all right everyone I am on the line with Nikita semyonov Nikita is CEO and co-founder of mem sequel Nikita welcome to this week in machine learning in AI thank you thank you for having me absolutely Nikita why don't we get started by having you tell us a little bit about your background you are the CEO of mem sequel but you've got a pretty technical background isn't that right that's right selling on my PhD in computer science from Saint Petersburg Russia and I moved to the States to work on Microsoft sequel server which I did for a number of years so have a very strong database and query processing background after that I moved over to Facebook where I was blown away by the magnitude of data problem Facebook was solving back then and since they only he only increased in magnitude seeing that and combining the visibility into those workloads and kind of making that assumption that in five years a lot more companies are going to be facing those challenges just like Facebook I decided to start a company and combine that that database background and expertise of building systems and the the early stages and the glimpse of the workloads that I saw at Facebook so I kind of knew where the world was going going into so that triggered my desire and gave me some insights into starting mmm sequel the company so we've been in it for six plus years and certainly validated some of the assumptions we had starting that journey hmm so I've been I've come across my sequel and I think of the company as an in-memory database you know it's also a little bit about what the focus is there and in particular you know what's the intersection between what you're doing and machine learning in AI are you seeing a lot of those types of workloads nowadays definitely that's certainly where our customers I'm moving towards but let me step back for a second and talk about them sequel and in-memory database and in technology okay we started as in purely in memory database and then since we've evolved to support a large class of applications that I built on top of mem sequel and in memory became an enabling technology but it's not the technology at mem sequel as a matter of fact the the key advantage that mem sequel brings to the world is the fact that it supports sequel which is a structured query language and the fact that it runs the database in a distributed environment so you can run them sequel on your laptop or you can run mem sequel on a thousand hosts that gives you an immense compute power to enable the new class of applications so let me talk about what kinds of applications we support certainly some of them have to do very little with AI and ml and nebs equal enables scale latency you even build very low latency applications on top of them sequel and that's where in-memory technology has come handy and you also can build applications that require very high levels of concurrency and that concurrency is enabled on top of the system of record so Maps equal support state and it supports full durable persistence sequel also allows you to build what we call real time applications and kind of the idea of real time applications is the opposite of batch so every time you need to do analytics and you do some sort of pre calculations upfront using Hadoop or data warehouses or any other offline and Batchelor ended technology that's basically our enemy so we're we're bringing the world to be completely real-time and perform all the computations that you need to live and we deliver it on you know extremely low latencies by leveraging an immense amount of compute and that we can do very very well because mmm sequel is a scalable technology that can run on on clusters of commodity hardware so now we're a and ml comes in well because we support this new class of applications the fact that our customers are incredibly forward-looking they want to do more with the technology and they want to blend classical database workloads with the new types of computations that are mostly stemmed from the AI and ml needs and one of the big ones that we see all the time is image recognition so we can talk we can talk a little more about it so let's say you have an application and you use that to power and use mem sequel to power that application the application is large scale there's a ton of data that's sorted mem sequel like I said earlier in memory is an enabling technology but it's not the technology you can actually put a lot more data into Memphis equal than the reason the cluster and what you want to do is you want to enable smart applications the ones that make decisions on either on behalf of a user or they provide recommendations or they provide some sort of search capabilities on top of unstructured and semi-structured data imagine an app that has a camera it brought something on your cell phone you snap a picture with this app and just in a few tens of milliseconds this app finds similar images to the one that you just took a picture of so why is it useful well it's useful because you just enabled visual search and the way we do it is we build we build some of the building blocks that allow you to run and operationalize and machine learning models and we build them straight into the database and now the database allows you to scale them and deliver very low latencies for this type of operations so that would be one example of an application I can give you another one in which we see a lot in the IOT space before we go into the next example can you drill down a little bit into what specifically mmm sequel is doing to enable the first example you know for example are there pre train models you know for image recognition or image similarity in this case built into the database kind of like you know you might think of a stored procedure or is it you know is there something else is it a different type of functionality yeah it's even it's of a lower level than that many people certainly support stored procedures and but in this particular case we implemented a few building blocks particularly dot product and Euclidean distance between between vectors oh they ain't it pretty low level yes and if you if you take a deep learning model and you look at the layers you know matrix multiplication tensor multiplication vector multiplication is the fundamental building block so what we do is as we train that model we take all the layers except for the very last one and apply it on the database of images that we have and that allows us to extract what we call a you know or anybody else calls a feature vector which is just a vector so once we have that and we have a model applying that model to an incoming image which you just took a picture with your cell phone will produce another feature vector and it just so happens that the multiplication of those two feature vectors normalized gives you the similarity score how how close those images are together so the heavy lifting of building a model belongs to somewhere else and the data may might as well be still stored up in them sequel and we enabling very fast data transfer in and out of mmm sequel in a parallel way so we can send it into spark or tensorflow or any other training framework but once it gets to operationalizing operationalizing that model and performing the last mile computation by really powering your app then MC will gives you that scale and it allows you to perform those computations pretty much at the memory bandwidth speed that the labels really low latencies for that last mile computation okay does it make sense okay no it does so your your computing these feature vectors is that happening as is that happening on write of new images or is it happening I'm assuming that's the way you would do it since you're anti batch you're not doing some big batch job that's like updating you know some column in your database with your feature vectors for all the images that are in there correct and you can do but you can do either but the typical workload is once you have that model you apply that model on right and we have technology it's called pipelines that allows you to perform arbitrary computations either in a stored procedure or an external piece of code that's where you can invoke all third party libraries to apply that computation and then the storm the feature vector in the database so you can do it and ingest and if they let's say you build something like you crawl the web or you crawl your your own product catalogs and once you identify those images you immediately stick them into the databases as you do that we trigger that computation so the feature vector arrives into the database at the same time instantly as the actual data and now it's immediately participates in all sorts of computations so that allows you to never have kind of stop and go computations you never do okay step one load all the data step 2 perform the kind of the batch computation on top of all the data into the database and step 3 do something else with it rather than it's all streamlined and it just flows in and out mmm and so how does this play out in the I or coyote case so in the IOT case they days and come in constantly so one of the use cases we have with a large energy company is to ingest IOT data from drill bits apparently in the in the world of fracking you tend to drill a lot more and then there is a non-trivial cost for a broken drill bit those those things are extremely expensive you have to stop your operation if the if the drill bit breaks so you're losing of not only on on the fact that you're fishing this thing out from the ground from hundreds and and or maybe even thousands of miles deep but you also not producing oil which is an operational cost so what we do we ingest that real-time data IOT data and we scouring that data applying a machine learning model in real time and then there's an application built that arrests the drill bits before it hits a problem just by measuring temperature and all the very you know temperature throughput all sorts of kind of vital signs of the drill bit of the drill bits as it goes through the ground and so that was step one step two is obviously you feed back all this you feed all this information back to direct drilling so in the world of fracking drilling is directional so it's not just vertical down into the ground but it's it's more like you change in the direction as you go and and drilling so using all that input you can direct the drill bit to make the whole operation looks more efficient and what are some of the algorithms that come into play in that use case so they started with you know like every typical data scientist started with some sort of linear regressions that they moved it to decision trees very quickly and now they experimenting with deep learning mm-hmm for that as well and the beauty of that of the solution that we presented is it integrates natively with all sorts of third-party libraries so we made the experimentation for them very very straightforward they started with SAS where they produce produce models and SAS as you know SAS is a proprietary technology but friends they've played with SPARC and now experimenting with tensorflow as well okay I was actually going to ask you about spark and how that fits in with your model do you I imagine you see it out in out working with customers is it a competitive technologies spark plus the rest of the Hadoop ecosystem or is it complimentary or how do you see that so all the Big Data technologies overlap a little bit in the case of spark I would say it's ninety ten so it's 10 percent competitive okay 90 percent complimentary and and here is why spark doesn't have state the state is usually stored somewhere else it's either HDFS in our relational database or you know s3 you know some sort of object store in the cloud so what we do in this case is we provide an extremely performant state and the combination of mem sequel and spark that's the 90% case they work really really well together because we give you that transactional scalable state and and nothing else on the market can give you that state so not only you you know you can store you can retrieve but you can also compute and we have a world-class sequel query processing engine that allows you to produce reports but also allows you to modify that state in a transactional fashion you can say begin transaction insert update delete you can run it at high concurrency and you can you have full durability and all sorts of guarantees for that data so that's where we're extremely complementary the typical deployment model is that there is a data Lake and the data Lake stores you know hundreds of terabytes and petabytes of data named sequel is deployed alongside of the data Lake to provide to power applications because you you cannot write applications on top of Hadoop because Hadoop is batch so now and then SPARC is the glue between those two and it allows you to have rapid data transfer all the data that is born in them sequel based on the application the interactions with applications that data is captured you can pick it up in spark very very easily we have a world-class spark connector drop it in the data lake for historical archival compliance type storage and then provide some and then perform some overnight batch computations take the results of those computations stick it into mem sequel and spark often times give you that unified API because through that API you can interact with them sequel you can interact with the data lake and it becomes kind of the go-to API for application developers and then people in this case just gives you sequel then you can attach a bi tool directly into mem sequel and you can scale the concurrency of you know data scientist attached that attach their bi tools to mem sequel and look at the reports and visualizations and is it primarily data scientists and folks that are and kind of end-user use of mem sequel are you also in this scenario attaching your traditional applications to mem sequel or using some other state technology for building applications that you know refer to this data well mostly it's actually application developers because names equal powers applications and because the nature of applications is changing all the time and we have higher scale requirements for the applications and then sequel is is a perfect technology to power applications like this I will give you a few more examples of such applications now because modern applications have AI and m/l requirements that's where that intersection comes in where you need to to have those models and that you produced somewhere in your in your data science lab and you want to operationalize those models and that's where a man sequel plays very very strongly so do you envision kind of along the lines of you know what sparks done kind of building higher level abstractions beyond dot product and Euclidean distance to enable folks to do the machine learning and AI more easily absolutely we have number of ideas that are circulating inside the engineering team and certainly influenced by by our customers and what they want to do as a technology well the customers C and M sequel is that very very fast scalable state that's that they can deploy inside their data centers or in the cloud on the cheap right because mem sequel provides world-class compression it has columnstore technology and that state is very fungible you because we support transactions you can change that state you can enrich that data with attributes that you compute on the fly etc etc etc now what people want to do is they want to perform computations that are beyond sequel so we're we're world-class in sequel query processing and that's great and our customers our customers want to do that and more and when they want to do more right now we resort to spark all right we say okay well deploy deploy spark alongside of mem sequel pull the data out and we'll we'll give you that data very quickly perform the computation put that data back and then leverage some of the building blocks then we have you know basic arithmetic operations vectorized operations vector operations to do the last mile computation to power your applications now like once you do that you just want to take that loop and you want to tighten it up so you start bringing all the computations that you today taking out of today you're taking data out of Mexico put it somewhere else in a temporary store like spark data frames and you want to perform those computations in place and there are two to interesting problems in that space one is what the API should be and today for machine learning nai the API is tend to be either spark driven or the Python library ecosystem driven you know the likes of non pi syuh pi pandas tensorflow so it seems like that the Python world lives in in universe the spark world lives in another universe and then there is the sequel universe that is pretty much ubiquitous because every database exposes sequel as an API and also data warehouses expose sequel as an API our our view that the spark universe nm sequel universe should be enabled by rapid data transfer between the two and then the Python universe should be enabled by pushing some of the computations that you express through Python API into the database and putting them on steroids now what exactly does that mean and can you give an example yeah totally so let's say you performing a non-trivial computation on your data let's say you have hundred terabytes of data you know certainly more data that it can fit on your on your laptop and your data scientist and as a data scientist your stack you live in the Python world mm-hmm and you're a big expert in the libraries like pandas numpy and Syfy and let's say you play with tensorflow as well so it's very natural for you to express computations on that data to perform you to perform training or perform a scoring on that data in in that Python world the problem is all the computation is a single-threaded and all the computations are in memory so so you're stuck at this point in time because you cannot access hundred terabytes of data because there's no way you can put hundred terabytes in memory and that's where we see the world is going to where you want those computations to become instant and parallelizable and scalable not so you can just instead of bringing that data into your high memory machine you can perform those computations in place in something that's very very scalable like x equal so you're going to see more and more of that happening over time you mentioned the the you know three different kind of modes of interacting with this for machine learning or in general spark Python and sequel are there any efforts to extend sequel to give it some kind of machine learning expressiveness like you know sequels got you know all kinds of aggregation operations like select average of X you know you know where whatever right I can imagine you know something like you know select predict you know why you know where whatever where you're kind of telling expressing in sequel that you want a prediction you know based on some something you know does that exist our folks playing with that today yeah so if you look at the history and you look at products like you know good old products like Terra data or Anna teaser or if you look at sequel server integration with our that certainly is something that that is happening we're trading is not happening in the database but scoring is that's a natural way to do things we achieve the same functionality through pipelines where we score data and ingest and so I think this is valuable and you'll see databases exposing more and more primitives the likes of dot product and Euclidean distance but you'll have instead of two you will have you know a hundred of those built in into the database and then you will see packages where it's just a package of story procedures which allows YouTube to to run those predictions and those will work really well for simple use cases now if you look at the world of a data scientist today they actually don't like that they like to live in the world of data frames where there's a lot more control over the types of computations that people are expressing people like to people understand that world of data frames very very well so I think the future is gonna be actually in paralyzing and speeding up computations inside data frames so so that that's just my opinion do you envision doing training directly in the database in the same way as as I describe the interaction with the data frames so if you want to enable inference in the database then you don't want to have the database you don't want to have the database to be a crutch where you're constantly fighting and trying to shoehorn that computation into sequel the natural ways of expressing those computations is operations on top of data frames however if you make the database naturally support those data frame computations then you have tremendous value from the fact that the database actually owns the state and you never need to transfer that state from your your storage to something else so you're bringing computations closer to data so that that's where I see the industry is moving in the next you know five years you just mentioned bringing computations closer to the data and that's obviously been one of the things that Hadoop ecosystem has you know made more readily accessible to folks this idea of data locality you know do you get to take advantage of that with machine learning types of models in general and this pipelining approach that you have in particular or are the pipelines run you know outside of you know separate from any concept of locality of data well locality is is is a loaded concept right so you can start with okay well I want to perform computation exactly on the same machine where the data is stored so that's kind of the extreme case of locality another way to think about is as you look at the at the computation plan let's say in the database terminology that would be a query plan so let's extend the definition of a sequel query plan to a broader concept of you performing some sort of arbitrary scalable computation and you know all the steps in the computation upfront so you can optimize those compute a and produce a query plan for those computations and you will run that query in the distributed environment now then you look at this and certain primitive computations you really really want to perform closer to the data because you will save tremendously on the amount of i/o that is happening for in the data transfer now from there if you do that you also deliver on the concurrency of such computations so now you can perform those computations at a high highly concurrent in highly concurrent environment where thousands of data scientists are attaching to the same data that is collected and centralized and stored in something like mem sequel and then and then they're performing their computations concurrently so there are no you opening up all the data to the organization and and not having any data silos so data locality for computations is is useful where it makes sense and not as useful in a way in a in a broader general purpose way and the perfect system would be engineered around pushing the computations closer to data where where it makes a ton of sense for example you do a lot of filtering or you performing a lot of what it's called group by operations and those operations make sense student to be done locally those operations make sense to be done in a vectorized fashion by leveraging you know the latest and greatest CPU instructions in the intel hardware and so on it makes sense to leverage indexes so you prune massive amounts of data so you don't even touch them for your computations and then from there it makes sense to bring all that data into a stateless distributed environment and perform the rest of the computations so that approach allows you allows you to build greater scale scalability for your system compared to a traditional database compared to Hadoop or compared to bark okay and now you just mentioned the taking advantage of the instruction sets of the underlying hardware and you guys are making an announcement with Intel at Amazon reinvent next week is it related to that area definitely Intel has been supporting vector instructions for some time for over a decade and every new generation of CPUs increases the size of the vector that you can use and perform vectorized operations on top of and where we stand right now it's 512 bits so compared to you know matrices in a GPU this is tiny however right when you perform a last mile computation like dot product it allows you to perform that computation much faster than the memory bandwidth that you have so you actually don't need more for that you know last mile computation then avx-512 because you're limitation is not your compute your limitation is memory bandwidth this is very very different when you train models and and perform computations for building you know deep learning that's where you perform a lot of tensor multiplication and they might have compute compared to the size of your data set is tremendously more when they performing something like dot product that's where you want GPUs okay but once the model is built and the model is trained you actually don't need GPUs to score that model in many many cases what that gives you is the it gives you the ability to to democratize that computation because Intel CPU is is everywhere you know if your machine has a GPU it also has an Intel CPU and what the point of that making that in many cases all you need is a CPU and you're not going to get the computation faster if you add a GPU to the system simply because it's the memory bandwidth that's less your bottleneck in the computation now we use avx-512 for vector dot product for you clearly and distance and for other vector operations that were built into the database we also use avx-512 for general purpose sequel computations and that allows you to just deliver on orders of magnitude faster sequel query processing that our competitors have and certainly what what Hadoop has our spark we're huge fans of that technology and we can't wait when Intel allows us to to perform computations on larger vectors not just 512 bits mmm can you be more specific in terms of some of the actual results you've seen in you know production systems in terms of performance differences yeah absolutely so with nem sequel 6.0 that we just released in October we can do things like a group by operation on 100 billion row table in a sub second so 100 billion ended billion yes Wow so it says a sub second operation that that runs something like give me an average stock price over a hundred billion data points and and groupid by security or by stock value the computation itself is relatively straightforward because we perform that computation quote-unquote on compressed data so and we're using those vectorized operations we achieve the performance of a billion operations per second per core so if you throw a hundred cores into the sector into the system and then sequel is extremely scalable so you can throw a hundred cores or you can throw a thousand course you can achieve those this type of performance on your system and so that's using the avx-512 instructions what were you seeing prior to that well there's a combination of two techniques that goes into into this type of performance the first is how can you perform operations on compressed data so if like I said earlier oftentimes it's the memory bandwidth that's the bottleneck of your computation so the better you compress but the better off you are in the final computation because at the end of the day you scan fewer bytes what exactly do we mean by compressor is a kind of your traditional you know binary compression or we also talking some kind of deduplication or something like that it's a combination so first of all the compression is called columnstore compression and then we also encode similar values to tell let's say you in that example that is said with stock trading you have 10,000 different stocks so if you have 10,000 different stocks then each individual stock can be encoded was just a few bits and so then you only use this many bits to represent each stock value and as you perform your computations you never go back decoding those bits into let's say an integer value or god forbid a string value of a stock right because those computation will become much more expensive so now that you perform computations on compressed data the second step that you do is you optimize your computation for as few branch mispredictions and as few cache misses as possible and those are in big deal because every time you have a branch misprediction you flush your cpu pipeline so that that slows down your computation so if you express your computation so there are no branch minute no branches basically and there are some very very cool papers out there and there's some innovation that we we've done here at nem sequel as well so that will be give you the second boost and then the third boost is now you take all those bits that you represent your values and you perform vector operations on top of those encoded values and then that's where you use avx-512 so now and that gives you kind of another I would say you know three to five X performance improvements just by using avx-512 but if you take that Plus you take operations on compress data plus you take care of branch misprediction you multiply those all those together and then you can routinely see two orders of magnitude improvement in performance Wow that's pretty fantastic kind of the point here is fancy hardware is cool and you want to use it where it's needed but also there's a lot of potential in the hardware you have at hand and if you take full advantage of it of that hardware you will have remarkable results mm-hmm oh that's great I guess as we you know as we kind of wrap up do you have any additional kind of advice to folks that are looking to you know get better you know just to take better advantage of you know whether it's their you know their data storage systems their you know hardware systems and to use these to make better you know predictions and better utilize machine learning in AI well my advice here is don't settle there are systems out there such as mint equal that allows you to take pretty much your end-to-end machine learning AI and application development pipeline and make them completely real-time so whatever you do in batch whatever you have to wait for what the right technology can be squished down to zero is that the technical term squish to zero say that and and that's where the world is going so will will will live in the world in the future where a compete where compute and storage are gonna be utilities and you know if you if you're willing to pay for compute you will have as much compute as you need at any concurrency as you need and any ladies latency as you need and the only thing that is gonna be bounded by is is the amount of money you pay for that computer mmm great well Nikita thank you so much for spending the time to chat with me I really appreciate it yeah absolutely thanks for having me all right everyone that's our show for today thanks so much for listening and for your continued feedback and support for more information on Makita or any of the topics covered in this episode head on over to tamil AI comm slash talk / 84 to follow along with our AWS reinvent series visit to Malaya comm slash reinvent of course we would love to receive your feedback or questions either via a comment on the show notes page or via Twitter - at to male eye or at Sam Charrington thanks once again - until Nirvana for their sponsorship of this series to learn more about deep lens and the other things they've been up to visit Intel nirvana calm and thank you once again for listening and catch you next timehello and welcome to another episode of we'll talk the podcast where I interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam Charrington this week on the podcast were featuring a series of conversations from the AWS reinvent conference in Las Vegas I had a great time at this event getting caught up on the new machine learning and AI products and services offered by AWS and its partners if you missed the news coming out of reinvent and want to know more about what one of the biggest AI platform providers is up to make sure you check out Monday's show - animal talk number 83 which was a roundtable discussion I held with Dave McCrory and Lawrence Chung we cover all of AWS s most important news including the new sage maker deep lens recognition video transcription service Alexa for business Greengrass MLM friends and more in this episode i'll be speaking with nikita semyonov co-founder and CEO of MEMS sequel a company offering a distributed memory optimized data warehouse of the same name Nikita and I take a deep dive into some of the features of their recently released vi auto version which supports built in vector operations to enable machine learning use cases like real-time image recognition visual search and predictive analytics for IOT we also discuss how to architect enterprise machine learning solutions around the data warehouse by including components like data lakes and spark finally we touch on some of the performance advantages that mem sequel is seen by implementing vector operations using Intel's latest avx2 and avx-512 instruction sets and speaking of Intel I'd like to thank our good friends over at Intel Nirvana for their sponsorship of this podcast and our reinvents series one of the big announcements from reinvent this year was the release of Amazon deep lens a fully programmable deep learning enabled wireless video camera designed to help developers learn and experiment with AI in the cloud and at the edge deep lens is powered by an Intel Atom x5 processor which delivers up to 100 gigaflops of processing power to onboard applications to learn more about deep lens and other interesting things Intel has been up to in the AI space make sure to check out Intel nirvana calm ok just a couple more quick announcements you may have heard me mention last time that over the weekend we hit a very exciting milestone for the podcast 1 million listens what an amazing way to close out an amazing year for the show it occurred to us that we'd hate to miss an opportunity to show you some love so we're launching a listener appreciation contest to celebrate the occasion to enter just tweet to us using the hashtag twill one mill every entry gets a fly twill one mill sticker plus a chance to win one of 10 limited run t-shirts commemorating the occasion we'll be giving away some other mystery prizes as well from the magic twill swag bag so you should definitely enter if you're not on Twitter or want more ways to enter visit twimble a i.com slash 1200 1 mil for the full rundown last but not least we are quickly approaching our final twill online meetup of the year which will be held on Wednesday December 13th at 3:00 p.m. Pacific time we'll start out by discussing the top ml and AI stories of 2017 and then for our main presentation Bruno Gonzales will be discussing the paper understanding deep learning requires rethinking generalization by Siwon Jiang from MIT and Google brain and others this will be a fun Meetup and one you don't want to miss so be sure to register at toiley Icom slash meetup if you haven't already done so and now onto the show all right everyone I am on the line with Nikita semyonov Nikita is CEO and co-founder of mem sequel Nikita welcome to this week in machine learning in AI thank you thank you for having me absolutely Nikita why don't we get started by having you tell us a little bit about your background you are the CEO of mem sequel but you've got a pretty technical background isn't that right that's right selling on my PhD in computer science from Saint Petersburg Russia and I moved to the States to work on Microsoft sequel server which I did for a number of years so have a very strong database and query processing background after that I moved over to Facebook where I was blown away by the magnitude of data problem Facebook was solving back then and since they only he only increased in magnitude seeing that and combining the visibility into those workloads and kind of making that assumption that in five years a lot more companies are going to be facing those challenges just like Facebook I decided to start a company and combine that that database background and expertise of building systems and the the early stages and the glimpse of the workloads that I saw at Facebook so I kind of knew where the world was going going into so that triggered my desire and gave me some insights into starting mmm sequel the company so we've been in it for six plus years and certainly validated some of the assumptions we had starting that journey hmm so I've been I've come across my sequel and I think of the company as an in-memory database you know it's also a little bit about what the focus is there and in particular you know what's the intersection between what you're doing and machine learning in AI are you seeing a lot of those types of workloads nowadays definitely that's certainly where our customers I'm moving towards but let me step back for a second and talk about them sequel and in-memory database and in technology okay we started as in purely in memory database and then since we've evolved to support a large class of applications that I built on top of mem sequel and in memory became an enabling technology but it's not the technology at mem sequel as a matter of fact the the key advantage that mem sequel brings to the world is the fact that it supports sequel which is a structured query language and the fact that it runs the database in a distributed environment so you can run them sequel on your laptop or you can run mem sequel on a thousand hosts that gives you an immense compute power to enable the new class of applications so let me talk about what kinds of applications we support certainly some of them have to do very little with AI and ml and nebs equal enables scale latency you even build very low latency applications on top of them sequel and that's where in-memory technology has come handy and you also can build applications that require very high levels of concurrency and that concurrency is enabled on top of the system of record so Maps equal support state and it supports full durable persistence sequel also allows you to build what we call real time applications and kind of the idea of real time applications is the opposite of batch so every time you need to do analytics and you do some sort of pre calculations upfront using Hadoop or data warehouses or any other offline and Batchelor ended technology that's basically our enemy so we're we're bringing the world to be completely real-time and perform all the computations that you need to live and we deliver it on you know extremely low latencies by leveraging an immense amount of compute and that we can do very very well because mmm sequel is a scalable technology that can run on on clusters of commodity hardware so now we're a and ml comes in well because we support this new class of applications the fact that our customers are incredibly forward-looking they want to do more with the technology and they want to blend classical database workloads with the new types of computations that are mostly stemmed from the AI and ml needs and one of the big ones that we see all the time is image recognition so we can talk we can talk a little more about it so let's say you have an application and you use that to power and use mem sequel to power that application the application is large scale there's a ton of data that's sorted mem sequel like I said earlier in memory is an enabling technology but it's not the technology you can actually put a lot more data into Memphis equal than the reason the cluster and what you want to do is you want to enable smart applications the ones that make decisions on either on behalf of a user or they provide recommendations or they provide some sort of search capabilities on top of unstructured and semi-structured data imagine an app that has a camera it brought something on your cell phone you snap a picture with this app and just in a few tens of milliseconds this app finds similar images to the one that you just took a picture of so why is it useful well it's useful because you just enabled visual search and the way we do it is we build we build some of the building blocks that allow you to run and operationalize and machine learning models and we build them straight into the database and now the database allows you to scale them and deliver very low latencies for this type of operations so that would be one example of an application I can give you another one in which we see a lot in the IOT space before we go into the next example can you drill down a little bit into what specifically mmm sequel is doing to enable the first example you know for example are there pre train models you know for image recognition or image similarity in this case built into the database kind of like you know you might think of a stored procedure or is it you know is there something else is it a different type of functionality yeah it's even it's of a lower level than that many people certainly support stored procedures and but in this particular case we implemented a few building blocks particularly dot product and Euclidean distance between between vectors oh they ain't it pretty low level yes and if you if you take a deep learning model and you look at the layers you know matrix multiplication tensor multiplication vector multiplication is the fundamental building block so what we do is as we train that model we take all the layers except for the very last one and apply it on the database of images that we have and that allows us to extract what we call a you know or anybody else calls a feature vector which is just a vector so once we have that and we have a model applying that model to an incoming image which you just took a picture with your cell phone will produce another feature vector and it just so happens that the multiplication of those two feature vectors normalized gives you the similarity score how how close those images are together so the heavy lifting of building a model belongs to somewhere else and the data may might as well be still stored up in them sequel and we enabling very fast data transfer in and out of mmm sequel in a parallel way so we can send it into spark or tensorflow or any other training framework but once it gets to operationalizing operationalizing that model and performing the last mile computation by really powering your app then MC will gives you that scale and it allows you to perform those computations pretty much at the memory bandwidth speed that the labels really low latencies for that last mile computation okay does it make sense okay no it does so your your computing these feature vectors is that happening as is that happening on write of new images or is it happening I'm assuming that's the way you would do it since you're anti batch you're not doing some big batch job that's like updating you know some column in your database with your feature vectors for all the images that are in there correct and you can do but you can do either but the typical workload is once you have that model you apply that model on right and we have technology it's called pipelines that allows you to perform arbitrary computations either in a stored procedure or an external piece of code that's where you can invoke all third party libraries to apply that computation and then the storm the feature vector in the database so you can do it and ingest and if they let's say you build something like you crawl the web or you crawl your your own product catalogs and once you identify those images you immediately stick them into the databases as you do that we trigger that computation so the feature vector arrives into the database at the same time instantly as the actual data and now it's immediately participates in all sorts of computations so that allows you to never have kind of stop and go computations you never do okay step one load all the data step 2 perform the kind of the batch computation on top of all the data into the database and step 3 do something else with it rather than it's all streamlined and it just flows in and out mmm and so how does this play out in the I or coyote case so in the IOT case they days and come in constantly so one of the use cases we have with a large energy company is to ingest IOT data from drill bits apparently in the in the world of fracking you tend to drill a lot more and then there is a non-trivial cost for a broken drill bit those those things are extremely expensive you have to stop your operation if the if the drill bit breaks so you're losing of not only on on the fact that you're fishing this thing out from the ground from hundreds and and or maybe even thousands of miles deep but you also not producing oil which is an operational cost so what we do we ingest that real-time data IOT data and we scouring that data applying a machine learning model in real time and then there's an application built that arrests the drill bits before it hits a problem just by measuring temperature and all the very you know temperature throughput all sorts of kind of vital signs of the drill bit of the drill bits as it goes through the ground and so that was step one step two is obviously you feed back all this you feed all this information back to direct drilling so in the world of fracking drilling is directional so it's not just vertical down into the ground but it's it's more like you change in the direction as you go and and drilling so using all that input you can direct the drill bit to make the whole operation looks more efficient and what are some of the algorithms that come into play in that use case so they started with you know like every typical data scientist started with some sort of linear regressions that they moved it to decision trees very quickly and now they experimenting with deep learning mm-hmm for that as well and the beauty of that of the solution that we presented is it integrates natively with all sorts of third-party libraries so we made the experimentation for them very very straightforward they started with SAS where they produce produce models and SAS as you know SAS is a proprietary technology but friends they've played with SPARC and now experimenting with tensorflow as well okay I was actually going to ask you about spark and how that fits in with your model do you I imagine you see it out in out working with customers is it a competitive technologies spark plus the rest of the Hadoop ecosystem or is it complimentary or how do you see that so all the Big Data technologies overlap a little bit in the case of spark I would say it's ninety ten so it's 10 percent competitive okay 90 percent complimentary and and here is why spark doesn't have state the state is usually stored somewhere else it's either HDFS in our relational database or you know s3 you know some sort of object store in the cloud so what we do in this case is we provide an extremely performant state and the combination of mem sequel and spark that's the 90% case they work really really well together because we give you that transactional scalable state and and nothing else on the market can give you that state so not only you you know you can store you can retrieve but you can also compute and we have a world-class sequel query processing engine that allows you to produce reports but also allows you to modify that state in a transactional fashion you can say begin transaction insert update delete you can run it at high concurrency and you can you have full durability and all sorts of guarantees for that data so that's where we're extremely complementary the typical deployment model is that there is a data Lake and the data Lake stores you know hundreds of terabytes and petabytes of data named sequel is deployed alongside of the data Lake to provide to power applications because you you cannot write applications on top of Hadoop because Hadoop is batch so now and then SPARC is the glue between those two and it allows you to have rapid data transfer all the data that is born in them sequel based on the application the interactions with applications that data is captured you can pick it up in spark very very easily we have a world-class spark connector drop it in the data lake for historical archival compliance type storage and then provide some and then perform some overnight batch computations take the results of those computations stick it into mem sequel and spark often times give you that unified API because through that API you can interact with them sequel you can interact with the data lake and it becomes kind of the go-to API for application developers and then people in this case just gives you sequel then you can attach a bi tool directly into mem sequel and you can scale the concurrency of you know data scientist attached that attach their bi tools to mem sequel and look at the reports and visualizations and is it primarily data scientists and folks that are and kind of end-user use of mem sequel are you also in this scenario attaching your traditional applications to mem sequel or using some other state technology for building applications that you know refer to this data well mostly it's actually application developers because names equal powers applications and because the nature of applications is changing all the time and we have higher scale requirements for the applications and then sequel is is a perfect technology to power applications like this I will give you a few more examples of such applications now because modern applications have AI and m/l requirements that's where that intersection comes in where you need to to have those models and that you produced somewhere in your in your data science lab and you want to operationalize those models and that's where a man sequel plays very very strongly so do you envision kind of along the lines of you know what sparks done kind of building higher level abstractions beyond dot product and Euclidean distance to enable folks to do the machine learning and AI more easily absolutely we have number of ideas that are circulating inside the engineering team and certainly influenced by by our customers and what they want to do as a technology well the customers C and M sequel is that very very fast scalable state that's that they can deploy inside their data centers or in the cloud on the cheap right because mem sequel provides world-class compression it has columnstore technology and that state is very fungible you because we support transactions you can change that state you can enrich that data with attributes that you compute on the fly etc etc etc now what people want to do is they want to perform computations that are beyond sequel so we're we're world-class in sequel query processing and that's great and our customers our customers want to do that and more and when they want to do more right now we resort to spark all right we say okay well deploy deploy spark alongside of mem sequel pull the data out and we'll we'll give you that data very quickly perform the computation put that data back and then leverage some of the building blocks then we have you know basic arithmetic operations vectorized operations vector operations to do the last mile computation to power your applications now like once you do that you just want to take that loop and you want to tighten it up so you start bringing all the computations that you today taking out of today you're taking data out of Mexico put it somewhere else in a temporary store like spark data frames and you want to perform those computations in place and there are two to interesting problems in that space one is what the API should be and today for machine learning nai the API is tend to be either spark driven or the Python library ecosystem driven you know the likes of non pi syuh pi pandas tensorflow so it seems like that the Python world lives in in universe the spark world lives in another universe and then there is the sequel universe that is pretty much ubiquitous because every database exposes sequel as an API and also data warehouses expose sequel as an API our our view that the spark universe nm sequel universe should be enabled by rapid data transfer between the two and then the Python universe should be enabled by pushing some of the computations that you express through Python API into the database and putting them on steroids now what exactly does that mean and can you give an example yeah totally so let's say you performing a non-trivial computation on your data let's say you have hundred terabytes of data you know certainly more data that it can fit on your on your laptop and your data scientist and as a data scientist your stack you live in the Python world mm-hmm and you're a big expert in the libraries like pandas numpy and Syfy and let's say you play with tensorflow as well so it's very natural for you to express computations on that data to perform you to perform training or perform a scoring on that data in in that Python world the problem is all the computation is a single-threaded and all the computations are in memory so so you're stuck at this point in time because you cannot access hundred terabytes of data because there's no way you can put hundred terabytes in memory and that's where we see the world is going to where you want those computations to become instant and parallelizable and scalable not so you can just instead of bringing that data into your high memory machine you can perform those computations in place in something that's very very scalable like x equal so you're going to see more and more of that happening over time you mentioned the the you know three different kind of modes of interacting with this for machine learning or in general spark Python and sequel are there any efforts to extend sequel to give it some kind of machine learning expressiveness like you know sequels got you know all kinds of aggregation operations like select average of X you know you know where whatever right I can imagine you know something like you know select predict you know why you know where whatever where you're kind of telling expressing in sequel that you want a prediction you know based on some something you know does that exist our folks playing with that today yeah so if you look at the history and you look at products like you know good old products like Terra data or Anna teaser or if you look at sequel server integration with our that certainly is something that that is happening we're trading is not happening in the database but scoring is that's a natural way to do things we achieve the same functionality through pipelines where we score data and ingest and so I think this is valuable and you'll see databases exposing more and more primitives the likes of dot product and Euclidean distance but you'll have instead of two you will have you know a hundred of those built in into the database and then you will see packages where it's just a package of story procedures which allows YouTube to to run those predictions and those will work really well for simple use cases now if you look at the world of a data scientist today they actually don't like that they like to live in the world of data frames where there's a lot more control over the types of computations that people are expressing people like to people understand that world of data frames very very well so I think the future is gonna be actually in paralyzing and speeding up computations inside data frames so so that that's just my opinion do you envision doing training directly in the database in the same way as as I describe the interaction with the data frames so if you want to enable inference in the database then you don't want to have the database you don't want to have the database to be a crutch where you're constantly fighting and trying to shoehorn that computation into sequel the natural ways of expressing those computations is operations on top of data frames however if you make the database naturally support those data frame computations then you have tremendous value from the fact that the database actually owns the state and you never need to transfer that state from your your storage to something else so you're bringing computations closer to data so that that's where I see the industry is moving in the next you know five years you just mentioned bringing computations closer to the data and that's obviously been one of the things that Hadoop ecosystem has you know made more readily accessible to folks this idea of data locality you know do you get to take advantage of that with machine learning types of models in general and this pipelining approach that you have in particular or are the pipelines run you know outside of you know separate from any concept of locality of data well locality is is is a loaded concept right so you can start with okay well I want to perform computation exactly on the same machine where the data is stored so that's kind of the extreme case of locality another way to think about is as you look at the at the computation plan let's say in the database terminology that would be a query plan so let's extend the definition of a sequel query plan to a broader concept of you performing some sort of arbitrary scalable computation and you know all the steps in the computation upfront so you can optimize those compute a and produce a query plan for those computations and you will run that query in the distributed environment now then you look at this and certain primitive computations you really really want to perform closer to the data because you will save tremendously on the amount of i/o that is happening for in the data transfer now from there if you do that you also deliver on the concurrency of such computations so now you can perform those computations at a high highly concurrent in highly concurrent environment where thousands of data scientists are attaching to the same data that is collected and centralized and stored in something like mem sequel and then and then they're performing their computations concurrently so there are no you opening up all the data to the organization and and not having any data silos so data locality for computations is is useful where it makes sense and not as useful in a way in a in a broader general purpose way and the perfect system would be engineered around pushing the computations closer to data where where it makes a ton of sense for example you do a lot of filtering or you performing a lot of what it's called group by operations and those operations make sense student to be done locally those operations make sense to be done in a vectorized fashion by leveraging you know the latest and greatest CPU instructions in the intel hardware and so on it makes sense to leverage indexes so you prune massive amounts of data so you don't even touch them for your computations and then from there it makes sense to bring all that data into a stateless distributed environment and perform the rest of the computations so that approach allows you allows you to build greater scale scalability for your system compared to a traditional database compared to Hadoop or compared to bark okay and now you just mentioned the taking advantage of the instruction sets of the underlying hardware and you guys are making an announcement with Intel at Amazon reinvent next week is it related to that area definitely Intel has been supporting vector instructions for some time for over a decade and every new generation of CPUs increases the size of the vector that you can use and perform vectorized operations on top of and where we stand right now it's 512 bits so compared to you know matrices in a GPU this is tiny however right when you perform a last mile computation like dot product it allows you to perform that computation much faster than the memory bandwidth that you have so you actually don't need more for that you know last mile computation then avx-512 because you're limitation is not your compute your limitation is memory bandwidth this is very very different when you train models and and perform computations for building you know deep learning that's where you perform a lot of tensor multiplication and they might have compute compared to the size of your data set is tremendously more when they performing something like dot product that's where you want GPUs okay but once the model is built and the model is trained you actually don't need GPUs to score that model in many many cases what that gives you is the it gives you the ability to to democratize that computation because Intel CPU is is everywhere you know if your machine has a GPU it also has an Intel CPU and what the point of that making that in many cases all you need is a CPU and you're not going to get the computation faster if you add a GPU to the system simply because it's the memory bandwidth that's less your bottleneck in the computation now we use avx-512 for vector dot product for you clearly and distance and for other vector operations that were built into the database we also use avx-512 for general purpose sequel computations and that allows you to just deliver on orders of magnitude faster sequel query processing that our competitors have and certainly what what Hadoop has our spark we're huge fans of that technology and we can't wait when Intel allows us to to perform computations on larger vectors not just 512 bits mmm can you be more specific in terms of some of the actual results you've seen in you know production systems in terms of performance differences yeah absolutely so with nem sequel 6.0 that we just released in October we can do things like a group by operation on 100 billion row table in a sub second so 100 billion ended billion yes Wow so it says a sub second operation that that runs something like give me an average stock price over a hundred billion data points and and groupid by security or by stock value the computation itself is relatively straightforward because we perform that computation quote-unquote on compressed data so and we're using those vectorized operations we achieve the performance of a billion operations per second per core so if you throw a hundred cores into the sector into the system and then sequel is extremely scalable so you can throw a hundred cores or you can throw a thousand course you can achieve those this type of performance on your system and so that's using the avx-512 instructions what were you seeing prior to that well there's a combination of two techniques that goes into into this type of performance the first is how can you perform operations on compressed data so if like I said earlier oftentimes it's the memory bandwidth that's the bottleneck of your computation so the better you compress but the better off you are in the final computation because at the end of the day you scan fewer bytes what exactly do we mean by compressor is a kind of your traditional you know binary compression or we also talking some kind of deduplication or something like that it's a combination so first of all the compression is called columnstore compression and then we also encode similar values to tell let's say you in that example that is said with stock trading you have 10,000 different stocks so if you have 10,000 different stocks then each individual stock can be encoded was just a few bits and so then you only use this many bits to represent each stock value and as you perform your computations you never go back decoding those bits into let's say an integer value or god forbid a string value of a stock right because those computation will become much more expensive so now that you perform computations on compressed data the second step that you do is you optimize your computation for as few branch mispredictions and as few cache misses as possible and those are in big deal because every time you have a branch misprediction you flush your cpu pipeline so that that slows down your computation so if you express your computation so there are no branch minute no branches basically and there are some very very cool papers out there and there's some innovation that we we've done here at nem sequel as well so that will be give you the second boost and then the third boost is now you take all those bits that you represent your values and you perform vector operations on top of those encoded values and then that's where you use avx-512 so now and that gives you kind of another I would say you know three to five X performance improvements just by using avx-512 but if you take that Plus you take operations on compress data plus you take care of branch misprediction you multiply those all those together and then you can routinely see two orders of magnitude improvement in performance Wow that's pretty fantastic kind of the point here is fancy hardware is cool and you want to use it where it's needed but also there's a lot of potential in the hardware you have at hand and if you take full advantage of it of that hardware you will have remarkable results mm-hmm oh that's great I guess as we you know as we kind of wrap up do you have any additional kind of advice to folks that are looking to you know get better you know just to take better advantage of you know whether it's their you know their data storage systems their you know hardware systems and to use these to make better you know predictions and better utilize machine learning in AI well my advice here is don't settle there are systems out there such as mint equal that allows you to take pretty much your end-to-end machine learning AI and application development pipeline and make them completely real-time so whatever you do in batch whatever you have to wait for what the right technology can be squished down to zero is that the technical term squish to zero say that and and that's where the world is going so will will will live in the world in the future where a compete where compute and storage are gonna be utilities and you know if you if you're willing to pay for compute you will have as much compute as you need at any concurrency as you need and any ladies latency as you need and the only thing that is gonna be bounded by is is the amount of money you pay for that computer mmm great well Nikita thank you so much for spending the time to chat with me I really appreciate it yeah absolutely thanks for having me all right everyone that's our show for today thanks so much for listening and for your continued feedback and support for more information on Makita or any of the topics covered in this episode head on over to tamil AI comm slash talk / 84 to follow along with our AWS reinvent series visit to Malaya comm slash reinvent of course we would love to receive your feedback or questions either via a comment on the show notes page or via Twitter - at to male eye or at Sam Charrington thanks once again - until Nirvana for their sponsorship of this series to learn more about deep lens and the other things they've been up to visit Intel nirvana calm and thank you once again for listening and catch you next time\n"