A Conversation with Glenn: The Intersection of Data Science and Insurance

In this conversation, we had the opportunity to sit down with Glenn, a data scientist at a large insurance company. We discussed his thoughts on what makes a good tech company, how he approaches job selection, and the trends that are particularly exciting in the insurance space.

So, what makes a good tech company? According to Glenn, it's not necessarily about being a tech company or having an exciting business model. Instead, he looks at the specific job and what kind of work he'll be doing on it. "If they make 95% of their business model from advertising, 95% of their data scientist is going to work on maximizing advertising revenue," he explained. "So, a lot of these large tech companies have done this for a decade or more, so you're building the 17th version of that model." In contrast, Glenn's own company, which is old but hasn't changed much in terms of its business model, means that he's often tackling challenges for the first time.

When it comes to job selection, Glenn emphasizes the importance of looking at what he'll be working on. "Don't just look at the company, look at what you're actually going to be working on," he advises. "Are you building the first model, the second model, or the 15th model in that area? What kind of interesting data do you get to work with? What kind of variety of models and other data science solutions are you going to build?" These questions help Glenn gauge whether a job is a good fit for him.

The Data Science Function: Setup for Success

Glenn also discussed the importance of setting up a successful data science function. "All their deploying all their models into a business and actually getting the benefit out of it," he said. "If they're not doing that, probably means that the existence of that data science function will eventually be questioned." To achieve this setup for success, Glenn's company has implemented various tools and software solutions.

One trend that Glenn is particularly excited about is mlops (model management platform). "We're just at the beginning of that journey," he said. "It's already made such a big improvement in the way data science functions in reality." Mlops platforms have improved the way models are deployed, monitored, and maintained, making it easier to integrate model development with production deployment.

Another trend that Glenn is excited about is post-deployment software solutions. "With model production, you know you should monitor that model," he explained. "I have a monitoring function that we put in place in the last two years, and now every model gets monitored for things like data drift and score drift." The company has also implemented reporting tools to ensure that business stakeholders are getting value from their models.

The Future of Data Science in Insurance

Glenn is particularly excited about the trends that are emerging in the insurance space. "Mlops engineer is probably one of the hardest professions," he said. "On the same level as data scientist." The intersection of mlops and data science is creating new opportunities for professionals who can bridge these two fields.

To stay connected with Glenn and learn more about his work, you can follow him on LinkedIn. He frequently posts actively on the platform, sharing insights and materials with his community. Additionally, Glenn's company regularly has job openings in various skill sets, including data science, mlops engineers, and project managers. They also have a large internship program and recently started an associate program, which provides opportunities for individuals to join the team while pursuing their master's degrees.

The conversation with Glenn highlights the importance of understanding what makes a good tech company, how to approach job selection, and the trends that are shaping the insurance space. By staying connected with professionals like Glenn and learning more about their work, we can gain insights into the future of data science in various industries.

"WEBVTTKind: captionsLanguage: enyou're listening to data framed a podcast by data camp in this show you'll hear all the latest trends and insights in data science whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization join us for in-depth discussions with data and analytics leaders at the Forefront of the data Revolution Let's Dive Right In hello everyone this is Adele data science evangelist and educator at data Camp when we talk about data maturity on the podcast and the process of becoming data driven we often lose sight of the sheer amount of effort and work it takes to build a strong data science function within an organization building data teams creating connection with the rest of the organization evangelizing the data team's work training others on data literacy and building a data culture is no easy feat but today's guest approaches building high impact data science functions with the clarity few have Glenn Hoffman is the chief analytics officer at New York Life Insurance he is an experienced senior executive in Insurance and financial services Who currently leads the corporate 50 person data science and AI function at New York Life he is responsible for the foundations relationships with many internal groups and a great variety of projects and also leads their data science academy their internal education program for all New York Life employees throughout the the episode we talk about how he built the team at New York Life Insurance the different skills they optimize for delivering career Pathways for data scientists building mlops and model governance teams how to build relationships and Market the work of the data team the ins and outs of the internal data science academy and much more if you enjoyed this episode make sure to rate subscribe and comment but only if you liked it now let's Dive Right In Glenn it's great to have you on the show great to be here thanks for inviting me I'm very excited to speak with you by your work leading data in New York Life Insurance how to organize effective data teams delivering impactful data science use cases within life insurance and much more but before can you give us a bit of a background about yourself and what got you to where you are today let me start with uh the current position and how I got to that and then I can go a little further back if needed yeah so I've been at New York Life a little over five years actually even before coming here I was watching the life insurance industry for a bit and was sort of you know excited about it and then uh it seemed like the right moment you know about 5 years ago to join the industry and do some data science because I mean life insurance is a it's a fairly traditional industry but now over the last few years it's been ripe for Change and the nice thing is it has a a great variety of interesting problems that we'll talk about more then we can build the first Solutions right so New York Life is a pretty big place you know it's one of the for 100 Financial companies so there's lots of challenges and most of them we tackle for the first time so I think that's really exciting to me you know to be at a place where you can be truly Innovative and build first solutions for really interesting problems so that's kind of why I joined the company and you know since then I've buil a function in the team a little further back I mean I've been in you know data science and what we used to call statistics for a long time I started out as a stats Professor you know a long long time ago did five years of that and then joined industry and basically did every job I was a a Hands-On predictive mod early in my career you know eventually got more and more responsibility and then probably over the last decade or so I've been building data science functions for for a few different companies that's really great and you mentioned here the exciting nature of working in data science and life insurance and I'm excited to First discuss that so I'd love to First understand the lay of the land when it comes to the use cases and applications of data science within the life insurance Industries what are some of the exciting data projects you've overseen at New York Life Insurance and where does data science and AI really provide value for Life insurers first of all you know my team the center for data science and AI at New York Life we are here to provide data science and AI expertise Solutions education and work with a variety of business partners to essentially build you know datadriven solutions that make decisions in all kinds of processes so so that's the idea the idea is to create value for the company you know mostly through data driven automated decisions that function in real time one batch and business processes I'll give a variety of examples of that and why they are exciting at least in my view right so there's a lot of different things we do we support a whole bunch of different areas in the company so for example the first one is our distribution system right we sell life insurance through agents and we have buil solutions for agent recruiting we've built solutions to predict agent productivity so it's exciting from a data perspective because it uses a lot of behavioral data and some transaction data stting from a perspective of algorithms because this is actually no network models that we do on that front then we have underwriting and you know for life insurance underwriting means mortality prediction so that is and triaging of various data sets so this means medical data right so from a data perspective exciting because it's a great variety of medical data and from an algorithm perspective it's you know survival modeling classical and machine learning type so that's cool a third example might be Co modeling right the last two years as you might imagine at a life insurance company we've had a pretty significant project trying to model Co mortality we've worked with medical doctor and government Affairs people and that is obviously you know very exciting very you know current data very interesting data and from an algorithm perspective we did hierarchical based pandemic modeling which is very very sophisticated statistical modeling and and then there is there's marketing modeling that we do there's flaud modeling you know flaud modeling is another very exciting problem because there you're looking for a needle and a Hast stack right the huge overing majority of transactions are perfectly fine no problem at all they should happen but every once in a while in a very small minority of cases there's somebody trying to attempt to you know fought and uh you know there's a whole another set of techniques that we use to try to figure that out right so this is just a handful of projects we have going on right now there's obviously much more but that's what shows the great variety and excitement in those projects both in the on the algorithm side as well as on the data side and certainly on the business problems that we're trying to solve that's awesome and you mentioned here a few use cases that create new value such as covid-19 modeling to use cases that accelerate value by reducing costs and generating more efficiencies so of course delivering on these use cases requires a high performing data science team and this is something I'm excited to deep dive with you on and really break down so I'd love it if you can walk us through what you found to be the Hallmarks of a successful and impactful data science function especially within large and complex Enterprises like New York Life Insurance I guess I start with saying it is the third data science function that I'm building in my career so I learned a lot from all the mistakes I made the first two times and I think we did a lot better this time Al I think there was three big aspects of putting a successful data science function together first is obviously Talent so hiring great data scientists and great data science managers and leaders is important but then also data scientists don't operate in a vacuum so also hiring a bunch of other skill sets to make it work so I'm thinking of you know machine learning operations Engineers data Engineers project managers model governments you know a little bit of training and development so I think all these are important talent to have to make a team successful the second is platform and Technology right we set up a platform both for you know model development as well as probably even more importantly for model deployment into production so you have all the you know the regular development platforms you know the python environment the all environment geospatial environment but then we really spent a lot of time over the last three years I would say on creating a deployment platform that allows us to easily put every model we built in production both is in production on our side as well as in production at the business partner that's you know using the models every day so that's the second one the third one is relationships especially in a large and complex organization like new of Life relationships inside the company to our various business partners and other stakeholders so we spend a lot of time and effort and communicating pretty consistently and frequently as our business partners both and theate ation and designing of projects execution of projects and then post project you know the use of our models and products so I'd love to unpack this with you you know given the diversity of the roles and stakeholders On Any Given data project or team how are these different roles organized at New York Life Insurance do you have a centralized data function you know like a center of excellence or an embedded model where these different roles are embedded in functional teams or something in between from a corporate perspective the function that I lead the center for data science AI is a corporate function right so we support you know almost all of the company with with data science AI so that's sort of centralized from a data perspective there is a corporate data team and then there's a few individual data teams that are closer to the business or the function right so I have a little data team in my organization that takes advantage of what the cor data team produces but then that's sort of the last mile of data processing if you want that we need and then there in my team I have a specific machine learning operations team uh so that has a leader you know who is an experienced engineer and he's got like I believe seven Engineers you know on his team that are completely specializing machine learning operations I have a a project management office that is you know L by a a manager and have several project managers on that have a change manager on that and then we have a development team that basically does a lot of train training and community building for my team as well as the rest of the company you know around data sign so the nice thing I mean I you know you have some scale you know that my team is about 55 people you know if you count fact then it's probably about 75 it creates a necessary scale to have career pathing for all these skill sets you know on the team so let's say you know a junior data scientist that might join the team or even a midcareer you know data scientist that joins the team has many options you know both from a the type of work they do in terms of what area theyve built models in and the company as well as a a career progress right so they become rly senior data scientists that works on more complex stuff they become a manager they can become a manager of managers you know all the way up to myspot right so we have a good career track for everybody you know on the team we also take care of continued education we do tuition reimbursement you allow people to go to conferences that they can talk with their skill set that's great and you mentioned here creating development tracks for a lot of the talent on the team when do you start thinking about creating these Pathways and when should data leaders building data functions start thinking about these Pathways and what are the lessons that you can share from your experience I think this is sort of important when you think about you have a centralized teams if you have you know small teams in different places you need to have at least one you know sort of central team that has critical math M to enable the career passing to enable the funding for platform uh and to enable all the support you know that you need I started thinking about that right from the start when I put the team together right so I designed my team or at least that you know for the most part in this way right from the start there are a few functions we added later as we gain you know scale but the general concept was there from beginning I think it is important because if you're joining a five person analytics team somewhere then you do have to wonder what are you going two years or three years down the line that's definitely true and harping on the mlops team and mlops functionality you've added in the data team at New York Life Insurance this strikes me as relative to where the industry is at as a sign of a mature data science function to have dedicated talents towards deployment so can you share the process of building such a function and what were some of the best practices you learned along the way in building an mop Steam from scratch you know I think it's been important for a while but now we have mag pedal technology that actually makes it a little easier to make this reality and that has evolved you know tremendously over just the last few years the importance for me and the reason why I put so much focus on it I see a lot of data science team out there that built really interesting exciting models and then never deploy them into production and data scientists get frustrated because they built something something that nobody uses and business gets frustrated because they don't get any value out of the data science team they built right so I I I wanted to avoid that completely and say the focus is on productionizing we want to build great models that we can have trust in and that are diligent and sound but they have to be Deployable otherwise there really no point so pretty early on you know as soon as the first model has got to a stage where we worry about you know where are they going to go started building this mlops function which sort of sits in between the data scientists and you know the pure technology folks the mlops function helps the data scientist works with the data scientist every day to build models that are ready for production first from a code perspective uh because you know Engineers can typically ride a bit better code than data scientists can but also from a data perspective is the data that's being used ready for production in that form and what are the concerns that are going to come up later when we productionize and then we also invested in platforms so now we have a kubernetes space you know deployment platform where we can promote code directly from python into the platform and probably a little bit of code polishing have these things be ready for production and then we build an API from our production platform to any production system in the company so our models can be used in real time by any production system in the company and that was really the key and creating value relatively quickly from everything that we build so I think it makes greatly satisfactory for data scientists because everything they build is being used by the company in production every day right so I think that creates a certain sense of satisfaction it's great for the business so you know when people ask me you know what kind of value are providing I can point to a dozen or more models that are in production that people are using every day so it's not a hard question to answer definitely it must be so exciting to show the ROI of a data team and I'd love to talk about building mlops teams for hours but given the topic today and discussing you know your work leading data science and New York Life Insurance I'd be remiss not to talk about the data culture component when building a high impact data team you've been very outspoken about the importance of transparency and education when it comes to the data team's work and impact because it ultimately means both buying from decision makers who own different lines of business and practitioners who are on the ground and predominantly relying on their own subject matter expertise so I'd love to first start off by understanding how you went about getting that buy in and partnering with these different audiences and getting them excited about the potential of data science yeah that is a very key thing that you know we spend a lot of time on to me it's a lot of communication a lot more than than you might think in a technical function so we have a what I call sort of a multi-level communication approach so no matter who you are on my team no matter what level you're on everybody has a set of stakeholders that we actually Define and we have an exercise about that every once in a while where we say okay these in my case probably like 30 or 40 people in the case of a junior data scientist maybe eight to 10 people those are the relationships you have have to maintain those real stakeholders you have to talk to them often and you know you have over time make them understand the value of data science for their business problem but also you have to understand their business problem and you have to understand what they worry about every day because communication can really only happen when most people understand a little bit of where the other person is coming from so you have this sort of multi-level strategy we get to ghetto frequently on the site of my team to actually strategize about who communicates with whom what do you communicating and you know how are we basically getting bu in on what we do the other principle that I have is I don't disend mediate my data scientist right so data scientists do talk directly to business partners Engineers do talk directly to business partners there's no like MBA translator in between I don't believe in that it does however mean that scientists do have to you know learn how to communicate fairly well there's uh pros and cons to each of those but that's the model that we have done and I guess the third thing I will say is you know so first was communication on multi level and often sing no it is just a mediation of technical people and then the third is we don't like go away and do our own thing and then come back with a solution right so every project has like weekly project meetings everything we do we communicate the stakeholders right away and get their Buy in often even on relatively small decisions on a project such that by the time you're halfway into the project it is not our project it is their project and they speak about it and when we go to like a status meeting in the business unit or something I prefer our business partners to present the project and not us to present the project because that's aend the idea that it's really their project so by the time you come around the end of it and you know a decision gets made on how to deploy it and how fast to deploy it it's their project they want to deploy no longer what we say and that's the best case that's such a awesome approach and following up here how have you been able to instill this mindset within the data scientist on your team especially the more technically minded ones and gear them to be relationship driven and value driven and not just focus on the technical aspects of their work you know it's constant reinforcement and it's not right for every single data scientist right so we try to recruit the people that are both technically very strong but also have the desire to solve business problems the desire to communicate the desire to teach sometimes right to desire to sort of advocate for their craft so to speak so recruiting has something to do with it and then of course we do a lot of training do a lot of soft skills training communication training PowerPoint training and then there's a lot of under job learning right learning from their manager obviously you know all of my managers are very strong in that so people can learn from the managers they get feedback it is something we focus on and you know that's how it happens another key component of creating a data culture as well is by ensuring that there there's a community of practice around data and that folks can feel like they can get involved so can you walk us through the different ways you've addressed this and how you got the general population at New York Life Insurance excited there's a lot of sort of community work so I have two people on my team that do nothing but training and Community for the rest of the company so on the training side we created this thing we call data science academy and it is a training program for really anybody in the entire company who has some interest in data science what we have done is we've put sort of curated learning tracks together mostly using existing online courses that we put together into a track we also created some of our own content from our projects that we do and we have learning tracks anywhere from like on the high end sort of the one year 10 course coding based data science uh learning track if somebody wants to make a career change becoming data scientist all the way to more of like a business style you know non-coding multi-course track to like a get to no track where if you just want to watch a few videos and look at a few articles about data science you can do that more recently I actually taught myself a half day data science course for all the executive officers of the New York Life and we got about those are very high level on busy people about half of the executive officers actually enrolled volunte into a data science course so that's you know the data science academy is one of the most popular learning programs in the entire company today so that creates a lot of branding for us internally it also creates people who are interested in data science at the various different business units that we want to collaborate with and it's always good to have that it's always good to have somebody to talk to in other units that have an interest in data science it makes it a lot easier for us to engage so that's one thing the second thing there is what we call the data science community so out of all these people that have some interest in data science in the entire company you know we created a mailing list and we have a lot of events so we put on seminars lunch and learns we bring in external speakers we interview people inside the company and that is really exciting and we in the days of Zoom meetings over the pandemic we have an average of like 300 people logging on to the zoom call that is a technical data science seminal that's a pretty good audience that part is is working really well and then of course you know for my team itself we have a lot of Po Technical non-technical Training as well so that's understood but this sort of Landing effort is more for the entire company I think it's important especially in a place like New York Life where where data science is just sort of you know in the first five years of development that's fantastic and I love the degree of personalization that you have in these different learning Pathways there's always a learning track for you no matter where you are at the organization but but also how you've paid attention to Executive training to generate that buyin and digging deep on that executive training component these are folks who are really busy they have a strong gut instinct that they developed over decades how do you create a learning program for such an audience and what does success look like for you in this case we originally thoughted planning it actually before the pandemic hit and then was supposed to be in person in a conference room for half a day you know with some neck breaks and all of that well you know then Co hit and that didn't work out so we had to reengineer the entire program for virtual experience one of the things I also should mention that we also use in the executive track we use for other purposes is for all of the large projects after the project we actually create a video and the video will have people from my team speak it will have our business partners speak might have somebody from technology speak and it's like a you know three to four minute kind of video not hard to watch and we can show it everybody and and that's impactful so we use the videos also in the executive offical train we used some general materials around what is data signs how is applied in various Industries and then I brought a lot of examples from from New York Life data science which is really relevant to them and uh to your question do you know what a success look like in a training like that so first of all if you've done trainings on Zoom I gu the first criteria is that people keep their camera on and they attend right so that's actually not as trivial as it sounds the second thing is already they asking a lot of questions during the course but the third thing that's proba most relevant is are they coming up with their own ideas of data science projects right so after getting an idea of what data science can do do they come with their own ideas you know at the end of the course or doing a discussion in the course and then follow up on those ideas after the course with you know a couple of meetings to explore new ideas that they conceived after learning a little more about data science so we had probably 20 different ideas coming out of these training sessions that they were interested in so that's how I would measure success of the training for that's such a great way to think about it because if you're able to create evangelists out of your partner Executives that will make life much easier for you as data leader and harping on the upscaling for the broader population what are the tools and skills you've prioritized and how has the upscaling program move the needle for you when it comes to creating a common data language in the organization so the skills sort of vary depending on the audience there right so I think for executive level folks or even like maybe director level folks if they all in the non-technical areas like sales or HR or those types of folks that we just want to create a general understanding of how to recognize a problem where there may be a data science solution right so understand enough about data science to recognize a problem that might have a data science and then for a little bit more technical folks it would be giving them a sense of what data science Solutions are looking like and and even for mid-level folks giving them an idea of what does it take to build a data science solution as well the types of people that need to be involved how early do these people need to get involved you know before the data is all screwed up and all that and really you know kind of enable them to Ay recognize our skill set as data scientist to be different from maybe a business analyst or from an act or from an engineer and recognize when they should get us involved and recognize the benefit of getting us involved then there was the other cohort of people that are thinking about a career change to data science for those folks we have to teach some real data science you know that is also something we can do so we talked about the skill element the culture element and to a certain extent the platform element but something that cuts across our discussion so far is the importance of building relationships so you've partnered with a lot of different folks within New York Life Insurance how do you manage these relationships and who are your main stakeholders and how do you ensure that the data teams road map is aligned with the business's priorities yeah okay there was a lot of stuff in there on the relationship side you basically I mean everybody on the team has you know a set of stakeholders to maintain relationships with right I mean for myself I probably have regular meetings or lunches or something this I don't know maybe 4 40 or 50 different executive officers in company right just to kind of understand what's going on in their unit you know what are their priorities where could I potentially help or my team could help and stay on top of what's going on in the business so that we can detect opportunities you know as soon as they arise for folks you know that reporting to me they have a large set of stakeholders also and they want to be a little more tactical and say okay you know what projects are you starting what's coming next year how can we help what's the status of current projects we're engaged in is everybody happy are you happy or your people happy you know to keep that going for more individual contributor Types on my team it will be the stakeholders of a particular project to make sure they talk to them not just in official meetings but maybe you know have a lunch with them have oneon-one meeting with them to just make sure that we hitting the Mark we're on the right track they understand what we're doing there's no gaps in knowledge or understanding uh no gaps and goals you know that's kind of the practical way that we do that on on the communication side now on the planning side that's I think the other part of your question there needs to be budget alignment when I say deployment you know it's nice to have a platform to deploy all the models but deploying models is not free right you also have to the budget to deploy so you know in our annual financial planning process I mean I don't only have to worry about my own budget that's complex enough but I have to worry about the budget of every one of my business partners too and make sure they have budgeted for the deployment of our models into their environment so that that will actually happen you know so there's a lot of coordination going on during the planning process and the budgeting process to make sure that you know you have to Budget on the tech side we have to Dutch budget in the business side they actually get miles deployed in the coming year and that's very important there's definitely a lot of complexity in managing these relationships and speaking of complexity I'd love it if you can also walk us through the challenges and working with data within an industry that is highly regulated and how do you ensure that you're consistently innovating responsibly that's uh very interesting and I guess more relevant topic every year I mean I want to make maybe two points on that one one is I mean regulation is there to protect consumers right so I mean New York live as a company is very much aligned with that in fact you know we're not a you know Market traded company we're what's called a mutual company so New York Life is actually owned by policy holders of New York Liv it was entally owned by our customers so hence our philosophy is very much to Delight by our customers and that is at least in principle you know the purpose of Regulation as well uh so we appreciate our wellth thought out regulation and we're quite happy you know to follow that and influence that in the right way it's very much aligned with all company so that's the first point the second point for you know engineers and data scientists is the uh compliance with regulation is not boring at all it's actually creates a lot of interesting analytical challenges in its own life so when you think about you know disp impact testing for protective classes that creates a whole new set of uh algorithms that you have to develop to do this intelligent right because this would impact is not sort of a you know black and white thing it's very it's complex this would impact of a model well a model is multivariant which pieces of that model are actually creating or not creating some potential disperate impact how do you Analyze That in the multivariate situation there's a complex modeling algorithm behind the Val you draw the line is not a qualitative Concept in fact is it's quantitative well you draw the line between you know some correlation that might indicate this put Impact versus not so it creates a lot of interesting analytical challenges I'd say that you know the head of my model government team is probably one of the most interesting positions on the whole team that's a great way of framing it you know we're constrain add a new later of complexity and Intrigue for different types of data science projects so can you walk me through the process of setting up a model governance team and the lessons that you can share with other data leaders here we set up a governments model govern team probably about two or three years ago you know once little organization was mature enough to be our own headcount on that it's what I call the first line of defense right so I have my own model governance team first line of defense then the company has a corporate model governance team that's the second line of fans and every model gets validated on my team so every model that gets built gets validated by people that had no involvement in building that's independent validation we look at data validation we look at model validation and then of course we look at any kinds of regulations or legislation that you know this may be subject to depending on the use case and the data that's in it so the leader of my model governance team has to be a very good data scientist who actually understands all the algorithms and the data but also is direct connected to our government Affairs team and the company and the legal Team stand regulation legislation not just of today but of tomorrow because these models we built they should be in production for many years so we have to look at you know what is being proposed as legislation in the state of Ohio you know in the next year or something like right is also Ste regulated so we don't just have one regulator I mean the federal government is also a regulator but we also have 50 other Regulators you know one in every state so that makes it pretty interesting job that's really fascinating now Glenn as we are reaching the end of our episode I'd love to ask you a few questions before you close out the first one is what would be your advice for any upand cominging data scientists Fielding choices between joining Tech or more you know quote unquote traditional Industries such as Finance or Insurance there is sort of the misconception that I want to debunk a little bit that you know if it's a tech company if exciting if it's you know insurance company it's boring that's not true at all right what you should look at you know what is this specific job and what kind of thing are you going to work on right because if they make 95% of their business model from advertising 95% of their data scientist is going to work on is maximization of advertising levenue so an individual job you know might be focusing on optimizing one single search term for the next year right so a lot of these large tech companies have done this for a decade or more so you're building the 17's version of that month right whereas at a place like New York Life you know you're building the first version of a model that has never been tackled before in a variety of different challenges the company is old you know 175 years old but the challenges we tackle from a data science perspective we're often tackling for the first time look at this specific job don't just look at the company look at what you're actually going to be working on are you building the first model the second model or the 15th model in that area what kind of interesting data do you get to work with what kind of variety of models and other data science Solutions are you going to build those are the people you're going to learn from and then you know obviously the the most important thing is the data science function setup for Success meaning all their deploying all their models into a business and actually get the benefit out of it because if they're not doing that probably means that the existence of that data science function will eventually be questioned and given the exciting use cases of data science and Ai and insurance that we just discussed what are some of the trends you're particularly excited about that will impact the insurance space yeah I mean you know like you said we can talk a long time about mlops so I think mlops is a huge Trend and we're just at the beginning of that journey and it's already made such a big Improvement in the way data science functions in reality right I mean now we have good platforms that you can deploy models on those platforms will probably get even better be more of an integrated way that you know integrates even better with the model development the thing that we're we're actually investing bit of money into now is what happens post deployment of the model in terms of software Solutions right so with the model production it's running now you know you should monitor that model I have a monitoring function that we put in place in the last two years and uh so every model now gets monitor both for things like data drift and score drift if something goes wrong with incoming data somebody gets loaded right away before we make a bunch of bad decisions and then also reporting so you know we want to play a model after the model gets deployed our business part is well I not that they're actually going to get the value that we promised right so we have to do a bunch of reporting on that production model we'd like to not write custom code for all of that would be a lot of work so we've been you know looking at software and actually we made a software decision that helps us Monitor and hold on models and production so that's all part of s the mlops process and I think you know mlops engineer is probably uh one of the hardest professions right on the same level dat scientist that's great now Glenn as we close out do you have any final call to action before we wrap up today's episode like some of the things I said if you want to kind of learn more over time definitely feel free to connect on LinkedIn with me I'm always happy to provide material to the community I post pretty actively on LinkedIn so that's a way to stay in touch also you know I would be of a Miss to not mention job opportunities on the team so I frequently have openings on the team on all the different skill sets that I mentioned so data science mops Engineers data Engineers project managers model governance at many different levels we also have a a large internship program that we run every summer and recently we started a what we call an associate program so starting this year we actually hire several people was a bachelor's degree in a technical area and then they spent three years on my team rotating through a few different jobs and at the same time doing a master's degree for which we paid the tuition so that's yet another you know great way to join the team and become a data scientist thank you so much Glenn for coming on data framed thanks so much for having me you've been listening to data framed a podcast by data camp keep connected with us by subscribing to the show in your favorite podcast player please give us a rating leave a comment and share episodes you love that helps us keep delivering insights into all things data thanks for listening until next timeyou're listening to data framed a podcast by data camp in this show you'll hear all the latest trends and insights in data science whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization join us for in-depth discussions with data and analytics leaders at the Forefront of the data Revolution Let's Dive Right In hello everyone this is Adele data science evangelist and educator at data Camp when we talk about data maturity on the podcast and the process of becoming data driven we often lose sight of the sheer amount of effort and work it takes to build a strong data science function within an organization building data teams creating connection with the rest of the organization evangelizing the data team's work training others on data literacy and building a data culture is no easy feat but today's guest approaches building high impact data science functions with the clarity few have Glenn Hoffman is the chief analytics officer at New York Life Insurance he is an experienced senior executive in Insurance and financial services Who currently leads the corporate 50 person data science and AI function at New York Life he is responsible for the foundations relationships with many internal groups and a great variety of projects and also leads their data science academy their internal education program for all New York Life employees throughout the the episode we talk about how he built the team at New York Life Insurance the different skills they optimize for delivering career Pathways for data scientists building mlops and model governance teams how to build relationships and Market the work of the data team the ins and outs of the internal data science academy and much more if you enjoyed this episode make sure to rate subscribe and comment but only if you liked it now let's Dive Right In Glenn it's great to have you on the show great to be here thanks for inviting me I'm very excited to speak with you by your work leading data in New York Life Insurance how to organize effective data teams delivering impactful data science use cases within life insurance and much more but before can you give us a bit of a background about yourself and what got you to where you are today let me start with uh the current position and how I got to that and then I can go a little further back if needed yeah so I've been at New York Life a little over five years actually even before coming here I was watching the life insurance industry for a bit and was sort of you know excited about it and then uh it seemed like the right moment you know about 5 years ago to join the industry and do some data science because I mean life insurance is a it's a fairly traditional industry but now over the last few years it's been ripe for Change and the nice thing is it has a a great variety of interesting problems that we'll talk about more then we can build the first Solutions right so New York Life is a pretty big place you know it's one of the for 100 Financial companies so there's lots of challenges and most of them we tackle for the first time so I think that's really exciting to me you know to be at a place where you can be truly Innovative and build first solutions for really interesting problems so that's kind of why I joined the company and you know since then I've buil a function in the team a little further back I mean I've been in you know data science and what we used to call statistics for a long time I started out as a stats Professor you know a long long time ago did five years of that and then joined industry and basically did every job I was a a Hands-On predictive mod early in my career you know eventually got more and more responsibility and then probably over the last decade or so I've been building data science functions for for a few different companies that's really great and you mentioned here the exciting nature of working in data science and life insurance and I'm excited to First discuss that so I'd love to First understand the lay of the land when it comes to the use cases and applications of data science within the life insurance Industries what are some of the exciting data projects you've overseen at New York Life Insurance and where does data science and AI really provide value for Life insurers first of all you know my team the center for data science and AI at New York Life we are here to provide data science and AI expertise Solutions education and work with a variety of business partners to essentially build you know datadriven solutions that make decisions in all kinds of processes so so that's the idea the idea is to create value for the company you know mostly through data driven automated decisions that function in real time one batch and business processes I'll give a variety of examples of that and why they are exciting at least in my view right so there's a lot of different things we do we support a whole bunch of different areas in the company so for example the first one is our distribution system right we sell life insurance through agents and we have buil solutions for agent recruiting we've built solutions to predict agent productivity so it's exciting from a data perspective because it uses a lot of behavioral data and some transaction data stting from a perspective of algorithms because this is actually no network models that we do on that front then we have underwriting and you know for life insurance underwriting means mortality prediction so that is and triaging of various data sets so this means medical data right so from a data perspective exciting because it's a great variety of medical data and from an algorithm perspective it's you know survival modeling classical and machine learning type so that's cool a third example might be Co modeling right the last two years as you might imagine at a life insurance company we've had a pretty significant project trying to model Co mortality we've worked with medical doctor and government Affairs people and that is obviously you know very exciting very you know current data very interesting data and from an algorithm perspective we did hierarchical based pandemic modeling which is very very sophisticated statistical modeling and and then there is there's marketing modeling that we do there's flaud modeling you know flaud modeling is another very exciting problem because there you're looking for a needle and a Hast stack right the huge overing majority of transactions are perfectly fine no problem at all they should happen but every once in a while in a very small minority of cases there's somebody trying to attempt to you know fought and uh you know there's a whole another set of techniques that we use to try to figure that out right so this is just a handful of projects we have going on right now there's obviously much more but that's what shows the great variety and excitement in those projects both in the on the algorithm side as well as on the data side and certainly on the business problems that we're trying to solve that's awesome and you mentioned here a few use cases that create new value such as covid-19 modeling to use cases that accelerate value by reducing costs and generating more efficiencies so of course delivering on these use cases requires a high performing data science team and this is something I'm excited to deep dive with you on and really break down so I'd love it if you can walk us through what you found to be the Hallmarks of a successful and impactful data science function especially within large and complex Enterprises like New York Life Insurance I guess I start with saying it is the third data science function that I'm building in my career so I learned a lot from all the mistakes I made the first two times and I think we did a lot better this time Al I think there was three big aspects of putting a successful data science function together first is obviously Talent so hiring great data scientists and great data science managers and leaders is important but then also data scientists don't operate in a vacuum so also hiring a bunch of other skill sets to make it work so I'm thinking of you know machine learning operations Engineers data Engineers project managers model governments you know a little bit of training and development so I think all these are important talent to have to make a team successful the second is platform and Technology right we set up a platform both for you know model development as well as probably even more importantly for model deployment into production so you have all the you know the regular development platforms you know the python environment the all environment geospatial environment but then we really spent a lot of time over the last three years I would say on creating a deployment platform that allows us to easily put every model we built in production both is in production on our side as well as in production at the business partner that's you know using the models every day so that's the second one the third one is relationships especially in a large and complex organization like new of Life relationships inside the company to our various business partners and other stakeholders so we spend a lot of time and effort and communicating pretty consistently and frequently as our business partners both and theate ation and designing of projects execution of projects and then post project you know the use of our models and products so I'd love to unpack this with you you know given the diversity of the roles and stakeholders On Any Given data project or team how are these different roles organized at New York Life Insurance do you have a centralized data function you know like a center of excellence or an embedded model where these different roles are embedded in functional teams or something in between from a corporate perspective the function that I lead the center for data science AI is a corporate function right so we support you know almost all of the company with with data science AI so that's sort of centralized from a data perspective there is a corporate data team and then there's a few individual data teams that are closer to the business or the function right so I have a little data team in my organization that takes advantage of what the cor data team produces but then that's sort of the last mile of data processing if you want that we need and then there in my team I have a specific machine learning operations team uh so that has a leader you know who is an experienced engineer and he's got like I believe seven Engineers you know on his team that are completely specializing machine learning operations I have a a project management office that is you know L by a a manager and have several project managers on that have a change manager on that and then we have a development team that basically does a lot of train training and community building for my team as well as the rest of the company you know around data sign so the nice thing I mean I you know you have some scale you know that my team is about 55 people you know if you count fact then it's probably about 75 it creates a necessary scale to have career pathing for all these skill sets you know on the team so let's say you know a junior data scientist that might join the team or even a midcareer you know data scientist that joins the team has many options you know both from a the type of work they do in terms of what area theyve built models in and the company as well as a a career progress right so they become rly senior data scientists that works on more complex stuff they become a manager they can become a manager of managers you know all the way up to myspot right so we have a good career track for everybody you know on the team we also take care of continued education we do tuition reimbursement you allow people to go to conferences that they can talk with their skill set that's great and you mentioned here creating development tracks for a lot of the talent on the team when do you start thinking about creating these Pathways and when should data leaders building data functions start thinking about these Pathways and what are the lessons that you can share from your experience I think this is sort of important when you think about you have a centralized teams if you have you know small teams in different places you need to have at least one you know sort of central team that has critical math M to enable the career passing to enable the funding for platform uh and to enable all the support you know that you need I started thinking about that right from the start when I put the team together right so I designed my team or at least that you know for the most part in this way right from the start there are a few functions we added later as we gain you know scale but the general concept was there from beginning I think it is important because if you're joining a five person analytics team somewhere then you do have to wonder what are you going two years or three years down the line that's definitely true and harping on the mlops team and mlops functionality you've added in the data team at New York Life Insurance this strikes me as relative to where the industry is at as a sign of a mature data science function to have dedicated talents towards deployment so can you share the process of building such a function and what were some of the best practices you learned along the way in building an mop Steam from scratch you know I think it's been important for a while but now we have mag pedal technology that actually makes it a little easier to make this reality and that has evolved you know tremendously over just the last few years the importance for me and the reason why I put so much focus on it I see a lot of data science team out there that built really interesting exciting models and then never deploy them into production and data scientists get frustrated because they built something something that nobody uses and business gets frustrated because they don't get any value out of the data science team they built right so I I I wanted to avoid that completely and say the focus is on productionizing we want to build great models that we can have trust in and that are diligent and sound but they have to be Deployable otherwise there really no point so pretty early on you know as soon as the first model has got to a stage where we worry about you know where are they going to go started building this mlops function which sort of sits in between the data scientists and you know the pure technology folks the mlops function helps the data scientist works with the data scientist every day to build models that are ready for production first from a code perspective uh because you know Engineers can typically ride a bit better code than data scientists can but also from a data perspective is the data that's being used ready for production in that form and what are the concerns that are going to come up later when we productionize and then we also invested in platforms so now we have a kubernetes space you know deployment platform where we can promote code directly from python into the platform and probably a little bit of code polishing have these things be ready for production and then we build an API from our production platform to any production system in the company so our models can be used in real time by any production system in the company and that was really the key and creating value relatively quickly from everything that we build so I think it makes greatly satisfactory for data scientists because everything they build is being used by the company in production every day right so I think that creates a certain sense of satisfaction it's great for the business so you know when people ask me you know what kind of value are providing I can point to a dozen or more models that are in production that people are using every day so it's not a hard question to answer definitely it must be so exciting to show the ROI of a data team and I'd love to talk about building mlops teams for hours but given the topic today and discussing you know your work leading data science and New York Life Insurance I'd be remiss not to talk about the data culture component when building a high impact data team you've been very outspoken about the importance of transparency and education when it comes to the data team's work and impact because it ultimately means both buying from decision makers who own different lines of business and practitioners who are on the ground and predominantly relying on their own subject matter expertise so I'd love to first start off by understanding how you went about getting that buy in and partnering with these different audiences and getting them excited about the potential of data science yeah that is a very key thing that you know we spend a lot of time on to me it's a lot of communication a lot more than than you might think in a technical function so we have a what I call sort of a multi-level communication approach so no matter who you are on my team no matter what level you're on everybody has a set of stakeholders that we actually Define and we have an exercise about that every once in a while where we say okay these in my case probably like 30 or 40 people in the case of a junior data scientist maybe eight to 10 people those are the relationships you have have to maintain those real stakeholders you have to talk to them often and you know you have over time make them understand the value of data science for their business problem but also you have to understand their business problem and you have to understand what they worry about every day because communication can really only happen when most people understand a little bit of where the other person is coming from so you have this sort of multi-level strategy we get to ghetto frequently on the site of my team to actually strategize about who communicates with whom what do you communicating and you know how are we basically getting bu in on what we do the other principle that I have is I don't disend mediate my data scientist right so data scientists do talk directly to business partners Engineers do talk directly to business partners there's no like MBA translator in between I don't believe in that it does however mean that scientists do have to you know learn how to communicate fairly well there's uh pros and cons to each of those but that's the model that we have done and I guess the third thing I will say is you know so first was communication on multi level and often sing no it is just a mediation of technical people and then the third is we don't like go away and do our own thing and then come back with a solution right so every project has like weekly project meetings everything we do we communicate the stakeholders right away and get their Buy in often even on relatively small decisions on a project such that by the time you're halfway into the project it is not our project it is their project and they speak about it and when we go to like a status meeting in the business unit or something I prefer our business partners to present the project and not us to present the project because that's aend the idea that it's really their project so by the time you come around the end of it and you know a decision gets made on how to deploy it and how fast to deploy it it's their project they want to deploy no longer what we say and that's the best case that's such a awesome approach and following up here how have you been able to instill this mindset within the data scientist on your team especially the more technically minded ones and gear them to be relationship driven and value driven and not just focus on the technical aspects of their work you know it's constant reinforcement and it's not right for every single data scientist right so we try to recruit the people that are both technically very strong but also have the desire to solve business problems the desire to communicate the desire to teach sometimes right to desire to sort of advocate for their craft so to speak so recruiting has something to do with it and then of course we do a lot of training do a lot of soft skills training communication training PowerPoint training and then there's a lot of under job learning right learning from their manager obviously you know all of my managers are very strong in that so people can learn from the managers they get feedback it is something we focus on and you know that's how it happens another key component of creating a data culture as well is by ensuring that there there's a community of practice around data and that folks can feel like they can get involved so can you walk us through the different ways you've addressed this and how you got the general population at New York Life Insurance excited there's a lot of sort of community work so I have two people on my team that do nothing but training and Community for the rest of the company so on the training side we created this thing we call data science academy and it is a training program for really anybody in the entire company who has some interest in data science what we have done is we've put sort of curated learning tracks together mostly using existing online courses that we put together into a track we also created some of our own content from our projects that we do and we have learning tracks anywhere from like on the high end sort of the one year 10 course coding based data science uh learning track if somebody wants to make a career change becoming data scientist all the way to more of like a business style you know non-coding multi-course track to like a get to no track where if you just want to watch a few videos and look at a few articles about data science you can do that more recently I actually taught myself a half day data science course for all the executive officers of the New York Life and we got about those are very high level on busy people about half of the executive officers actually enrolled volunte into a data science course so that's you know the data science academy is one of the most popular learning programs in the entire company today so that creates a lot of branding for us internally it also creates people who are interested in data science at the various different business units that we want to collaborate with and it's always good to have that it's always good to have somebody to talk to in other units that have an interest in data science it makes it a lot easier for us to engage so that's one thing the second thing there is what we call the data science community so out of all these people that have some interest in data science in the entire company you know we created a mailing list and we have a lot of events so we put on seminars lunch and learns we bring in external speakers we interview people inside the company and that is really exciting and we in the days of Zoom meetings over the pandemic we have an average of like 300 people logging on to the zoom call that is a technical data science seminal that's a pretty good audience that part is is working really well and then of course you know for my team itself we have a lot of Po Technical non-technical Training as well so that's understood but this sort of Landing effort is more for the entire company I think it's important especially in a place like New York Life where where data science is just sort of you know in the first five years of development that's fantastic and I love the degree of personalization that you have in these different learning Pathways there's always a learning track for you no matter where you are at the organization but but also how you've paid attention to Executive training to generate that buyin and digging deep on that executive training component these are folks who are really busy they have a strong gut instinct that they developed over decades how do you create a learning program for such an audience and what does success look like for you in this case we originally thoughted planning it actually before the pandemic hit and then was supposed to be in person in a conference room for half a day you know with some neck breaks and all of that well you know then Co hit and that didn't work out so we had to reengineer the entire program for virtual experience one of the things I also should mention that we also use in the executive track we use for other purposes is for all of the large projects after the project we actually create a video and the video will have people from my team speak it will have our business partners speak might have somebody from technology speak and it's like a you know three to four minute kind of video not hard to watch and we can show it everybody and and that's impactful so we use the videos also in the executive offical train we used some general materials around what is data signs how is applied in various Industries and then I brought a lot of examples from from New York Life data science which is really relevant to them and uh to your question do you know what a success look like in a training like that so first of all if you've done trainings on Zoom I gu the first criteria is that people keep their camera on and they attend right so that's actually not as trivial as it sounds the second thing is already they asking a lot of questions during the course but the third thing that's proba most relevant is are they coming up with their own ideas of data science projects right so after getting an idea of what data science can do do they come with their own ideas you know at the end of the course or doing a discussion in the course and then follow up on those ideas after the course with you know a couple of meetings to explore new ideas that they conceived after learning a little more about data science so we had probably 20 different ideas coming out of these training sessions that they were interested in so that's how I would measure success of the training for that's such a great way to think about it because if you're able to create evangelists out of your partner Executives that will make life much easier for you as data leader and harping on the upscaling for the broader population what are the tools and skills you've prioritized and how has the upscaling program move the needle for you when it comes to creating a common data language in the organization so the skills sort of vary depending on the audience there right so I think for executive level folks or even like maybe director level folks if they all in the non-technical areas like sales or HR or those types of folks that we just want to create a general understanding of how to recognize a problem where there may be a data science solution right so understand enough about data science to recognize a problem that might have a data science and then for a little bit more technical folks it would be giving them a sense of what data science Solutions are looking like and and even for mid-level folks giving them an idea of what does it take to build a data science solution as well the types of people that need to be involved how early do these people need to get involved you know before the data is all screwed up and all that and really you know kind of enable them to Ay recognize our skill set as data scientist to be different from maybe a business analyst or from an act or from an engineer and recognize when they should get us involved and recognize the benefit of getting us involved then there was the other cohort of people that are thinking about a career change to data science for those folks we have to teach some real data science you know that is also something we can do so we talked about the skill element the culture element and to a certain extent the platform element but something that cuts across our discussion so far is the importance of building relationships so you've partnered with a lot of different folks within New York Life Insurance how do you manage these relationships and who are your main stakeholders and how do you ensure that the data teams road map is aligned with the business's priorities yeah okay there was a lot of stuff in there on the relationship side you basically I mean everybody on the team has you know a set of stakeholders to maintain relationships with right I mean for myself I probably have regular meetings or lunches or something this I don't know maybe 4 40 or 50 different executive officers in company right just to kind of understand what's going on in their unit you know what are their priorities where could I potentially help or my team could help and stay on top of what's going on in the business so that we can detect opportunities you know as soon as they arise for folks you know that reporting to me they have a large set of stakeholders also and they want to be a little more tactical and say okay you know what projects are you starting what's coming next year how can we help what's the status of current projects we're engaged in is everybody happy are you happy or your people happy you know to keep that going for more individual contributor Types on my team it will be the stakeholders of a particular project to make sure they talk to them not just in official meetings but maybe you know have a lunch with them have oneon-one meeting with them to just make sure that we hitting the Mark we're on the right track they understand what we're doing there's no gaps in knowledge or understanding uh no gaps and goals you know that's kind of the practical way that we do that on on the communication side now on the planning side that's I think the other part of your question there needs to be budget alignment when I say deployment you know it's nice to have a platform to deploy all the models but deploying models is not free right you also have to the budget to deploy so you know in our annual financial planning process I mean I don't only have to worry about my own budget that's complex enough but I have to worry about the budget of every one of my business partners too and make sure they have budgeted for the deployment of our models into their environment so that that will actually happen you know so there's a lot of coordination going on during the planning process and the budgeting process to make sure that you know you have to Budget on the tech side we have to Dutch budget in the business side they actually get miles deployed in the coming year and that's very important there's definitely a lot of complexity in managing these relationships and speaking of complexity I'd love it if you can also walk us through the challenges and working with data within an industry that is highly regulated and how do you ensure that you're consistently innovating responsibly that's uh very interesting and I guess more relevant topic every year I mean I want to make maybe two points on that one one is I mean regulation is there to protect consumers right so I mean New York live as a company is very much aligned with that in fact you know we're not a you know Market traded company we're what's called a mutual company so New York Life is actually owned by policy holders of New York Liv it was entally owned by our customers so hence our philosophy is very much to Delight by our customers and that is at least in principle you know the purpose of Regulation as well uh so we appreciate our wellth thought out regulation and we're quite happy you know to follow that and influence that in the right way it's very much aligned with all company so that's the first point the second point for you know engineers and data scientists is the uh compliance with regulation is not boring at all it's actually creates a lot of interesting analytical challenges in its own life so when you think about you know disp impact testing for protective classes that creates a whole new set of uh algorithms that you have to develop to do this intelligent right because this would impact is not sort of a you know black and white thing it's very it's complex this would impact of a model well a model is multivariant which pieces of that model are actually creating or not creating some potential disperate impact how do you Analyze That in the multivariate situation there's a complex modeling algorithm behind the Val you draw the line is not a qualitative Concept in fact is it's quantitative well you draw the line between you know some correlation that might indicate this put Impact versus not so it creates a lot of interesting analytical challenges I'd say that you know the head of my model government team is probably one of the most interesting positions on the whole team that's a great way of framing it you know we're constrain add a new later of complexity and Intrigue for different types of data science projects so can you walk me through the process of setting up a model governance team and the lessons that you can share with other data leaders here we set up a governments model govern team probably about two or three years ago you know once little organization was mature enough to be our own headcount on that it's what I call the first line of defense right so I have my own model governance team first line of defense then the company has a corporate model governance team that's the second line of fans and every model gets validated on my team so every model that gets built gets validated by people that had no involvement in building that's independent validation we look at data validation we look at model validation and then of course we look at any kinds of regulations or legislation that you know this may be subject to depending on the use case and the data that's in it so the leader of my model governance team has to be a very good data scientist who actually understands all the algorithms and the data but also is direct connected to our government Affairs team and the company and the legal Team stand regulation legislation not just of today but of tomorrow because these models we built they should be in production for many years so we have to look at you know what is being proposed as legislation in the state of Ohio you know in the next year or something like right is also Ste regulated so we don't just have one regulator I mean the federal government is also a regulator but we also have 50 other Regulators you know one in every state so that makes it pretty interesting job that's really fascinating now Glenn as we are reaching the end of our episode I'd love to ask you a few questions before you close out the first one is what would be your advice for any upand cominging data scientists Fielding choices between joining Tech or more you know quote unquote traditional Industries such as Finance or Insurance there is sort of the misconception that I want to debunk a little bit that you know if it's a tech company if exciting if it's you know insurance company it's boring that's not true at all right what you should look at you know what is this specific job and what kind of thing are you going to work on right because if they make 95% of their business model from advertising 95% of their data scientist is going to work on is maximization of advertising levenue so an individual job you know might be focusing on optimizing one single search term for the next year right so a lot of these large tech companies have done this for a decade or more so you're building the 17's version of that month right whereas at a place like New York Life you know you're building the first version of a model that has never been tackled before in a variety of different challenges the company is old you know 175 years old but the challenges we tackle from a data science perspective we're often tackling for the first time look at this specific job don't just look at the company look at what you're actually going to be working on are you building the first model the second model or the 15th model in that area what kind of interesting data do you get to work with what kind of variety of models and other data science Solutions are you going to build those are the people you're going to learn from and then you know obviously the the most important thing is the data science function setup for Success meaning all their deploying all their models into a business and actually get the benefit out of it because if they're not doing that probably means that the existence of that data science function will eventually be questioned and given the exciting use cases of data science and Ai and insurance that we just discussed what are some of the trends you're particularly excited about that will impact the insurance space yeah I mean you know like you said we can talk a long time about mlops so I think mlops is a huge Trend and we're just at the beginning of that journey and it's already made such a big Improvement in the way data science functions in reality right I mean now we have good platforms that you can deploy models on those platforms will probably get even better be more of an integrated way that you know integrates even better with the model development the thing that we're we're actually investing bit of money into now is what happens post deployment of the model in terms of software Solutions right so with the model production it's running now you know you should monitor that model I have a monitoring function that we put in place in the last two years and uh so every model now gets monitor both for things like data drift and score drift if something goes wrong with incoming data somebody gets loaded right away before we make a bunch of bad decisions and then also reporting so you know we want to play a model after the model gets deployed our business part is well I not that they're actually going to get the value that we promised right so we have to do a bunch of reporting on that production model we'd like to not write custom code for all of that would be a lot of work so we've been you know looking at software and actually we made a software decision that helps us Monitor and hold on models and production so that's all part of s the mlops process and I think you know mlops engineer is probably uh one of the hardest professions right on the same level dat scientist that's great now Glenn as we close out do you have any final call to action before we wrap up today's episode like some of the things I said if you want to kind of learn more over time definitely feel free to connect on LinkedIn with me I'm always happy to provide material to the community I post pretty actively on LinkedIn so that's a way to stay in touch also you know I would be of a Miss to not mention job opportunities on the team so I frequently have openings on the team on all the different skill sets that I mentioned so data science mops Engineers data Engineers project managers model governance at many different levels we also have a a large internship program that we run every summer and recently we started a what we call an associate program so starting this year we actually hire several people was a bachelor's degree in a technical area and then they spent three years on my team rotating through a few different jobs and at the same time doing a master's degree for which we paid the tuition so that's yet another you know great way to join the team and become a data scientist thank you so much Glenn for coming on data framed thanks so much for having me you've been listening to data framed a podcast by data camp keep connected with us by subscribing to the show in your favorite podcast player please give us a rating leave a comment and share episodes you love that helps us keep delivering insights into all things data thanks for listening until next time\n"