PySpark Tutorial - Why learn how to build recommendation engines

Welcome to Building Recommendation Engines using Alternating Least Squares (ALS) in PI SPARC

You are probably already familiar with the output of these types of recommendation engines, where websites tell you something along the lines of "if you like that then you'll probably like this". You've likely seen these types of recommendations on your favorite retail or media streaming websites. These recommendations are generated through different types of data that you as a user provide, either directly or indirectly, when you purchase something online, watch a movie, or even read an article.

You're often given a chance to rate the item on a scale of one to five stars, a thumbs up or thumbs down, or some other type of rating scale based on your feedback from these types of rating systems. Companies can learn a lot about your preferences and offer you recommendations based on preferences of users that are similar to you. For example, if your movie streaming service sees that you like The Dark Knight and Iron Man and did not like Tangled, it also sees other users that also like The Dark Knight and Iron Man and also did not like Tangled. The ALS algorithm would see that you and these other users have similar tastes.

It would then look at the movies that you have not yet seen and see which ones are the highest rated among those similar users and offer them as recommendations to you. This is why websites will often say things like "because you like that movie, we think you'll like this movie" or "users like you also watch this movie". These types of rating systems are extremely powerful. In fact, an article published by McKinsey & Company in October of 2013 stated that 35% of what customers buy on Amazon and 75% of what they watch on Netflix come from product recommendations based on algorithms such as the one you are going to be learning in this course.

That's a powerful use of data. With this course, you will learn how to do this in addition to other alternate uses of recommendation algorithms that can be extremely useful for purposes as broad as feature space reduction, image compression, mathematical user and product grouping, latent feature discovery, and more. You're going to learn some of these in this course.

This tutorial is intended for those who have experience with SPARC and Python and understand the fundamentals of machine learning. If needed, you can refer to our Data Camps introduction to PI SPARC course, their intermediate Python for data science course, and their supervised machine learning with Python's scikit-learn course for a good starting point. Welcome to this course on building recommendation engines using Alternating Least Squares (ALS) in PI SPARC.

"WEBVTTKind: captionsLanguage: enhi welcome to this course on building recommendation engines using alternating least squares or ALS in PI SPARC you're probably already familiar with the output of these types of recommendation engines where website tells you something along the lines of if you like that then you'll probably like this you've likely seen these types of recommendations on your favorite retail or media streaming websites these recommendations are generated through different types of data that you as a user provide either directly or indirectly when you purchase something online or watch a movie or even read an article you're often given a chance to rate that item on a scale of one to five stars a thumbs up or thumbs down or some other type of rating scale based on your feedback from these types of rating systems companies can learn a lot about your preferences and offer you recommendations based on preferences of users that are similar to you for example if your movie streaming service sees that you like The Dark Knight and Iron Man and did not like tangled and it also sees other users that also like the Dark Knight and Iron Man and also did not like tangled the ALS algorithm would see that you and these other users have similar tastes it would then look at the movies that you have not yet seen and see which ones are the highest rated among those similar users and offer them as recommendations to you this is why websites will often say things like because you like that movie we think you'll like this movie or users like you also watch this movie these types of rating systems are extremely powerful in fact an article published by McKinsey & Company in October of 2013 stated that 35% of what customers buy on Amazon and 75% of what they watch on Netflix come from product recommendations based on algorithms such as the one you are going to be learning in this course that's a powerful use of data and with this course you will learn how to do this in addition to this there are alternate uses of recommendation algorithms that can be extremely useful for purposes as broad as feature space reduction image compression mathematical user and product grouping latent feature discovery and you're going to learn some of these in this course this tutorial is intended for those that have experience with SPARC and Python and understand the fundamentals of machine learning if needed some good introductory resources our data camps introduction to PI SPARC course their intermediate Python for data science course and their supervised machine learning with pythons scikit-learn coursehi welcome to this course on building recommendation engines using alternating least squares or ALS in PI SPARC you're probably already familiar with the output of these types of recommendation engines where website tells you something along the lines of if you like that then you'll probably like this you've likely seen these types of recommendations on your favorite retail or media streaming websites these recommendations are generated through different types of data that you as a user provide either directly or indirectly when you purchase something online or watch a movie or even read an article you're often given a chance to rate that item on a scale of one to five stars a thumbs up or thumbs down or some other type of rating scale based on your feedback from these types of rating systems companies can learn a lot about your preferences and offer you recommendations based on preferences of users that are similar to you for example if your movie streaming service sees that you like The Dark Knight and Iron Man and did not like tangled and it also sees other users that also like the Dark Knight and Iron Man and also did not like tangled the ALS algorithm would see that you and these other users have similar tastes it would then look at the movies that you have not yet seen and see which ones are the highest rated among those similar users and offer them as recommendations to you this is why websites will often say things like because you like that movie we think you'll like this movie or users like you also watch this movie these types of rating systems are extremely powerful in fact an article published by McKinsey & Company in October of 2013 stated that 35% of what customers buy on Amazon and 75% of what they watch on Netflix come from product recommendations based on algorithms such as the one you are going to be learning in this course that's a powerful use of data and with this course you will learn how to do this in addition to this there are alternate uses of recommendation algorithms that can be extremely useful for purposes as broad as feature space reduction image compression mathematical user and product grouping latent feature discovery and you're going to learn some of these in this course this tutorial is intended for those that have experience with SPARC and Python and understand the fundamentals of machine learning if needed some good introductory resources our data camps introduction to PI SPARC course their intermediate Python for data science course and their supervised machine learning with pythons scikit-learn course\n"