Crowding problem _ t-SNE _ Dimensionality Reduction _ Applied AI Course

The Objective of Testing the Concept (TC) of Neighborhood Preservation

In the context of testing the concept (TC) of neighborhood preservation, it is essential to understand what constitutes a neighborhood and how it relates to the overall objective of preserving all neighborhoods. A neighborhood, in this context, refers to a group of points that are at a certain distance from each other, typically specified as D.

For instance, if we consider point X1, its neighborhood would include all points that are at a distance of D or less than D from X1. This means that the neighborhood of X1 includes points X2 and X4, as they are both within the specified distance D from X1. Similarly, the neighborhood of X2 would include points X1, X3, and X4, while the neighborhood of X3 would include points X1 and X2, and so on.

When we project this concept into a one-dimensional space, it becomes clear that the preservation of neighborhoods is crucial for ensuring that all points are treated equally. If we consider point X1, its neighborhood is well-preserved, as it consists of points X2 and X4, which are both at a distance D from X1. However, when we extend this concept to point X2, we find that its neighborhood has changed due to the introduction of new points.

In particular, the neighborhood of X2 now includes points X1, X3, and X4, whereas before it only included X1 and X3. This change in the neighborhood of X2 is a result of the overlap between X2 and X4, which is at a distance of 0 from each other. Similarly, the neighborhood of X3 has changed, as it now includes points X1 and X2, whereas before it only included X1.

However, the neighborhood of X4 remains unchanged, as it still consists of points X1, X3, and X2. This highlights the importance of preserving neighborhoods for all points, not just individual points. If we do not ensure that the neighborhoods are preserved, we risk introducing inconsistencies and inaccuracies in our analysis.

The preservation of neighborhoods is also crucial when it comes to visualizing the concept of TC. In particular, if we have two sets of points, each with a different configuration, they may not be considered equivalent even if they satisfy all the constraints of the problem. For instance, consider two configurations of points as shown below:

In this case, the two configurations are not the same, despite satisfying all the constraints of the problem. This is because the neighborhoods of the points in each configuration are different.

Therefore, it is essential to ensure that the neighborhoods are preserved for all points, regardless of their configuration or position. By doing so, we can accurately visualize and analyze the concept of TC, which is critical for solving problems in various fields.

The preservation of neighborhood is only one concept that is used to make this work. The primary objective of testing the concept (TC) is to ensure that all neighborhoods are preserved for all points, not just individual points. If this objective is not met, the entire analysis may be flawed and inaccurate.

"WEBVTTKind: captionsLanguage: enso for a crowding in TC video there's a question asked on YouTube by madre de krishna chaitanya ready at YouTube and the question is very interesting the question goes as follows question is imagine we are taking a 2d data set of four points X 1 X 2 X 3 X 4 which are on on a on a square right where the distances between X 1 X 4 is d X 3 X 4 is d X 3 X 2 is D similarly X 1 X 2 is d now when we try to convert this data from 2d or when you try to predict the data from 2d to 1d in Teasley I said that this is one projection but what's happening here is I put X 4 here right so what's happening here if you notice is the question that the Chaitanya asked is why are we putting X 4 here why can't we put X 4 on top of X 2 so let's say if I place X 4 on top of X 2 in the video at 6 minutes 14 seconds I explain that the objective of TC is to preserve all the neighborhood's right so let's read write down the neighborhood's what is a neighborhood of point X 1 so let's say human neighborhood only is about points which are in distance D right so the neighborhood of point X 1 is X 2 and X 4 right including this is intruding right similarly what is the neighborhood of X 2 the neighborhood of X 2 is again X 1 and X 3 similarly neighborhood of X 3 is X 4 and X 2 and neighborhood of X 4 itself is X 1 and X 3 this isn't true D now when we project it into one being let's assume I don't place my X 4 here I place X 4 on top of X 2 right now one one good thing that's happening here is for 4 X 2 sorry for X 1 both X 2 and X 4 are at distance D it's satisfying that constraint but has to satisfy for all the neighborhoods now with this with this solution let's look at the neighborhoods in our in our in our 1d space what is the neighborhood of x1 my neighborhood of x1 is all the points which light a distance D so my neighborhood of X 1 is X 2 X 4 and these two are same so my neighborhood of X 1 is well preserved right so because this consists set set of 2 points X 2 X 4 my 4 X 1 everything looks good for X 1 life is good the neighborhood is perfectly preserved but we also need the neighborhood to be preserved for all the points not just one point as we clearly stated it at 6 minutes 14 seconds in the original video right what about X 2 now what is the neighborhood of X 2 neighborhood of X 2 now is which all points are at a distance of D or less than D right you have X 1 you have X 3 and X 4 because X 4 is exactly overlapping with X 2 which at a distance of 0 now if you notice here in the original 2d space you had only X 1 and X 3 at a distance of D here you have the neighborhood has changed because you have new point which is X 4 here so if you look at it from x 2.is perspective the neighborhood is not preserved because I have a new point here which is which is so because if you see here I have only X 1 X 3 here I have an extra point X 4 which should not be the case if the neighborhood is perfectly preserved what about neighborhood of X 3 what happens to my X 3 my X 3 has X 2 and X 4 my X 3 has X 2 and X 4 so 1 X 3 snai Burwood is perfectly preserved what about X what about X 4 what is my what is my X fourth neighborhood my X 4 has X 1 X 3 and X X 1 X 2 and X 3 all three of them in the neighborhood because X 2 and X 4 over overlapping right but here I have only X 1 and X 3 here I have a new point which is X 2 so my X force neighborhood is not also being preserved so if you overlap X 2 and X 4 while your N 1 while you are X 1 and X 3s are preserved or neighborhoods the neighborhoods of X 1 and x3 are preserved for x2 and x4 they are not preserved at all preservation basically means that these two sets should be exactly same that's an important distinction and the neighborhoods of all the points should be preserved if your tea sneeze is working perfectly right and let's not forget the primary objective of testing the primary objective of tea Snee is to visualize now if I give you two visualizations of points wherein I give one visualization of points as this the other visualization of points as this one on top of other these two are not the same forget about the dimensions let's assume I have three points like this right if I place X whatever X to some some configuration this is in no way similar to this configuration of points whatever way you arrange them but that doesn't matter so the whole objective of teasley is to keep the alignment same and neighborhood is only one concept that we are using to make it work the preservation of neighborhood needs to happen needs to hold for all the points not just one point I hope that answers your question Krishna Chaitanya and I hope others who have the same question also could benefit from this videoso for a crowding in TC video there's a question asked on YouTube by madre de krishna chaitanya ready at YouTube and the question is very interesting the question goes as follows question is imagine we are taking a 2d data set of four points X 1 X 2 X 3 X 4 which are on on a on a square right where the distances between X 1 X 4 is d X 3 X 4 is d X 3 X 2 is D similarly X 1 X 2 is d now when we try to convert this data from 2d or when you try to predict the data from 2d to 1d in Teasley I said that this is one projection but what's happening here is I put X 4 here right so what's happening here if you notice is the question that the Chaitanya asked is why are we putting X 4 here why can't we put X 4 on top of X 2 so let's say if I place X 4 on top of X 2 in the video at 6 minutes 14 seconds I explain that the objective of TC is to preserve all the neighborhood's right so let's read write down the neighborhood's what is a neighborhood of point X 1 so let's say human neighborhood only is about points which are in distance D right so the neighborhood of point X 1 is X 2 and X 4 right including this is intruding right similarly what is the neighborhood of X 2 the neighborhood of X 2 is again X 1 and X 3 similarly neighborhood of X 3 is X 4 and X 2 and neighborhood of X 4 itself is X 1 and X 3 this isn't true D now when we project it into one being let's assume I don't place my X 4 here I place X 4 on top of X 2 right now one one good thing that's happening here is for 4 X 2 sorry for X 1 both X 2 and X 4 are at distance D it's satisfying that constraint but has to satisfy for all the neighborhoods now with this with this solution let's look at the neighborhoods in our in our in our 1d space what is the neighborhood of x1 my neighborhood of x1 is all the points which light a distance D so my neighborhood of X 1 is X 2 X 4 and these two are same so my neighborhood of X 1 is well preserved right so because this consists set set of 2 points X 2 X 4 my 4 X 1 everything looks good for X 1 life is good the neighborhood is perfectly preserved but we also need the neighborhood to be preserved for all the points not just one point as we clearly stated it at 6 minutes 14 seconds in the original video right what about X 2 now what is the neighborhood of X 2 neighborhood of X 2 now is which all points are at a distance of D or less than D right you have X 1 you have X 3 and X 4 because X 4 is exactly overlapping with X 2 which at a distance of 0 now if you notice here in the original 2d space you had only X 1 and X 3 at a distance of D here you have the neighborhood has changed because you have new point which is X 4 here so if you look at it from x 2.is perspective the neighborhood is not preserved because I have a new point here which is which is so because if you see here I have only X 1 X 3 here I have an extra point X 4 which should not be the case if the neighborhood is perfectly preserved what about neighborhood of X 3 what happens to my X 3 my X 3 has X 2 and X 4 my X 3 has X 2 and X 4 so 1 X 3 snai Burwood is perfectly preserved what about X what about X 4 what is my what is my X fourth neighborhood my X 4 has X 1 X 3 and X X 1 X 2 and X 3 all three of them in the neighborhood because X 2 and X 4 over overlapping right but here I have only X 1 and X 3 here I have a new point which is X 2 so my X force neighborhood is not also being preserved so if you overlap X 2 and X 4 while your N 1 while you are X 1 and X 3s are preserved or neighborhoods the neighborhoods of X 1 and x3 are preserved for x2 and x4 they are not preserved at all preservation basically means that these two sets should be exactly same that's an important distinction and the neighborhoods of all the points should be preserved if your tea sneeze is working perfectly right and let's not forget the primary objective of testing the primary objective of tea Snee is to visualize now if I give you two visualizations of points wherein I give one visualization of points as this the other visualization of points as this one on top of other these two are not the same forget about the dimensions let's assume I have three points like this right if I place X whatever X to some some configuration this is in no way similar to this configuration of points whatever way you arrange them but that doesn't matter so the whole objective of teasley is to keep the alignment same and neighborhood is only one concept that we are using to make it work the preservation of neighborhood needs to happen needs to hold for all the points not just one point I hope that answers your question Krishna Chaitanya and I hope others who have the same question also could benefit from this video\n"