The Power of J: Selecting Columns and Performing Operations

In this section, we will delve into the world of J, a powerful programming language used for data manipulation and analysis. We have already explored how to select rows using row numbers in the I argument, which is the first argument in Data Table. In this section, we'll cover the second argument, J, which allows us to perform various operations on columns.

Selecting Columns

One of the simplest things you can do in J is to select columns. When working with a data table like DT, which has three columns A, B, and C, selecting just these two columns involves wrapping them around with dots followed by parentheses. This tells J to return only the specified columns. For example, `A.B` would return column B of the data table. By doing this, we can easily access and manipulate specific columns within our dataset.

Beyond Selecting Columns: Calling Functions

In addition to selecting columns, J also allows us to call functions on them, treating the columns as variables. In a previous section, we demonstrated how to select rows using row numbers in the I argument. Here, we'll explore how to select columns and perform various operations on them using functions. Using the same data table DT from our previous exploration, we have three columns A, B, and C. Instead of just selecting these columns, we can also aggregate values across multiple columns by calling functions on them.

Summing Values Across Multiple Columns

To demonstrate how to call functions on columns, let's consider summing values across multiple columns. We'll start with the function `sum`, which adds up all values in a given column. Applying this function to column A, we get the result of `(1+2+3+4+5)`, which equals 15. Similarly, applying the function `mean` to column C gives us an average value of `(6+7+8+9+10)/5 = 8`. Notice that in both cases, J automatically assigns column names to the resulting values, ensuring clarity and consistency throughout our analysis.

Providing Column Names for Results

When we perform operations on columns without explicitly providing column names, Data Table will automatically generate these names. However, it's always a good practice to include them, especially when working with complex datasets or collaborating with others. This helps maintain organization and readability in our results.

Computing on Multiple Columns: Length Considerations

Another important aspect of J is how it handles operations across multiple columns. While computing on multiple columns can be beneficial for performing more complex analyses, the resulting output may not always have the same length as the shorter column involved. In such cases, J will automatically recycle the length of the shorter column to match that of the larger one, ensuring consistency throughout our results.

Using Functions in J: A Demonstration

To further illustrate how functions can be used in J, let's consider another example involving column B and column C. We select column B and apply the `sum` function on column C, which yields a single value equal to `(6+7+8+9+10)`. In this case, because the length of column B is longer than that of column C, J reuses the values from column C to fill in any missing values within column B. This recycling process allows us to maintain data integrity while performing multiple operations in a single statement.

Plotting Data: A Side Effect

J also supports plotting data as part of its capabilities. When using the `plot` function without wrapping it around dot parentheses, J serves as a side effect, returning a value but also displaying the plot directly. This allows us to visually inspect our results while still being able to work with them programmatically.

Customizing Output: Setting Return Value

In some cases, we might want to customize how J outputs its values or results. To achieve this, we can set the `return` value explicitly, ensuring that only specific output is generated. For instance, if we use a function like `histogram`, which returns values invisibly by default, we need to manually specify that we want these values displayed.

The Capabilities of J: Putting it All Together

J's capabilities extend far beyond simple data manipulation and selection. With its powerful functions for aggregation, plotting, and customization, it provides an incredibly versatile toolset for working with Data Table. From summing values across multiple columns to creating plots and customizing output, J offers the flexibility to tackle even the most complex analyses.

"WEBVTTKind: captionsLanguage: enin the previous section Matt showed you how to select rows using row numbers in the I argument which is the first argument in data table in this section we'll cover the second argument which is J the simplest thing what you can do in J is to select columns here's the data table DT where you have three columns a B and C to select the columns B and C we just have to wrap them around with dots followed by a parenthesis in addition to selecting columns we can also call functions in J because the columns are as if they are variables which we just saw in the previous section using the same data table from the previous section we have three columns a B and C now instead of just selecting on columns will aggregate on columns a and C we call the function sum on column a that adds up the values 1 plus 2 Plus 3 plus 4 plus 5 and returns 15 similarly because the function mean on column C which averages the values 6 7 8 9 and 10 which gives 14 divided by 5 equals 8 not that we provided total and mean as a column names that have been assigned as the column names in the result now if we did not provide any column names data table will automatically generate those values for us and computing on multiple columns does not all return the same length the length of the shorter column gets recycled to match the length of the larger column using the same data table DT with 3 columns a B and C we select column B and calculate sum on column C now some 1 column C return a single value which is 6 plus 7 plus 8 plus 9 plus 10 equals 14 and the length of column B is Phi and therefore sum of C value gets recycled to fit the length of column B in fact you can do pretty much anything in J it doesn't even have to return a value for example let's do DT of plot of a comma C and it just plots the column a in the x-axis and column C in the y axis note that we didn't have to wrap plot with the dot parentheses syntax because here we use plot as a side effect and returns a value now in fact we can also have multiple expressions wrapped within curly braces so for example we print Colin a here on to the console with returns one two three four and five followed by the plotting of histogram on column C here however histogram returns the values invisibly and therefore we explicitly set the value return value now in jailin the previous section Matt showed you how to select rows using row numbers in the I argument which is the first argument in data table in this section we'll cover the second argument which is J the simplest thing what you can do in J is to select columns here's the data table DT where you have three columns a B and C to select the columns B and C we just have to wrap them around with dots followed by a parenthesis in addition to selecting columns we can also call functions in J because the columns are as if they are variables which we just saw in the previous section using the same data table from the previous section we have three columns a B and C now instead of just selecting on columns will aggregate on columns a and C we call the function sum on column a that adds up the values 1 plus 2 Plus 3 plus 4 plus 5 and returns 15 similarly because the function mean on column C which averages the values 6 7 8 9 and 10 which gives 14 divided by 5 equals 8 not that we provided total and mean as a column names that have been assigned as the column names in the result now if we did not provide any column names data table will automatically generate those values for us and computing on multiple columns does not all return the same length the length of the shorter column gets recycled to match the length of the larger column using the same data table DT with 3 columns a B and C we select column B and calculate sum on column C now some 1 column C return a single value which is 6 plus 7 plus 8 plus 9 plus 10 equals 14 and the length of column B is Phi and therefore sum of C value gets recycled to fit the length of column B in fact you can do pretty much anything in J it doesn't even have to return a value for example let's do DT of plot of a comma C and it just plots the column a in the x-axis and column C in the y axis note that we didn't have to wrap plot with the dot parentheses syntax because here we use plot as a side effect and returns a value now in fact we can also have multiple expressions wrapped within curly braces so for example we print Colin a here on to the console with returns one two three four and five followed by the plotting of histogram on column C here however histogram returns the values invisibly and therefore we explicitly set the value return value now in jail\n"