Ready for R (2024-02-05): Skim Your Data

One of my favorite tools: {skimr}

Please note that I am working with an experimental API package for my mailing list, and I am debugging visibility issues. Please let me know if you can’t read any commands or output. You can see the web version of this by clicking on the Ready4R Banner above.

I’m going to highlight the {skimr} package, which I try to always use when I encounter new data. It’s invaluable to see the big picture of the data. You can install it using:

install.packages("skimr")

Let’s start out with the 80 cereals dataset (available here). Looking at the description, we know there should be at least 3 categorical variables (manufacturer, type, and shelf). Let’s keep that in mind when start looking at the data. I’m going to change the data types of all of these to factors. In the case of shelf, we’ll cast the variable as an ordered factor type.

library(tidyverse)

cereals <- readr::read_csv("data/cereal.csv", ) |> 
  janitor::clean_names() |>
  mutate(shelf = factor(shelf, ordered=TRUE)) |>
  mutate(across(c("manufacturer", "type"), as.factor))

Now that we have the data loaded correctly, we can now take a look at it. You may be used to using summary() on your data. skimr::skim() is like that, but gives you variable summaries based on variable type (character, factor, and numeric in this dataset).

When you use it, you’ll usually call it like the following: skimr::skim(cereals). But to make it easier to explain, I’ll separate it out into the different summary types.

Overall Summary

The first one is the overall data.frame summary:

skim_output <- skimr::skim(cereals)
summary(skim_output)

You can get lots of important information here: dimensions, and number of different variable types. If these variable counts by type aren’t what I expect, that’s usually a signal I’ll need to do some variable transformation.

Character Summary

Let’s take a look at the character variable summary.

skimr::yank(skim_output, "character")

Variable type: character

There’s only one character variable, name. We expect this to be unique, since the variable is defined as 1 cereal per row. The n_unique confirms this (77 rows, 77 unique values).

Factor Summary

When we take a look at the factor level summary, things start to get interesting:

skimr::yank(skim_output, "factor")

Variable type: factor

You can see that the categories for each variable, such as manufacturer are sorted by frequency. There’s also information about whether the variable is missing any values, which can be helpful.

Numeric Summary

I get really excited about the numeric summary.

skimr::yank(skim_output, "numeric")

Variable type: numeric

This table contains most of the usual values of summary() (quantiles, mean, sd, etc). But I love the hist column because it shows a small histogram of the distribution of numeric values, which is really helpful to see if the distributions are skewed.

{skimr} helps you get to know your data

skimr::skim() is usually one of the first things I do when I load data into R. It helps me confirm facts from the dataset description, such as uniqueness of columns, but also gives me a very helpful overview.

{skimr} is maintained by rOpenSci. They’re a really great group that develop very handy R Packages and I suggest you check them out!

That’s it for today

That’s it for this week’s newsletter. I hope to talk more about handy Exploratory Data Analysis tools and the mindset of Exploratory Data Analysis. I believe that getting good at Exploratory Data Analysis helps build your confidence as an analyst.