Data Workflow + Tidy data + Wrangling

MP223 - Applied Econometrics Methods for the Social Sciences

Eduard Bukin

R setup

library(tidyverse)       # for data wrangling

# set default theme and larger font size for ggplot2
ggplot2::theme_set(ggplot2::theme_minimal(base_size = 16))

# set default figure parameters for knitr
knitr::opts_chunk$set(
  fig.width = 8,
  fig.asp = 0.618,
  fig.retina = 3,
  dpi = 300,
  out.width = "80%"
)

Introduction

Data analysis workflow

Tidy data (1/5)

Tidy data (2/5) wide format

Tidy data (3/5) long format

Tidy data (4/5) transformation

Tidy data (5/5). Learn more?

Read:

• R4DS Ch. 12. Tidy data

Follow slides, videos and exercises:

• Chapter 7.1.2 Wrangling and tidying data in Data Science in a Box

Wrangling

Wrangling: See also

Read:

• R4DS: R for data science by Hadley Wickham and Garrett Grolemund (book’s source code) (Wickham and Grolemund 2017)
• R4DS Ch 9. Wrangle

Watch

• Grammar of data wrangling video + slides

• webinar: Data wrangling with R and RStudio

Data wrangling

dplyr package

Does data wrangling.

• Documentation tidyverse/dplyr + Source code

• Cheat sheets Data transformation with dplyr

• Learn by doing R4DS Ch 5. Data transformation

dplyr package: getting started

Article: Introduction to dplyr

Cheat sheets: Data transformation with dplyr

Webinar: Data wrangling with R and RStudio by Garrett Grolemund

Interactive exercises (repeated on following slides)

• Working with Tibbles
• Isolating Data with dplyr
• Filter observations
• Derive Information with dplyr
• Summarizing data

Readings:

• R4DS Chapter 5 Data transformation

dplyr::rename() Rename columns

• Read about in R4DS Ch. 5.4 + function reference (see examples!)

dplyr::mutate() Create, modify, and delete columns

• Read about in R4DS Ch. 5.5 + function reference

• Articles: Add new columns with mutate and Column-wise operations

• Interactive exercises: Derive Information with dplyr, Summarizing data and Road Traffic Accidents

dplyr::filter() (1/2) subset rows using column values

• Read about in R4DS Ch. 5.2 + function reference (see examples!)

• Article: Filter rows with filter()

Interactive exercises: Isolating Data with dplyr and Filter observations.

dplyr::filter() (2/2) relies on logical operators

• See: ?Comparison and ?base::Logic

dplyr::select() Subset columns using their names and types

• Read about in R4DS Ch. 5.4 + function reference (see examples!)

Interactive exercises: Isolating Data with dplyr

Left for later

• dplyr::summarise()
• dplyr::group_by()
• dplyr::arrange()
• dplyr::pull()
• dplyr::distinct()
• dplyr::count()

Summary statistics

skimr package

Does Summary statistics

• Documentation ropensci/skimr + Source code
• Learn by doing Using Skimr

Key functions: skimr::skim().

Correlation

Correlation

Does Summary statistics

• Documentation easystats/correlation + Source code

• Learn by doing Using correlation

Key functions: correlation::correlation().

Histogram

Box plot

Scatter plot

References

References

Wickham, Hadley, and Garrett Grolemund. 2017. R for Data Science. O’Reilly Media. http://r4ds.had.co.nz/.