---
title: "Getting Started"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
library(CDMConnector)
if (Sys.getenv("EUNOMIA_DATA_FOLDER") == "") Sys.setenv("EUNOMIA_DATA_FOLDER" = tempdir())
if (!dir.exists(Sys.getenv("EUNOMIA_DATA_FOLDER"))) dir.create(Sys.getenv("EUNOMIA_DATA_FOLDER"))
if (!eunomia_is_available()) downloadEunomiaData()

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

## Creating a reference to the OMOP CDM

The CDMConnector package provides tools for working with OMOP Common Data Model (CDM) tables using familiar [dplyr](https://dplyr.tidyverse.org) syntax. After connecting to a database containing  data mapped to the OMOP CDM, we can use `cdm_from_con` from CDMConnector to create a CDM reference. This CDM reference is a single object that contains table references along with specific metadata.

For this example, we’ll use the Eunomia data contained in a [duckdb](https://duckdb.org/) database. First, if you haven't previously done so, we need to download the data. And once downloaded, add the path to our Renviron.

```{r, message=FALSE, warning=FALSE}
library(CDMConnector)
library(dplyr)
```

```{r, eval=FALSE}
downloadEunomiaData(
  pathToData = here::here(), # change to the location you want to save the data
  overwrite = TRUE
)
# once downloaded, save path to your Renviron: EUNOMIA_DATA_FOLDER="......"
# (and then restart R)
```

With the Eunomia data now downloaded, we can connect to the database and create our reference.

```{r}
con <- DBI::dbConnect(duckdb::duckdb(), dbdir = eunomia_dir())
cdm <- cdm_from_con(con, cdm_schema = "main")
cdm
```

Individual CDM table references can be accessed using `$` and piped to dplyr verbs.

```{r}
cdm$person %>% 
  glimpse()
```

## Selecting a subset of CDM tables

If you do not need references to all tables you can easily select only a subset of tables to include in the CDM reference. The `cdm_tables` argument of `cdm_from_con` supports the [tidyselect selection language](https://tidyselect.r-lib.org/reference/language.html) and provides a new selection helper: `tbl_group`.

```{r}
cdm_from_con(con, cdm_tables = c("person", "observation_period")) # character vector
cdm_from_con(con, cdm_tables = starts_with("concept")) # tables that start with 'concept'
cdm_from_con(con, cdm_tables = contains("era")) # tables that contain the substring 'era'
cdm_from_con(con, cdm_tables = matches("person|period")) # regular expression
```

Predefined sets of tables can also be selected using `tbl_group` which supports several subsets of the CDM: "all", "clinical", "vocab", "derived", and "default".

```{r}
# pre-defined groups
cdm_from_con(con, cdm_tables = tbl_group("clinical")) 
cdm_from_con(con, cdm_tables = tbl_group("vocab")) 
```

The default set of CDM tables included in a CDM object is:

```{r}
tbl_group("default")
```

## Including existing cohort tables in the CDM reference

It is common to use one or more cohort tables along with the CDM. We can include existing cohort tables by specifying the schema in which they reside and their name like so:
```{r, echo=FALSE}
cohort <- tibble(cohort_definition_id = 1L,
                 subject_id = 1L:2L,
                 cohort_start_date = c(Sys.Date(), as.Date("2020-02-03")),
                 cohort_end_date = c(Sys.Date(), as.Date("2020-11-04")))

invisible(DBI::dbExecute(con, "create schema write_schema;"))

DBI::dbWriteTable(con, DBI::Id(schema = "write_schema", table_name = "cohort"), cohort)

```

```{r}
cdm <- cdm_from_con(con, 
                    cdm_tables = c("person", "observation_period"), 
                    write_schema = "write_schema",
                    cohort_tables = "cohort") 

cdm$cohort
```

## Extracting data

There are two ways to extract subsets of the CDM.

-   `collect` pulls data into R

-   `stow` saves the cdm subset to a set of files on disk in either parquet, feather, or csv format

Note, in either case, you should think carefully about the data you are extracting and make sure to only get the data that you require (and will fit into memory!)

```{r}
local_cdm <- cdm %>% 
  collect()

# The cdm tables are now dataframes
local_cdm$person[1:4, 1:4] 
```

```{r}
save_path <- file.path(tempdir(), "tmp")
dir.create(save_path)

cdm %>% 
  stow(path = save_path)

list.files(save_path)
```

## Closing connections

Close the database connection with `dbDisconnect`. After the connection is closed the cdm object can no longer be used.

```{r}
DBI::dbDisconnect(con, shutdown = TRUE)
```
