---
title: "Data preparation"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Data preparation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(echo = TRUE, fig.align = "center", out.width = "100%")
knitr::opts_chunk$set(comment = "#>")
old_options <- options(width = 100)
```

`spifa()` expects data in a specific shape: one row per respondent, and the binary item
responses stored as a single *matrix-valued column* (`nobs x nitems`) rather than one
column per item. This is the same convention base R uses for multivariate `lm()`. This
vignette builds a small dataset acceptable by `spifa()`.

```{r}
library(spifa)
library(sf)
```

## Simulating a raw questionnaire

Five binary items, a household `wealth` covariate, and GPS coordinates, for 20 households:

```{r}
set.seed(1)
nobs <- 20
nitems <- 5

raw <- data.frame(
  id = seq_len(nobs),
  wealth = rnorm(nobs),
  lon = runif(nobs, -71.70, -71.68),
  lat = runif(nobs, -7.06, -7.04),
  item1 = rbinom(nobs, 1, 0.1) == 1,
  item2 = rbinom(nobs, 1, 0.2) == 1,
  item3 = rbinom(nobs, 1, 0.3) == 1,
  item4 = rbinom(nobs, 1, 0.4) == 1,
  item5 = rbinom(nobs, 1, 0.5) == 1
)
raw
```

## Data for `spifa()`

We extract the item as a matrix, and then provide it to a new `data.frame` to later
convert to `sf`:

```{r}
# get items as a matrix
items <- subset(raw, select = grep("^item", names(raw))) |>
  as.matrix() |>
  unname()
# create data.frame and geo-reference it
data <- data.frame(items = I(items), wealth = raw$wealth, x = raw$lon, y = raw$lat) |>
    st_as_sf(coords = c("x", "y"), crs = 4326)
data
```

Notice that we can easily access the response `items`:

```{r}
data$items
```

This dataset can directly be used to fit a model:

```{r}
spifa(items ~ wealth, data = data, nfactors = 2, niter = 5)
```

Alternatively, you could also create the `sf` data first and then assign the items:

```{r}
data <- data.frame(wealth = raw$wealth, x = raw$lon, y = raw$lat) |>
    st_as_sf(coords = c("x", "y"), crs = 4326)
data$items <- items
data
```

```{r}
spifa(items ~ wealth, data = data, nfactors = 2, niter = 5)
```

## Missing values

### Missing item responses

Individual missing item responses (`NA`) don't need to be dropped or
imputed -- `spifa()` handles them natively, treating a missing response as
an unobserved auxiliary variable to be sampled along with everything else,
rather than requiring complete rows:

```{r}
data_na <- data
data_na$items[1:5, 1] <- NA

fit_na <- spifa(items ~ 1, data = data_na, nfactors = 2, ngp = 0, niter = 5)
fit_na
```

As you can see in `Dimensions`, the missing responses are not removed but rather
modelled.

### Missing predictor values

Missing *predictor* values behave differently, and are worth calling out:
a `NA` in a predictor column drops that respondent from the whole fit:

```{r}
data_na <- data
data_na$wealth[1:5] <- NA

fit_na <- spifa(items ~ wealth, data = data_na, nfactors = 2, ngp = 0, niter = 5)
fit_na
```

As you can see in `Dimensions`, the 5 respondents with a missing `wealth` value are
dropped -- 15 respondents remain, unlike the item-response case above.

```{r, include = FALSE}
options(old_options)
```