---
title: "Getting Started with kindling"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started with kindling}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
    collapse = TRUE,
    comment = "#>",
    eval = identical(Sys.getenv("NOT_CRAN"), "true"),
    fig.width = 7,
    fig.height = 5,
    warning = FALSE,
    message = FALSE
)

fansi::set_knit_hooks(knitr::knit_hooks)
```

## Introduction

`{kindling}` bridges the gap between `{torch}` and `{tidymodels}`, providing a streamlined interface for building, training, and tuning deep learning models. This vignette will guide you through the basic usage.

## Installation

You can install `{kindling}` on CRAN:

``` r
install.packages('kindling')
```

Or install the development version from GitHub:

```{r eval = FALSE}
# install.packages("pak")
pak::pak("joshuamarie/kindling")
## devtools::install_github("joshuamarie/kindling") 
```

## Before using {kindling}

```{r}
library(kindling)
```

Before starting, you need to install LibTorch first, the backend of PyTorch, which is also the backend of `{torch}` R package:

``` r
torch::install_torch()
```

## Main Features

Current `{kindling}` supports the following: 

-   Code generation of `{torch}` expression
-   Multiple architectures available

    -  Base models interface: feedforward networks (MLP/DNN/FFNN) and recurrent variants (RNN, LSTM, GRU)
    -  Generalized neural network trainer that has the same topology as MLPs

-   Native support for R ML workflows and pipelines (currently `{tidymodels}`; `{mlr3}` planned)
-   Fine-grained control over network depth, layer sizes, and activation functions
-   GPU acceleration support via `{torch}` tensors

### What it doesn't support

As of `{kindling}` >0.3.0, it supports most of NN architectures thanks to its versatility, as long as they follow typical MLP's topology. This package, however, does not support the following:

1.  Residual Networks (ResNet)
2.  Automatic Integration (AutoInt)
3.  Self-Attention and Inter-sample Attention Transformer (Saint)

To use all of these, you might want to take an interest towards `{brulee}` package instead. The said NN architectures above are available on version 1.0.0 (and later) release. 

## Usage: Three Levels of Interaction

`{kindling}` is powered by R's metaprogramming capabilities through *code generation*. Generated `torch::nn_module()` expressions power the training functions, which in turn serve as engines for `{tidymodels}` integration. This architecture gives you flexibility to work at whatever abstraction level suits your task.

### Level 1: Code Generation for `torch::nn_module`

At the lowest level, you can generate raw `torch::nn_module` code for maximum customization. Functions ending with `_generator` return unevaluated expressions you can inspect, modify, or execute.

Here's how to generate a feedforward network specification:

```{r, collapse=FALSE, comment=""}
ffnn_generator(
    nn_name = "MyFFNN",
    hd_neurons = c(64, 32, 16),
    no_x = 10,
    no_y = 1,
    activations = 'relu'
)
```

This creates a three-hidden-layer network (64 - 32 - 16 neurons) that takes 10 inputs and produces 1 output. Each hidden layer uses ReLU activation, while the output layer remains "untransformed".

### Level 2: Direct Training Interface

Skip the code generation and train models directly with your data. This approach handles all the `{torch}` boilerplate when training the models internally.

Let's classify iris species:

```{r, comment="", collapse=FALSE} 
model = ffnn(
    Species ~ .,
    data = iris,
    hidden_neurons = c(10, 15, 7),
    activations = act_funs(relu, softshrink[lambd = 0.5], elu), 
    loss = "cross_entropy",
    epochs = 100
)

model
```

> For parametric activation functions like softshrink, which contains `"lambd"` ($\lambda$) as its parameter (the default is 1), use indexed syntax (available on v0.3.x+) e.g. `softshrink[lambd = 0.5]` or `softshrink[0.5]`, or a string literal expression e.g. `"softshrink(lambd = 0.5)"`, to transmute the parameter value. See `?kindling::act_funs()` for more details. 

Evaluate the prediction through `predict()`. The `predict()` method is extended for fitted models through its `newdata` argument. 

Two kinds of `predict()` usage: 

1.  **Without `newdata`** predictions default to the training data. 

    ```{r}
    predict(model) |>
        (\(x) table(actual = iris$Species, predicted = x))()
    ```


2.  **With `newdata`** simply pass the new data frame as the new reference. 

    ```{r}
    sample_iris = dplyr::slice_sample(iris, n = 10, by = Species)

    predict(model, newdata = sample_iris) |>
        (\(x) table(actual = sample_iris$Species, predicted = x))()
    ```


### Level 3: Conventional tidymodels Integration

Work with neural networks just like any other `{parsnip}` model. This unlocks the entire `{tidymodels}` toolkit for preprocessing, cross-validation, and model evaluation.

```{r}
# library(kindling)
# library(parsnip)
# library(yardstick)
box::use(
    kindling[mlp_kindling, rnn_kindling, act_funs, args],
    parsnip[fit, augment],
    yardstick[metrics]
)
data(Ionosphere, package = "mlbench")

ionosphere_data = Ionosphere[, -2]

# Train a feedforward network with parsnip
mlp_kindling(
    mode = "classification",
    hidden_neurons = c(128, 64),
    activations = act_funs(relu, softshrink[lambd = 0.5]),
    epochs = 100
) |>
    fit(Class ~ ., data = ionosphere_data) |>
    augment(new_data = ionosphere_data) |>
    metrics(truth = Class, estimate = .pred_class)

# Or try a recurrent architecture (demonstrative example with tabular data)
rnn_kindling(
    mode = "classification",
    hidden_neurons = c(128, 64),
    activations = act_funs(relu, elu),
    epochs = 100,
    rnn_type = "gru"
) |>
    fit(Class ~ ., data = ionosphere_data) |>
    augment(new_data = ionosphere_data) |>
    metrics(truth = Class, estimate = .pred_class)
```

## Hyperparameter Tuning & Resampling

The package has integration with `{tidymodels}`, so it supports hyperparameter tuning via `{tune}` with searchable parameters.

The current searchable parameters under `{kindling}`: 

-   Layer widths (neurons per layer)
-   Network depth (number of hidden layers)
-   Activation function combinations
-   Output activation
-   Optimizer (Type of optimization algorithm)
-   Bias (choose between the presence and the absence of the bias term)
-   Validation Split Proportion
-   Bidirectional (boolean; only for RNN)

The searchable parameters outside from `{kindling}`, i.e. under `{dials}` package such as `learn_rate()` also supported. 

Here's an example: 

```{r, collapse=FALSE, comment=""}
# library(tidymodels)
box::use(
    kindling[
        mlp_kindling, hidden_neurons, activations, output_activation, grid_depth
    ],
    parsnip[fit, augment],
    recipes[recipe],
    workflows[workflow, add_recipe, add_model],
    rsample[vfold_cv],
    tune[tune_grid, tune, select_best, finalize_workflow],
    dials[grid_random],
    yardstick[accuracy, roc_auc, metric_set, metrics]
)

mlp_tune_spec = mlp_kindling(
    mode = "classification",
    hidden_neurons = tune(),
    activations = tune(),
    output_activation = tune()
)

iris_folds = vfold_cv(iris, v = 3)
nn_wf = workflow() |>
    add_recipe(recipe(Species ~ ., data = iris)) |>
    add_model(mlp_tune_spec)

nn_grid_depth = grid_depth(
    hidden_neurons(c(32L, 128L)),
    activations(c("relu", "elu")),
    output_activation(c("sigmoid", "linear")),
    n_hlayer = 2,
    size = 10,
    type = "latin_hypercube"
)

# This is supported but limited to 1 hidden layer only
## nn_grid = grid_random(
##     hidden_neurons(c(32L, 128L)),
##     activations(c("relu", "elu")),
##     output_activation(c("sigmoid", "linear")),
##     size = 10
## )

nn_tunes = tune::tune_grid(
    nn_wf,
    iris_folds,
    grid = nn_grid_depth
)

best_nn = select_best(nn_tunes)
best_nn
final_nn = finalize_workflow(nn_wf, best_nn)
final_nn_model = fit(final_nn, data = iris)
final_nn_model

final_nn_model |>
    augment(new_data = iris) |>
    metrics(truth = Species, estimate = .pred_class)
```

Resampling strategies from `{rsample}` will enable robust cross-validation workflows, orchestrated through the `{tune}` and `{dials}` APIs.

## Variable Importance

`{kindling}` integrates with established variable importance methods from `{NeuralNetTools}` and `{vip}` to interpret trained neural networks. Two primary algorithms are available:

1.  Garson's Algorithm

    ```{r}
    garson(model, bar_plot = FALSE)
    ```

2.  Olden's Algorithm

    ```{r}
    olden(model, bar_plot = FALSE)
    ```

### Integration with {vip}

For users working within the `{tidymodels}` ecosystem, `{kindling}` models work seamlessly with the `{vip}` package:

```{r, fig.height=6, fig.width=8}
#| label: example
#| fig.alt: "Variable Importance Plot"
box::use(
    vip[vi, vip]
)

vi(model) |> 
    vip()
```

*Note: Weight caching increases memory usage proportional to network size. Only enable it when you plan to compute variable importance multiple times on the same model.*

## Learn More

- Visit the package website: https://kindling.joshuamarie.com
- Report issues: https://github.com/joshuamarie/kindling/issues