# Dataset splitting

mlpack provides a simple functions for splitting a dataset into a training set
and a test set.

 * [`Split()`](#split): split a dataset into a training set
   and test set, optionally with labels and weights.

 * [`StratifiedSplit()`](#stratifiedsplit): perform a stratified
   split, ensuring that the training and test set have the same ratios of each
   label.

---

## `Split()`

 * `Split(input, trainData, testData, testRatio, shuffleData=true)`
 * `Split(input, inputLabels, trainData, testData, trainLabels, testLabels, testRatio, shuffleData=true)`
 * `Split(input, inputLabels, inputWeights, trainData, testData, trainLabels, testLabels, trainWeights, testWeights, testRatio, shuffleData=true)`
   - Perform a standard train/test split, with a factor of `testRatio` of the
     data stored in the test set.
   - If `shuffleData` is `false`, the first points in the dataset will be used
     for the training set, and the last points for the test set.
   - `inputLabels` can be specified if there are also labels that should be
     split; `inputLabels.n_cols` should be equal to `input.n_cols`.
   - `inputWeights` can be specified if there are also instance weights that
     should be split; `inputWeights.n_cols` should be equal to `input.n_cols`.

See more details on each parameter in the [Parameters](#parameters) section.

---

## `StratifiedSplit()`

 * `StratifiedSplit(input, inputLabels, trainData, testData, trainLabels,
   testLabels, testRatio, shuffleData=true)`
   - Perform a stratified train/test split, with a factor of `testRatio` of the
     dataset stored in the test set.
   - A stratified split ensures that the ratio of classes in the training and
     test sets matches the ratio of classes in the original dataset.  This can
     be useful for highly imbalanced datasets.
   - If `shuffleData` is `false`, the first points in the dataset for each class
     will be used for the training set, and the last points for the test set.

See more details on each parameter in the [Parameters](#parameters) section.

---

## Parameters

| **name** | **type** | **description** | **default** |
|----------|----------|-----------------|-------------|
| `input` | [`arma::mat`](../matrices.md) | [Column-major](../matrices.md#representing-data-in-mlpack) data matrix. | _(N/A)_ |
| `inputLabels` | [`arma::Row<size_t>`](../matrices.md) | Labels for data matrix.  Should have length `data.n_cols`. | _(N/A)_ |
| `inputWeights` | [`arma::rowvec`](../matrices.md) | Weights for data matrix. Should have length `data.n_cols`. | _(N/A)_ |
| `trainData` | [`arma::mat&`](../matrices.md) | Matrix to store training points in.  Will be set to size `data.n_rows` x `((1.0 - testRatio) * data.n_cols)`. | _(N/A)_ |
| `testData` | [`arma::mat&`](../matrices.md) | Matrix to store test points in. Will be set to size `data.n_rows` x `testRatio * data.n_cols`. | _(N/A)_ |
| `trainLabels` | [`arma::Row<size_t>&`](../matrices.md) | Vector to store training labels in.  Will be set to length `trainData.n_cols`. | _(N/A)_ |
| `testLabels` | [`arma::Row<size_t>&`](../matrices.md) | Vector to store test labels in.  Will be set to length `testData.n_cols`. | _(N/A)_ |
| `trainWeights` | [`arma::rowvec&`](../matrices.md) | Vector to store training weights in.  Will be set to length `trainData.n_cols`. | _(N/A)_ |
| `testWeights` | [`arma::rowvec&`](../matrices.md) | Vector to store test weights in.  Will be set to length `testData.n_cols`. | _(N/A)_ |
| `testRatio` | `double` | Fraction of columns in `input` to use for test set. Typically between 0.1 and 0.25. | _(N/A)_ |
| `shuffleData` | `bool` | If `true`, then training and test sets are sampled randomly from `input`. | `true` |

***Notes:***

 - Any matrix or cube type matching the Armadillo API can be used for `input`,
   `trainData`, and `testData` (e.g. `arma::fmat`, `arma::sp_mat`, `arma::cube`,
   etc.).
    * All three matrices must have the same type.
    * [`arma::field<>`](https://arma.sourceforge.net/docs.html#field) types can
      also be used.

 - Any row vector, matrix, or cube type matching the Armadillo API can be used
   for `labels`, `trainLabels`, and `testLabels` (e.g. `arma::urowvec`,
   `arma::Row<unsigned short>`, `arma::cube`, etc.).
    * All three label parameters must have the same type.
    * [`arma::field<>`](https://arma.sourceforge.net/docs.html#field) types may
      also be used, so long as the given `field` has the same number of columns
      as `input`.

 - Any row vector, matrix, or cube type matching the Armadillo API can be used
   for `weights`, trainWeights`, and `testWeights` (e.g. `arma::frowvec`,
   `arma::fmat`, `arma::cube`, etc.).
    * All three weight parameters must have the same type.
    * [`arma::field<>`](https://arma.sourceforge.net/docs.html#field) types may
      also be used, so long as the given `field` has the same number of columns
      as `input`.

 - Splitting is done on *columns*; so, if vector types are given, they should be
   row vectors (e.g. `arma::rowvec`, `arma::Row<size_t>`, etc.).

## Example usage

Split the unlabeled `cloud` dataset, using 20% of the dataset for the test set.

```c++
// See https://datasets.mlpack.org/cloud.csv.
arma::mat dataset;
mlpack::Load("cloud.csv", dataset, mlpack::Fatal);

arma::mat trainData, testData;

// Split the data, using 20% of the data for the test set.
mlpack::Split(dataset, trainData, testData, 0.2);

// Print the size of each matrix.
std::cout << "Full data size:     " << dataset.n_rows << " x " << dataset.n_cols
    << "." << std::endl;

std::cout << "Training data size: " << trainData.n_rows << " x "
    << trainData.n_cols << "." << std::endl;
std::cout << "Test data size:     " << testData.n_rows << " x "
    << testData.n_cols << "." << std::endl;
```

---

Split the mixed categorical `telecom_churn` dataset and associated responses for
regression, using 25% of the dataset for the test set.  Use 32-bit floating
point elements to represent both the data and responses.

```c++
// See https://datasets.mlpack.org/telecom_churn.arff.
arma::fmat dataset;
mlpack::TextOptions opts = mlpack::Fatal + mlpack::Categorical;
mlpack::Load("telecom_churn.arff", dataset, opts);

// See https://datasets.mlpack.org/telecom_churn.responses.csv.
arma::frowvec labels;
mlpack::Load("telecom_churn.responses.csv", labels, mlpack::Fatal);

arma::fmat trainData, testData;
arma::frowvec trainLabels, testLabels;

// Split the data, using 25% of the data for the test set.
// Note that Split() can accept many different types for the data and the
// labels---here we pass arma::frowvec instead of arma::Row<size_t>.
mlpack::Split(dataset, labels, trainData, testData, trainLabels,
    testLabels, 0.25);

// Print the size of each matrix.
std::cout << "Full data size:       " << dataset.n_rows << " x "
    << dataset.n_cols << "." << std::endl;
std::cout << "Full labels size:     " << labels.n_elem << "." << std::endl;

std::cout << std::endl;

std::cout << "Training data size:   " << trainData.n_rows << " x "
    << trainData.n_cols << "." << std::endl;
std::cout << "Training labels size: " << trainLabels.n_elem << "." << std::endl;
std::cout << "Test data size:     " << testData.n_rows << " x "
    << testData.n_cols << "." << std::endl;
std::cout << "Test labels size:   " << testLabels.n_elem << "." << std::endl;
```

---

Split the `movielens` dataset, which is a sparse matrix.  Don't shuffle when
splitting.

```c++
// See https://datasets.mlpack.org/movielens-100k.csv.
arma::sp_mat dataset;
mlpack::Load("movielens-100k.csv", dataset, mlpack::Fatal);

arma::sp_mat trainData, testData;

// Split the dataset without shuffling.
mlpack::Split(dataset, trainData, testData, 0.2, false);

// Print the first point of the dataset and the training set (these will be the
// same because we did not shuffle during splitting).
std::cout << "First point of full dataset:" << std::endl;
std::cout << dataset.col(0).t() << std::endl;

std::cout << "First point of training set:" << std::endl;
std::cout << trainData.col(0).t() << std::endl;

// Print the last point of the dataset and the test set (these will also be the
// same).
std::cout << "Last point of full dataset:" << std::endl;
std::cout << dataset.col(dataset.n_cols - 1).t() << std::endl;

std::cout << "Last point of test set:" << std::endl;
std::cout << testData.col(testData.n_cols - 1).t() << std::endl;

```

---

Perform a stratified sampling of the `covertype` dataset, printing the
percentage of each class in the original dataset and in the split datasets.

```c++
// See https://datasets.mlpack.org/covertype.data.csv.
arma::mat dataset;
mlpack::Load("covertype.data.csv", dataset, mlpack::Fatal);

// See https://datasets.mlpack.org/covertype.labels.csv.
arma::Row<size_t> labels;
mlpack::Load("covertype.labels.csv", labels, mlpack::Fatal);

arma::mat trainData, testData;
arma::Row<size_t> trainLabels, testLabels;

// Perform a stratified split, keeping 15% of the data as a test set.
mlpack::StratifiedSplit(dataset, labels, trainData, testData, trainLabels,
    testLabels, 0.15);

// Now compute the percentage of each label in the dataset.
const size_t numClasses = arma::max(labels) + 1;
arma::vec classPercentages(numClasses);
for (size_t i = 0; i < labels.n_elem; ++i)
  ++classPercentages[(size_t) labels[i]];
classPercentages /= labels.n_elem;

std::cout << "Percentages of each class in the full dataset:" << std::endl;
for (size_t i = 0; i < numClasses; ++i)
{
  std::cout << " - Class " << i << ": " << 100.0 * classPercentages[i] << "%."
      << std::endl;
}

// Now compute the percentage of each label in the training set.
classPercentages.zeros();
for (size_t i = 0; i < trainLabels.n_elem; ++i)
  ++classPercentages[(size_t) trainLabels[i]];
classPercentages /= trainLabels.n_elem;

std::cout << "Percentages of each class in the training set:" << std::endl;
for (size_t i = 0; i < numClasses; ++i)
{
  std::cout << " - Class " << i << ": " << 100.0 * classPercentages[i] << "%."
      << std::endl;
}

// Finally compute the percentage of each label in the test set.
classPercentages.zeros();
for (size_t i = 0; i < testLabels.n_elem; ++i)
  ++classPercentages[(size_t) testLabels[i]];
classPercentages /= testLabels.n_elem;

std::cout << "Percentages of each class in the training set:" << std::endl;
for (size_t i = 0; i < numClasses; ++i)
{
  std::cout << " - Class " << i << ": " << 100.0 * classPercentages[i] << "%."
      << std::endl;
}
```
