{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e183dc18",
   "metadata": {},
   "source": [
    "# Predicting Columns in a Table - Deployment Optimization\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/autogluon/autogluon/blob/master/docs/tutorials/tabular/advanced/tabular-deployment.ipynb)\n",
    "[![Open In SageMaker Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/autogluon/autogluon/blob/master/docs/tutorials/tabular/advanced/tabular-deployment.ipynb)\n",
    "\n",
    "\n",
    "\n",
    "This tutorial will cover how to perform the end-to-end AutoML process to create an optimized and deployable\n",
    "AutoGluon artifact for production usage.\n",
    "\n",
    "This tutorial assumes you have already read [Predicting Columns in a Table - Quick Start](../tabular-quick-start.ipynb) and [Predicting Columns in a Table - In Depth](../tabular-indepth.ipynb).\n",
    "\n",
    "## Fitting a TabularPredictor\n",
    "\n",
    "We will again use the AdultIncome dataset as in the previous tutorials and train a predictor\n",
    "to predict whether the person's income exceeds $50,000 or not, which is recorded in the `class` column of this table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa00faab-252f-44c9-b8f7-57131aa8251c",
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "!pip install autogluon.tabular[all]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f16e9582",
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogluon.tabular import TabularDataset, TabularPredictor\n",
    "train_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')\n",
    "label = 'class'\n",
    "subsample_size = 500  # subsample subset of data for faster demo, try setting this to much larger values\n",
    "train_data = train_data.sample(n=subsample_size, random_state=0)\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0fb53c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "save_path = 'agModels-predictClass-deployment'  # specifies folder to store trained models\n",
    "predictor = TabularPredictor(label=label, path=save_path).fit(train_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18c14745",
   "metadata": {},
   "source": [
    "Next, load separate test data to demonstrate how to make predictions on new examples at inference time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "132eb22b",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')\n",
    "y_test = test_data[label]  # values to predict\n",
    "test_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d936a72",
   "metadata": {},
   "source": [
    "We use our trained models to make predictions on the new data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7318e0d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = TabularPredictor.load(save_path)  # unnecessary, just demonstrates how to load previously-trained predictor from file\n",
    "\n",
    "y_pred = predictor.predict(test_data)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88ad9bde",
   "metadata": {},
   "source": [
    "We can use leaderboard to evaluate the performance of each individual trained model on our labeled test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80076630",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.leaderboard(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fffd5255",
   "metadata": {},
   "source": [
    "## Snapshot a Predictor with .clone()\n",
    "\n",
    "Now that we have a working predictor artifact, we may want to alter it in a variety of ways to better suite our needs.\n",
    "For example, we may want to delete certain models to reduce disk usage via `.delete_models()`,\n",
    "or train additional models on top of the ones we already have via `.fit_extra()`.\n",
    "\n",
    "While you can do all of these operations on your predictor,\n",
    "you may want to be able to be able to revert to a prior state of the predictor in case something goes wrong.\n",
    "This is where `predictor.clone()` comes in.\n",
    "\n",
    "`predictor.clone()` allows you to create a snapshot of the given predictor,\n",
    "cloning the artifacts of the predictor to a new location.\n",
    "You can then freely play around with the predictor and always load \n",
    "the earlier snapshot in case you want to undo your actions.\n",
    "\n",
    "All you need to do to clone a predictor is specify a new directory path to clone to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "056bd2d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "save_path_clone = save_path + '-clone'\n",
    "# will return the path to the cloned predictor, identical to save_path_clone\n",
    "path_clone = predictor.clone(path=save_path_clone)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e1157f1",
   "metadata": {},
   "source": [
    "Note that this logic doubles disk usage, as it completely clones\n",
    "every predictor artifact on disk to make an exact replica.\n",
    "\n",
    "Now we can load the cloned predictor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09e7a267",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone = TabularPredictor.load(path=path_clone)\n",
    "# You can alternatively load the cloned TabularPredictor at the time of cloning:\n",
    "# predictor_clone = predictor.clone(path=save_path_clone, return_clone=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef68f0f",
   "metadata": {},
   "source": [
    "We can see that the cloned predictor has the same leaderboard and functionality as the original:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "891fbca3",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_clone = predictor.predict(test_data)\n",
    "y_pred_clone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64c7d4b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred.equals(y_pred_clone)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3005b9dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone.leaderboard(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c9baf43",
   "metadata": {},
   "source": [
    "Now let's do some extra logic with the clone, such as calling refit_full:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c410403",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone.refit_full()\n",
    "\n",
    "predictor_clone.leaderboard(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e1ffdb5",
   "metadata": {},
   "source": [
    "We can see that we were able to fit additional models, but for whatever reason we may want to undo this operation.\n",
    "\n",
    "Luckily, our original predictor is untouched!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67976b0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.leaderboard(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e2a5624",
   "metadata": {},
   "source": [
    "We can simply clone a new predictor from our original, and we will no longer be impacted\n",
    "by the call to refit_full on the prior clone.\n",
    "\n",
    "## Snapshot a deployment optimized Predictor via .clone_for_deployment()\n",
    "\n",
    "Instead of cloning an exact copy, we can instead clone a copy\n",
    "which has the minimal set of artifacts needed to do prediction.\n",
    "\n",
    "Note that this optimized clone will have very limited functionality outside of calling predict and predict_proba.\n",
    "For example, it will be unable to train more models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a12802a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "save_path_clone_opt = save_path + '-clone-opt'\n",
    "# will return the path to the cloned predictor, identical to save_path_clone_opt\n",
    "path_clone_opt = predictor.clone_for_deployment(path=save_path_clone_opt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c19918c",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone_opt = TabularPredictor.load(path=path_clone_opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e17ba759-33c3-4b0e-958e-4b414c229895",
   "metadata": {},
   "source": [
    "To avoid loading the model in every prediction call, we can persist the model in memory by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8963a672-50fe-437b-9719-4cbec06041ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone_opt.persist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8eff20c",
   "metadata": {},
   "source": [
    "We can see that the optimized clone still makes the same predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bdac8628",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_clone_opt = predictor_clone_opt.predict(test_data)\n",
    "y_pred_clone_opt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2078f4bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred.equals(y_pred_clone_opt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ee8f96b",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone_opt.leaderboard(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dcd851d",
   "metadata": {},
   "source": [
    "We can check the disk usage of the optimized clone compared to the original:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "034b4583",
   "metadata": {},
   "outputs": [],
   "source": [
    "size_original = predictor.disk_usage()\n",
    "size_opt = predictor_clone_opt.disk_usage()\n",
    "print(f'Size Original:  {size_original} bytes')\n",
    "print(f'Size Optimized: {size_opt} bytes')\n",
    "print(f'Optimized predictor achieved a {round((1 - (size_opt/size_original)) * 100, 1)}% reduction in disk usage.')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9d2ad58",
   "metadata": {},
   "source": [
    "We can also investigate the difference in the files that exist in the original and optimized predictor.\n",
    "\n",
    "Original:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "adce7bf1",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.disk_usage_per_file()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b35fb10",
   "metadata": {},
   "source": [
    "Optimized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5362017b",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone_opt.disk_usage_per_file()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b2a84d-e48b-4603-a396-b33a65d0918b",
   "metadata": {},
   "source": [
    "## Compile models for maximized inference speed\n",
    "\n",
    "In order to further improve inference efficiency, we can call `.compile()` to automatically\n",
    "convert sklearn function calls into their ONNX equivalents.\n",
    "Note that this is currently an experimental feature, which only improves RandomForest and TabularNeuralNetwork models.\n",
    "The compilation and inference speed acceleration require installation of `skl2onnx` and `onnxruntime` packages.\n",
    "To install supported versions of these packages automatically, we can call `pip install autogluon.tabular[skl2onnx]`\n",
    "on top of an existing AutoGluon installation, or `pip install autogluon.tabular[all,skl2onnx]` on a new AutoGluon installation.\n",
    "\n",
    "It is important to make sure the predictor is cloned, because once the models are compiled, it won't support fitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbf8805f-8f0f-433f-b843-8d04a434c0d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor_clone_opt.compile()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b21aa01-5d1c-42b8-bea5-dbe38ff10df9",
   "metadata": {},
   "source": [
    "With the compiled predictor, the prediction results might not be exactly the same but should be very close."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d732ef25-9bf1-4c9f-a225-b874388ddc0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_compile_opt = predictor_clone_opt.predict(test_data)\n",
    "y_pred_compile_opt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffc87467",
   "metadata": {},
   "source": [
    "Now all that is left is to upload the optimized predictor to a centralized storage location such as S3.\n",
    "To use this predictor in a new machine / system, simply download the artifact to local disk and load the predictor.\n",
    "Ensure that when loading a predictor you use the same Python version\n",
    "and AutoGluon version used during training to avoid instability."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}