{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "18c7e4db",
   "metadata": {},
   "source": [
    "# Adding a custom model to AutoGluon (Advanced)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/autogluon/autogluon/blob/master/docs/tutorials/tabular/advanced/tabular-custom-model-advanced.ipynb)\n",
    "[![Open In SageMaker Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/autogluon/autogluon/blob/master/docs/tutorials/tabular/advanced/tabular-custom-model-advanced.ipynb)\n",
    "\n",
    "\n",
    "\n",
    "**Tip**: If you are new to AutoGluon, review [Predicting Columns in a Table - Quick Start](../tabular-quick-start.ipynb) to learn the basics of the AutoGluon API.\n",
    "\n",
    "In this tutorial we will cover advanced custom model options that go beyond the topics covered in [Adding a custom model to AutoGluon](tabular-custom-model.ipynb).\n",
    "\n",
    "It is assumed that you have fully read through [Adding a custom model to AutoGluon](tabular-custom-model.ipynb) prior to this tutorial.\n",
    "\n",
    "## Loading the data\n",
    "\n",
    "First we will load the data. For this tutorial we will use the adult income dataset because it has a mix of integer, float, and categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa00faab-252f-44c9-b8f7-57131aa8251c",
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "!pip install autogluon.tabular[all]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34a395db",
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogluon.tabular import TabularDataset\n",
    "\n",
    "train_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')  # can be local CSV file as well, returns Pandas DataFrame\n",
    "test_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')  # another Pandas DataFrame\n",
    "label = 'class'  # specifies which column do we want to predict\n",
    "train_data = train_data.sample(n=1000, random_state=0)  # subsample for faster demo\n",
    "\n",
    "train_data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce03ca04",
   "metadata": {},
   "source": [
    "## Force features to be passed to models without preprocessing / dropping\n",
    "\n",
    "Reasons why you would want to do this is if you have model logic that requires a particular column to always be present,\n",
    "regardless of its content. For example, if you are fine-tuning a pre-trained language model that expects\n",
    "a feature indicating the language of the text in a given row which dictates how the text is preprocessed,\n",
    "but training data only includes one language, without this adjustment\n",
    "the language identifier feature would be dropped prior to fitting the model.\n",
    "\n",
    "### Force features to not be dropped in model-specific preprocessing\n",
    "\n",
    "To avoid dropping features in custom models due to having only 1 unique value,\n",
    "add the following `_get_default_auxiliary_params` method to your custom model class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4c94075",
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogluon.core.models import AbstractModel\n",
    "\n",
    "class DummyModel(AbstractModel):\n",
    "    def _fit(self, X, **kwargs):\n",
    "        print(f'Before {self.__class__.__name__} Preprocessing ({len(X.columns)} features):\\n\\t{list(X.columns)}')\n",
    "        X = self.preprocess(X)\n",
    "        print(f'After  {self.__class__.__name__} Preprocessing ({len(X.columns)} features):\\n\\t{list(X.columns)}')\n",
    "        print(X.head(5))\n",
    "\n",
    "class DummyModelKeepUnique(DummyModel):\n",
    "    def _get_default_auxiliary_params(self) -> dict:\n",
    "        default_auxiliary_params = super()._get_default_auxiliary_params()\n",
    "        extra_auxiliary_params = dict(\n",
    "            drop_unique=False,  # Whether to drop features that have only 1 unique value, default is True\n",
    "        )\n",
    "        default_auxiliary_params.update(extra_auxiliary_params)\n",
    "        return default_auxiliary_params"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbdd281c",
   "metadata": {},
   "source": [
    "### Force features to not be dropped in global preprocessing\n",
    "\n",
    "While the above fix for model-specific preprocessing works if the feature is still present after global preprocessing,\n",
    "it won't help if the feature was already dropped before getting to the model. For this, we need to\n",
    "create a new feature generator class\n",
    "which separates the preprocessing logic between normal features and user override features.\n",
    "\n",
    "Here is an example implementation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec38d411",
   "metadata": {},
   "outputs": [],
   "source": [
    "# WARNING: To use this in practice, you must put this code in a separate python file\n",
    "#  from the main process and import it or else it will not be serializable.)\n",
    "from autogluon.features import BulkFeatureGenerator, AutoMLPipelineFeatureGenerator, IdentityFeatureGenerator\n",
    "\n",
    "\n",
    "class CustomFeatureGeneratorWithUserOverride(BulkFeatureGenerator):\n",
    "    def __init__(self, automl_generator_kwargs: dict = None, **kwargs):\n",
    "        generators = self._get_default_generators(automl_generator_kwargs=automl_generator_kwargs)\n",
    "        super().__init__(generators=generators, **kwargs)\n",
    "\n",
    "    def _get_default_generators(self, automl_generator_kwargs: dict = None):\n",
    "        if automl_generator_kwargs is None:\n",
    "            automl_generator_kwargs = dict()\n",
    "\n",
    "        generators = [\n",
    "            [\n",
    "                # Preprocessing logic that handles normal features\n",
    "                AutoMLPipelineFeatureGenerator(banned_feature_special_types=['user_override'], **automl_generator_kwargs),\n",
    "\n",
    "                # Preprocessing logic that handles special features user wishes to treat separately, here we simply skip preprocessing for these features.\n",
    "                IdentityFeatureGenerator(infer_features_in_args=dict(required_special_types=['user_override'])),\n",
    "            ],\n",
    "        ]\n",
    "        return generators"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f44833e4",
   "metadata": {},
   "source": [
    "The above code splits the preprocessing logic of a feature\n",
    "depending on if it is tagged with the `'user_override'` special type in feature metadata.\n",
    "To tag three features `['age', 'native-country', 'dummy_feature']` in this way,\n",
    "you can do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7186bf6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# add a useless dummy feature to show that it is not dropped in preprocessing\n",
    "train_data['dummy_feature'] = 'dummy value'\n",
    "test_data['dummy_feature'] = 'dummy value'\n",
    "\n",
    "from autogluon.tabular import FeatureMetadata\n",
    "feature_metadata = FeatureMetadata.from_df(train_data)\n",
    "\n",
    "print('Before inserting overrides:')\n",
    "print(feature_metadata)\n",
    "\n",
    "feature_metadata = feature_metadata.add_special_types(\n",
    "    {\n",
    "        'age': ['user_override'],\n",
    "        'native-country': ['user_override'],\n",
    "        'dummy_feature': ['user_override'],\n",
    "    }\n",
    ")\n",
    "\n",
    "print('After inserting overrides:')\n",
    "print(feature_metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00f9a085",
   "metadata": {},
   "source": [
    "Note that this is only one example implementation of a custom feature generator that has bifurcated preprocessing logic.\n",
    "Users can make their tagging and feature generator logic arbitrarily complex to fit their needs.\n",
    "In this example, we perform the standard preprocessing on non-tagged features, and for tagged features we pass\n",
    "them through `IdentityFeatureGenerator` which is a no-op logic that does not alter the features in any way.\n",
    "Instead of an `IdentityFeatureGenerator`, you could use any kind of feature generator to suite your needs.\n",
    "\n",
    "### Putting it all together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e99b1c82",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separate features and labels\n",
    "X = train_data.drop(columns=[label])\n",
    "y = train_data[label]\n",
    "X_test = test_data.drop(columns=[label])\n",
    "y_test = test_data[label]\n",
    "\n",
    "# preprocess the label column, as done in the prior custom model tutorial\n",
    "from autogluon.core.data import LabelCleaner\n",
    "from autogluon.core.utils import infer_problem_type\n",
    "# Construct a LabelCleaner to neatly convert labels to float/integers during model training/inference, can also use to inverse_transform back to original.\n",
    "problem_type = infer_problem_type(y=y)  # Infer problem type (or else specify directly)\n",
    "label_cleaner = LabelCleaner.construct(problem_type=problem_type, y=y)\n",
    "y_preprocessed = label_cleaner.transform(y)\n",
    "y_test_preprocessed = label_cleaner.transform(y_test)\n",
    "\n",
    "# Make sure to specify your custom feature metadata to the feature generator\n",
    "my_custom_feature_generator = CustomFeatureGeneratorWithUserOverride(feature_metadata_in=feature_metadata)\n",
    "\n",
    "X_preprocessed = my_custom_feature_generator.fit_transform(X)\n",
    "X_test_preprocessed = my_custom_feature_generator.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afaa4806",
   "metadata": {},
   "source": [
    "Notice how the user_override features were not preprocessed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a1e0f0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(X_preprocessed.columns))\n",
    "X_preprocessed.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb0fd121",
   "metadata": {},
   "source": [
    "Now lets see what happens when we send this data to fit a dummy model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29c1528c",
   "metadata": {},
   "outputs": [],
   "source": [
    "dummy_model = DummyModel()\n",
    "dummy_model.fit(X=X, y=y, feature_metadata=my_custom_feature_generator.feature_metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a81e1f9",
   "metadata": {},
   "source": [
    "Notice how the model dropped `dummy_feature` during the preprocess call. Now lets see what happens if we use `DummyModelKeepUnique`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "538515dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "dummy_model_keep_unique = DummyModelKeepUnique()\n",
    "dummy_model_keep_unique.fit(X=X, y=y, feature_metadata=my_custom_feature_generator.feature_metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cde6a33",
   "metadata": {},
   "source": [
    "Now `dummy_feature` is no longer dropped!\n",
    "\n",
    "The above code logic can be re-used for testing your own complex model implementations,\n",
    "simply replace `DummyModelKeepUnique` with your custom model and check that it keeps the features you want to use.\n",
    "\n",
    "### Keeping Features via TabularPredictor\n",
    "\n",
    "Now let's demonstrate how to do this via TabularPredictor in far fewer lines of code.\n",
    "Note that this code will raise an exception if ran in this tutorial because the\n",
    "custom model and feature generator must exist in other files for them to be serializable.\n",
    "Therefore, we will not run the code in the tutorial.\n",
    "(It will also raise an exception because DummyModel isn't a real model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a13c1ca",
   "metadata": {},
   "source": [
    "```\n",
    "from autogluon.tabular import TabularPredictor\n",
    "\n",
    "feature_generator = CustomFeatureGeneratorWithUserOverride()\n",
    "predictor = TabularPredictor(label=label)\n",
    "predictor.fit(\n",
    "    train_data=train_data,\n",
    "    feature_metadata=feature_metadata,  # feature metadata with your overrides\n",
    "    feature_generator=feature_generator,  # your custom feature generator that handles the overrides\n",
    "    hyperparameters={\n",
    "        'GBM': {},  # Can fit your custom model alongside default models\n",
    "        DummyModel: {},  # Will drop dummy_feature\n",
    "        DummyModelKeepUnique: {},  # Will not drop dummy_feature\n",
    "        # DummyModel: {'ag_args_fit': {'drop_unique': False}},  # This is another way to get same result as using DummyModelKeepUnique\n",
    "    }\n",
    ")\n",
    "```\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}