.. _sec_automm_textprediction_beginner:

AutoMM for Text - Quick Start
=============================


``MultiModalPredictor`` can solve problems where the data are either
image, text, numerical values, or categorical features. To get started,
we first demonstrate how to use it to solve problems that only contain
text. We pick two classical NLP problems for the purpose of
demonstration:

-  `Sentiment
   Analysis <https://en.wikipedia.org/wiki/Sentiment_analysis>`__
-  `Sentence Similarity <https://arxiv.org/abs/1910.03940>`__

Here, we format the NLP datasets as data tables where the feature
columns contain text fields and the label column contain numerical
(regression) / categorical (classification) values. Each row in the
table corresponds to one training sample.

.. code:: python

    %matplotlib inline
    
    import numpy as np
    import warnings
    import matplotlib.pyplot as plt
    
    warnings.filterwarnings('ignore')
    np.random.seed(123)

Sentiment Analysis Task
-----------------------

First, we consider the Stanford Sentiment Treebank
(`SST <https://nlp.stanford.edu/sentiment/>`__) dataset, which consists
of movie reviews and their associated sentiment. Given a new movie
review, the goal is to predict the sentiment reflected in the text (in
this case a **binary classification**, where reviews are labeled as 1 if
they convey a positive opinion and labeled as 0 otherwise). Let’s first
load and look at the data, noting the labels are stored in a column
called **label**.

.. code:: python

    from autogluon.core.utils.loaders import load_pd
    train_data = load_pd.load('https://autogluon-text.s3-accelerate.amazonaws.com/glue/sst/train.parquet')
    test_data = load_pd.load('https://autogluon-text.s3-accelerate.amazonaws.com/glue/sst/dev.parquet')
    subsample_size = 1000  # subsample data for faster demo, try setting this to larger values
    train_data = train_data.sample(n=subsample_size, random_state=0)
    train_data.head(10)




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentence</th>
          <th>label</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>43787</th>
          <td>very pleasing at its best moments</td>
          <td>1</td>
        </tr>
        <tr>
          <th>16159</th>
          <td>, american chai is enough to make you put away...</td>
          <td>0</td>
        </tr>
        <tr>
          <th>59015</th>
          <td>too much like an infomercial for ram dass 's l...</td>
          <td>0</td>
        </tr>
        <tr>
          <th>5108</th>
          <td>a stirring visual sequence</td>
          <td>1</td>
        </tr>
        <tr>
          <th>67052</th>
          <td>cool visual backmasking</td>
          <td>1</td>
        </tr>
        <tr>
          <th>35938</th>
          <td>hard ground</td>
          <td>0</td>
        </tr>
        <tr>
          <th>49879</th>
          <td>the striking , quietly vulnerable personality ...</td>
          <td>1</td>
        </tr>
        <tr>
          <th>51591</th>
          <td>pan nalin 's exposition is beautiful and myste...</td>
          <td>1</td>
        </tr>
        <tr>
          <th>56780</th>
          <td>wonderfully loopy</td>
          <td>1</td>
        </tr>
        <tr>
          <th>28518</th>
          <td>most beautiful , evocative</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    </div>



Above the data happen to be stored in the
`Parquet <https://databricks.com/glossary/what-is-parquet>`__ format,
but you can also directly ``load()`` data from a
`CSV <https://en.wikipedia.org/wiki/Comma-separated_values>`__ file or
other equivalent formats. While here we load files from `AWS S3 cloud
storage <https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html>`__,
these could instead be local files on your machine. After loading,
``train_data`` is simply a `Pandas
DataFrame <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html>`__,
where each row represents a different training example.

Training
~~~~~~~~

To ensure this tutorial runs quickly, we simply call ``fit()`` with a
subset of 1000 training examples and limit its runtime to approximately
1 minute. To achieve reasonable performance in your applications, you
are recommended to set much longer ``time_limit`` (eg. 1 hour), or do
not specify ``time_limit`` at all (``time_limit=None``).

.. code:: python

    from autogluon.multimodal import MultiModalPredictor
    import uuid
    model_path = f"./tmp/{uuid.uuid4().hex}-automm_sst"
    predictor = MultiModalPredictor(label='label', eval_metric='acc', path=model_path)
    predictor.fit(train_data, time_limit=180)


.. parsed-literal::
    :class: output

    Global seed set to 123
    Auto select gpus: [0]
    Using 16bit native Automatic Mixed Precision (AMP)
    GPU available: True (cuda), used: True
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs
    HPU available: False, using: 0 HPUs
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    
      | Name              | Type                         | Params
    -------------------------------------------------------------------
    0 | model             | HFAutoModelForTextPrediction | 108 M 
    1 | validation_metric | Accuracy                     | 0     
    2 | loss_func         | CrossEntropyLoss             | 0     
    -------------------------------------------------------------------
    108 M     Trainable params
    0         Non-trainable params
    108 M     Total params
    217.786   Total estimated model params size (MB)
    Epoch 0, global step 3: 'val_acc' reached 0.59500 (best 0.59500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=0-step=3.ckpt' as top 3
    Epoch 0, global step 7: 'val_acc' reached 0.68000 (best 0.68000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=0-step=7.ckpt' as top 3
    Epoch 1, global step 10: 'val_acc' reached 0.72000 (best 0.72000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=1-step=10.ckpt' as top 3
    Epoch 1, global step 14: 'val_acc' reached 0.82000 (best 0.82000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=1-step=14.ckpt' as top 3
    Epoch 2, global step 17: 'val_acc' reached 0.89000 (best 0.89000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=2-step=17.ckpt' as top 3
    Epoch 2, global step 21: 'val_acc' reached 0.93500 (best 0.93500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=2-step=21.ckpt' as top 3
    Epoch 3, global step 24: 'val_acc' reached 0.91000 (best 0.93500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=3-step=24.ckpt' as top 3
    Epoch 3, global step 28: 'val_acc' reached 0.93000 (best 0.93500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=3-step=28.ckpt' as top 3
    Epoch 4, global step 31: 'val_acc' reached 0.93500 (best 0.93500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=4-step=31.ckpt' as top 3
    Epoch 4, global step 35: 'val_acc' reached 0.93500 (best 0.93500), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/b9845fff44d6403eac6c4ef91b8b100c-automm_sst/epoch=4-step=35.ckpt' as top 3
    Epoch 5, global step 38: 'val_acc' was not in top 3
    Epoch 5, global step 42: 'val_acc' was not in top 3
    Epoch 6, global step 45: 'val_acc' was not in top 3
    Epoch 6, global step 49: 'val_acc' was not in top 3
    Epoch 7, global step 52: 'val_acc' was not in top 3
    Epoch 7, global step 56: 'val_acc' was not in top 3




.. parsed-literal::
    :class: output

    <autogluon.multimodal.predictor.MultiModalPredictor at 0x7f3e4096a3a0>



Above we specify that: the column named **label** contains the label
values to predict, AutoGluon should optimize its predictions for the
accuracy evaluation metric, trained models should be saved in the
**automm_sst** folder, and training should run for around 60 seconds.

Evaluation
~~~~~~~~~~

After training, we can easily evaluate our predictor on separate test
data formatted similarly to our training data.

.. code:: python

    test_score = predictor.evaluate(test_data)
    print(test_score)


.. parsed-literal::
    :class: output

    {'acc': 0.8669724770642202}


By default, ``evaluate()`` will report the evaluation metric previously
specified, which is ``accuracy`` in our example. You may also specify
additional metrics, e.g. F1 score, when calling evaluate.

.. code:: python

    test_score = predictor.evaluate(test_data, metrics=['acc', 'f1'])
    print(test_score)


.. parsed-literal::
    :class: output

    {'acc': 0.8669724770642202, 'f1': 0.8760683760683761}


Prediction
~~~~~~~~~~

And you can easily obtain predictions from these models by calling
``predictor.predict()``.

.. code:: python

    sentence1 = "it's a charming and often affecting journey."
    sentence2 = "It's slow, very, very, very slow."
    predictions = predictor.predict({'sentence': [sentence1, sentence2]})
    print('"Sentence":', sentence1, '"Predicted Sentiment":', predictions[0])
    print('"Sentence":', sentence2, '"Predicted Sentiment":', predictions[1])


.. parsed-literal::
    :class: output

    "Sentence": it's a charming and often affecting journey. "Predicted Sentiment": 1
    "Sentence": It's slow, very, very, very slow. "Predicted Sentiment": 0


For classification tasks, you can ask for predicted class-probabilities
instead of predicted classes.

.. code:: python

    probs = predictor.predict_proba({'sentence': [sentence1, sentence2]})
    print('"Sentence":', sentence1, '"Predicted Class-Probabilities":', probs[0])
    print('"Sentence":', sentence2, '"Predicted Class-Probabilities":', probs[1])


.. parsed-literal::
    :class: output

    "Sentence": it's a charming and often affecting journey. "Predicted Class-Probabilities": [0.00267822 0.9973219 ]
    "Sentence": It's slow, very, very, very slow. "Predicted Class-Probabilities": [0.9867517  0.01324833]


We can just as easily produce predictions over an entire dataset.

.. code:: python

    test_predictions = predictor.predict(test_data)
    test_predictions.head()




.. parsed-literal::
    :class: output

    0    1
    1    1
    2    1
    3    1
    4    0
    Name: label, dtype: int64



Save and Load
~~~~~~~~~~~~~

The trained predictor is automatically saved at the end of ``fit()``,
and you can easily reload it.

.. warning::

   ``MultiModalPredictor.load()`` used ``pickle`` module implicitly,
   which is known to be insecure. It is possible to construct malicious
   pickle data which will execute arbitrary code during unpickling.
   Never load data that could have come from an untrusted source, or
   that could have been tampered with. **Only load data you trust.**

.. code:: python

    loaded_predictor = MultiModalPredictor.load(model_path)
    loaded_predictor.predict_proba({'sentence': [sentence1, sentence2]})




.. parsed-literal::
    :class: output

    array([[0.00267822, 0.9973219 ],
           [0.9867517 , 0.01324833]], dtype=float32)



You can also save the predictor to any location by calling ``.save()``.

.. code:: python

    new_model_path = f"./tmp/{uuid.uuid4().hex}-automm_sst"
    loaded_predictor.save(new_model_path)
    loaded_predictor2 = MultiModalPredictor.load(new_model_path)
    loaded_predictor2.predict_proba({'sentence': [sentence1, sentence2]})




.. parsed-literal::
    :class: output

    array([[0.00267822, 0.9973219 ],
           [0.9867517 , 0.01324833]], dtype=float32)



.. _sec_automm_textprediction_extract_embedding:

Extract Embeddings
~~~~~~~~~~~~~~~~~~


You can also use a trained predictor to extract embeddings that maps
each row of the data table to an embedding vector extracted from
intermediate neural network representations of the row.

.. code:: python

    embeddings = predictor.extract_embedding(test_data)
    print(embeddings.shape)


.. parsed-literal::
    :class: output

    (872, 768)


Here, we use TSNE to visualize these extracted embeddings. We can see
that there are two clusters corresponding to our two labels, since this
network has been trained to discriminate between these labels.

.. code:: python

    from sklearn.manifold import TSNE
    X_embedded = TSNE(n_components=2, random_state=123).fit_transform(embeddings)
    for val, color in [(0, 'red'), (1, 'blue')]:
        idx = (test_data['label'].to_numpy() == val).nonzero()
        plt.scatter(X_embedded[idx, 0], X_embedded[idx, 1], c=color, label=f'label={val}')
    plt.legend(loc='best')




.. parsed-literal::
    :class: output

    <matplotlib.legend.Legend at 0x7f3e04096580>




.. figure:: output_beginner_text_e1426f_23_1.png


.. _sec_automm_textprediction_continuous_training:

Continuous Training
~~~~~~~~~~~~~~~~~~~


You can also load a predictor and call ``.fit()`` again to continue
training the same predictor with new data.

.. code:: python

    new_predictor = MultiModalPredictor.load(new_model_path)
    new_predictor.fit(train_data, time_limit=30)
    test_score = new_predictor.evaluate(test_data, metrics=['acc', 'f1'])
    print(test_score)


.. parsed-literal::
    :class: output

    Global seed set to 123
    A new predictor save path is created.This is to prevent you to overwrite previous predictor saved here.You could check current save path at predictor._save_path.If you still want to use this path, set resume=True
    Auto select gpus: [0]
    Using 16bit native Automatic Mixed Precision (AMP)
    GPU available: True (cuda), used: True
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs
    HPU available: False, using: 0 HPUs
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    
      | Name              | Type                         | Params
    -------------------------------------------------------------------
    0 | model             | HFAutoModelForTextPrediction | 108 M 
    1 | validation_metric | Accuracy                     | 0     
    2 | loss_func         | CrossEntropyLoss             | 0     
    -------------------------------------------------------------------
    108 M     Trainable params
    0         Non-trainable params
    108 M     Total params
    217.786   Total estimated model params size (MB)
    Epoch 0, global step 3: 'val_acc' reached 0.95000 (best 0.95000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/AutogluonModels/ag-20221117_040515/epoch=0-step=3.ckpt' as top 3
    Epoch 0, global step 7: 'val_acc' reached 0.90000 (best 0.95000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/AutogluonModels/ag-20221117_040515/epoch=0-step=7.ckpt' as top 3
    Epoch 1, global step 10: 'val_acc' reached 0.87500 (best 0.95000), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/AutogluonModels/ag-20221117_040515/epoch=1-step=10.ckpt' as top 3
    Time limit reached. Elapsed time is 0:00:33. Signaling Trainer to stop.
    Epoch 1, global step 10: 'val_acc' was not in top 3


.. parsed-literal::
    :class: output

    {'acc': 0.8818807339449541, 'f1': 0.8871851040525739}


Sentence Similarity Task
------------------------

Next, let’s use MultiModalPredictor to train a model for evaluating how
semantically similar two sentences are. We use the `Semantic Textual
Similarity
Benchmark <http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark>`__
dataset for illustration.

.. code:: python

    sts_train_data = load_pd.load('https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/train.parquet')[['sentence1', 'sentence2', 'score']]
    sts_test_data = load_pd.load('https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/dev.parquet')[['sentence1', 'sentence2', 'score']]
    sts_train_data.head(10)




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sentence1</th>
          <th>sentence2</th>
          <th>score</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>A plane is taking off.</td>
          <td>An air plane is taking off.</td>
          <td>5.00</td>
        </tr>
        <tr>
          <th>1</th>
          <td>A man is playing a large flute.</td>
          <td>A man is playing a flute.</td>
          <td>3.80</td>
        </tr>
        <tr>
          <th>2</th>
          <td>A man is spreading shreded cheese on a pizza.</td>
          <td>A man is spreading shredded cheese on an uncoo...</td>
          <td>3.80</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Three men are playing chess.</td>
          <td>Two men are playing chess.</td>
          <td>2.60</td>
        </tr>
        <tr>
          <th>4</th>
          <td>A man is playing the cello.</td>
          <td>A man seated is playing the cello.</td>
          <td>4.25</td>
        </tr>
        <tr>
          <th>5</th>
          <td>Some men are fighting.</td>
          <td>Two men are fighting.</td>
          <td>4.25</td>
        </tr>
        <tr>
          <th>6</th>
          <td>A man is smoking.</td>
          <td>A man is skating.</td>
          <td>0.50</td>
        </tr>
        <tr>
          <th>7</th>
          <td>The man is playing the piano.</td>
          <td>The man is playing the guitar.</td>
          <td>1.60</td>
        </tr>
        <tr>
          <th>8</th>
          <td>A man is playing on a guitar and singing.</td>
          <td>A woman is playing an acoustic guitar and sing...</td>
          <td>2.20</td>
        </tr>
        <tr>
          <th>9</th>
          <td>A person is throwing a cat on to the ceiling.</td>
          <td>A person throws a cat on the ceiling.</td>
          <td>5.00</td>
        </tr>
      </tbody>
    </table>
    </div>



In this data, the column named **score** contains numerical values
(which we’d like to predict) that are human-annotated similarity scores
for each given pair of sentences.

.. code:: python

    print('Min score=', min(sts_train_data['score']), ', Max score=', max(sts_train_data['score']))


.. parsed-literal::
    :class: output

    Min score= 0.0 , Max score= 5.0


Let’s train a regression model to predict these scores. Note that we
only need to specify the label column and AutoGluon automatically
determines the type of prediction problem and an appropriate loss
function. Once again, you should increase the short ``time_limit`` below
to obtain reasonable performance in your own applications.

.. code:: python

    sts_model_path = f"./tmp/{uuid.uuid4().hex}-automm_sts"
    predictor_sts = MultiModalPredictor(label='score', path=sts_model_path)
    predictor_sts.fit(sts_train_data, time_limit=60)


.. parsed-literal::
    :class: output

    Global seed set to 123
    Auto select gpus: [0]
    Using 16bit native Automatic Mixed Precision (AMP)
    GPU available: True (cuda), used: True
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs
    HPU available: False, using: 0 HPUs
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    
      | Name              | Type                         | Params
    -------------------------------------------------------------------
    0 | model             | HFAutoModelForTextPrediction | 108 M 
    1 | validation_metric | MeanSquaredError             | 0     
    2 | loss_func         | MSELoss                      | 0     
    -------------------------------------------------------------------
    108 M     Trainable params
    0         Non-trainable params
    108 M     Total params
    217.785   Total estimated model params size (MB)
    Epoch 0, global step 20: 'val_rmse' reached 0.56009 (best 0.56009), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/5838d17807684e12b7b04b0595a32808-automm_sts/epoch=0-step=20.ckpt' as top 3
    Epoch 0, global step 40: 'val_rmse' reached 0.47620 (best 0.47620), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/5838d17807684e12b7b04b0595a32808-automm_sts/epoch=0-step=40.ckpt' as top 3
    Epoch 1, global step 61: 'val_rmse' reached 0.44776 (best 0.44776), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/5838d17807684e12b7b04b0595a32808-automm_sts/epoch=1-step=61.ckpt' as top 3
    Time limit reached. Elapsed time is 0:01:03. Signaling Trainer to stop.
    Epoch 1, global step 61: 'val_rmse' reached 0.44776 (best 0.44776), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/text_prediction/tmp/5838d17807684e12b7b04b0595a32808-automm_sts/epoch=1-step=61-v1.ckpt' as top 3




.. parsed-literal::
    :class: output

    <autogluon.multimodal.predictor.MultiModalPredictor at 0x7f3e03f8e490>



We again evaluate our trained model’s performance on separate test data.
Below we choose to compute the following metrics: RMSE, Pearson
Correlation, and Spearman Correlation.

.. code:: python

    test_score = predictor_sts.evaluate(sts_test_data, metrics=['rmse', 'pearsonr', 'spearmanr'])
    print('RMSE = {:.2f}'.format(test_score['rmse']))
    print('PEARSONR = {:.4f}'.format(test_score['pearsonr']))
    print('SPEARMANR = {:.4f}'.format(test_score['spearmanr']))


.. parsed-literal::
    :class: output

    RMSE = 0.69
    PEARSONR = 0.8910
    SPEARMANR = 0.8945


Let’s use our model to predict the similarity score between a few
sentences.

.. code:: python

    sentences = ['The child is riding a horse.',
                 'The young boy is riding a horse.',
                 'The young man is riding a horse.',
                 'The young man is riding a bicycle.']
    
    score1 = predictor_sts.predict({'sentence1': [sentences[0]],
                                    'sentence2': [sentences[1]]}, as_pandas=False)
    
    score2 = predictor_sts.predict({'sentence1': [sentences[0]],
                                    'sentence2': [sentences[2]]}, as_pandas=False)
    
    score3 = predictor_sts.predict({'sentence1': [sentences[0]],
                                    'sentence2': [sentences[3]]}, as_pandas=False)
    print(score1, score2, score3)


.. parsed-literal::
    :class: output

    4.7364306 3.304551 0.8223398


Although the ``MultiModalPredictor`` currently supports classification
and regression tasks, it can directly be used for many NLP tasks if you
properly format them into a data table. Note that there can be many text
columns in this data table. Refer to the `MultiModalPredictor
documentation <../../api/autogluon.predictor.html#autogluon.multimodal.MultiModalPredictor.fit>`__
to see all available methods/options.

Unlike ``TabularPredictor`` which trains/ensembles different types of
models, ``MultiModalPredictor`` focuses on selecting and finetuning deep
learning based models. Internally, it integrates with
`timm <https://github.com/rwightman/pytorch-image-models>`__ ,
`huggingface/transformers <https://github.com/huggingface/transformers>`__,
`openai/clip <https://github.com/openai/CLIP>`__ as the model zoo.

Other Examples
--------------

You may go to `AutoMM
Examples <https://github.com/awslabs/autogluon/tree/master/examples/automm>`__
to explore other examples about AutoMM.

Customization
-------------

To learn how to customize AutoMM, please refer to
:ref:`sec_automm_customization`.