# Version 0.5.1

## Changes

### AutoMM

Changed to a new namespace `autogluon.multimodal` (AutoMM), which is a deep learning "model zoo" of model zoos. On one hand, AutoMM can automatically train deep models for unimodal (image-only, text-only or tabular-only) problems. On the other hand, AutoMM can automatically solve multimodal (any combinations of image, text, and tabular) problems by fusing multiple deep learning models. In addition, AutoMM can be used as a base model in AutoGluon Tabular and participate in the model ensemble.

#### New features

- Supported zero-shot learning with CLIP ([#1922](https://github.com/autogluon/autogluon/pull/1922)) [@zhiqiangdon](https://github.com/zhiqiangdon)
  - Users can directly perform zero-shot image classification with the [CLIP model](https://arxiv.org/abs/2103.00020). Moreover, users can extract image and text embeddings with CLIP to do image-to-text or text-to-image retrieval. 

- Improved efficient finetuning
  - Support “bit_fit”, “norm_fit“, “lora”, “lora_bias”, “lora_norm”. In four multilingual datasets ([xnli](https://huggingface.co/datasets/xnli), [stsb_multi_mt](https://huggingface.co/datasets/stsb_multi_mt), [paws-x](https://huggingface.co/datasets/paws-x), [amazon_reviews_multi](https://huggingface.co/datasets/amazon_reviews_multi)), “lora_bias”, which is a combination of [LoRA](https://arxiv.org/abs/2106.09685) and [BitFit](https://arxiv.org/abs/2106.10199), achieved the best overall performance. Compared to finetuning the whole network, “lora_bias” will only finetune **<0.5%** of the network parameters and can achieve comparable performance on “stsb_multi_mt” ([#1780](https://github.com/autogluon/autogluon/pull/1780), [#1809](https://github.com/autogluon/autogluon/pull/1809)). [@Raldir](https://github.com/Raldir) [@zhiqiangdon](https://github.com/zhiqiangdon)
  - Support finetuning the [mT5-XL](https://huggingface.co/google/mt5-xl) model that has 1.7B parameters on a single NVIDIA G4 GPU. In AutoMM, we only use the T5-encoder (1.7B parameters) like [Sentence-T5](https://aclanthology.org/2022.findings-acl.146.pdf). ([#1933](https://github.com/autogluon/autogluon/pull/1933)) [@sxjscience](https://github.com/sxjscience)

- Added more data augmentation techniques
  - [Mixup](https://arxiv.org/pdf/1710.09412.pdf) for image data. ([#1730](https://github.com/autogluon/autogluon/pull/1730)) [@Linuxdex](https://github.com/Linuxdex)
  - [TrivialAugment](https://arxiv.org/pdf/2103.10158.pdf) for both image and text data. ([#1792](https://github.com/autogluon/autogluon/pull/1792)) [@lzcemma](https://github.com/lzcemma)
  - [Easy text augmentations](https://arxiv.org/pdf/1901.11196.pdf). ([#1756](https://github.com/autogluon/autogluon/pull/1756)) [@lzcemma](https://github.com/lzcemma)

- Enhanced teacher-student model distillation
  - Support distilling the knowledge from a unimodal/multimodal teacher model to a student model. ([#1670](https://github.com/autogluon/autogluon/pull/1670), [#1895](https://github.com/autogluon/autogluon/pull/1895)) [@zhiqiangdon](https://github.com/zhiqiangdon)

#### More tutorials and examples

- [Beginner tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/index.html) of applying AutoMM to image, text, or multimodal (including tabular) data. ([#1861](https://github.com/autogluon/autogluon/pull/1861), [#1908](https://github.com/autogluon/autogluon/pull/1908), [#1858](https://github.com/autogluon/autogluon/pull/1858), [#1869](https://github.com/autogluon/autogluon/pull/1869)) [@bryanyzhu](https://github.com/bryanyzhu) [@sxjscience](https://github.com/sxjscience) [@zhiqiangdon](https://github.com/zhiqiangdon)

- [A zero-shot image classification tutorial](https://auto.gluon.ai/0.5.1/tutorials/multimodal/clip_zeroshot.html) with the CLIP model. ([#1942](https://github.com/autogluon/autogluon/pull/1942)) [@bryanyzhu](https://github.com/bryanyzhu)

- A tutorial of using [CLIP model to extract embeddings](https://auto.gluon.ai/0.5.1/tutorials/multimodal/clip_embedding.html) for image-text retrieval. ([#1957](https://github.com/autogluon/autogluon/pull/1957)) [@bryanyzhu](https://github.com/bryanyzhu)

- [A tutorial](https://auto.gluon.ai/0.5.1/tutorials/multimodal/customization.html) to introduce comprehensive AutoMM configurations ([#1861](https://github.com/autogluon/autogluon/pull/1861)). [@zhiqiangdon](https://github.com/zhiqiangdon)

- [AutoMM for tabular data examples](https://github.com/autogluon/autogluon/tree/master/examples/automm/tabular_dl) ([#1752](https://github.com/autogluon/autogluon/pull/1752), [#1893](https://github.com/autogluon/autogluon/pull/1893), [#1903](https://github.com/autogluon/autogluon/pull/1903)). [@yiqings](https://github.com/yiqings)

- [AutoMM distillation example](https://github.com/autogluon/autogluon/tree/master/examples/automm/distillation) ([#1846](https://github.com/autogluon/autogluon/pull/1846)). [@FANGAreNotGnu](https://github.com/FANGAreNotGnu)

- A Kaggle notebook about how to use AutoMM to predict pet adoption: https://www.kaggle.com/code/linuxdex/use-autogluon-to-predict-pet-adoption. The model achieves the score equivalent to **top 1% (20th/3537) in this kernel-only competition (test data is only available in the kernel without internet access)** ([#1796](https://github.com/autogluon/autogluon/pull/1796), [#1847](https://github.com/autogluon/autogluon/pull/1847), [#1894](https://github.com/autogluon/autogluon/pull/1894), [#1943](https://github.com/autogluon/autogluon/pull/1943)). [@Linuxdex](https://github.com/Linuxdex)


### TimeSeries

We are happy to announce AutoGluon-TimeSeries! Starting with v0.5, AutoGluon now supports AutoML for time series forecasting, 
leveraging both statistical forecasting methods such as ETS and ARIMA, as well as modern deep learning architectures
through [GluonTS](https://ts.gluon.ai/stable/). The new module also features a weighted ensemble of time series models, 
and is geared towards probabilistic (quantile) forecasting to enable many use cases from demand and supply chain forecasting 
to financial applications.

All time series forecasting tasks are supported via the familiar AutoGluon interface, through the
`TimeSeriesPredictor` class. Start forecasting today with the AutoGluon-TimeSeries 
[quick start guide](https://auto.gluon.ai/0.5.1/tutorials/timeseries/forecasting-quickstart.html).

Contributor List: [@canerturkmen](https://github.com/canerturkmen), [@huibinshen](https://github.com/huibinshen), [@Innixma](https://github.com/Innixma), [@yinweisu](https://github.com/yinweisu), [@shchur](https://github.com/shchur), [@gradientsky](https://github.com/gradientsky)  

#### Fixes and enhancements in v0.5.1

- Add early stopping for AutoGluon-TimeSeries models ([#1917](https://github.com/autogluon/autogluon/pull/1917)) [@huibinshen](https://github.com/huibinshen)
- Allow for automatically inferring seasonality period from `TimeSeriesDataFrame` index in `AutoETS`, intelligently setting seasonality to be used in ETS models by default. ([#1914](https://github.com/autogluon/autogluon/pull/1914)) [@canerturkmen](https://github.com/canerturkmen), [@shchur](https://github.com/shchur)
- Changes in model presets, enabling `ARIMA` and GluonTS's `Transformer` models to be enabled by default and removing `MQCNN` models ([#1914](https://github.com/autogluon/autogluon/pull/1914)). [@canerturkmen](https://github.com/canerturkmen), [@shchur](https://github.com/shchur)
- Fix for an issue that affected data sets with custom target column names when using `TimeSeriesPredictor` ([#1901](https://github.com/autogluon/autogluon/pull/1901)) [@canerturkmen](https://github.com/canerturkmen)
- Capping `gluonts`, `sktime` versions ([#1914](https://github.com/autogluon/autogluon/pull/1914), [#1916](https://github.com/autogluon/autogluon/pull/1916)) [@yinweisu](https://github.com/yinweisu), [@canerturkmen](https://github.com/canerturkmen), [@shchur](https://github.com/shchur)