Version 1.2.0

We’re happy to announce the AutoGluon 1.2.0 release.

AutoGluon 1.2 contains massive improvements to both Tabular and TimeSeries modules, each achieving a 70% win-rate vs AutoGluon 1.1. This release additionally adds support for Python 3.12 and drops support for Python 3.8.

This release contains 186 commits from 19 contributors! See the full commit change-log here: https://github.com/autogluon/autogluon/compare/v1.1.1…v1.2.0

Join the community:
Get the latest updates: Twitter

Loading models trained on older versions of AutoGluon is not supported. Please re-train models using AutoGluon 1.2.

For Tabular, we encompass the primary enhancements of the new TabPFNMix tabular foundation model and parallel fit strategy into the new "experimental_quality" preset to ensure a smooth transition period for those who wish to try the new cutting edge features. We will be using this release to gather feedback prior to incorporating these features into the other presets. We also introduce a new stack layer model pruning technique that results in a 3x inference speedup on small datasets with zero performance loss and greatly improved post-hoc calibration across the board, particularly on small datasets.

For TimeSeries, we introduce Chronos-Bolt, our latest foundation model integrated into AutoGluon, with massive improvements to both accuracy and inference speed compared to Chronos, along with fine-tuning capabilities. We also added covariate regressor support!

We are also excited to announce AutoGluon-Assistant (AG-A), our first venture into the realm of Automated Data Science.

See more details in the Spotlights below!

Spotlight

AutoGluon Becomes the Golden Standard for Competition ML in 2024

Before diving into the new features of 1.2, we would like to start by highlighting the wide-spread adoption AutoGluon has received on competition ML sites like Kaggle in 2024. Across all of 2024, AutoGluon was used to achieve a top 3 finish in 15 out of 18 tabular Kaggle competitions, including 7 first place finishes, and was never outside the top 1% of private leaderboard placements, with an average of over 1000 competing human teams in each competition. In the $75,000 prize money 2024 Kaggle AutoML Grand Prix, AutoGluon was used by the 1st, 2nd, and 3rd place teams, with the 2nd place team led by two AutoGluon developers: Lennart Purucker and Nick Erickson! For comparison, in 2023 AutoGluon achieved only 1 first place and 1 second place solution. We attribute the bulk of this increase to the improvements seen in AutoGluon 1.0 and beyond.

We’d like to emphasize that these results are achieved via human expert interaction with AutoGluon and other tools, and often includes manual feature engineering and hyperparameter tuning to get the most out of AutoGluon. To see a live tracking of all AutoGluon solution placements on Kaggle, refer to our AWESOME.md ML competition section where we provide links to all solution write-ups.

AutoGluon-Assistant: Automating Data Science with AutoGluon and LLMs

We are excited to share the release of a new AutoGluon-Assistant module (AG-A), powered by LLMs from AWS Bedrock or OpenAI. AutoGluon-Assistant empowers users to solve tabular machine learning problems using only natural language descriptions, in zero lines of code with our simple user interface. Fully autonomous AG-A outperforms 74% of human ML practitioners in Kaggle competitions and secured a live top 10 finish in the $75,000 prize money 2024 Kaggle AutoML Grand Prix competition as Team AGA 🤖!

TabularPredictor presets=”experimental_quality”

TabularPredictor has a new "experimental_quality" preset that offers even better predictive quality than "best_quality". On the AutoMLBenchmark, we observe a 70% winrate vs best_quality when running for 4 hours on a 64 CPU machine. This preset is a testing ground for cutting edge features and models which we hope to incorporate into best_quality for future releases. We recommend to use a machine with at least 16 CPU cores, 64 GB of memory, and a 4 hour+ time_limit to get the most benefit out of experimental_quality. Please let us know via a GitHub issue if you run into any problems running the experimental_quality preset.

TabPFNMix: A Foundation Model for Tabular Data

TabPFNMix is the first tabular foundation model created by the AutoGluon team, and was pre-trained exclusively on synthetic data. The model builds upon the prior work of TabPFN and TabForestPFN. TabPFNMix to the best of our knowledge achieves a new state-of-the-art for individual open source model performance on datasets between 1000 and 10000 samples, and also supports regression tasks! Across the 109 classification datasets with less than or equal to 10000 training samples in TabRepo, fine-tuned TabPFNMix outperforms all prior models, with a 64% win-rate vs the strongest tree model, CatBoost, and a 61% win-rate vs fine-tuned TabForestPFN.

The model is available via the TABPFNMIX hyperparameters key, and is used in the new experimental_quality preset. We recommend using this model for datasets smaller than 50,000 training samples, ideally with a large time limit and 64+ GB of memory. This work is still in the early stages, and we appreciate any feedback from the community to help us iterate and improve for future releases. You can learn more by going to our HuggingFace model page for the model (tabpfn-mix-1.0-classifier, tabpfn-mix-1.0-regressor). Give us a like on HuggingFace if you want to see more! A paper is planned in future to provide more details about the model.

fit_strategy=”parallel”

AutoGluon’s TabularPredictor now supports the new fit argument fit_strategy and the new "parallel" option, enabled by default in the new experimental_quality preset. For machines with 16 or more CPU cores, the parallel fit strategy offers a major speedup over the previous "sequential" strategy. We estimate with 64 CPU cores that most datasets will experience a 2-4x speedup, with the speedup getting larger as CPU cores increase.

Chronos-Bolt⚡: a 250x faster, more accurate Chronos model

Chronos-Bolt is our latest foundation model for forecasting that has been integrated into AutoGluon. It is based on the T5 encoder-decoder architecture and has been trained on nearly 100 billion time series observations. It chunks the historical time series context into patches of multiple observations, which are then input into the encoder. The decoder then uses these representations to directly generate quantile forecasts across multiple future steps—a method known as direct multi-step forecasting. Chronos-Bolt models are up to 250 times faster and 20 times more memory-efficient than the original Chronos models of the same size.

The following plot compares the inference time of Chronos-Bolt against the original Chronos models for forecasting 1024 time series with a context length of 512 observations and a prediction horizon of 64 steps.

Chronos-Bolt models are not only significantly faster but also more accurate than the original Chronos models. The following plot reports the probabilistic and point forecasting performance of Chronos-Bolt in terms of the Weighted Quantile Loss (WQL) and the Mean Absolute Scaled Error (MASE), respectively, aggregated over 27 datasets (see the Chronos paper for details on this benchmark). Remarkably, despite having no prior exposure to these datasets during training, the zero-shot Chronos-Bolt models outperform commonly used statistical models and deep learning models that have been trained on these datasets (highlighted by *). Furthermore, they also perform better than other FMs, denoted by a +, which indicates that these models were pretrained on certain datasets in our benchmark and are not entirely zero-shot. Notably, Chronos-Bolt (Base) also surpasses the original Chronos (Large) model in terms of the forecasting accuracy while being over 600 times faster.

Chronos-Bolt models are now available through AutoGluon in four sizes—Tiny (9M), Mini (21M), Small (48M), and Base (205M)—and can also be used on the CPU. With the addition of Chronos-Bolt models and other enhancements, AutoGluon v1.2 achieves a 70%+ win rate against the previous release!

In addition to the new Chronos-Bolt models, we have also added support for effortless fine-tuning of Chronos and Chronos-Bolt models. Check out the updated Chronos tutorial to learn how to use and fine-tune Chronos-Bolt models.

Time Series Covariate Regressors

We have added support for covariate regressors for all forecasting models. Covariate regressors are tabular regression models that can be combined with univariate forecasting models to incorporate exogenous information. These are particularly useful for foundation models like Chronos-Bolt, which rely solely on the target time series’ historical data and cannot directly use exogenous information (such as holidays or promotions). To improve the predictions of univariate models when covariates are available, a covariate regressor is first fit on the known covariates and static features to predict the target column at each time step. The predictions of the covariate regressor are then subtracted from the target column, and the univariate model then forecasts the residuals. The Chronos tutorial showcases how covariate regressors can be used with Chronos-Bolt.

General

Improvements

  • Update full_install.sh to install AutoGluon in parallel and to use uv, resulting in much faster source installation times. @Innixma (#4582, #4587, #4592)

Dependencies

  • Python 3.12 support added. @suzhoum (#4536)

  • Python 3.8 support dropped. @prateekdesai04 (#4512)

  • Update numpy to >=1.25.0,<2.1.4. @suzhoum (#4538)

  • Update scipy to >=1.5.4,<1.16. @suzhoum (#4538)

  • Update torch to >=2.2,<2.6. @tonyhoo (#4360, #4612)

  • Update ray to >=2.10.0,<2.40. @suzhoum, @Innixma (#4302, #4688)

  • Update scikit-learn to >=1.4.0,<1.5.3. @prateekdesai04 (#4420, #4570)

  • Update matplotlib to >=3.7.0,<3.11. @suzhoum (#4511)

  • Update pyarrow to >=15.0.0. @prateekdesai04 (#4520)

  • Update psutil to >=5.7.3,<7.0.0. @prateekdesai04 (#4570)

  • Update Pillow to >=10.0.1,<12. @prateekdesai04 (#4570)

  • Update xgboost to >=1.6,<2.2. @prateekdesai04 (#4570)

  • Update torchvision to >=0.16.0,<0.21.0. @Innixma (#4579)

  • Update nltk to >=3.4.5,<3.9. @tonyhoo (#4604)

  • Update timm to >=0.9.5,<1.0.7. @prateekdesai04 (#4580)

  • Update lightning to >=2.2,<2.6. @tonyhoo (#4612)

  • Update async_timeout to >=4.0,<6. @tonyhoo (#4612)

  • Update transformers to >4.38.0,<5. @tonyhoo (#4612)

  • Update accelerate to >=0.34.0,<1.0. @cheungdaven @tonyhoo @shchur (#4596, #4612, #4676)

  • Update lightgbm to >=4.0,<4.6. @Innixma (#4688)

  • Update scikit-learn-intelex to >=2024.0,<2025.1. @Innixma (#4688)

Documentation

  • Update install instructions to use proper torch and ray versions. @Innixma (#4581)

  • Add +cpu tag for cpu installation guide. @tonyhoo (#4554)

  • Add SECURITY.md for vulnerability reporting. @tonyhoo (#4298)

Fixes and Improvements

  • Speed up DropDuplicatesFeatureGenerator fit time by 2x+. @shchur (#4543)

  • Add compute_metric as a replacement for compute_weighted_metric with improved compatibility across the project. @Innixma (#4631)

  • Enhanced generate_train_test_split. @Innixma (#4478)

Tabular

New Features

  • Add TabPFNMix model. Try it out with presets="experimental". @xiyuanzh @Innixma (#4671, #4694)

  • Parallel model fit support. Try it out with fit_strategy="parallel". @LennartPurucker @Innixma (#4606)

  • Predictor callbacks support. @Innixma (#4327, #4473)

  • Learning curve generation feature. @adibiasio @Innixma (#4411, #4635)

  • Set calibrate_decision_threshold="auto" by default, and improve decision threshold calibration. This dramatically improves results when the eval_metric is f1 and balanced_accuracy for binary classification. @Innixma (#4632)

  • Add roc_auc_ovo and roc_auc_ovr metrics. @Innixma (#4248)

  • Add support for custom memory (soft) limits. @LennartPurucker (#4333)

  • Add ag.compile hyperparameter to models to enable compiling at fit time rather than with predictor.compile. @Innixma (#4354)

  • Add AdaptiveES support to NN_TORCH and increase max_epochs from 500 to 1000, enabled by default. @Innixma (#4436)

  • Add support for controlling repeated cross-validation behavior via delay_bag_sets fit argument. Set default to False (previously True). @LennartPurucker (#4552)

  • Make positive_class an init argument of TabularPredictor. @Innixma (#4445)

  • Add AdamW support to NN_TORCH model. @Innixma (#4610)

Documentation

  • Added a tutorial with a deep dive on how AutoGluon works. @rey-allan (#4284)

Fixes and Improvements

  • (Major) Fix stacker max_models logic for a 3x inference speedup. @Innixma (#4290)

  • (Major) Speed up EnsembleSelection fitting speed by 2x+. @nathanaelbosch (#4367)

  • (Major) Dramatically improve temperature scaling performance by using the best iteration instead of the last iteration’s temperature. @LennartPurucker (#4396)

  • (Major) Automatically skip temperature scaling if negative temperature is found. @Innixma (#4397)

  • (Major) Fix roc_auc metric to use macro for multiclass instead of weighted. @LennartPurucker (#4407)

  • (Major) Ensure refit_full respects user specified num_cpus and num_gpus. @Innixma (#4495)

  • (Major) Refactor TabularDataset. Now TabularDataset will always return a pandas DataFrame object when initialized, to simplify various documentation and improve IDE debugging visualization compatibility. @Innixma (#4613)

  • Fix bug where validation data is not used when in HPO mode when no search space is provided for the model. @echowve (#4667)

  • Set num_bag_sets=1 by default, to avoid num_bag_sets>1 being used if the user doesn’t use a preset and sets num_bag_folds>=2. @Innixma (#4446)

  • Fix FASTAI crash when a column contains only a single unique value + NaNs. @Innixma (#4584)

  • Fix torch seed accidentally being updated on model.score calls in NN_TORCH. @adibiasio (#4391)

  • Fix LightGBM predict_proba quantile output dtype. @Innixma (#4272)

  • Fix incorrect return type for predict_multi for regression. @Innixma (#4450)

  • Improved error messages when given invalid hyperparameters. @Innixma (#4258)

  • Improved user specified num_cpus and num_gpus sanity checking. @Innixma (#4277)

  • Add readable error message for invalid models in predictor.persist calls. @Innixma (#4285)

  • Improve logging for invalid label columns. @Innixma (#4287)

  • Improve NN_TORCH timeout logging. @Innixma (#4382)

  • Add toggle raise_on_no_models_fitted to control if AutoGluon errors when no models are fit. @LennartPurucker (#4389)

  • Make raise_on_no_models_fitted=True by default. Was False in previous release. @Innixma (#4400)

  • Add utility methods to FeatureMetadata. @Innixma (#4401)

  • Fix feature pruning crashing on Windows. @Innixma (#4405)

  • Add valid_stacker and use_orig_features model options. @Innixma (#4444)

  • Improve reliability of predictor.predict_proba_multi in edge-case scenarios. @Innixma (#4527)

  • Remove ensemble cascade support. @Innixma (#4548)

  • Fix edgecase crash during label column handling if it is a pandas category dtype with 0 instances of a category. @Innixma (#4583)

  • Enable aarch64 platform build. @abhishek-iitmadras (#4663)

  • Minor fixes. @Innixma @LennartPurucker @shchur @rsj123 (#4224, #4317, #4335, #4352, #4353, #4379, #4384, #4474, #4485, #4675, #4682, #4700)

  • Minor unit tests, documentation, and cleanup. @Innixma @abhishek-iitmadras (#4398, #4399, #4402, #4498, #4546, #4547, #4549, #4687, #4690, #4692)

TimeSeries

New Features

  • Add fine-tuning support for Chronos and Chronos-Bolt models @abdulfatir (#4608, #4645, #4653, #4655, #4659, #4661, #4673, #4677)

  • Add Chronos-Bolt @canerturkmen (#4625)

  • TimeSeriesPredictor.leaderboard now can compute extra metrics and return hyperparameters for each model @shchur (#4481)

  • Add target_scaler support for all forecasting models @shchur (#4460, #4644)

  • Add covariate_regressor support for all forecasting models @shchur (#4566, #4641)

  • Add method to convert a TimeSeriesDataFrame to a regular pd.DataFrame @shchur (#4415)

  • [experimental] Add the weighted cumulative error forecasting metric @shchur (#4594)

  • [experimental] Allow custom ensemble model types for time series @shchur (#4662)

Fixes and Improvements

  • Update presets @canerturkmen @shchur (#4656, #4658, #4666, #4672)

  • Unify all Croston models into a single class @shchur (#4564)

  • Bump statsforecast version to 1.7 @canerturkmen @shchur (#4194, #4357)

  • Fix deep learning models failing if item_ids have StringDtype @rsj123 (#4539)

  • Update logic for inferring the time series frequency @shchur (#4540)

  • Speed up and reduce memory usage of the TimeSeriesFeatureGenerator preprocessing logic @shchur (#4557)

  • Update to GluonTS v0.16.0 @shchur (#4628)

  • Refactor GluonTS default parameter handling, update TiDE parameters @canerturkmen (#4640)

  • Move covariate scaling logic into a separate class @shchur (#4634)

  • Prune timeseries unit and smoke tests @canerturkmen (#4650)

  • Minor fixes @abdulfatir @canerturkmen @shchur (#4259, #4299, #4395, #4386, #4409, #4533, #4565, #4633, #4647)

Multimodal

Fixes and Improvements

  • Fix Missing Validation Metric While Resuming A Model Failed At Checkpoint Fusing Stage by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4449

  • Add coco_root for better support for custom dataset in COCO format. by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/3809

  • Add COCO Format Saving Support and Update Object Detection I/O Handling by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/3811

  • Skip MMDet Config Files While Checking with bandit by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4630

  • Fix Logloss Bug and Refine Compute Score Logics by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4629

  • Fix Index Typo in Tutorial by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4642

  • Fix Proba Metrics for Multiclass by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4643

  • Support torch 2.4 by @tonyhoo in https://github.com/autogluon/autogluon/pull/4360

  • Add Installation Guide for Object Detection in Tutorial by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4430

  • Add Bandit Warning Mitigation for Internal torch.save and torch.load Usage by @tonyhoo in https://github.com/autogluon/autogluon/pull/4502

  • update accelerate version range by @cheungdaven in https://github.com/autogluon/autogluon/pull/4596

  • Bound nltk version to avoid verbose logging issue by @tonyhoo in https://github.com/autogluon/autogluon/pull/4604

  • Upgrade TIMM by @prateekdesai04 in https://github.com/autogluon/autogluon/pull/4580

  • Key dependency updates in _setup_utils.py for v1.2 release by @tonyhoo in https://github.com/autogluon/autogluon/pull/4612

  • Configurable Number of Checkpoints to Keep per HPO Trial by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4615

  • Refactor Metrics for Each Problem Type by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4616

  • Fix Torch Version and Colab Installation for Object Detection by @FANGAreNotGnu in https://github.com/autogluon/autogluon/pull/4447

Special Thanks

Contributors

Full Contributor List (ordered by # of commits):

@Innixma @shchur @prateekdesai04 @tonyhoo @FangAreNotGnu @suzhoum @abdulfatir @canerturkmen @LennartPurucker @abhishek-iitmadras @adibiasio @rsj123 @nathanaelbosch @cheungdaven @lostella @zkalson @rey-allan @echowve @xiyuanzh

New Contributors

  • @nathanaelbosch made their first contribution in https://github.com/autogluon/autogluon/pull/4366

  • @adibiasio made their first contribution in https://github.com/autogluon/autogluon/pull/4391

  • @abdulfatir made their first contribution in https://github.com/autogluon/autogluon/pull/4608

  • @echowve made their first contribution in https://github.com/autogluon/autogluon/pull/4667

  • @abhishek-iitmadras made their first contribution in https://github.com/autogluon/autogluon/pull/4685

  • @xiyuanzh made their first contribution in https://github.com/autogluon/autogluon/pull/4694