.. _sec_imgquick: Image Prediction - Quick Start ============================== In this quick start, we’ll use the task of image classification to illustrate how to use AutoGluon’s APIs. This tutorial demonstrates how to load images and corresponding labels into AutoGluon and use this data to obtain a neural network that can classify new images. This is different from traditional machine learning where we need to manually define the neural network and then specify the hyperparameters in the training process. Instead, with just a single call to AutoGluon’s `fit <../../api/autogluon.predictor.html#autogluon.vision.ImagePredictor.fit>`__ function, AutoGluon automatically trains many models with different hyperparameter configurations and returns the model that achieved the highest level of accuracy. .. code:: python import autogluon.core as ag from autogluon.vision import ImagePredictor, ImageDataset .. parsed-literal:: :class: output /home/ci/opt/venv/lib/python3.8/site-packages/gluoncv/__init__.py:40: UserWarning: Both `mxnet==1.9.1` and `torch==1.12.1+cu102` are installed. You might encounter increased GPU memory footprint if both framework are used at the same time. warnings.warn(f'Both `mxnet=={mx.__version__}` and `torch=={torch.__version__}` are installed. ' INFO:matplotlib.font_manager:generated new fontManager INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmplx1thlno INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmplx1thlno/_remote_module_non_scriptable.py INFO:root:Generating grammar tables from /usr/lib/python3.8/lib2to3/Grammar.txt INFO:root:Generating grammar tables from /usr/lib/python3.8/lib2to3/PatternGrammar.txt Create Image Dataset -------------------- For demonstration purposes, we use a subset of the `Shopee-IET dataset `__ from Kaggle. Each image in this data depicts a clothing item and the corresponding label specifies its clothing category. Our subset of the data contains the following possible labels: ``BabyPants``, ``BabyShirt``, ``womencasualshoes``, ``womenchiffontop``. We can load a dataset by downloading a url data automatically: .. code:: python train_dataset, _, test_dataset = ImageDataset.from_folders('https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip') print(train_dataset) .. parsed-literal:: :class: output Downloading /home/ci/.gluoncv/archive/shopee-iet.zip from https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip... .. parsed-literal:: :class: output 100%|██████████| 40895/40895 [00:01<00:00, 21837.15KB/s] .. parsed-literal:: :class: output data/ ├── test/ └── train/ image label 0 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 0 1 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 0 2 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 0 3 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 0 4 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 0 .. ... ... 795 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 3 796 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 3 797 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 3 798 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 3 799 /home/ci/.gluoncv/datasets/shopee-iet/data/tra... 3 [800 rows x 2 columns] Use AutoGluon to Fit Models --------------------------- Now, we fit a classifier using AutoGluon as follows: .. code:: python predictor = ImagePredictor() # since the original dataset does not provide validation split, the `fit` function splits it randomly with 90/10 ratio predictor.fit(train_dataset, hyperparameters={'epochs': 2}) # you can trust the default config, we reduce the # epoch to save some build time .. parsed-literal:: :class: output AutoGluon ImagePredictor will be deprecated in v0.7. Please use AutoGluon MultiModalPredictor instead for more functionalities and better support. Visit https://auto.gluon.ai/stable/tutorials/multimodal/index.html for more details! ImagePredictor sets accuracy as default eval_metric for classification problems. `time_limit=auto` set to `time_limit=7200`. Reset labels to [0, 1, 2, 3] Randomly split train_data into train[720]/validation[80] splits. The number of requested GPUs is greater than the number of available GPUs.Reduce the number to 1 Starting fit without HPO INFO:TorchImageClassificationEstimator:modified configs( != ): { INFO:TorchImageClassificationEstimator:root.img_cls.model resnet101 != resnet50 INFO:TorchImageClassificationEstimator:root.train.early_stop_patience -1 != 10 INFO:TorchImageClassificationEstimator:root.train.batch_size 32 != 16 INFO:TorchImageClassificationEstimator:root.train.epochs 200 != 2 INFO:TorchImageClassificationEstimator:root.train.early_stop_max_value 1.0 != inf INFO:TorchImageClassificationEstimator:root.train.early_stop_baseline 0.0 != -inf INFO:TorchImageClassificationEstimator:root.misc.num_workers 4 != 8 INFO:TorchImageClassificationEstimator:root.misc.seed 42 != 623 INFO:TorchImageClassificationEstimator:} INFO:TorchImageClassificationEstimator:Saved config to /home/ci/autogluon/docs/_build/eval/tutorials/image_prediction/25a8cf38/.trial_0/config.yaml Downloading: "https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/resnet50_a1_0-14fe96d1.pth" to /home/ci/.cache/torch/hub/checkpoints/resnet50_a1_0-14fe96d1.pth INFO:TorchImageClassificationEstimator:Model resnet50 created, param count: 23516228 INFO:TorchImageClassificationEstimator:AMP not enabled. Training in float32. INFO:TorchImageClassificationEstimator:Disable EMA as it is not supported for now. INFO:TorchImageClassificationEstimator:Start training from [Epoch 0] INFO:TorchImageClassificationEstimator:[Epoch 0] training: accuracy=0.247222 INFO:TorchImageClassificationEstimator:[Epoch 0] speed: 88 samples/sec time cost: 7.979265 INFO:TorchImageClassificationEstimator:[Epoch 0] validation: top1=0.312500 top5=1.000000 INFO:TorchImageClassificationEstimator:[Epoch 0] Current best top-1: 0.312500 vs previous -inf, saved to /home/ci/autogluon/docs/_build/eval/tutorials/image_prediction/25a8cf38/.trial_0/best_checkpoint.pkl INFO:TorchImageClassificationEstimator:[Epoch 1] training: accuracy=0.475000 INFO:TorchImageClassificationEstimator:[Epoch 1] speed: 96 samples/sec time cost: 7.297479 INFO:TorchImageClassificationEstimator:[Epoch 1] validation: top1=0.625000 top5=1.000000 INFO:TorchImageClassificationEstimator:[Epoch 1] Current best top-1: 0.625000 vs previous 0.312500, saved to /home/ci/autogluon/docs/_build/eval/tutorials/image_prediction/25a8cf38/.trial_0/best_checkpoint.pkl INFO:TorchImageClassificationEstimator:Applying the state from the best checkpoint... Finished, total runtime is 22.22 s { 'best_config': { 'batch_size': 16, 'dist_ip_addrs': None, 'early_stop_baseline': -inf, 'early_stop_max_value': inf, 'early_stop_patience': 10, 'epochs': 2, 'final_fit': False, 'gpus': [0], 'lr': 0.01, 'model': 'resnet50', 'ngpus_per_trial': 8, 'nthreads_per_trial': 128, 'num_workers': 8, 'searcher': 'random', 'seed': 623, 'time_limits': 7200}, 'total_time': 16.20781970024109, 'train_acc': 0.475, 'valid_acc': 0.625} .. parsed-literal:: :class: output Within ``fit``, the dataset is automatically split into training and validation sets. The model with the best hyperparameter configuration is selected based on its performance on the validation set. The best model is finally retrained on our entire dataset (i.e., merging training+validation) using the best configuration. The best Top-1 accuracy achieved on the validation set is as follows: .. code:: python fit_result = predictor.fit_summary() print('Top-1 train acc: %.3f, val acc: %.3f' %(fit_result['train_acc'], fit_result['valid_acc'])) .. parsed-literal:: :class: output Top-1 train acc: 0.475, val acc: 0.625 Predict on a New Image ---------------------- Given an example image, we can easily use the final model to ``predict`` the label (and the conditional class-probability denoted as ``score``): .. code:: python image_path = test_dataset.iloc[0]['image'] result = predictor.predict(image_path) print(result) .. parsed-literal:: :class: output 0 2 Name: label, dtype: int64 If probabilities of all categories are needed, you can call ``predict_proba``: .. code:: python proba = predictor.predict_proba(image_path) print(proba) .. parsed-literal:: :class: output 0 1 2 3 0 0.219222 0.286474 0.318675 0.175629 You can also feed in multiple images all together, let’s use images in test dataset as an example: .. code:: python bulk_result = predictor.predict(test_dataset) print(bulk_result) .. parsed-literal:: :class: output 0 2 1 1 2 2 3 2 4 1 .. 75 3 76 3 77 3 78 3 79 2 Name: label, Length: 80, dtype: int64 An extra column will be included in bulk prediction, to indicate the corresponding image for the row. There will be (# image) rows in the result, each row includes ``class``, ``score``, ``id`` and ``image`` for prediction class, prediction confidence, class id, and image path respectively. Generate image features with a classifier ----------------------------------------- Extracting representation from the whole image learned by a model is also very useful. We provide ``predict_feature`` function to allow predictor to return the N-dimensional image feature where ``N`` depends on the model(usually a 512 to 2048 length vector) .. code:: python image_path = test_dataset.iloc[0]['image'] feature = predictor.predict_feature(image_path) print(feature) .. parsed-literal:: :class: output image_feature \ 0 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... image 0 /home/ci/.gluoncv/datasets/shopee-iet/data/tes... Evaluate on Test Dataset ------------------------ You can evaluate the classifier on a test dataset rather than retrieving the predictions. The validation and test top-1 accuracy are: .. code:: python test_acc = predictor.evaluate(test_dataset) print('Top-1 test acc: %.3f' % test_acc['top1']) .. parsed-literal:: :class: output INFO:TorchImageClassificationEstimator:[Epoch 1] validation: top1=0.637500 top5=1.000000 .. parsed-literal:: :class: output Top-1 test acc: 0.637 Save and load classifiers ------------------------- You can directly save the instances of classifiers: .. warning:: ``ImagePredictor.load()`` used ``pickle`` module implicitly, which is known to be insecure. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never load data that could have come from an untrusted source, or that could have been tampered with. **Only load data you trust.** .. code:: python filename = 'predictor.ag' predictor.save(filename) predictor_loaded = ImagePredictor.load(filename) # use predictor_loaded as usual result = predictor_loaded.predict(image_path) print(result) .. parsed-literal:: :class: output 0 2 Name: label, dtype: int64