Image Prediction - Quick Start¶
In this quick start, we’ll use the task of image classification to illustrate how to use AutoGluon’s APIs. This tutorial demonstrates how to load images and corresponding labels into AutoGluon and use this data to obtain a neural network that can classify new images. This is different from traditional machine learning where we need to manually define the neural network and then specify the hyperparameters in the training process. Instead, with just a single call to AutoGluon’s fit function, AutoGluon automatically trains many models with different hyperparameter configurations and returns the model that achieved the highest level of accuracy.
import autogluon.core as ag
from autogluon.vision import ImagePredictor, ImageDataset
/var/lib/jenkins/workspace/workspace/autogluon-tutorial-image-classification-v3/venv/lib/python3.7/site-packages/gluoncv/__init__.py:40: UserWarning: Both mxnet==1.7.0 and torch==1.7.1+cu101 are installed. You might encounter increased GPU memory footprint if both framework are used at the same time.
warnings.warn(f'Both mxnet=={mx.__version__} and torch=={torch.__version__} are installed. '
Create Image Dataset¶
For demonstration purposes, we use a subset of the Shopee-IET
dataset
from Kaggle. Each image in this data depicts a clothing item and the
corresponding label specifies its clothing category. Our subset of the
data contains the following possible labels: BabyPants,
BabyShirt, womencasualshoes, womenchiffontop.
We can load a dataset by downloading a url data automatically:
train_dataset, _, test_dataset = ImageDataset.from_folders('https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip')
print(train_dataset)
data/
├── test/
└── train/
image label
0 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 0
1 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 0
2 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 0
3 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 0
4 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 0
.. ... ...
795 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 3
796 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 3
797 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 3
798 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 3
799 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/... 3
[800 rows x 2 columns]
Use AutoGluon to Fit Models¶
Now, we fit a classifier using AutoGluon as follows:
predictor = ImagePredictor()
# since the original dataset does not provide validation split, the `fit` function splits it randomly with 90/10 ratio
predictor.fit(train_dataset, hyperparameters={'epochs': 2}) # you can trust the default config, we reduce the # epoch to save some build time
time_limit=auto set to time_limit=7200.
Reset labels to [0, 1, 2, 3]
Randomly split train_data into train[720]/validation[80] splits.
The number of requested GPUs is greater than the number of available GPUs.Reduce the number to 1
Starting fit without HPO
modified configs(<old> != <new>): {
root.misc.num_workers 4 != 8
root.misc.seed 42 != 626
root.train.early_stop_patience -1 != 10
root.train.early_stop_max_value 1.0 != inf
root.train.batch_size 32 != 16
root.train.epochs 200 != 2
root.train.early_stop_baseline 0.0 != -inf
root.img_cls.model resnet101 != resnet50
}
Saved config to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-image-classification-v3/docs/_build/eval/tutorials/image_prediction/a3a13a5a/.trial_0/config.yaml
Model resnet50 created, param count: 23516228
AMP not enabled. Training in float32.
Disable EMA as it is not supported for now.
Start training from [Epoch 0]
[Epoch 0] training: accuracy=0.234722
[Epoch 0] speed: 90 samples/sec time cost: 7.788882
[Epoch 0] validation: top1=0.437500 top5=1.000000
[Epoch 0] Current best top-1: 0.437500 vs previous -inf, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-image-classification-v3/docs/_build/eval/tutorials/image_prediction/a3a13a5a/.trial_0/best_checkpoint.pkl
[Epoch 1] training: accuracy=0.444444
[Epoch 1] speed: 94 samples/sec time cost: 7.458240
[Epoch 1] validation: top1=0.675000 top5=1.000000
[Epoch 1] Current best top-1: 0.675000 vs previous 0.437500, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-image-classification-v3/docs/_build/eval/tutorials/image_prediction/a3a13a5a/.trial_0/best_checkpoint.pkl
Applying the state from the best checkpoint...
Finished, total runtime is 23.58 s
{ 'best_config': { 'batch_size': 16,
'dist_ip_addrs': None,
'early_stop_baseline': -inf,
'early_stop_max_value': inf,
'early_stop_patience': 10,
'epochs': 2,
'final_fit': False,
'gpus': [0],
'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-image-classification-v3/docs/_build/eval/tutorials/image_prediction/a3a13a5a',
'lr': 0.01,
'model': 'resnet50',
'ngpus_per_trial': 8,
'nthreads_per_trial': 128,
'num_trials': 1,
'num_workers': 8,
'problem_type': 'multiclass',
'scheduler': 'local',
'search_strategy': 'random',
'searcher': 'random',
'seed': 626,
'time_limits': 7200,
'wall_clock_tick': 1630109030.0705285},
'total_time': 16.659470081329346,
'train_acc': 0.4444444444444444,
'valid_acc': 0.675}
<autogluon.vision.predictor.predictor.ImagePredictor at 0x7f3a7cc37d50>
Within fit, the dataset is automatically split into training and
validation sets. The model with the best hyperparameter configuration is
selected based on its performance on the validation set. The best model
is finally retrained on our entire dataset (i.e., merging
training+validation) using the best configuration.
The best Top-1 accuracy achieved on the validation set is as follows:
fit_result = predictor.fit_summary()
print('Top-1 train acc: %.3f, val acc: %.3f' %(fit_result['train_acc'], fit_result['valid_acc']))
Top-1 train acc: 0.444, val acc: 0.675
Predict on a New Image¶
Given an example image, we can easily use the final model to predict
the label (and the conditional class-probability denoted as score):
image_path = test_dataset.iloc[0]['image']
result = predictor.predict(image_path)
print(result)
0 1
Name: label, dtype: int64
If probabilities of all categories are needed, you can call
predict_proba:
proba = predictor.predict_proba(image_path)
print(proba)
0 1 2 3
0 0.250368 0.32417 0.191457 0.234005
You can also feed in multiple images all together, let’s use images in test dataset as an example:
bulk_result = predictor.predict(test_dataset)
print(bulk_result)
0 1
1 1
2 1
3 2
4 1
..
75 1
76 3
77 1
78 3
79 3
Name: label, Length: 80, dtype: int64
An extra column will be included in bulk prediction, indicate the
corresponding image for the row. There will be (# image) rows in the
result, each row includes class, score, id and image for
prediction class, prediction confidence, class id, and image path
respectively.
Generate image features with a classifier¶
Extracting representation from the whole image learned by a model is
also very useful. We provide predict_feature function to allow
predictor to return the N-dimensional image feature where N depends
on the model(usually a 512 to 2048 length vector)
image_path = test_dataset.iloc[0]['image']
feature = predictor.predict_feature(image_path)
print(feature)
image_feature 0 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...
image
0 /var/lib/jenkins/.gluoncv/datasets/shopee-iet/...
Evaluate on Test Dataset¶
You can evaluate the classifier on a test dataset rather than retrieving the predictions.
The validation and test top-1 accuracy are:
test_acc = predictor.evaluate(test_dataset)
print('Top-1 test acc: %.3f' % test_acc['top1'])
[Epoch 1] validation: top1=0.600000 top5=1.000000
Top-1 test acc: 0.600
Save and load classifiers¶
You can directly save the instances of classifiers:
filename = 'predictor.ag'
predictor.save(filename)
predictor_loaded = ImagePredictor.load(filename)
# use predictor_loaded as usual
result = predictor_loaded.predict(image_path)
print(result)
0 1
Name: label, dtype: int64