AutoMM Detection - Quick Start on a Tiny COCO Format Dataset¶
In this section, our goal is to fast finetune a pretrained model on a small dataset in COCO format, and evaluate on its test set. Both training and test sets are in COCO format. See Convert Data to COCO Format for how to convert other datasets to COCO format.
To start, let’s import MultiModalPredictor:
from autogluon.multimodal import MultiModalPredictor
Make sure mmcv-full and mmdet are installed:
!mim install mmcv-full
!pip install mmdet
Looking in links: https://download.openmmlab.com/mmcv/dist/cu102/torch1.12.0/index.html
Requirement already satisfied: mmcv-full in /home/ci/opt/venv/lib/python3.8/site-packages (1.7.0)
Requirement already satisfied: opencv-python>=3 in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (4.6.0.66)
Requirement already satisfied: yapf in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (0.32.0)
Requirement already satisfied: pyyaml in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (5.4.1)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (1.22.4)
Requirement already satisfied: addict in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (2.4.0)
Requirement already satisfied: Pillow in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (9.0.1)
Requirement already satisfied: packaging in /home/ci/opt/venv/lib/python3.8/site-packages (from mmcv-full) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/ci/opt/venv/lib/python3.8/site-packages (from packaging->mmcv-full) (3.0.9)
Requirement already satisfied: mmdet in /home/ci/opt/venv/lib/python3.8/site-packages (2.25.3)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.8/site-packages (from mmdet) (3.6.2)
Requirement already satisfied: pycocotools in /home/ci/opt/venv/lib/python3.8/site-packages (from mmdet) (2.0.6)
Requirement already satisfied: six in /home/ci/opt/venv/lib/python3.8/site-packages (from mmdet) (1.16.0)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.8/site-packages (from mmdet) (1.22.4)
Requirement already satisfied: terminaltables in /home/ci/opt/venv/lib/python3.8/site-packages (from mmdet) (3.1.10)
Requirement already satisfied: packaging>=20.0 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (21.3)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (1.0.6)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (2.8.2)
Requirement already satisfied: pyparsing>=2.2.1 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (3.0.9)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (4.38.0)
Requirement already satisfied: pillow>=6.2.0 in /home/ci/opt/venv/lib/python3.8/site-packages (from matplotlib->mmdet) (9.0.1)
And also import some other packages that will be used in this tutorial:
import os
import time
from autogluon.core.utils.loaders import load_zip
We have the sample dataset ready in the cloud. Let’s download it:
zip_file = "https://automl-mm-bench.s3.amazonaws.com/object_detection_dataset/tiny_motorbike_coco.zip"
download_dir = "./tiny_motorbike_coco"
load_zip.unzip(zip_file, unzip_dir=download_dir)
data_dir = os.path.join(download_dir, "tiny_motorbike")
train_path = os.path.join(data_dir, "Annotations", "trainval_cocoformat.json")
test_path = os.path.join(data_dir, "Annotations", "test_cocoformat.json")
Downloading ./tiny_motorbike_coco/file.zip from https://automl-mm-bench.s3.amazonaws.com/object_detection_dataset/tiny_motorbike_coco.zip...
100%|██████████| 21.8M/21.8M [00:00<00:00, 53.9MiB/s]
While using COCO format dataset, the input is the json annotation file
of the dataset split. In this example, trainval_cocoformat.json is
the annotation file of the train-and-validate split, and
test_cocoformat.json is the annotation file of the test split.
We select the YOLOv3 with MobileNetV2 as backbone, and input resolution is 320x320, pretrained on COCO dataset. With this setting, it is fast to finetune or inference, and easy to deploy. And we use all the GPUs (if any):
checkpoint_name = "yolov3_mobilenetv2_320_300e_coco"
num_gpus = -1 # use all GPUs
We create the MultiModalPredictor with selected checkpoint name and
number of GPUs. We need to specify the problem_type to
"object_detection", and also provide a sample_data_path for the
predictor to infer the catgories of the dataset. Here we provide the
train_path, and it also works using any other split of this dataset.
And we also provide a path to save the predictor. It will be saved
to a automatically generated directory with timestamp under
AutogluonModels if path is not specified.
# Init predictor
import uuid
model_path = f"./tmp/{uuid.uuid4().hex}-quick_start_tutorial_temp_save"
predictor = MultiModalPredictor(
hyperparameters={
"model.mmdet_image.checkpoint_name": checkpoint_name,
"env.num_gpus": num_gpus,
},
problem_type="object_detection",
sample_data_path=train_path,
path=model_path,
)
/home/ci/autogluon/multimodal/src/autogluon/multimodal/predictor.py:433: UserWarning: Running object detection. Make sure that you have installed mmdet and mmcv-full, by running 'mim install mmcv-full' and 'pip install mmdet'
warnings.warn(
processing yolov3_mobilenetv2_320_300e_coco...
Output()
[32mSuccessfully downloaded yolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth to /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
[32mSuccessfully dumped yolov3_mobilenetv2_320_300e_coco.py to /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
load checkpoint from local path: yolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth
The model and loaded state dict do not match exactly
size mismatch for bbox_head.convs_pred.0.weight: copying a param with shape torch.Size([255, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([45, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.0.bias: copying a param with shape torch.Size([255]) from checkpoint, the shape in current model is torch.Size([45]).
size mismatch for bbox_head.convs_pred.1.weight: copying a param with shape torch.Size([255, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([45, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.1.bias: copying a param with shape torch.Size([255]) from checkpoint, the shape in current model is torch.Size([45]).
size mismatch for bbox_head.convs_pred.2.weight: copying a param with shape torch.Size([255, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([45, 96, 1, 1]).
size mismatch for bbox_head.convs_pred.2.bias: copying a param with shape torch.Size([255]) from checkpoint, the shape in current model is torch.Size([45]).
We set the learning rate to be 2e-4. Note that we use a two-stage
learning rate option during finetuning by default, and the model head
will have 100x learning rate. Using a two-stage learning rate with high
learning rate only on head layers makes the model converge faster during
finetuning. It usually gives better performance as well, especially on
small datasets with hundreds or thousands of images. We also set the
epoch to be 15 and batch_size to be 32. We also compute the time of the
fit process here for better understanding the speed. We run it on a
g4.2xlarge EC2 machine on AWS, and part of the command outputs are shown
below:
start = time.time()
# Fit
predictor.fit(
train_path,
hyperparameters={
"optimization.learning_rate": 2e-4, # we use two stage and detection head has 100x lr
"optimization.max_epochs": 30,
"env.per_gpu_batch_size": 32, # decrease it when model is large
},
)
train_end = time.time()
Global seed set to 123
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
GPU available: True (cuda), used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs HPU available: False, using: 0 HPUs LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] | Name | Type | Params ----------------------------------------------------------------------- 0 | model | MMDetAutoModelForObjectDetection | 3.7 M 1 | validation_metric | MeanMetric | 0 ----------------------------------------------------------------------- 3.7 M Trainable params 0 Non-trainable params 3.7 M Total params 14.706 Total estimated model params size (MB) /home/ci/opt/venv/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (5) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch. rank_zero_warn( Epoch 0, global step 1: 'val_direct_loss' reached 34813.25391 (best 34813.25391), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=0-step=1.ckpt' as top 1 Epoch 1, global step 2: 'val_direct_loss' reached 13185.68750 (best 13185.68750), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=1-step=2.ckpt' as top 1 Epoch 1, global step 3: 'val_direct_loss' reached 5299.51514 (best 5299.51514), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=1-step=3.ckpt' as top 1 Epoch 2, global step 4: 'val_direct_loss' reached 2527.24023 (best 2527.24023), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=2-step=4.ckpt' as top 1 Epoch 2, global step 5: 'val_direct_loss' reached 1455.05627 (best 1455.05627), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=2-step=5.ckpt' as top 1 Epoch 3, global step 6: 'val_direct_loss' reached 1257.15979 (best 1257.15979), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=3-step=6.ckpt' as top 1 Epoch 3, global step 7: 'val_direct_loss' reached 974.69373 (best 974.69373), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=3-step=7.ckpt' as top 1 Epoch 4, global step 8: 'val_direct_loss' was not in top 1 Epoch 4, global step 9: 'val_direct_loss' reached 945.76221 (best 945.76221), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=4-step=9.ckpt' as top 1 Epoch 5, global step 10: 'val_direct_loss' was not in top 1 Epoch 5, global step 11: 'val_direct_loss' was not in top 1 Epoch 6, global step 12: 'val_direct_loss' reached 923.07587 (best 923.07587), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=6-step=12.ckpt' as top 1 Epoch 6, global step 13: 'val_direct_loss' was not in top 1 Epoch 7, global step 14: 'val_direct_loss' reached 917.54089 (best 917.54089), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=7-step=14.ckpt' as top 1 Epoch 7, global step 15: 'val_direct_loss' was not in top 1 Epoch 8, global step 16: 'val_direct_loss' was not in top 1 Epoch 8, global step 17: 'val_direct_loss' reached 861.14917 (best 861.14917), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=8-step=17.ckpt' as top 1 Epoch 9, global step 18: 'val_direct_loss' reached 810.36340 (best 810.36340), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=9-step=18.ckpt' as top 1 Epoch 9, global step 19: 'val_direct_loss' was not in top 1 Epoch 10, global step 20: 'val_direct_loss' was not in top 1 Epoch 10, global step 21: 'val_direct_loss' was not in top 1 Epoch 11, global step 22: 'val_direct_loss' reached 792.89148 (best 792.89148), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=11-step=22.ckpt' as top 1 Epoch 11, global step 23: 'val_direct_loss' reached 748.54730 (best 748.54730), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=11-step=23.ckpt' as top 1 Epoch 12, global step 24: 'val_direct_loss' was not in top 1 Epoch 12, global step 25: 'val_direct_loss' reached 718.71021 (best 718.71021), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=12-step=25.ckpt' as top 1 Epoch 13, global step 26: 'val_direct_loss' was not in top 1 Epoch 13, global step 27: 'val_direct_loss' reached 624.69897 (best 624.69897), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=13-step=27.ckpt' as top 1 Epoch 14, global step 28: 'val_direct_loss' was not in top 1 Epoch 14, global step 29: 'val_direct_loss' was not in top 1 Epoch 15, global step 30: 'val_direct_loss' was not in top 1 Epoch 15, global step 31: 'val_direct_loss' was not in top 1 Epoch 16, global step 32: 'val_direct_loss' was not in top 1 Epoch 16, global step 33: 'val_direct_loss' was not in top 1 Epoch 17, global step 34: 'val_direct_loss' was not in top 1 Epoch 17, global step 35: 'val_direct_loss' was not in top 1 Epoch 18, global step 36: 'val_direct_loss' was not in top 1 Epoch 18, global step 37: 'val_direct_loss' was not in top 1 Epoch 19, global step 38: 'val_direct_loss' was not in top 1 Epoch 19, global step 39: 'val_direct_loss' was not in top 1 Epoch 20, global step 40: 'val_direct_loss' was not in top 1 Epoch 20, global step 41: 'val_direct_loss' was not in top 1 Epoch 21, global step 42: 'val_direct_loss' was not in top 1 Epoch 21, global step 43: 'val_direct_loss' was not in top 1 Epoch 22, global step 44: 'val_direct_loss' was not in top 1 Epoch 22, global step 45: 'val_direct_loss' was not in top 1 Epoch 23, global step 46: 'val_direct_loss' was not in top 1 Epoch 23, global step 47: 'val_direct_loss' reached 600.80170 (best 600.80170), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=23-step=47.ckpt' as top 1 Epoch 24, global step 48: 'val_direct_loss' reached 576.06274 (best 576.06274), saving model to '/home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/epoch=24-step=48.ckpt' as top 1 Epoch 24, global step 49: 'val_direct_loss' was not in top 1 Epoch 25, global step 50: 'val_direct_loss' was not in top 1 Epoch 25, global step 51: 'val_direct_loss' was not in top 1 Epoch 26, global step 52: 'val_direct_loss' was not in top 1 Epoch 26, global step 53: 'val_direct_loss' was not in top 1 Epoch 27, global step 54: 'val_direct_loss' was not in top 1 Epoch 27, global step 55: 'val_direct_loss' was not in top 1 Epoch 28, global step 56: 'val_direct_loss' was not in top 1 Epoch 28, global step 57: 'val_direct_loss' was not in top 1 Epoch 29, global step 58: 'val_direct_loss' was not in top 1 Epoch 29, global step 59: 'val_direct_loss' was not in top 1 Trainer.fit stopped: max_epochs=30 reached.
Notice that at the end of each progress bar, if the checkpoint at
current stage is saved, it prints the model’s save path. In this
example, it’s ./quick_start_tutorial_temp_save.
Print out the time and we can see that it’s fast!
print("This finetuning takes %.2f seconds." % (train_end - start))
This finetuning takes 149.00 seconds.
To evaluate the model we just trained, run following code.
And the evaluation results are shown in command line output. The first line is mAP in COCO standard, and the second line is mAP in VOC standard (or mAP50). For more details about these metrics, see COCO’s evaluation guideline. Note that for presenting a fast finetuning we use 15 epochs, you could get better result on this dataset by simply increasing the epochs.
predictor.evaluate(test_path)
eval_end = time.time()
loading annotations into memory... Done (t=0.00s) creating index... index created! saving file at /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/tmp/e484e890c9b24af9b2b5b8554dde2b06-quick_start_tutorial_temp_save/object_detection_result_cache.json loading annotations into memory... Done (t=0.00s) creating index... index created! Loading and preparing results... DONE (t=0.01s) creating index... index created! Running per image evaluation... Evaluate annotation type bbox DONE (t=0.19s). Accumulating evaluation results... DONE (t=0.06s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.112 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.291 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.038 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.036 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.314 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.096 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.181 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.196 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.086 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.241 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.431
Print out the evaluation time:
print("The evaluation takes %.2f seconds." % (eval_end - train_end))
The evaluation takes 1.62 seconds.
We can load a new predictor with previous save_path, and we can also reset the number of GPUs to use if not all the devices are available:
# Load and reset num_gpus
new_predictor = MultiModalPredictor.load(model_path)
new_predictor.set_num_gpus(1)
processing yolov3_mobilenetv2_320_300e_coco...
[32myolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth exists in /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
[32mSuccessfully dumped yolov3_mobilenetv2_320_300e_coco.py to /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
Evaluating the new predictor gives us exactly the same result:
# Evaluate new predictor
new_predictor.evaluate(test_path)
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
WARNING:automm:A new predictor save path is created.This is to prevent you to overwrite previous predictor saved here.You could check current save path at predictor._save_path.If you still want to use this path, set resume=True
saving file at /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/AutogluonModels/ag-20221117_040114/object_detection_result_cache.json loading annotations into memory... Done (t=0.00s) creating index... index created! Loading and preparing results... DONE (t=0.01s) creating index... index created! Running per image evaluation... Evaluate annotation type bbox DONE (t=0.19s). Accumulating evaluation results... DONE (t=0.06s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.112 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.291 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.038 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.036 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.314 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.096 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.181 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.196 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.086 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.241 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.431
{'map': 0.11220779598447833}
If we set validation metric to "map" (Mean Average Precision), and
max epochs to 50, the predictor will have better performance with
the same pretrained model (YOLOv3). We trained it offline and uploaded
to S3. To load and check the result:
# Load Trained Predictor from S3
zip_file = "https://automl-mm-bench.s3.amazonaws.com/object_detection/quick_start/AP50_433.zip"
download_dir = "./AP50_433"
load_zip.unzip(zip_file, unzip_dir=download_dir)
better_predictor = MultiModalPredictor.load("./AP50_433/quick_start_tutorial_temp_save")
better_predictor.set_num_gpus(1)
# Evaluate new predictor
better_predictor.evaluate(test_path)
Downloading ./AP50_433/file.zip from https://automl-mm-bench.s3.amazonaws.com/object_detection/quick_start/AP50_433.zip...
100%|██████████| 27.8M/27.8M [00:00<00:00, 64.1MiB/s]
/home/ci/opt/venv/lib/python3.8/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator LabelEncoder from version 1.0.2 when using version 1.1.3. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
warnings.warn(
/home/ci/opt/venv/lib/python3.8/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator StandardScaler from version 1.0.2 when using version 1.1.3. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
warnings.warn(
processing yolov3_mobilenetv2_320_300e_coco...
[32myolov3_mobilenetv2_320_300e_coco_20210719_215349-d18dff72.pth exists in /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
[32mSuccessfully dumped yolov3_mobilenetv2_320_300e_coco.py to /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start[0m
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
WARNING:automm:A new predictor save path is created.This is to prevent you to overwrite previous predictor saved here.You could check current save path at predictor._save_path.If you still want to use this path, set resume=True
saving file at /home/ci/autogluon/docs/_build/eval/tutorials/multimodal/object_detection/quick_start/AutogluonModels/ag-20221117_040120/object_detection_result_cache.json loading annotations into memory... Done (t=0.00s) creating index... index created! Loading and preparing results... DONE (t=0.01s) creating index... index created! Running per image evaluation... Evaluate annotation type bbox DONE (t=0.17s). Accumulating evaluation results... DONE (t=0.06s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.195 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.433 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.135 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.036 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.206 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.450 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.158 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.231 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.244 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.138 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.295 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.508
{'map': 0.19495386487978572}
For how to set those hyperparameters and finetune the model with higher performance, see AutoMM Detection - High Performance Finetune on COCO Format Dataset.
Other Examples¶
You may go to AutoMM Examples to explore other examples about AutoMM.
Customization¶
To learn how to customize AutoMM, please refer to Customize AutoMM.
Citation¶
@misc{redmon2018yolov3,
title={YOLOv3: An Incremental Improvement},
author={Joseph Redmon and Ali Farhadi},
year={2018},
eprint={1804.02767},
archivePrefix={arXiv},
primaryClass={cs.CV}
}