AutoMM Detection - Quick Start on a Tiny COCO Format Dataset¶
In this section, our goal is to fast finetune a pretrained model on a small dataset in COCO format, and evaluate on its test set. Both training and test sets are in COCO format. See Convert Data to COCO Format for how to convert other datasets to COCO format.
Setting up the imports¶
To start, make sure mmcv and mmdet are installed.
Note: MMDet is no longer actively maintained and is only compatible with MMCV version 2.1.0. Installation can be problematic due to CUDA version compatibility issues. For best results:
Use CUDA 12.4 with PyTorch 2.5
Before installation, run:
pip install -U pip setuptools wheel sudo apt-get install -y ninja-build gcc g++
This will help prevent MMCV installation from hanging during wheel building.
After installation in Jupyter notebook, restart the kernel for changes to take effect.
# Update package tools and install build dependencies
!pip install -U pip setuptools wheel
!sudo apt-get install -y ninja-build gcc g++
# Install MMCV
!python3 -m mim install "mmcv==2.1.0"
# For Google Colab users: If the above fails, use this alternative MMCV installation
# pip install "mmcv==2.1.0" -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1.0/index.html
# Install MMDet
!python3 -m pip install "mmdet==3.2.0"
# Install MMEngine (version >=0.10.6 for PyTorch 2.5 compatibility)
!python3 -m pip install "mmengine>=0.10.6"
Show code cell output
Requirement already satisfied: pip in /home/ci/opt/venv/lib/python3.11/site-packages (25.1)
Requirement already satisfied: setuptools in /home/ci/opt/venv/lib/python3.11/site-packages (80.1.0)
Requirement already satisfied: wheel in /home/ci/opt/venv/lib/python3.11/site-packages (0.45.1)
/usr/bin/sh: 1: sudo: not found
Looking in links: https://download.openmmlab.com/mmcv/dist/cu124/torch2.6.0/index.html
Requirement already satisfied: mmcv==2.1.0 in /home/ci/opt/venv/lib/python3.11/site-packages (2.1.0)
Requirement already satisfied: addict in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (2.4.0)
Requirement already satisfied: mmengine>=0.3.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (0.10.7)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (2.1.3)
Requirement already satisfied: packaging in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (24.2)
Requirement already satisfied: Pillow in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (11.2.1)
Requirement already satisfied: pyyaml in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (6.0.2)
Requirement already satisfied: yapf in /home/ci/opt/venv/lib/python3.11/site-packages (from mmcv==2.1.0) (0.43.0)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (3.10.1)
Requirement already satisfied: rich in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (14.0.0)
Requirement already satisfied: termcolor in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (3.1.0)
Requirement already satisfied: opencv-python>=3 in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.3.0->mmcv==2.1.0) (4.11.0.86)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (4.57.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.4.8)
Requirement already satisfied: pyparsing>=2.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (3.2.3)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in /home/ci/opt/venv/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->mmengine>=0.3.0->mmcv==2.1.0) (1.17.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.3.0->mmcv==2.1.0) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.3.0->mmcv==2.1.0) (2.19.1)
Requirement already satisfied: mdurl~=0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich->mmengine>=0.3.0->mmcv==2.1.0) (0.1.2)
Requirement already satisfied: platformdirs>=3.5.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from yapf->mmcv==2.1.0) (4.3.7)
Requirement already satisfied: mmdet==3.2.0 in /home/ci/opt/venv/lib/python3.11/site-packages (3.2.0)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (3.10.1)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (2.1.3)
Requirement already satisfied: pycocotools in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (2.0.8)
Requirement already satisfied: scipy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (1.15.2)
Requirement already satisfied: shapely in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (2.1.0)
Requirement already satisfied: six in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (1.17.0)
Requirement already satisfied: terminaltables in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (3.1.10)
Requirement already satisfied: tqdm in /home/ci/opt/venv/lib/python3.11/site-packages (from mmdet==3.2.0) (4.67.1)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (4.57.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (1.4.8)
Requirement already satisfied: packaging>=20.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (24.2)
Requirement already satisfied: pillow>=8 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (11.2.1)
Requirement already satisfied: pyparsing>=2.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (3.2.3)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmdet==3.2.0) (2.9.0.post0)
Requirement already satisfied: mmengine>=0.10.6 in /home/ci/opt/venv/lib/python3.11/site-packages (0.10.7)
Requirement already satisfied: addict in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (2.4.0)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (3.10.1)
Requirement already satisfied: numpy in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (2.1.3)
Requirement already satisfied: pyyaml in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (6.0.2)
Requirement already satisfied: rich in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (14.0.0)
Requirement already satisfied: termcolor in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (3.1.0)
Requirement already satisfied: yapf in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (0.43.0)
Requirement already satisfied: opencv-python>=3 in /home/ci/opt/venv/lib/python3.11/site-packages (from mmengine>=0.10.6) (4.11.0.86)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (4.57.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (1.4.8)
Requirement already satisfied: packaging>=20.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (24.2)
Requirement already satisfied: pillow>=8 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (11.2.1)
Requirement already satisfied: pyparsing>=2.3.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (3.2.3)
Requirement already satisfied: python-dateutil>=2.7 in /home/ci/opt/venv/lib/python3.11/site-packages (from matplotlib->mmengine>=0.10.6) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in /home/ci/opt/venv/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->mmengine>=0.10.6) (1.17.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.10.6) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/ci/opt/venv/lib/python3.11/site-packages (from rich->mmengine>=0.10.6) (2.19.1)
Requirement already satisfied: mdurl~=0.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich->mmengine>=0.10.6) (0.1.2)
Requirement already satisfied: platformdirs>=3.5.1 in /home/ci/opt/venv/lib/python3.11/site-packages (from yapf->mmengine>=0.10.6) (4.3.7)
To start, let’s import MultiModalPredictor:
from autogluon.multimodal import MultiModalPredictor
And also import some other packages that will be used in this tutorial:
import os
import time
from autogluon.core.utils.loaders import load_zip
Downloading Data¶
We have the sample dataset ready in the cloud. Let’s download it:
zip_file = "https://automl-mm-bench.s3.amazonaws.com/object_detection_dataset/tiny_motorbike_coco.zip"
download_dir = "./tiny_motorbike_coco"
load_zip.unzip(zip_file, unzip_dir=download_dir)
data_dir = os.path.join(download_dir, "tiny_motorbike")
train_path = os.path.join(data_dir, "Annotations", "trainval_cocoformat.json")
test_path = os.path.join(data_dir, "Annotations", "test_cocoformat.json")
Downloading ./tiny_motorbike_coco/file.zip from https://automl-mm-bench.s3.amazonaws.com/object_detection_dataset/tiny_motorbike_coco.zip...
0%| | 0.00/21.8M [00:00<?, ?iB/s]
50%|█████ | 11.0M/21.8M [00:00<00:00, 110MiB/s]
100%|██████████| 21.8M/21.8M [00:00<00:00, 71.0MiB/s]
Dataset Format¶
For COCO format datasets, provide JSON annotation files for each split:
trainval_cocoformat.json: train and validation datatest_cocoformat.json: test data
Model Selection¶
We use the medium_quality preset which features:
Base model: YOLOX-large (pretrained on COCO)
Benefits: Fast finetuning, quick inference, easy deployment
Alternative presets available:
high_quality: DINO-Resnet50 modelbest_quality: DINO-SwinL model
Both alternatives offer improved performance at the cost of slower processing and higher GPU memory requirements.
presets = "medium_quality"
When creating the MultiModalPredictor, specify these essential parameters:
problem_type="object_detection"to define the taskpresets="medium_quality"for presets selectionsample_data_pathpointing to any dataset split (typically train_path) to infer object categoriespath(optional) to set a custom save location
If no path is specified, the model will be automatically saved to a timestamped directory under AutogluonModels/.
# Init predictor
import uuid
model_path = f"./tmp/{uuid.uuid4().hex}-quick_start_tutorial_temp_save"
predictor = MultiModalPredictor(
problem_type="object_detection",
sample_data_path=train_path,
presets=presets,
path=model_path,
)
Finetuning the Model¶
The model uses optimized preset configurations for learning rate, epochs, and batch size. By default, it employs a two-stage learning rate strategy:
Model head layers use 100x higher learning rate This approach accelerates convergence and typically improves performance, especially for small datasets (hundreds to thousands of images)
Timing results below are from a test run on AWS g4.2xlarge EC2 instance:
start = time.time()
predictor.fit(train_path) # Fit
train_end = time.time()
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Downloading yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth from https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_l_8x8_300e_coco/yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth...
Loads checkpoint by local backend from path: yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth
The model and loaded state dict do not match exactly
size mismatch for bbox_head.multi_level_conv_cls.0.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([10, 256, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.0.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([10]).
size mismatch for bbox_head.multi_level_conv_cls.1.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([10, 256, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.1.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([10]).
size mismatch for bbox_head.multi_level_conv_cls.2.weight: copying a param with shape torch.Size([80, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([10, 256, 1, 1]).
size mismatch for bbox_head.multi_level_conv_cls.2.bias: copying a param with shape torch.Size([80]) from checkpoint, the shape in current model is torch.Size([10]).
=================== System Info ===================
AutoGluon Version: 1.3.0b20250501
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Wed Mar 12 14:53:59 UTC 2025
CPU Count: 8
Pytorch Version: 2.6.0+cu124
CUDA Version: 12.4
Memory Avail: 28.40 GB / 30.95 GB (91.8%)
Disk Space Avail: WARNING, an exception (FileNotFoundError) occurred while attempting to get available disk space. Consider opening a GitHub Issue.
===================================================
Using default root folder: ./tiny_motorbike_coco/tiny_motorbike/Annotations/... Specify `model.mmdet_image.coco_root=...` in hyperparameters if you think it is wrong.
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/object_detection/quick_start/tmp/e4aa24301d1d4c2da309e2e5335d1c79-quick_start_tutorial_temp_save
```
Seed set to 0
0%| | 0.00/217M [00:00<?, ?iB/s]
0%| | 327k/217M [00:00<01:08, 3.15MiB/s]
0%| | 769k/217M [00:00<00:56, 3.82MiB/s]
1%| | 1.36M/217M [00:00<00:45, 4.73MiB/s]
1%| | 2.13M/217M [00:00<00:36, 5.88MiB/s]
1%|▏ | 3.13M/217M [00:00<00:29, 7.33MiB/s]
2%|▏ | 4.46M/217M [00:00<00:22, 9.32MiB/s]
3%|▎ | 6.14M/217M [00:00<00:17, 11.8MiB/s]
4%|▍ | 8.19M/217M [00:00<00:14, 14.5MiB/s]
5%|▍ | 10.4M/217M [00:00<00:12, 16.7MiB/s]
6%|▌ | 12.6M/217M [00:01<00:11, 18.2MiB/s]
7%|▋ | 14.9M/217M [00:01<00:10, 19.5MiB/s]
8%|▊ | 17.1M/217M [00:01<00:10, 19.9MiB/s]
9%|▉ | 19.3M/217M [00:01<00:09, 19.9MiB/s]
10%|▉ | 21.6M/217M [00:01<00:09, 20.7MiB/s]
11%|█ | 23.8M/217M [00:01<00:09, 20.7MiB/s]
12%|█▏ | 26.1M/217M [00:01<00:08, 21.4MiB/s]
13%|█▎ | 28.3M/217M [00:01<00:08, 21.6MiB/s]
14%|█▍ | 30.6M/217M [00:01<00:08, 22.0MiB/s]
15%|█▌ | 32.8M/217M [00:01<00:08, 21.7MiB/s]
16%|█▌ | 35.1M/217M [00:02<00:08, 22.0MiB/s]
17%|█▋ | 37.4M/217M [00:02<00:08, 21.7MiB/s]
18%|█▊ | 39.6M/217M [00:02<00:08, 21.9MiB/s]
19%|█▉ | 41.8M/217M [00:02<00:08, 20.9MiB/s]
20%|██ | 44.1M/217M [00:02<00:08, 21.5MiB/s]
21%|██▏ | 46.3M/217M [00:02<00:10, 15.6MiB/s]
22%|██▏ | 48.5M/217M [00:02<00:09, 17.2MiB/s]
23%|██▎ | 50.7M/217M [00:02<00:09, 18.4MiB/s]
24%|██▍ | 53.0M/217M [00:03<00:08, 19.4MiB/s]
25%|██▌ | 55.0M/217M [00:03<00:08, 19.4MiB/s]
26%|██▋ | 57.3M/217M [00:03<00:07, 20.3MiB/s]
27%|██▋ | 59.4M/217M [00:03<00:07, 20.2MiB/s]
28%|██▊ | 61.4M/217M [00:03<00:07, 19.7MiB/s]
29%|██▉ | 63.7M/217M [00:03<00:07, 20.5MiB/s]
30%|███ | 66.0M/217M [00:03<00:07, 21.3MiB/s]
31%|███▏ | 68.2M/217M [00:03<00:07, 21.1MiB/s]
32%|███▏ | 70.5M/217M [00:03<00:06, 21.5MiB/s]
33%|███▎ | 72.7M/217M [00:03<00:06, 21.4MiB/s]
34%|███▍ | 74.9M/217M [00:04<00:06, 21.8MiB/s]
36%|███▌ | 77.1M/217M [00:04<00:06, 21.9MiB/s]
37%|███▋ | 79.4M/217M [00:04<00:06, 21.6MiB/s]
38%|███▊ | 81.6M/217M [00:04<00:06, 21.8MiB/s]
39%|███▊ | 83.9M/217M [00:04<00:06, 22.0MiB/s]
40%|███▉ | 86.1M/217M [00:04<00:05, 21.9MiB/s]
41%|████ | 88.3M/217M [00:04<00:05, 21.7MiB/s]
42%|████▏ | 90.4M/217M [00:04<00:05, 21.7MiB/s]
43%|████▎ | 92.6M/217M [00:04<00:05, 21.6MiB/s]
44%|████▎ | 94.8M/217M [00:04<00:05, 21.5MiB/s]
45%|████▍ | 97.0M/217M [00:05<00:05, 21.9MiB/s]
46%|████▌ | 99.2M/217M [00:05<00:05, 21.5MiB/s]
47%|████▋ | 101M/217M [00:05<00:05, 21.2MiB/s]
48%|████▊ | 104M/217M [00:05<00:05, 21.6MiB/s]
49%|████▉ | 106M/217M [00:05<00:05, 21.8MiB/s]
50%|████▉ | 108M/217M [00:05<00:04, 22.1MiB/s]
51%|█████ | 110M/217M [00:05<00:04, 21.5MiB/s]
52%|█████▏ | 113M/217M [00:05<00:04, 21.3MiB/s]
53%|█████▎ | 115M/217M [00:05<00:04, 20.9MiB/s]
54%|█████▍ | 117M/217M [00:05<00:04, 21.3MiB/s]
55%|█████▍ | 119M/217M [00:06<00:04, 21.6MiB/s]
56%|█████▌ | 122M/217M [00:06<00:04, 22.1MiB/s]
57%|█████▋ | 124M/217M [00:06<00:04, 21.9MiB/s]
58%|█████▊ | 126M/217M [00:06<00:04, 22.5MiB/s]
59%|█████▉ | 128M/217M [00:06<00:03, 22.5MiB/s]
60%|██████ | 131M/217M [00:06<00:03, 22.5MiB/s]
61%|██████ | 133M/217M [00:06<00:03, 22.0MiB/s]
62%|██████▏ | 135M/217M [00:06<00:03, 22.4MiB/s]
63%|██████▎ | 138M/217M [00:06<00:03, 22.4MiB/s]
64%|██████▍ | 140M/217M [00:07<00:03, 21.9MiB/s]
65%|██████▌ | 142M/217M [00:07<00:03, 20.8MiB/s]
66%|██████▋ | 144M/217M [00:07<00:03, 21.4MiB/s]
67%|██████▋ | 146M/217M [00:07<00:03, 21.2MiB/s]
69%|██████▊ | 149M/217M [00:07<00:03, 21.8MiB/s]
70%|██████▉ | 151M/217M [00:07<00:03, 21.9MiB/s]
71%|███████ | 154M/217M [00:07<00:02, 22.3MiB/s]
72%|███████▏ | 156M/217M [00:07<00:02, 22.6MiB/s]
73%|███████▎ | 158M/217M [00:07<00:02, 22.7MiB/s]
74%|███████▍ | 161M/217M [00:07<00:02, 22.7MiB/s]
75%|███████▍ | 163M/217M [00:08<00:02, 23.0MiB/s]
76%|███████▌ | 165M/217M [00:08<00:02, 22.5MiB/s]
77%|███████▋ | 167M/217M [00:08<00:02, 22.4MiB/s]
78%|███████▊ | 170M/217M [00:08<00:02, 22.4MiB/s]
79%|███████▉ | 172M/217M [00:08<00:02, 22.3MiB/s]
80%|████████ | 174M/217M [00:08<00:01, 22.0MiB/s]
81%|████████ | 176M/217M [00:08<00:01, 22.3MiB/s]
82%|████████▏ | 179M/217M [00:08<00:01, 22.3MiB/s]
83%|████████▎ | 181M/217M [00:08<00:01, 22.2MiB/s]
84%|████████▍ | 183M/217M [00:08<00:01, 21.7MiB/s]
85%|████████▌ | 185M/217M [00:09<00:01, 21.4MiB/s]
86%|████████▋ | 188M/217M [00:09<00:01, 21.7MiB/s]
87%|████████▋ | 190M/217M [00:09<00:01, 22.2MiB/s]
88%|████████▊ | 192M/217M [00:09<00:01, 22.0MiB/s]
90%|████████▉ | 194M/217M [00:09<00:01, 22.4MiB/s]
91%|█████████ | 197M/217M [00:09<00:00, 22.4MiB/s]
92%|█████████▏| 199M/217M [00:09<00:00, 22.3MiB/s]
93%|█████████▎| 201M/217M [00:09<00:00, 22.1MiB/s]
94%|█████████▎| 204M/217M [00:09<00:00, 22.3MiB/s]
95%|█████████▍| 206M/217M [00:09<00:00, 22.2MiB/s]
96%|█████████▌| 208M/217M [00:10<00:00, 22.5MiB/s]
97%|█████████▋| 210M/217M [00:10<00:00, 22.2MiB/s]
98%|█████████▊| 213M/217M [00:10<00:00, 22.6MiB/s]
99%|█████████▉| 215M/217M [00:10<00:00, 22.7MiB/s]
GPU Count: 1
GPU Count to be Used: 1
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params | Mode
-------------------------------------------------------------------------------
0 | model | MMDetAutoModelForObjectDetection | 54.2 M | train
1 | validation_metric | MeanAveragePrecision | 0 | train
-------------------------------------------------------------------------------
54.2 M Trainable params
0 Non-trainable params
54.2 M Total params
216.620 Total estimated model params size (MB)
592 Modules in train mode
0 Modules in eval mode
/home/ci/opt/venv/lib/python3.11/site-packages/mmdet/models/backbones/csp_darknet.py:118: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(enabled=False):
/home/ci/opt/venv/lib/python3.11/site-packages/torch/functional.py:539: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:3637.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[8], line 2
1 start = time.time()
----> 2 predictor.fit(train_path) # Fit
3 train_end = time.time()
File ~/autogluon/multimodal/src/autogluon/multimodal/predictor.py:540, in MultiModalPredictor.fit(self, train_data, presets, tuning_data, max_num_tuning_data, id_mappings, time_limit, save_path, hyperparameters, column_types, holdout_frac, teacher_predictor, seed, standalone, hyperparameter_tune_kwargs, clean_ckpts, predictions, labels, predictors)
537 assert isinstance(predictors, list)
538 learners = [ele if isinstance(ele, str) else ele._learner for ele in predictors]
--> 540 self._learner.fit(
541 train_data=train_data,
542 presets=presets,
543 tuning_data=tuning_data,
544 max_num_tuning_data=max_num_tuning_data,
545 time_limit=time_limit,
546 save_path=save_path,
547 hyperparameters=hyperparameters,
548 column_types=column_types,
549 holdout_frac=holdout_frac,
550 teacher_learner=teacher_learner,
551 seed=seed,
552 standalone=standalone,
553 hyperparameter_tune_kwargs=hyperparameter_tune_kwargs,
554 clean_ckpts=clean_ckpts,
555 id_mappings=id_mappings,
556 predictions=predictions,
557 labels=labels,
558 learners=learners,
559 )
561 return self
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/object_detection.py:243, in ObjectDetectionLearner.fit(self, train_data, presets, tuning_data, max_num_tuning_data, time_limit, save_path, hyperparameters, column_types, holdout_frac, seed, standalone, hyperparameter_tune_kwargs, clean_ckpts, **kwargs)
236 self.fit_sanity_check()
237 self.prepare_fit_args(
238 time_limit=time_limit,
239 seed=seed,
240 standalone=standalone,
241 clean_ckpts=clean_ckpts,
242 )
--> 243 fit_returns = self.execute_fit()
244 self.on_fit_end(
245 training_start=training_start,
246 strategy=fit_returns.get("strategy", None),
(...)
249 clean_ckpts=clean_ckpts,
250 )
252 return self
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:577, in BaseLearner.execute_fit(self)
575 return dict()
576 else:
--> 577 attributes = self.fit_per_run(**self._fit_args)
578 self.update_attributes(**attributes) # only update attributes for non-HPO mode
579 return attributes
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/object_detection.py:438, in ObjectDetectionLearner.fit_per_run(self, max_time, save_path, ckpt_path, resume, enable_progress_bar, seed, hyperparameters, advanced_hyperparameters, config, df_preprocessor, data_processors, model, standalone, clean_ckpts)
419 config = self.post_update_config_per_run(
420 config=config,
421 num_gpus=num_gpus,
422 precision=precision,
423 strategy=strategy,
424 )
425 trainer = self.init_trainer_per_run(
426 num_gpus=num_gpus,
427 config=config,
(...)
435 enable_progress_bar=enable_progress_bar,
436 )
--> 438 self.run_trainer(
439 trainer=trainer,
440 litmodule=litmodule,
441 datamodule=datamodule,
442 ckpt_path=ckpt_path,
443 resume=resume,
444 )
445 self.on_fit_per_run_end(
446 save_path=save_path,
447 standalone=standalone,
(...)
452 model=model,
453 )
455 return dict(
456 config=config,
457 df_preprocessor=df_preprocessor,
(...)
461 strategy=strategy,
462 )
File ~/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:1211, in BaseLearner.run_trainer(self, trainer, litmodule, datamodule, ckpt_path, resume, pred_writer, is_train)
1209 warnings.filterwarnings("ignore", filter)
1210 if is_train:
-> 1211 trainer.fit(
1212 litmodule,
1213 datamodule=datamodule,
1214 ckpt_path=ckpt_path if resume else None, # this is to resume training that was broken accidentally
1215 )
1216 else:
1217 blacklist_msgs = []
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:561, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
559 self.training = True
560 self.should_stop = False
--> 561 call._call_and_handle_interrupt(
562 self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
563 )
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py:48, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
46 if trainer.strategy.launcher is not None:
47 return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
---> 48 return trainer_fn(*args, **kwargs)
50 except _TunerExitException:
51 _call_teardown_hook(trainer)
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:599, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
592 download_model_from_registry(ckpt_path, self)
593 ckpt_path = self._checkpoint_connector._select_ckpt_path(
594 self.state.fn,
595 ckpt_path,
596 model_provided=True,
597 model_connected=self.lightning_module is not None,
598 )
--> 599 self._run(model, ckpt_path=ckpt_path)
601 assert self.state.stopped
602 self.training = False
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:1012, in Trainer._run(self, model, ckpt_path)
1007 self._signal_connector.register_signal_handlers()
1009 # ----------------------------
1010 # RUN THE TRAINER
1011 # ----------------------------
-> 1012 results = self._run_stage()
1014 # ----------------------------
1015 # POST-Training CLEAN UP
1016 # ----------------------------
1017 log.debug(f"{self.__class__.__name__}: trainer tearing down")
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py:1056, in Trainer._run_stage(self)
1054 self._run_sanity_check()
1055 with torch.autograd.set_detect_anomaly(self._detect_anomaly):
-> 1056 self.fit_loop.run()
1057 return None
1058 raise RuntimeError(f"Unexpected state {self.state}")
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py:216, in _FitLoop.run(self)
214 try:
215 self.on_advance_start()
--> 216 self.advance()
217 self.on_advance_end()
218 except StopIteration:
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py:455, in _FitLoop.advance(self)
453 with self.trainer.profiler.profile("run_training_epoch"):
454 assert self._data_fetcher is not None
--> 455 self.epoch_loop.run(self._data_fetcher)
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py:150, in _TrainingEpochLoop.run(self, data_fetcher)
148 while not self.done:
149 try:
--> 150 self.advance(data_fetcher)
151 self.on_advance_end(data_fetcher)
152 except StopIteration:
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py:282, in _TrainingEpochLoop.advance(self, data_fetcher)
280 else:
281 dataloader_iter = None
--> 282 batch, _, __ = next(data_fetcher)
283 # TODO: we should instead use the batch_idx returned by the fetcher, however, that will require saving the
284 # fetcher state so that the batch_idx is correct after restarting
285 batch_idx = self.batch_idx + 1
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py:134, in _PrefetchDataFetcher.__next__(self)
131 self.done = not self.batches
132 elif not self.done:
133 # this will run only when no pre-fetching was done.
--> 134 batch = super().__next__()
135 else:
136 # the iterator is empty
137 raise StopIteration
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py:61, in _DataFetcher.__next__(self)
59 self._start_profiler()
60 try:
---> 61 batch = next(self.iterator)
62 except StopIteration:
63 self.done = True
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py:341, in CombinedLoader.__next__(self)
339 def __next__(self) -> _ITERATOR_RETURN:
340 assert self._iterator is not None
--> 341 out = next(self._iterator)
342 if isinstance(self._iterator, _Sequential):
343 return out
File ~/opt/venv/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py:78, in _MaxSizeCycle.__next__(self)
76 for i in range(n):
77 try:
---> 78 out[i] = next(self.iterators[i])
79 except StopIteration:
80 self._consumed[i] = True
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:708, in _BaseDataLoaderIter.__next__(self)
705 if self._sampler_iter is None:
706 # TODO(https://github.com/pytorch/pytorch/issues/76750)
707 self._reset() # type: ignore[call-arg]
--> 708 data = self._next_data()
709 self._num_yielded += 1
710 if (
711 self._dataset_kind == _DatasetKind.Iterable
712 and self._IterableDataset_len_called is not None
713 and self._num_yielded > self._IterableDataset_len_called
714 ):
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:1480, in _MultiProcessingDataLoaderIter._next_data(self)
1478 del self._task_info[idx]
1479 self._rcvd_idx += 1
-> 1480 return self._process_data(data)
File ~/opt/venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:1505, in _MultiProcessingDataLoaderIter._process_data(self, data)
1503 self._try_put_index()
1504 if isinstance(data, ExceptionWrapper):
-> 1505 data.reraise()
1506 return data
File ~/opt/venv/lib/python3.11/site-packages/torch/_utils.py:733, in ExceptionWrapper.reraise(self)
729 except TypeError:
730 # If the exception takes multiple arguments, don't try to
731 # instantiate since we don't know how to
732 raise RuntimeError(msg) from None
--> 733 raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/opt/venv/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 146, in _load_item
return self.__getitem__((idx + 1) % self.__len__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 167, in __getitem__
results = copy.deepcopy(self._load_item(idx))
^^^^^^^^^^^^^^^^^^^^
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 148, in _load_item
raise e
File "/home/ci/autogluon/multimodal/src/autogluon/multimodal/data/dataset_mmlab/multi_image_mix_dataset.py", line 134, in _load_item
per_ret = apply_data_processor(
^^^^^^^^^^^^^^^^^^^^^
TypeError: apply_data_processor() got an unexpected keyword argument 'feature_modalities'
Notice that at the end of each progress bar, if the checkpoint at current stage is saved,
it prints the model’s save path.
In this example, it’s ./quick_start_tutorial_temp_save.
Print out the time and we can see that it’s fast!
print("This finetuning takes %.2f seconds." % (train_end - start))
Evaluation¶
To evaluate the model we just trained, run following code.
And the evaluation results are shown in command line output. The first line is mAP in COCO standard, and the second line is mAP in VOC standard (or mAP50). For more details about these metrics, see COCO’s evaluation guideline. Note that for presenting a fast finetuning we use presets “medium_quality”, you could get better result on this dataset by simply using “high_quality” or “best_quality” presets, or customize your own model and hyperparameter settings: Customization, and some other examples at Fast Fine-tune Coco or High Performance Fine-tune Coco.
predictor.evaluate(test_path)
eval_end = time.time()
Print out the evaluation time:
print("The evaluation takes %.2f seconds." % (eval_end - train_end))
We can load a new predictor with previous save path, and we can also reset the number of used GPUs if not all the devices are available:
# Load and reset num_gpus
new_predictor = MultiModalPredictor.load(model_path)
new_predictor.set_num_gpus(1)
Evaluating the new predictor gives us exactly the same result:
# Evaluate new predictor
new_predictor.evaluate(test_path)
For how to set the hyperparameters and finetune the model with higher performance, see AutoMM Detection - High Performance Finetune on COCO Format Dataset.
Inference¶
Let’s perform predictions using our finetuned model. The predictor can process the entire test set with a single command:
pred = predictor.predict(test_path)
print(len(pred)) # Number of predictions
print(pred[:3]) # Sample of first 3 predictions
The predictor returns predictions as a pandas DataFrame with two columns:
image: Contains path to each input imagebboxes: Contains list of detected objects, where each object is a dictionary:{ "class": "predicted_class_name", "bbox": [x1, y1, x2, y2], # Coordinates of Upper Left and Bottom Right corners "score": confidence_score }
By default, predictions are returned but not saved. To save detection results, use the save parameter in your predict call.
pred = predictor.predict(test_path, save_results=True, as_coco=False)
The predictions can be saved in two formats:
CSV file: Matches the DataFrame structure with image and bboxes columns
COCO JSON: Standard COCO format annotation file
This works with any predictor configuration (pretrained or finetuned models).
Visualizing Results¶
To run visualizations, ensure that you have opencv installed. If you haven’t already, install opencv by running
!pip install opencv-python
To visualize the detection bounding boxes, run the following:
from autogluon.multimodal.utils import ObjectDetectionVisualizer
conf_threshold = 0.4 # Specify a confidence threshold to filter out unwanted boxes
image_result = pred.iloc[30]
img_path = image_result.image # Select an image to visualize
visualizer = ObjectDetectionVisualizer(img_path) # Initialize the Visualizer
out = visualizer.draw_instance_predictions(image_result, conf_threshold=conf_threshold) # Draw detections
visualized = out.get_image() # Get the visualized image
from PIL import Image
from IPython.display import display
img = Image.fromarray(visualized, 'RGB')
display(img)
Testing on Your Own Data¶
You can also predict on your own images with various input format. The follow is an example:
Download the example image:
from autogluon.multimodal import download
image_url = "https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/detection/street_small.jpg"
test_image = download(image_url)
Run inference on data in a json file of COCO format (See Convert Data to COCO Format for more details about COCO format). Note that since the root is by default the parent folder of the annotation file, here we put the annotation file in a folder:
import json
# create a input file for demo
data = {"images": [{"id": 0, "width": -1, "height": -1, "file_name": test_image}], "categories": []}
os.mkdir("input_data_for_demo")
input_file = "input_data_for_demo/demo_annotation.json"
with open(input_file, "w+") as f:
json.dump(data, f)
pred_test_image = predictor.predict(input_file)
print(pred_test_image)
Run inference on data in a list of image file names:
pred_test_image = predictor.predict([test_image])
print(pred_test_image)
Other Examples¶
You may go to AutoMM Examples to explore other examples about AutoMM.
Customization¶
To learn how to customize AutoMM, please refer to Customize AutoMM.
Citation¶
@article{DBLP:journals/corr/abs-2107-08430,
author = {Zheng Ge and
Songtao Liu and
Feng Wang and
Zeming Li and
Jian Sun},
title = {{YOLOX:} Exceeding {YOLO} Series in 2021},
journal = {CoRR},
volume = {abs/2107.08430},
year = {2021},
url = {https://arxiv.org/abs/2107.08430},
eprinttype = {arXiv},
eprint = {2107.08430},
timestamp = {Tue, 05 Apr 2022 14:09:44 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-2107-08430.bib},
bibsource = {dblp computer science bibliography, https://dblp.org},
}