.. _sec_customobj: Searchable Objects ================== When defining custom Python objects such as network architectures, or specialized optimizers, it may be hard to decide what values to set for all of their attributes. AutoGluon provides an API that allows you to instead specify a search space of possible values to consider for such attributes, within which the optimal value will be automatically searched for at runtime. This tutorial demonstrates how easy this is to do, without having to modify your existing code at all! Example for Constructing a Network ---------------------------------- This tutorial covers an example of selecting a neural network's architecture as a hyperparameter optimization (HPO) task. If you are interested in efficient neural architecture search (NAS), please refer to this other tutorial instead: ``sec_proxyless``\ \_ . CIFAR ResNet in GluonCV ~~~~~~~~~~~~~~~~~~~~~~~ GluonCV provides `CIFARResNet `__, which allow user to specify how many layers at each stage. For example, we can construct a CIFAR ResNet with only 1 layer per stage: .. code:: python from gluoncv.model_zoo.cifarresnet import CIFARResNetV1, CIFARBasicBlockV1 layers = [1, 1, 1] channels = [16, 16, 32, 64] net = CIFARResNetV1(CIFARBasicBlockV1, layers, channels) We can visualize the network: .. code:: python import autogluon.core as ag from autogluon.vision.utils import plot_network plot_network(net, (1, 3, 32, 32)) .. figure:: output_object_d3e86d_3_0.svg Searchable Network Architecture Using AutoGluon Object ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :func:`autogluon.obj` enables customized search space to any user defined class. It can also be used within ``autogluon.Categorical()`` if you have multiple networks to choose from. .. code:: python @ag.obj( nstage1=ag.space.Int(2, 4), nstage2=ag.space.Int(2, 4), ) class MyCifarResNet(CIFARResNetV1): def __init__(self, nstage1, nstage2): nstage3 = 9 - nstage1 - nstage2 layers = [nstage1, nstage2, nstage3] channels = [16, 16, 32, 64] super().__init__(CIFARBasicBlockV1, layers=layers, channels=channels) Create one network instance and print the configuration space: .. code:: python mynet=MyCifarResNet() print(mynet.cs) .. parsed-literal:: :class: output Configuration space object: Hyperparameters: nstage1, Type: UniformInteger, Range: [2, 4], Default: 3 nstage2, Type: UniformInteger, Range: [2, 4], Default: 3 We can also overwrite existing search spaces: .. code:: python mynet1 = MyCifarResNet(nstage1=1, nstage2=ag.space.Int(5, 10)) print(mynet1.cs) .. parsed-literal:: :class: output Configuration space object: Hyperparameters: nstage2, Type: UniformInteger, Range: [5, 10], Default: 8 Decorate Existing Class ~~~~~~~~~~~~~~~~~~~~~~~ We can also use :func:`autogluon.obj` to easily decorate any existing classes. For example, if we want to search learning rate and weight decay for Adam optimizer, we only need to add a decorator: .. code:: python from mxnet import optimizer as optim @ag.obj() class Adam(optim.Adam): pass Then we can create an instance: .. code:: python myoptim = Adam(learning_rate=ag.Real(1e-2, 1e-1, log=True), wd=ag.Real(1e-5, 1e-3, log=True)) print(myoptim.cs) .. parsed-literal:: :class: output Configuration space object: Hyperparameters: learning_rate, Type: UniformFloat, Range: [0.01, 0.1], Default: 0.0316227766, on log-scale wd, Type: UniformFloat, Range: [1e-05, 0.001], Default: 0.0001, on log-scale Launch Experiments Using AutoGluon Object ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ AutoGluon Object is compatible with Fit API in AutoGluon tasks, and also works with user-defined training scripts using :func:`autogluon.autogluon_register_args`. We can start fitting: .. code:: python from autogluon.vision import ImagePredictor classifier = ImagePredictor().fit('cifar10', hyperparameters={'net': mynet, 'optimizer': myoptim, 'epochs': 1}, ngpus_per_trial=1) .. parsed-literal:: :class: output INFO:root:`time_limit=auto` set to `time_limit=7200`. INFO:gluoncv.auto.tasks.image_classification:Starting fit without HPO INFO:ImageClassificationEstimator:modified configs( != ): { INFO:ImageClassificationEstimator:root.valid.batch_size 128 != 16 INFO:ImageClassificationEstimator:root.valid.num_workers 4 != 8 INFO:ImageClassificationEstimator:root.img_cls.model resnet50_v1 != resnet50_v1b INFO:ImageClassificationEstimator:root.train.num_training_samples 1281167 != -1 INFO:ImageClassificationEstimator:root.train.num_workers 4 != 8 INFO:ImageClassificationEstimator:root.train.lr 0.1 != 0.01 INFO:ImageClassificationEstimator:root.train.rec_val ~/.mxnet/datasets/imagenet/rec/val.rec != auto INFO:ImageClassificationEstimator:root.train.rec_val_idx ~/.mxnet/datasets/imagenet/rec/val.idx != auto INFO:ImageClassificationEstimator:root.train.rec_train_idx ~/.mxnet/datasets/imagenet/rec/train.idx != auto INFO:ImageClassificationEstimator:root.train.rec_train ~/.mxnet/datasets/imagenet/rec/train.rec != auto INFO:ImageClassificationEstimator:root.train.data_dir ~/.mxnet/datasets/imagenet != auto INFO:ImageClassificationEstimator:root.train.epochs 10 != 1 INFO:ImageClassificationEstimator:root.train.batch_size 128 != 16 INFO:ImageClassificationEstimator:} INFO:ImageClassificationEstimator:Saved config to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7/.trial_0/config.yaml INFO:ImageClassificationEstimator:Start training from [Epoch 0] INFO:ImageClassificationEstimator:Epoch[0] Batch [49] Speed: 101.968123 samples/sec accuracy=0.172500 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [99] Speed: 104.189805 samples/sec accuracy=0.233750 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [149] Speed: 103.551351 samples/sec accuracy=0.285417 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [199] Speed: 103.123701 samples/sec accuracy=0.324375 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [249] Speed: 102.370974 samples/sec accuracy=0.352250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [299] Speed: 101.884047 samples/sec accuracy=0.366875 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [349] Speed: 101.210340 samples/sec accuracy=0.385714 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [399] Speed: 100.752868 samples/sec accuracy=0.401406 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [449] Speed: 100.180295 samples/sec accuracy=0.415833 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [499] Speed: 99.668112 samples/sec accuracy=0.430125 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [549] Speed: 98.953313 samples/sec accuracy=0.440455 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [599] Speed: 98.563180 samples/sec accuracy=0.452708 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [649] Speed: 97.931173 samples/sec accuracy=0.463173 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [699] Speed: 97.352515 samples/sec accuracy=0.472321 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [749] Speed: 96.948858 samples/sec accuracy=0.481833 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [799] Speed: 96.024894 samples/sec accuracy=0.490000 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [849] Speed: 95.392738 samples/sec accuracy=0.497721 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [899] Speed: 94.932330 samples/sec accuracy=0.503611 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [949] Speed: 94.393813 samples/sec accuracy=0.508947 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [999] Speed: 93.593267 samples/sec accuracy=0.514625 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1049] Speed: 93.001790 samples/sec accuracy=0.519345 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1099] Speed: 92.470108 samples/sec accuracy=0.523807 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1149] Speed: 91.992084 samples/sec accuracy=0.529348 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1199] Speed: 91.871178 samples/sec accuracy=0.533333 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1249] Speed: 92.222805 samples/sec accuracy=0.538000 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1299] Speed: 92.173491 samples/sec accuracy=0.543654 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1349] Speed: 92.101029 samples/sec accuracy=0.547685 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1399] Speed: 91.971739 samples/sec accuracy=0.550402 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1449] Speed: 91.761918 samples/sec accuracy=0.553922 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1499] Speed: 91.604789 samples/sec accuracy=0.557542 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1549] Speed: 91.590236 samples/sec accuracy=0.560645 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1599] Speed: 91.920071 samples/sec accuracy=0.563359 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1649] Speed: 92.296159 samples/sec accuracy=0.567121 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1699] Speed: 92.169475 samples/sec accuracy=0.571250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1749] Speed: 92.085320 samples/sec accuracy=0.573929 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1799] Speed: 91.824013 samples/sec accuracy=0.577292 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1849] Speed: 91.679084 samples/sec accuracy=0.579392 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1899] Speed: 91.657517 samples/sec accuracy=0.581908 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1949] Speed: 91.391104 samples/sec accuracy=0.584006 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [1999] Speed: 91.453672 samples/sec accuracy=0.586250 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2049] Speed: 91.294017 samples/sec accuracy=0.588232 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2099] Speed: 91.958108 samples/sec accuracy=0.591131 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2149] Speed: 92.314833 samples/sec accuracy=0.593198 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2199] Speed: 92.359445 samples/sec accuracy=0.595114 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2249] Speed: 92.511702 samples/sec accuracy=0.597139 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2299] Speed: 92.934818 samples/sec accuracy=0.599592 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2349] Speed: 92.797128 samples/sec accuracy=0.601543 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2399] Speed: 92.975538 samples/sec accuracy=0.603099 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2449] Speed: 92.942589 samples/sec accuracy=0.604898 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2499] Speed: 93.073967 samples/sec accuracy=0.606150 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2549] Speed: 93.137603 samples/sec accuracy=0.608333 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2599] Speed: 93.001870 samples/sec accuracy=0.610192 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2649] Speed: 93.230029 samples/sec accuracy=0.612099 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2699] Speed: 93.208268 samples/sec accuracy=0.614352 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2749] Speed: 93.180000 samples/sec accuracy=0.615727 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2799] Speed: 93.205168 samples/sec accuracy=0.617455 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2849] Speed: 93.021740 samples/sec accuracy=0.618838 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2899] Speed: 93.086325 samples/sec accuracy=0.620905 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2949] Speed: 93.154892 samples/sec accuracy=0.622500 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [2999] Speed: 93.090589 samples/sec accuracy=0.624104 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3049] Speed: 93.265959 samples/sec accuracy=0.625676 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3099] Speed: 93.191317 samples/sec accuracy=0.627460 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3149] Speed: 93.251405 samples/sec accuracy=0.628988 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3199] Speed: 93.293091 samples/sec accuracy=0.630449 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3249] Speed: 93.154261 samples/sec accuracy=0.631827 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3299] Speed: 93.344509 samples/sec accuracy=0.632803 lr=0.010000 INFO:ImageClassificationEstimator:Epoch[0] Batch [3349] Speed: 93.142614 samples/sec accuracy=0.634067 lr=0.010000 INFO:ImageClassificationEstimator:[Epoch 0] training: accuracy=0.634593 INFO:ImageClassificationEstimator:[Epoch 0] speed: 94 samples/sec time cost: 594.446127 INFO:ImageClassificationEstimator:[Epoch 0] validation: top1=0.902000 top5=0.997000 INFO:ImageClassificationEstimator:[Epoch 0] Current best top-1: 0.902000 vs previous 0.000000, saved to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7/.trial_0/best_checkpoint.pkl INFO:ImageClassificationEstimator:Pickled to /var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7/.trial_0/best_checkpoint.pkl INFO:gluoncv.auto.tasks.image_classification:Finished, total runtime is 602.53 s INFO:gluoncv.auto.tasks.image_classification:{ 'best_config': { 'batch_size': 16, 'custom_net': MyCifarResNet( (features): HybridSequential( (0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (3): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (3): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (4): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (5): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (6): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (7): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (4): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW) ) (output): Dense(64 -> 10, linear) ), 'custom_optimizer': <__main__.Adam object at 0x7f32a8489850>, 'dist_ip_addrs': None, 'epochs': 1, 'estimator': , 'final_fit': False, 'gpus': [0], 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7', 'lr': 0.01, 'model': 'resnet50_v1b', 'ngpus_per_trial': 1, 'nthreads_per_trial': 128, 'num_trials': 1, 'num_workers': 8, 'search_strategy': 'random', 'seed': 277, 'time_limits': 7200, 'wall_clock_tick': 1615358651.6753}, 'total_time': 586.4399874210358, 'train_acc': 0.6345925925925926, 'valid_acc': 0.902} .. code:: python print(classifier.fit_summary()) .. parsed-literal:: :class: output {'train_acc': 0.6345925925925926, 'valid_acc': 0.902, 'total_time': 586.4399874210358, 'best_config': {'model': 'resnet50_v1b', 'lr': 0.01, 'num_trials': 1, 'epochs': 1, 'batch_size': 16, 'nthreads_per_trial': 128, 'ngpus_per_trial': 1, 'time_limits': 7200, 'search_strategy': 'random', 'dist_ip_addrs': None, 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7', 'custom_net': MyCifarResNet( (features): HybridSequential( (0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (3): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (3): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (4): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (5): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (6): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (7): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (4): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW) ) (output): Dense(64 -> 10, linear) ), 'custom_optimizer': <__main__.Adam object at 0x7f32a8489850>, 'num_workers': 8, 'gpus': [0], 'seed': 277, 'final_fit': False, 'estimator': , 'wall_clock_tick': 1615358651.6753}, 'fit_history': {'train_acc': 0.6345925925925926, 'valid_acc': 0.902, 'total_time': 586.4399874210358, 'best_config': {'model': 'resnet50_v1b', 'lr': 0.01, 'num_trials': 1, 'epochs': 1, 'batch_size': 16, 'nthreads_per_trial': 128, 'ngpus_per_trial': 1, 'time_limits': 7200, 'search_strategy': 'random', 'dist_ip_addrs': None, 'log_dir': '/var/lib/jenkins/workspace/workspace/autogluon-tutorial-course-v3/docs/_build/eval/tutorials/course/95a139f7', 'custom_net': MyCifarResNet( (features): HybridSequential( (0): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(16 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (3): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(16 -> 32, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (1): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (2): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (3): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (4): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (5): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (6): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) (7): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(32 -> 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (4): HybridSequential( (0): CIFARBasicBlockV1( (body): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) (2): Activation(relu) (3): Conv2D(64 -> 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (4): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) (downsample): HybridSequential( (0): Conv2D(32 -> 64, kernel_size=(1, 1), stride=(2, 2), bias=False) (1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=None) ) ) ) (5): GlobalAvgPool2D(size=(1, 1), stride=(1, 1), padding=(0, 0), ceil_mode=True, global_pool=True, pool_type=avg, layout=NCHW) ) (output): Dense(64 -> 10, linear) ), 'custom_optimizer': <__main__.Adam object at 0x7f32a8489850>, 'num_workers': 8, 'gpus': [0], 'seed': 277, 'final_fit': False, 'estimator': , 'wall_clock_tick': 1615358651.6753}}}