.. _sec_proxyless:

How to Use ENAS/ProxylessNAS in Ten Minutes
===========================================


What is the Key Idea of ENAS and ProxylessNAS?
----------------------------------------------

Traditional reinforcement learning-based neural architecture search
learns an architecture controller by iteratively sampling the
architecture and training the model to get final reward to update the
controller. It is extremely expensive process due to training CNN.

.. figure:: https://raw.githubusercontent.com/zhanghang1989/AutoGluonWebdata/master/docs/tutorial/proxyless.png

   ProxylessNAS

Recent work of ENAS and ProxylessNAS construct an over-parameterized
network (supernet) and share the weights across different architecture
to speed up the search speed. The reward is calculated every few
iterations instead of every training period.

Import MXNet and AutoGluon:

.. code:: python

    import autogluon as ag
    import mxnet as mx
    import mxnet.gluon.nn as nn

How to Construct a SuperNet
---------------------------

Basic NN blocks for CNN.

.. code:: python

    class Identity(mx.gluon.HybridBlock):
        def hybrid_forward(self, F, x):
            return x
    
    class ConvBNReLU(mx.gluon.HybridBlock):
        def __init__(self, in_channels, channels, kernel, stride):
            super().__init__()
            padding = (kernel - 1) // 2
            self.conv = nn.Conv2D(channels, kernel, stride, padding, in_channels=in_channels)
            self.bn = nn.BatchNorm(in_channels=channels)
            self.relu = nn.Activation('relu')
        def hybrid_forward(self, F, x):
            return self.relu(self.bn(self.conv(x)))


.. parsed-literal::
    :class: output

    /var/lib/jenkins/miniconda3/envs/autogluon_docs-v0_0_15/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
      and should_run_async(code)


AutoGluon ENAS Unit
~~~~~~~~~~~~~~~~~~~

.. code:: python

    from autogluon.contrib.enas import *
    
    @enas_unit()
    class ResUnit(mx.gluon.HybridBlock):
        def __init__(self, in_channels, channels, hidden_channels, kernel, stride):
            super().__init__()
            self.conv1 = ConvBNReLU(in_channels, hidden_channels, kernel, stride)
            self.conv2 = ConvBNReLU(hidden_channels, channels, kernel, 1)
            if in_channels == channels and stride == 1:
                self.shortcut = Identity()
            else:
                self.shortcut = nn.Conv2D(channels, 1, stride, in_channels=in_channels)
        def hybrid_forward(self, F, x):
            return self.conv2(self.conv1(x)) + self.shortcut(x)

AutoGluon Sequntial
~~~~~~~~~~~~~~~~~~~

Creating a ENAS network using Sequential Block:

.. code:: python

    mynet = ENAS_Sequential(
        ResUnit(1, 8, hidden_channels=ag.space.Categorical(4, 8), kernel=ag.space.Categorical(3, 5), stride=2),
        ResUnit(8, 8, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
        ResUnit(8, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
        ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
        ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
        nn.GlobalAvgPool2D(),
        nn.Flatten(),
        nn.Activation('relu'),
        nn.Dense(10, in_units=16),
    )
    
    mynet.initialize()
    
    #mynet.graph


.. parsed-literal::
    :class: output

    /var/lib/jenkins/miniconda3/envs/autogluon_docs-v0_0_15/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
      and should_run_async(code)


Evaluate Network Latency and Define Reward Function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

    x = mx.nd.random.uniform(shape=(1, 1, 28, 28))
    y = mynet.evaluate_latency(x)

Show the latencies:

.. code:: python

    print('Average latency is {:.2f} ms, latency of the current architecture is {:.2f} ms'.format(mynet.avg_latency, mynet.latency))


.. parsed-literal::
    :class: output

    Average latency is 4.57 ms, latency of the current architecture is 5.19 ms


We also provide number of params

.. code:: python

    mynet.nparams




.. parsed-literal::
    :class: output

    8714



Define the reward function:

.. code:: python

    reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.1)

Start the Training
------------------

Construct experiment scheduler, which automatically creates an RL
controller based on user-defined search space.

.. code:: python

    scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                               reward_fn=reward_fn, batch_size=128, num_gpus=1,
                               warmup_epochs=0, epochs=1, controller_lr=3e-3,
                               plot_frequency=10, update_arch_frequency=5)


.. parsed-literal::
    :class: output

    get_built_in_dataset mnist
    get_built_in_dataset mnist


Start the training:

.. code:: python

    scheduler.run()


.. parsed-literal::
    :class: output

      0%|          | 0/1 [00:00<?, ?it/s]
      0%|          | 0/468 [00:00<?, ?it/s]
    avg reward: 0.09:   0%|          | 0/468 [00:00<?, ?it/s]
    avg reward: 0.09:   0%|          | 1/468 [00:00<05:25,  1.43it/s]
    avg reward: 0.09:   0%|          | 1/468 [00:00<05:25,  1.43it/s]
    avg reward: 0.09:   0%|          | 1/468 [00:00<05:25,  1.43it/s]
    avg reward: 0.09:   0%|          | 1/468 [00:00<05:25,  1.43it/s]
    avg reward: 0.09:   0%|          | 1/468 [00:00<05:25,  1.43it/s]
    avg reward: 0.09:   1%|          | 5/468 [00:00<03:49,  2.02it/s]
    avg reward: 0.10:   1%|          | 5/468 [00:00<03:49,  2.02it/s]
    avg reward: 0.10:   1%|          | 5/468 [00:01<03:49,  2.02it/s]
    avg reward: 0.10:   1%|▏         | 7/468 [00:01<02:54,  2.65it/s]
    avg reward: 0.10:   1%|▏         | 7/468 [00:01<02:54,  2.65it/s]
    avg reward: 0.10:   1%|▏         | 7/468 [00:01<02:54,  2.65it/s]
    avg reward: 0.10:   1%|▏         | 7/468 [00:01<02:54,  2.65it/s]
    avg reward: 0.10:   1%|▏         | 7/468 [00:01<02:54,  2.65it/s]
    avg reward: 0.10:   2%|▏         | 11/468 [00:01<02:08,  3.55it/s]
    avg reward: 0.10:   2%|▏         | 11/468 [00:01<02:08,  3.55it/s]
    avg reward: 0.10:   2%|▏         | 11/468 [00:01<02:08,  3.55it/s]
    avg reward: 0.10:   2%|▏         | 11/468 [00:01<02:08,  3.55it/s]
    avg reward: 0.10:   2%|▏         | 11/468 [00:01<02:08,  3.55it/s]
    avg reward: 0.10:   3%|▎         | 15/468 [00:01<01:33,  4.84it/s]
    avg reward: 0.11:   3%|▎         | 15/468 [00:01<01:33,  4.84it/s]
    avg reward: 0.11:   3%|▎         | 15/468 [00:01<01:33,  4.84it/s]
    avg reward: 0.11:   4%|▎         | 17/468 [00:01<01:18,  5.71it/s]
    avg reward: 0.11:   4%|▎         | 17/468 [00:01<01:18,  5.71it/s]
    avg reward: 0.11:   4%|▎         | 17/468 [00:01<01:18,  5.71it/s]
    avg reward: 0.11:   4%|▎         | 17/468 [00:01<01:18,  5.71it/s]
    avg reward: 0.11:   4%|▎         | 17/468 [00:01<01:18,  5.71it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:01<01:03,  7.04it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:01<01:03,  7.04it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:01<01:03,  7.04it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:01<01:03,  7.04it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:01<01:03,  7.04it/s]
    avg reward: 0.11:   4%|▍         | 21/468 [00:02<01:03,  7.04it/s]
    avg reward: 0.11:   6%|▌         | 26/468 [00:02<00:51,  8.62it/s]
    avg reward: 0.11:   6%|▌         | 26/468 [00:02<00:51,  8.62it/s]
    avg reward: 0.11:   6%|▌         | 26/468 [00:02<00:51,  8.62it/s]
    avg reward: 0.11:   6%|▌         | 26/468 [00:02<00:51,  8.62it/s]
    avg reward: 0.11:   6%|▌         | 26/468 [00:02<00:51,  8.62it/s]
    avg reward: 0.11:   6%|▋         | 30/468 [00:02<00:38, 11.26it/s]
    avg reward: 0.17:   6%|▋         | 30/468 [00:02<00:38, 11.26it/s]
    avg reward: 0.17:   6%|▋         | 30/468 [00:02<00:38, 11.26it/s]
    avg reward: 0.17:   6%|▋         | 30/468 [00:02<00:38, 11.26it/s]
    avg reward: 0.17:   7%|▋         | 33/468 [00:02<00:36, 11.78it/s]
    avg reward: 0.17:   7%|▋         | 33/468 [00:02<00:36, 11.78it/s]
    avg reward: 0.17:   7%|▋         | 33/468 [00:02<00:36, 11.78it/s]
    avg reward: 0.24:   7%|▋         | 33/468 [00:02<00:36, 11.78it/s]
    avg reward: 0.24:   8%|▊         | 36/468 [00:02<00:34, 12.39it/s]
    avg reward: 0.24:   8%|▊         | 36/468 [00:02<00:34, 12.39it/s]
    avg reward: 0.24:   8%|▊         | 36/468 [00:02<00:34, 12.39it/s]
    avg reward: 0.24:   8%|▊         | 36/468 [00:02<00:34, 12.39it/s]
    avg reward: 0.24:   8%|▊         | 36/468 [00:02<00:34, 12.39it/s]
    avg reward: 0.24:   9%|▊         | 40/468 [00:02<00:27, 15.62it/s]
    avg reward: 0.39:   9%|▊         | 40/468 [00:02<00:27, 15.62it/s]
    avg reward: 0.39:   9%|▊         | 40/468 [00:02<00:27, 15.62it/s]
    avg reward: 0.39:   9%|▊         | 40/468 [00:02<00:27, 15.62it/s]
    avg reward: 0.39:   9%|▉         | 43/468 [00:02<00:29, 14.62it/s]
    avg reward: 0.39:   9%|▉         | 43/468 [00:02<00:29, 14.62it/s]
    avg reward: 0.39:   9%|▉         | 43/468 [00:03<00:29, 14.62it/s]
    avg reward: 0.42:   9%|▉         | 43/468 [00:03<00:29, 14.62it/s]
    avg reward: 0.42:  10%|▉         | 46/468 [00:03<00:28, 14.61it/s]
    avg reward: 0.42:  10%|▉         | 46/468 [00:03<00:28, 14.61it/s]
    avg reward: 0.42:  10%|▉         | 46/468 [00:03<00:28, 14.61it/s]
    avg reward: 0.42:  10%|▉         | 46/468 [00:03<00:28, 14.61it/s]
    avg reward: 0.42:  10%|▉         | 46/468 [00:03<00:28, 14.61it/s]
    avg reward: 0.42:  11%|█         | 50/468 [00:03<00:23, 18.00it/s]
    avg reward: 0.49:  11%|█         | 50/468 [00:03<00:23, 18.00it/s]
    avg reward: 0.49:  11%|█         | 50/468 [00:03<00:23, 18.00it/s]
    avg reward: 0.49:  11%|█         | 50/468 [00:03<00:23, 18.00it/s]
    avg reward: 0.49:  11%|█▏        | 53/468 [00:03<00:26, 15.82it/s]
    avg reward: 0.49:  11%|█▏        | 53/468 [00:03<00:26, 15.82it/s]
    avg reward: 0.49:  11%|█▏        | 53/468 [00:03<00:26, 15.82it/s]
    avg reward: 0.60:  11%|█▏        | 53/468 [00:03<00:26, 15.82it/s]
    avg reward: 0.60:  12%|█▏        | 56/468 [00:03<00:26, 15.31it/s]
    avg reward: 0.60:  12%|█▏        | 56/468 [00:03<00:26, 15.31it/s]
    avg reward: 0.60:  12%|█▏        | 56/468 [00:03<00:26, 15.31it/s]
    avg reward: 0.60:  12%|█▏        | 56/468 [00:03<00:26, 15.31it/s]
    avg reward: 0.60:  12%|█▏        | 56/468 [00:03<00:26, 15.31it/s]
    avg reward: 0.60:  13%|█▎        | 60/468 [00:03<00:21, 18.77it/s]
    avg reward: 0.64:  13%|█▎        | 60/468 [00:04<00:21, 18.77it/s]
    avg reward: 0.64:  13%|█▎        | 60/468 [00:04<00:21, 18.77it/s]
    avg reward: 0.64:  13%|█▎        | 60/468 [00:04<00:21, 18.77it/s]
    avg reward: 0.64:  13%|█▎        | 63/468 [00:04<00:29, 13.73it/s]
    avg reward: 0.64:  13%|█▎        | 63/468 [00:04<00:29, 13.73it/s]
    avg reward: 0.64:  13%|█▎        | 63/468 [00:04<00:29, 13.73it/s]
    avg reward: 0.73:  13%|█▎        | 63/468 [00:04<00:29, 13.73it/s]
    avg reward: 0.73:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.73:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.73:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.73:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.73:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.77:  14%|█▍        | 66/468 [00:04<00:30, 13.26it/s]
    avg reward: 0.77:  15%|█▌        | 71/468 [00:04<00:27, 14.43it/s]
    avg reward: 0.77:  15%|█▌        | 71/468 [00:04<00:27, 14.43it/s]
    avg reward: 0.77:  15%|█▌        | 71/468 [00:04<00:27, 14.43it/s]
    avg reward: 0.77:  15%|█▌        | 71/468 [00:04<00:27, 14.43it/s]
    avg reward: 0.77:  15%|█▌        | 71/468 [00:04<00:27, 14.43it/s]
    avg reward: 0.77:  16%|█▌        | 75/468 [00:04<00:22, 17.65it/s]
    avg reward: 0.81:  16%|█▌        | 75/468 [00:05<00:22, 17.65it/s]
    avg reward: 0.81:  16%|█▌        | 75/468 [00:05<00:22, 17.65it/s]
    avg reward: 0.81:  16%|█▌        | 75/468 [00:05<00:22, 17.65it/s]
    avg reward: 0.81:  17%|█▋        | 78/468 [00:05<00:25, 15.38it/s]
    avg reward: 0.81:  17%|█▋        | 78/468 [00:05<00:25, 15.38it/s]
    avg reward: 0.81:  17%|█▋        | 78/468 [00:05<00:25, 15.38it/s]
    avg reward: 0.83:  17%|█▋        | 78/468 [00:05<00:25, 15.38it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:05<00:26, 14.76it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:05<00:26, 14.76it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:05<00:26, 14.76it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:05<00:26, 14.76it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:05<00:26, 14.76it/s]
    avg reward: 0.83:  18%|█▊        | 85/468 [00:05<00:21, 17.83it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:21, 17.83it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:21, 17.83it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:21, 17.83it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 15.99it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 15.99it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 15.99it/s]
    avg reward: 0.87:  19%|█▉        | 88/468 [00:05<00:23, 15.99it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.15it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.15it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.15it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.15it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.15it/s]
    avg reward: 0.90:  19%|█▉        | 91/468 [00:06<00:24, 15.15it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  21%|██        | 96/468 [00:06<00:23, 15.78it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.89:  22%|██▏       | 101/468 [00:06<00:22, 16.59it/s]
    avg reward: 0.89:  23%|██▎       | 106/468 [00:06<00:21, 17.12it/s]
    avg reward: 0.89:  23%|██▎       | 106/468 [00:06<00:21, 17.12it/s]
    avg reward: 0.89:  23%|██▎       | 106/468 [00:06<00:21, 17.12it/s]
    avg reward: 0.89:  23%|██▎       | 106/468 [00:06<00:21, 17.12it/s]
    avg reward: 0.89:  23%|██▎       | 106/468 [00:06<00:21, 17.12it/s]
    avg reward: 0.89:  24%|██▎       | 110/468 [00:06<00:17, 20.51it/s]
    avg reward: 0.89:  24%|██▎       | 110/468 [00:06<00:17, 20.51it/s]
    avg reward: 0.89:  24%|██▎       | 110/468 [00:06<00:17, 20.51it/s]
    avg reward: 0.89:  24%|██▎       | 110/468 [00:07<00:17, 20.51it/s]
    avg reward: 0.89:  24%|██▍       | 113/468 [00:07<00:20, 17.49it/s]
    avg reward: 0.89:  24%|██▍       | 113/468 [00:07<00:20, 17.49it/s]
    avg reward: 0.89:  24%|██▍       | 113/468 [00:07<00:20, 17.49it/s]
    avg reward: 0.89:  24%|██▍       | 113/468 [00:07<00:20, 17.49it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  25%|██▍       | 116/468 [00:07<00:21, 16.23it/s]
    avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 16.96it/s]
    avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 16.96it/s]
    avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 16.96it/s]
    avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 16.96it/s]
    avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 16.96it/s]
    avg reward: 0.89:  27%|██▋       | 125/468 [00:07<00:16, 20.23it/s]
    avg reward: 0.90:  27%|██▋       | 125/468 [00:07<00:16, 20.23it/s]
    avg reward: 0.90:  27%|██▋       | 125/468 [00:07<00:16, 20.23it/s]
    avg reward: 0.90:  27%|██▋       | 125/468 [00:07<00:16, 20.23it/s]
    avg reward: 0.90:  27%|██▋       | 128/468 [00:07<00:19, 17.35it/s]
    avg reward: 0.90:  27%|██▋       | 128/468 [00:07<00:19, 17.35it/s]
    avg reward: 0.90:  27%|██▋       | 128/468 [00:07<00:19, 17.35it/s]
    avg reward: 0.91:  27%|██▋       | 128/468 [00:08<00:19, 17.35it/s]
    avg reward: 0.91:  28%|██▊       | 131/468 [00:08<00:20, 16.12it/s]
    avg reward: 0.91:  28%|██▊       | 131/468 [00:08<00:20, 16.12it/s]
    avg reward: 0.91:  28%|██▊       | 131/468 [00:08<00:20, 16.12it/s]
    avg reward: 0.91:  28%|██▊       | 131/468 [00:08<00:20, 16.12it/s]
    avg reward: 0.91:  28%|██▊       | 131/468 [00:08<00:20, 16.12it/s]
    avg reward: 0.91:  29%|██▉       | 135/468 [00:08<00:17, 19.48it/s]
    avg reward: 0.90:  29%|██▉       | 135/468 [00:08<00:17, 19.48it/s]
    avg reward: 0.90:  29%|██▉       | 135/468 [00:08<00:17, 19.48it/s]
    avg reward: 0.90:  29%|██▉       | 135/468 [00:08<00:17, 19.48it/s]
    avg reward: 0.90:  29%|██▉       | 138/468 [00:08<00:19, 16.74it/s]
    avg reward: 0.90:  29%|██▉       | 138/468 [00:08<00:19, 16.74it/s]
    avg reward: 0.90:  29%|██▉       | 138/468 [00:08<00:19, 16.74it/s]
    avg reward: 0.91:  29%|██▉       | 138/468 [00:08<00:19, 16.74it/s]
    avg reward: 0.91:  30%|███       | 141/468 [00:08<00:20, 15.76it/s]
    avg reward: 0.91:  30%|███       | 141/468 [00:08<00:20, 15.76it/s]
    avg reward: 0.91:  30%|███       | 141/468 [00:08<00:20, 15.76it/s]
    avg reward: 0.91:  30%|███       | 141/468 [00:08<00:20, 15.76it/s]
    avg reward: 0.91:  30%|███       | 141/468 [00:08<00:20, 15.76it/s]
    avg reward: 0.91:  31%|███       | 145/468 [00:08<00:16, 19.10it/s]
    avg reward: 0.93:  31%|███       | 145/468 [00:08<00:16, 19.10it/s]
    avg reward: 0.93:  31%|███       | 145/468 [00:08<00:16, 19.10it/s]
    avg reward: 0.93:  31%|███       | 145/468 [00:08<00:16, 19.10it/s]
    avg reward: 0.93:  32%|███▏      | 148/468 [00:08<00:19, 16.67it/s]
    avg reward: 0.93:  32%|███▏      | 148/468 [00:08<00:19, 16.67it/s]
    avg reward: 0.93:  32%|███▏      | 148/468 [00:09<00:19, 16.67it/s]
    avg reward: 0.93:  32%|███▏      | 148/468 [00:09<00:19, 16.67it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  32%|███▏      | 151/468 [00:09<00:20, 15.61it/s]
    avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:19, 16.31it/s]
    avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:19, 16.31it/s]
    avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:19, 16.31it/s]
    avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:19, 16.31it/s]
    avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:19, 16.31it/s]
    avg reward: 0.93:  34%|███▍      | 160/468 [00:09<00:15, 19.78it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 19.78it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 19.78it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 19.78it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.94it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.94it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.94it/s]
    avg reward: 0.95:  35%|███▍      | 163/468 [00:10<00:18, 16.94it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 15.94it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 15.94it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 15.94it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 15.94it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 15.94it/s]
    avg reward: 0.95:  36%|███▋      | 170/468 [00:10<00:15, 19.34it/s]
    avg reward: 0.95:  36%|███▋      | 170/468 [00:10<00:15, 19.34it/s]
    avg reward: 0.95:  36%|███▋      | 170/468 [00:10<00:15, 19.34it/s]
    avg reward: 0.95:  36%|███▋      | 170/468 [00:10<00:15, 19.34it/s]
    avg reward: 0.95:  37%|███▋      | 173/468 [00:10<00:17, 17.17it/s]
    avg reward: 0.95:  37%|███▋      | 173/468 [00:10<00:17, 17.17it/s]
    avg reward: 0.95:  37%|███▋      | 173/468 [00:10<00:17, 17.17it/s]
    avg reward: 0.95:  37%|███▋      | 173/468 [00:10<00:17, 17.17it/s]
    avg reward: 0.95:  38%|███▊      | 176/468 [00:10<00:17, 16.28it/s]
    avg reward: 0.95:  38%|███▊      | 176/468 [00:10<00:17, 16.28it/s]
    avg reward: 0.95:  38%|███▊      | 176/468 [00:10<00:17, 16.28it/s]
    avg reward: 0.95:  38%|███▊      | 176/468 [00:10<00:17, 16.28it/s]
    avg reward: 0.95:  38%|███▊      | 176/468 [00:10<00:17, 16.28it/s]
    avg reward: 0.95:  38%|███▊      | 180/468 [00:10<00:14, 19.40it/s]
    avg reward: 0.95:  38%|███▊      | 180/468 [00:10<00:14, 19.40it/s]
    avg reward: 0.95:  38%|███▊      | 180/468 [00:10<00:14, 19.40it/s]
    avg reward: 0.95:  38%|███▊      | 180/468 [00:10<00:14, 19.40it/s]
    avg reward: 0.95:  39%|███▉      | 183/468 [00:10<00:16, 17.01it/s]
    avg reward: 0.95:  39%|███▉      | 183/468 [00:10<00:16, 17.01it/s]
    avg reward: 0.95:  39%|███▉      | 183/468 [00:10<00:16, 17.01it/s]
    avg reward: 0.96:  39%|███▉      | 183/468 [00:11<00:16, 17.01it/s]
    avg reward: 0.96:  40%|███▉      | 186/468 [00:11<00:18, 15.56it/s]
    avg reward: 0.96:  40%|███▉      | 186/468 [00:11<00:18, 15.56it/s]
    avg reward: 0.96:  40%|███▉      | 186/468 [00:11<00:18, 15.56it/s]
    avg reward: 0.96:  40%|███▉      | 186/468 [00:11<00:18, 15.56it/s]
    avg reward: 0.96:  40%|███▉      | 186/468 [00:11<00:18, 15.56it/s]
    avg reward: 0.96:  41%|████      | 190/468 [00:11<00:14, 18.93it/s]
    avg reward: 0.96:  41%|████      | 190/468 [00:11<00:14, 18.93it/s]
    avg reward: 0.96:  41%|████      | 190/468 [00:11<00:14, 18.93it/s]
    avg reward: 0.96:  41%|████      | 190/468 [00:11<00:14, 18.93it/s]
    avg reward: 0.96:  41%|████      | 193/468 [00:11<00:16, 16.54it/s]
    avg reward: 0.96:  41%|████      | 193/468 [00:11<00:16, 16.54it/s]
    avg reward: 0.96:  41%|████      | 193/468 [00:11<00:16, 16.54it/s]
    avg reward: 0.96:  41%|████      | 193/468 [00:11<00:16, 16.54it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:17, 15.70it/s]
    avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:16, 16.43it/s]
    avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:16, 16.43it/s]
    avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:16, 16.43it/s]
    avg reward: 0.95:  43%|████▎     | 201/468 [00:12<00:16, 16.43it/s]
    avg reward: 0.95:  43%|████▎     | 201/468 [00:12<00:16, 16.43it/s]
    avg reward: 0.95:  44%|████▍     | 205/468 [00:12<00:13, 19.72it/s]
    avg reward: 0.96:  44%|████▍     | 205/468 [00:12<00:13, 19.72it/s]
    avg reward: 0.96:  44%|████▍     | 205/468 [00:12<00:13, 19.72it/s]
    avg reward: 0.96:  44%|████▍     | 205/468 [00:12<00:13, 19.72it/s]
    avg reward: 0.96:  44%|████▍     | 208/468 [00:12<00:14, 17.51it/s]
    avg reward: 0.96:  44%|████▍     | 208/468 [00:12<00:14, 17.51it/s]
    avg reward: 0.96:  44%|████▍     | 208/468 [00:12<00:14, 17.51it/s]
    avg reward: 0.95:  44%|████▍     | 208/468 [00:12<00:14, 17.51it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  45%|████▌     | 211/468 [00:12<00:22, 11.37it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  46%|████▌     | 216/468 [00:13<00:19, 12.96it/s]
    avg reward: 0.95:  47%|████▋     | 221/468 [00:13<00:17, 14.29it/s]
    avg reward: 0.95:  47%|████▋     | 221/468 [00:13<00:17, 14.29it/s]
    avg reward: 0.95:  47%|████▋     | 221/468 [00:13<00:17, 14.29it/s]
    avg reward: 0.95:  47%|████▋     | 221/468 [00:13<00:17, 14.29it/s]
    avg reward: 0.95:  47%|████▋     | 221/468 [00:13<00:17, 14.29it/s]
    avg reward: 0.95:  48%|████▊     | 225/468 [00:13<00:13, 17.60it/s]
    avg reward: 0.95:  48%|████▊     | 225/468 [00:13<00:13, 17.60it/s]
    avg reward: 0.95:  48%|████▊     | 225/468 [00:13<00:13, 17.60it/s]
    avg reward: 0.95:  48%|████▊     | 225/468 [00:13<00:13, 17.60it/s]
    avg reward: 0.95:  49%|████▊     | 228/468 [00:13<00:15, 15.67it/s]
    avg reward: 0.95:  49%|████▊     | 228/468 [00:13<00:15, 15.67it/s]
    avg reward: 0.95:  49%|████▊     | 228/468 [00:13<00:15, 15.67it/s]
    avg reward: 0.96:  49%|████▊     | 228/468 [00:13<00:15, 15.67it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:16, 14.75it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:16, 14.75it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:16, 14.75it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:16, 14.75it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:16, 14.75it/s]
    avg reward: 0.96:  49%|████▉     | 231/468 [00:14<00:16, 14.75it/s]
    avg reward: 0.96:  50%|█████     | 236/468 [00:14<00:14, 15.80it/s]
    avg reward: 0.96:  50%|█████     | 236/468 [00:14<00:14, 15.80it/s]
    avg reward: 0.96:  50%|█████     | 236/468 [00:14<00:14, 15.80it/s]
    avg reward: 0.96:  50%|█████     | 236/468 [00:14<00:14, 15.80it/s]
    avg reward: 0.96:  50%|█████     | 236/468 [00:14<00:14, 15.80it/s]
    avg reward: 0.96:  51%|█████▏    | 240/468 [00:14<00:11, 19.18it/s]
    avg reward: 0.96:  51%|█████▏    | 240/468 [00:14<00:11, 19.18it/s]
    avg reward: 0.96:  51%|█████▏    | 240/468 [00:14<00:11, 19.18it/s]
    avg reward: 0.96:  51%|█████▏    | 240/468 [00:14<00:11, 19.18it/s]
    avg reward: 0.96:  52%|█████▏    | 243/468 [00:14<00:13, 16.61it/s]
    avg reward: 0.96:  52%|█████▏    | 243/468 [00:14<00:13, 16.61it/s]
    avg reward: 0.96:  52%|█████▏    | 243/468 [00:14<00:13, 16.61it/s]
    avg reward: 0.97:  52%|█████▏    | 243/468 [00:14<00:13, 16.61it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  53%|█████▎    | 246/468 [00:14<00:14, 15.04it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:14<00:13, 15.80it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:14<00:13, 15.80it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:15<00:13, 15.80it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:15<00:13, 15.80it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:15<00:13, 15.80it/s]
    avg reward: 0.97:  54%|█████▎    | 251/468 [00:15<00:13, 15.80it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  55%|█████▍    | 256/468 [00:15<00:12, 16.77it/s]
    avg reward: 0.97:  56%|█████▌    | 261/468 [00:15<00:11, 17.39it/s]
    avg reward: 0.97:  56%|█████▌    | 261/468 [00:15<00:11, 17.39it/s]
    avg reward: 0.97:  56%|█████▌    | 261/468 [00:15<00:11, 17.39it/s]
    avg reward: 0.97:  56%|█████▌    | 261/468 [00:15<00:11, 17.39it/s]
    avg reward: 0.97:  56%|█████▌    | 261/468 [00:15<00:11, 17.39it/s]
    avg reward: 0.97:  57%|█████▋    | 265/468 [00:15<00:09, 20.69it/s]
    avg reward: 0.97:  57%|█████▋    | 265/468 [00:15<00:09, 20.69it/s]
    avg reward: 0.97:  57%|█████▋    | 265/468 [00:15<00:09, 20.69it/s]
    avg reward: 0.97:  57%|█████▋    | 265/468 [00:15<00:09, 20.69it/s]
    avg reward: 0.97:  57%|█████▋    | 268/468 [00:15<00:11, 17.34it/s]
    avg reward: 0.97:  57%|█████▋    | 268/468 [00:15<00:11, 17.34it/s]
    avg reward: 0.97:  57%|█████▋    | 268/468 [00:15<00:11, 17.34it/s]
    avg reward: 0.97:  57%|█████▋    | 268/468 [00:16<00:11, 17.34it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  58%|█████▊    | 271/468 [00:16<00:12, 16.17it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  59%|█████▉    | 276/468 [00:16<00:11, 16.60it/s]
    avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.96:  60%|██████    | 281/468 [00:16<00:10, 17.32it/s]
    avg reward: 0.96:  61%|██████    | 286/468 [00:16<00:10, 17.87it/s]
    avg reward: 0.96:  61%|██████    | 286/468 [00:16<00:10, 17.87it/s]
    avg reward: 0.96:  61%|██████    | 286/468 [00:16<00:10, 17.87it/s]
    avg reward: 0.96:  61%|██████    | 286/468 [00:16<00:10, 17.87it/s]
    avg reward: 0.96:  61%|██████    | 286/468 [00:16<00:10, 17.87it/s]
    avg reward: 0.97:  61%|██████    | 286/468 [00:17<00:10, 17.87it/s]
    avg reward: 0.97:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.97:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.97:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.97:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.97:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.98:  62%|██████▏   | 291/468 [00:17<00:09, 18.02it/s]
    avg reward: 0.98:  63%|██████▎   | 296/468 [00:17<00:09, 18.36it/s]
    avg reward: 0.98:  63%|██████▎   | 296/468 [00:17<00:09, 18.36it/s]
    avg reward: 0.98:  63%|██████▎   | 296/468 [00:17<00:09, 18.36it/s]
    avg reward: 0.98:  63%|██████▎   | 296/468 [00:17<00:09, 18.36it/s]
    avg reward: 0.98:  63%|██████▎   | 296/468 [00:17<00:09, 18.36it/s]
    avg reward: 0.98:  64%|██████▍   | 300/468 [00:17<00:07, 21.77it/s]
    avg reward: 0.98:  64%|██████▍   | 300/468 [00:17<00:07, 21.77it/s]
    avg reward: 0.98:  64%|██████▍   | 300/468 [00:17<00:07, 21.77it/s]
    avg reward: 0.98:  64%|██████▍   | 300/468 [00:17<00:07, 21.77it/s]
    avg reward: 0.98:  65%|██████▍   | 303/468 [00:17<00:09, 17.45it/s]
    avg reward: 0.98:  65%|██████▍   | 303/468 [00:17<00:09, 17.45it/s]
    avg reward: 0.98:  65%|██████▍   | 303/468 [00:17<00:09, 17.45it/s]
    avg reward: 0.99:  65%|██████▍   | 303/468 [00:17<00:09, 17.45it/s]
    avg reward: 0.99:  65%|██████▌   | 306/468 [00:17<00:10, 15.94it/s]
    avg reward: 0.99:  65%|██████▌   | 306/468 [00:17<00:10, 15.94it/s]
    avg reward: 0.99:  65%|██████▌   | 306/468 [00:18<00:10, 15.94it/s]
    avg reward: 0.99:  65%|██████▌   | 306/468 [00:18<00:10, 15.94it/s]
    avg reward: 0.99:  65%|██████▌   | 306/468 [00:18<00:10, 15.94it/s]
    avg reward: 0.99:  66%|██████▌   | 310/468 [00:18<00:08, 19.35it/s]
    avg reward: 0.99:  66%|██████▌   | 310/468 [00:18<00:08, 19.35it/s]
    avg reward: 0.99:  66%|██████▌   | 310/468 [00:18<00:08, 19.35it/s]
    avg reward: 0.99:  66%|██████▌   | 310/468 [00:18<00:08, 19.35it/s]
    avg reward: 0.99:  67%|██████▋   | 313/468 [00:18<00:08, 17.32it/s]
    avg reward: 0.99:  67%|██████▋   | 313/468 [00:18<00:08, 17.32it/s]
    avg reward: 0.99:  67%|██████▋   | 313/468 [00:18<00:08, 17.32it/s]
    avg reward: 0.98:  67%|██████▋   | 313/468 [00:18<00:08, 17.32it/s]
    avg reward: 0.98:  68%|██████▊   | 316/468 [00:18<00:09, 16.00it/s]
    avg reward: 0.98:  68%|██████▊   | 316/468 [00:18<00:09, 16.00it/s]
    avg reward: 0.98:  68%|██████▊   | 316/468 [00:18<00:09, 16.00it/s]
    avg reward: 0.98:  68%|██████▊   | 316/468 [00:18<00:09, 16.00it/s]
    avg reward: 0.98:  68%|██████▊   | 316/468 [00:18<00:09, 16.00it/s]
    avg reward: 0.98:  68%|██████▊   | 320/468 [00:18<00:07, 19.10it/s]
    avg reward: 0.99:  68%|██████▊   | 320/468 [00:18<00:07, 19.10it/s]
    avg reward: 0.99:  68%|██████▊   | 320/468 [00:18<00:07, 19.10it/s]
    avg reward: 0.99:  68%|██████▊   | 320/468 [00:18<00:07, 19.10it/s]
    avg reward: 0.99:  69%|██████▉   | 323/468 [00:18<00:08, 16.79it/s]
    avg reward: 0.99:  69%|██████▉   | 323/468 [00:18<00:08, 16.79it/s]
    avg reward: 0.99:  69%|██████▉   | 323/468 [00:18<00:08, 16.79it/s]
    avg reward: 0.99:  69%|██████▉   | 323/468 [00:19<00:08, 16.79it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  70%|██████▉   | 326/468 [00:19<00:08, 15.84it/s]
    avg reward: 0.99:  71%|███████   | 331/468 [00:19<00:08, 16.52it/s]
    avg reward: 0.99:  71%|███████   | 331/468 [00:19<00:08, 16.52it/s]
    avg reward: 0.99:  71%|███████   | 331/468 [00:19<00:08, 16.52it/s]
    avg reward: 0.99:  71%|███████   | 331/468 [00:19<00:08, 16.52it/s]
    avg reward: 0.99:  71%|███████   | 331/468 [00:19<00:08, 16.52it/s]
    avg reward: 0.99:  72%|███████▏  | 335/468 [00:19<00:06, 19.96it/s]
    avg reward: 0.99:  72%|███████▏  | 335/468 [00:19<00:06, 19.96it/s]
    avg reward: 0.99:  72%|███████▏  | 335/468 [00:19<00:06, 19.96it/s]
    avg reward: 0.99:  72%|███████▏  | 335/468 [00:19<00:06, 19.96it/s]
    avg reward: 0.99:  72%|███████▏  | 338/468 [00:19<00:07, 16.99it/s]
    avg reward: 0.99:  72%|███████▏  | 338/468 [00:19<00:07, 16.99it/s]
    avg reward: 0.99:  72%|███████▏  | 338/468 [00:19<00:07, 16.99it/s]
    avg reward: 0.99:  72%|███████▏  | 338/468 [00:19<00:07, 16.99it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:19<00:08, 15.81it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:19<00:08, 15.81it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:19<00:08, 15.81it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:19<00:08, 15.81it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:19<00:08, 15.81it/s]
    avg reward: 0.99:  73%|███████▎  | 341/468 [00:20<00:08, 15.81it/s]
    avg reward: 0.99:  74%|███████▍  | 346/468 [00:20<00:07, 16.54it/s]
    avg reward: 0.99:  74%|███████▍  | 346/468 [00:20<00:07, 16.54it/s]
    avg reward: 0.99:  74%|███████▍  | 346/468 [00:20<00:07, 16.54it/s]
    avg reward: 0.99:  74%|███████▍  | 346/468 [00:20<00:07, 16.54it/s]
    avg reward: 0.99:  74%|███████▍  | 346/468 [00:20<00:07, 16.54it/s]
    avg reward: 0.99:  75%|███████▍  | 350/468 [00:20<00:05, 20.06it/s]
    avg reward: 0.99:  75%|███████▍  | 350/468 [00:20<00:05, 20.06it/s]
    avg reward: 0.99:  75%|███████▍  | 350/468 [00:20<00:05, 20.06it/s]
    avg reward: 0.99:  75%|███████▍  | 350/468 [00:20<00:05, 20.06it/s]
    avg reward: 0.99:  75%|███████▌  | 353/468 [00:20<00:06, 16.74it/s]
    avg reward: 0.99:  75%|███████▌  | 353/468 [00:20<00:06, 16.74it/s]
    avg reward: 0.99:  75%|███████▌  | 353/468 [00:20<00:06, 16.74it/s]
    avg reward: 0.99:  75%|███████▌  | 353/468 [00:20<00:06, 16.74it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:20<00:07, 15.98it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:20<00:07, 15.98it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:20<00:07, 15.98it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:20<00:07, 15.98it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:20<00:07, 15.98it/s]
    avg reward: 0.99:  76%|███████▌  | 356/468 [00:21<00:07, 15.98it/s]
    avg reward: 0.99:  77%|███████▋  | 361/468 [00:21<00:06, 16.50it/s]
    avg reward: 0.99:  77%|███████▋  | 361/468 [00:21<00:06, 16.50it/s]
    avg reward: 0.99:  77%|███████▋  | 361/468 [00:21<00:06, 16.50it/s]
    avg reward: 0.99:  77%|███████▋  | 361/468 [00:21<00:06, 16.50it/s]
    avg reward: 0.99:  77%|███████▋  | 361/468 [00:21<00:06, 16.50it/s]
    avg reward: 0.99:  78%|███████▊  | 365/468 [00:21<00:05, 19.92it/s]
    avg reward: 1.00:  78%|███████▊  | 365/468 [00:21<00:05, 19.92it/s]
    avg reward: 1.00:  78%|███████▊  | 365/468 [00:21<00:05, 19.92it/s]
    avg reward: 1.00:  78%|███████▊  | 365/468 [00:21<00:05, 19.92it/s]
    avg reward: 1.00:  79%|███████▊  | 368/468 [00:21<00:05, 17.23it/s]
    avg reward: 1.00:  79%|███████▊  | 368/468 [00:21<00:05, 17.23it/s]
    avg reward: 1.00:  79%|███████▊  | 368/468 [00:21<00:05, 17.23it/s]
    avg reward: 1.00:  79%|███████▊  | 368/468 [00:21<00:05, 17.23it/s]
    avg reward: 1.00:  79%|███████▉  | 371/468 [00:21<00:06, 15.88it/s]
    avg reward: 1.00:  79%|███████▉  | 371/468 [00:21<00:06, 15.88it/s]
    avg reward: 1.00:  79%|███████▉  | 371/468 [00:21<00:06, 15.88it/s]
    avg reward: 1.00:  79%|███████▉  | 371/468 [00:21<00:06, 15.88it/s]
    avg reward: 1.00:  79%|███████▉  | 371/468 [00:21<00:06, 15.88it/s]
    avg reward: 1.00:  80%|████████  | 375/468 [00:21<00:04, 19.27it/s]
    avg reward: 1.00:  80%|████████  | 375/468 [00:21<00:04, 19.27it/s]
    avg reward: 1.00:  80%|████████  | 375/468 [00:21<00:04, 19.27it/s]
    avg reward: 1.00:  80%|████████  | 375/468 [00:21<00:04, 19.27it/s]
    avg reward: 1.00:  81%|████████  | 378/468 [00:21<00:05, 17.01it/s]
    avg reward: 1.00:  81%|████████  | 378/468 [00:21<00:05, 17.01it/s]
    avg reward: 1.00:  81%|████████  | 378/468 [00:21<00:05, 17.01it/s]
    avg reward: 1.00:  81%|████████  | 378/468 [00:22<00:05, 17.01it/s]
    avg reward: 1.00:  81%|████████▏ | 381/468 [00:22<00:05, 15.83it/s]
    avg reward: 1.00:  81%|████████▏ | 381/468 [00:22<00:05, 15.83it/s]
    avg reward: 1.00:  81%|████████▏ | 381/468 [00:22<00:05, 15.83it/s]
    avg reward: 1.00:  81%|████████▏ | 381/468 [00:22<00:05, 15.83it/s]
    avg reward: 1.00:  81%|████████▏ | 381/468 [00:22<00:05, 15.83it/s]
    avg reward: 1.00:  82%|████████▏ | 385/468 [00:22<00:04, 19.25it/s]
    avg reward: 1.01:  82%|████████▏ | 385/468 [00:22<00:04, 19.25it/s]
    avg reward: 1.01:  82%|████████▏ | 385/468 [00:22<00:04, 19.25it/s]
    avg reward: 1.01:  82%|████████▏ | 385/468 [00:22<00:04, 19.25it/s]
    avg reward: 1.01:  83%|████████▎ | 388/468 [00:22<00:04, 17.28it/s]
    avg reward: 1.01:  83%|████████▎ | 388/468 [00:22<00:04, 17.28it/s]
    avg reward: 1.01:  83%|████████▎ | 388/468 [00:22<00:04, 17.28it/s]
    avg reward: 1.01:  83%|████████▎ | 388/468 [00:22<00:04, 17.28it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  84%|████████▎ | 391/468 [00:22<00:04, 16.21it/s]
    avg reward: 1.01:  85%|████████▍ | 396/468 [00:22<00:04, 16.79it/s]
    avg reward: 1.01:  85%|████████▍ | 396/468 [00:22<00:04, 16.79it/s]
    avg reward: 1.01:  85%|████████▍ | 396/468 [00:22<00:04, 16.79it/s]
    avg reward: 1.01:  85%|████████▍ | 396/468 [00:22<00:04, 16.79it/s]
    avg reward: 1.01:  85%|████████▍ | 396/468 [00:23<00:04, 16.79it/s]
    avg reward: 1.01:  85%|████████▌ | 400/468 [00:23<00:03, 20.06it/s]
    avg reward: 1.01:  85%|████████▌ | 400/468 [00:23<00:03, 20.06it/s]
    avg reward: 1.01:  85%|████████▌ | 400/468 [00:23<00:03, 20.06it/s]
    avg reward: 1.01:  85%|████████▌ | 400/468 [00:23<00:03, 20.06it/s]
    avg reward: 1.01:  86%|████████▌ | 403/468 [00:23<00:03, 16.67it/s]
    avg reward: 1.01:  86%|████████▌ | 403/468 [00:23<00:03, 16.67it/s]
    avg reward: 1.01:  86%|████████▌ | 403/468 [00:23<00:03, 16.67it/s]
    avg reward: 1.01:  86%|████████▌ | 403/468 [00:23<00:03, 16.67it/s]
    avg reward: 1.01:  87%|████████▋ | 406/468 [00:23<00:03, 15.64it/s]
    avg reward: 1.01:  87%|████████▋ | 406/468 [00:23<00:03, 15.64it/s]
    avg reward: 1.01:  87%|████████▋ | 406/468 [00:23<00:03, 15.64it/s]
    avg reward: 1.01:  87%|████████▋ | 406/468 [00:23<00:03, 15.64it/s]
    avg reward: 1.01:  87%|████████▋ | 406/468 [00:23<00:03, 15.64it/s]
    avg reward: 1.01:  88%|████████▊ | 410/468 [00:23<00:03, 18.84it/s]
    avg reward: 1.01:  88%|████████▊ | 410/468 [00:23<00:03, 18.84it/s]
    avg reward: 1.01:  88%|████████▊ | 410/468 [00:23<00:03, 18.84it/s]
    avg reward: 1.01:  88%|████████▊ | 410/468 [00:23<00:03, 18.84it/s]
    avg reward: 1.01:  88%|████████▊ | 413/468 [00:23<00:03, 16.75it/s]
    avg reward: 1.01:  88%|████████▊ | 413/468 [00:23<00:03, 16.75it/s]
    avg reward: 1.01:  88%|████████▊ | 413/468 [00:23<00:03, 16.75it/s]
    avg reward: 1.01:  88%|████████▊ | 413/468 [00:24<00:03, 16.75it/s]
    avg reward: 1.01:  89%|████████▉ | 416/468 [00:24<00:03, 15.26it/s]
    avg reward: 1.01:  89%|████████▉ | 416/468 [00:24<00:03, 15.26it/s]
    avg reward: 1.01:  89%|████████▉ | 416/468 [00:24<00:03, 15.26it/s]
    avg reward: 1.01:  89%|████████▉ | 416/468 [00:24<00:03, 15.26it/s]
    avg reward: 1.01:  89%|████████▉ | 416/468 [00:24<00:03, 15.26it/s]
    avg reward: 1.01:  90%|████████▉ | 420/468 [00:24<00:02, 17.37it/s]
    avg reward: 1.01:  90%|████████▉ | 420/468 [00:24<00:02, 17.37it/s]
    avg reward: 1.01:  90%|████████▉ | 420/468 [00:24<00:02, 17.37it/s]
    avg reward: 1.01:  90%|████████▉ | 420/468 [00:24<00:02, 17.37it/s]
    avg reward: 1.01:  90%|█████████ | 423/468 [00:24<00:03, 14.67it/s]
    avg reward: 1.01:  90%|█████████ | 423/468 [00:24<00:03, 14.67it/s]
    avg reward: 1.01:  90%|█████████ | 423/468 [00:24<00:03, 14.67it/s]
    avg reward: 1.01:  90%|█████████ | 423/468 [00:24<00:03, 14.67it/s]
    avg reward: 1.01:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.01:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.01:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.01:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.01:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.02:  91%|█████████ | 426/468 [00:24<00:02, 14.02it/s]
    avg reward: 1.02:  92%|█████████▏| 431/468 [00:24<00:02, 15.36it/s]
    avg reward: 1.02:  92%|█████████▏| 431/468 [00:25<00:02, 15.36it/s]
    avg reward: 1.02:  92%|█████████▏| 431/468 [00:25<00:02, 15.36it/s]
    avg reward: 1.02:  92%|█████████▏| 431/468 [00:25<00:02, 15.36it/s]
    avg reward: 1.02:  92%|█████████▏| 431/468 [00:25<00:02, 15.36it/s]
    avg reward: 1.02:  93%|█████████▎| 435/468 [00:25<00:01, 18.52it/s]
    avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 18.52it/s]
    avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 18.52it/s]
    avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 18.52it/s]
    avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.61it/s]
    avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.61it/s]
    avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.61it/s]
    avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.61it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 14.91it/s]
    avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.04it/s]
    avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.04it/s]
    avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.04it/s]
    avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.04it/s]
    avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.04it/s]
    avg reward: 1.01:  96%|█████████▌| 450/468 [00:25<00:00, 19.39it/s]
    avg reward: 1.02:  96%|█████████▌| 450/468 [00:26<00:00, 19.39it/s]
    avg reward: 1.02:  96%|█████████▌| 450/468 [00:26<00:00, 19.39it/s]
    avg reward: 1.02:  96%|█████████▌| 450/468 [00:26<00:00, 19.39it/s]
    avg reward: 1.02:  97%|█████████▋| 453/468 [00:26<00:00, 17.16it/s]
    avg reward: 1.02:  97%|█████████▋| 453/468 [00:26<00:00, 17.16it/s]
    avg reward: 1.02:  97%|█████████▋| 453/468 [00:26<00:00, 17.16it/s]
    avg reward: 1.01:  97%|█████████▋| 453/468 [00:26<00:00, 17.16it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 15.84it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01:  99%|█████████▊| 461/468 [00:26<00:00, 16.72it/s]
    avg reward: 1.01: 100%|█████████▉| 466/468 [00:26<00:00, 17.39it/s]
    avg reward: 1.01: 100%|█████████▉| 466/468 [00:26<00:00, 17.39it/s]
    avg reward: 1.01: 100%|██████████| 468/468 [00:26<00:00, 17.36it/s]
    
      0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 1.0:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.98828125:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.9869791666666666:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.98828125:   0%|          | 0/79 [00:00<?, ?it/s]        
    Val Acc: 0.9875:   0%|          | 0/79 [00:00<?, ?it/s]    
    Val Acc: 0.98828125:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.98828125:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.9877232142857143:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.9892578125:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]      
    Val Acc: 0.9895833333333334:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.98828125:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]        
    Val Acc: 0.9879261363636364:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.98828125:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]        
    Val Acc: 0.9873798076923077:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.9849330357142857:   8%|▊         | 6/79 [00:00<00:01, 59.27it/s]
    Val Acc: 0.9849330357142857:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]
    Val Acc: 0.984375:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]          
    Val Acc: 0.98388671875:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]
    Val Acc: 0.984375:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]     
    Val Acc: 0.9848090277777778:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]
    Val Acc: 0.984375:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]          
    Val Acc: 0.983984375:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]
    Val Acc: 0.9840029761904762:  18%|█▊        | 14/79 [00:00<00:01, 62.36it/s]
    Val Acc: 0.9840029761904762:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9836647727272727:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9836956521739131:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9837239583333334:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.98375:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]           
    Val Acc: 0.9837740384615384:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9837962962962963:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9838169642857143:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9841056034482759:  27%|██▋       | 21/79 [00:00<00:00, 63.46it/s]
    Val Acc: 0.9841056034482759:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.984375:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]          
    Val Acc: 0.983366935483871:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.983154296875:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]   
    Val Acc: 0.982717803030303:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.9829963235294118:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.9823660714285715:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.9826388888888888:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.9826858108108109:  37%|███▋      | 29/79 [00:00<00:00, 64.64it/s]
    Val Acc: 0.9826858108108109:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9829358552631579:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9827724358974359:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9828125:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]         
    Val Acc: 0.9828506097560976:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9830729166666666:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9831031976744186:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9825994318181818:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]
    Val Acc: 0.9828125:  47%|████▋     | 37/79 [00:00<00:00, 65.07it/s]         
    Val Acc: 0.9828125:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9828464673913043:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9825465425531915:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9825846354166666:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9829400510204082:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.98265625:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]        
    Val Acc: 0.9820772058823529:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9822716346153846:  57%|█████▋    | 45/79 [00:00<00:00, 65.35it/s]
    Val Acc: 0.9822716346153846:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9823113207547169:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9820601851851852:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9821022727272727:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9820033482142857:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9817708333333334:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9815463362068966:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9810646186440678:  66%|██████▌   | 52/79 [00:00<00:00, 66.64it/s]
    Val Acc: 0.9810646186440678:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9809895833333333:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9806608606557377:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9805947580645161:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9807787698412699:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9805908203125:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]   
    Val Acc: 0.9807692307692307:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.9807054924242424:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s]
    Val Acc: 0.980643656716418:  75%|███████▍  | 59/79 [00:00<00:00, 66.70it/s] 
    Val Acc: 0.980643656716418:  85%|████████▍ | 67/79 [00:00<00:00, 67.68it/s]
    Val Acc: 0.9805836397058824:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.9807518115942029:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.9809151785714286:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.980743838028169:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s] 
    Val Acc: 0.9807942708333334:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.9810573630136986:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.9811021959459459:  85%|████████▍ | 67/79 [00:01<00:00, 67.68it/s]
    Val Acc: 0.9811021959459459:  94%|█████████▎| 74/79 [00:01<00:00, 67.11it/s]
    Val Acc: 0.9810416666666667:  94%|█████████▎| 74/79 [00:01<00:00, 67.11it/s]
    Val Acc: 0.9810855263157895:  94%|█████████▎| 74/79 [00:01<00:00, 67.11it/s]
    Val Acc: 0.9811282467532467:  94%|█████████▎| 74/79 [00:01<00:00, 67.11it/s]
    Val Acc: 0.981270032051282:  94%|█████████▎| 74/79 [00:01<00:00, 67.11it/s] 
    Val Acc: 0.9813: 100%|██████████| 79/79 [00:01<00:00, 66.31it/s]
    epoch 0, val_acc: 0.98, avg reward: 1.01: 100%|██████████| 1/1 [00:28<00:00, 28.20s/it]


The resulting architecture is:

.. code:: python

    mynet.graph




.. figure:: output_enas_proxylessnas_ad55b9_21_0.svg



**Change the reward trade-off:**

.. code:: python

    reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.8)
    mynet.initialize(force_reinit=True)
    scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                               reward_fn=reward_fn, batch_size=128, num_gpus=1,
                               warmup_epochs=0, epochs=1, controller_lr=3e-3,
                               plot_frequency=10, update_arch_frequency=5)
    scheduler.run()


.. parsed-literal::
    :class: output

    /var/lib/jenkins/miniconda3/envs/autogluon_docs-v0_0_15/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
      and should_run_async(code)
    get_built_in_dataset mnist
    get_built_in_dataset mnist
      0%|          | 0/1 [00:00<?, ?it/s]
      0%|          | 0/468 [00:00<?, ?it/s]
    avg reward: 0.14:   0%|          | 0/468 [00:00<?, ?it/s]
    avg reward: 0.14:   0%|          | 1/468 [00:00<02:27,  3.18it/s]
    avg reward: 0.14:   0%|          | 1/468 [00:00<02:27,  3.18it/s]
    avg reward: 0.14:   0%|          | 1/468 [00:00<02:27,  3.18it/s]
    avg reward: 0.14:   0%|          | 1/468 [00:00<02:27,  3.18it/s]
    avg reward: 0.14:   0%|          | 1/468 [00:00<02:27,  3.18it/s]
    avg reward: 0.14:   1%|          | 5/468 [00:00<01:45,  4.37it/s]
    avg reward: 0.14:   1%|          | 5/468 [00:00<01:45,  4.37it/s]
    avg reward: 0.14:   1%|          | 5/468 [00:00<01:45,  4.37it/s]
    avg reward: 0.14:   1%|▏         | 7/468 [00:00<01:27,  5.27it/s]
    avg reward: 0.14:   1%|▏         | 7/468 [00:00<01:27,  5.27it/s]
    avg reward: 0.14:   1%|▏         | 7/468 [00:00<01:27,  5.27it/s]
    avg reward: 0.14:   1%|▏         | 7/468 [00:00<01:27,  5.27it/s]
    avg reward: 0.13:   1%|▏         | 7/468 [00:00<01:27,  5.27it/s]
    avg reward: 0.13:   2%|▏         | 11/468 [00:00<01:09,  6.54it/s]
    avg reward: 0.13:   2%|▏         | 11/468 [00:00<01:09,  6.54it/s]
    avg reward: 0.13:   2%|▏         | 11/468 [00:00<01:09,  6.54it/s]
    avg reward: 0.13:   2%|▏         | 11/468 [00:00<01:09,  6.54it/s]
    avg reward: 0.13:   3%|▎         | 14/468 [00:00<00:53,  8.50it/s]
    avg reward: 0.13:   3%|▎         | 14/468 [00:01<00:53,  8.50it/s]
    avg reward: 0.12:   3%|▎         | 14/468 [00:01<00:53,  8.50it/s]
    avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:51,  8.76it/s]
    avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:51,  8.76it/s]
    avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:51,  8.76it/s]
    avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:51,  8.76it/s]
    avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:51,  8.76it/s]
    avg reward: 0.12:   4%|▍         | 20/468 [00:01<00:39, 11.41it/s]
    avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:39, 11.41it/s]
    avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:39, 11.41it/s]
    avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:39, 11.41it/s]
    avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:37, 12.02it/s]
    avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:37, 12.02it/s]
    avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:37, 12.02it/s]
    avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:37, 12.02it/s]
    avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:35, 12.52it/s]
    avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:35, 12.52it/s]
    avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:35, 12.52it/s]
    avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:35, 12.52it/s]
    avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:35, 12.52it/s]
    avg reward: 0.13:   6%|▋         | 30/468 [00:01<00:27, 15.70it/s]
    avg reward: 0.16:   6%|▋         | 30/468 [00:02<00:27, 15.70it/s]
    avg reward: 0.16:   6%|▋         | 30/468 [00:02<00:27, 15.70it/s]
    avg reward: 0.16:   6%|▋         | 30/468 [00:02<00:27, 15.70it/s]
    avg reward: 0.16:   7%|▋         | 33/468 [00:02<00:29, 14.84it/s]
    avg reward: 0.16:   7%|▋         | 33/468 [00:02<00:29, 14.84it/s]
    avg reward: 0.16:   7%|▋         | 33/468 [00:02<00:29, 14.84it/s]
    avg reward: 0.26:   7%|▋         | 33/468 [00:02<00:29, 14.84it/s]
    avg reward: 0.26:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.26:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.26:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.26:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.26:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.44:   8%|▊         | 36/468 [00:02<00:29, 14.48it/s]
    avg reward: 0.44:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.44:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.44:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.44:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.44:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.55:   9%|▉         | 41/468 [00:02<00:27, 15.46it/s]
    avg reward: 0.55:  10%|▉         | 46/468 [00:02<00:25, 16.40it/s]
    avg reward: 0.55:  10%|▉         | 46/468 [00:02<00:25, 16.40it/s]
    avg reward: 0.55:  10%|▉         | 46/468 [00:02<00:25, 16.40it/s]
    avg reward: 0.55:  10%|▉         | 46/468 [00:02<00:25, 16.40it/s]
    avg reward: 0.55:  10%|▉         | 46/468 [00:02<00:25, 16.40it/s]
    avg reward: 0.55:  11%|█         | 50/468 [00:02<00:21, 19.46it/s]
    avg reward: 0.61:  11%|█         | 50/468 [00:03<00:21, 19.46it/s]
    avg reward: 0.61:  11%|█         | 50/468 [00:03<00:21, 19.46it/s]
    avg reward: 0.61:  11%|█         | 50/468 [00:03<00:21, 19.46it/s]
    avg reward: 0.61:  11%|█▏        | 53/468 [00:03<00:24, 17.26it/s]
    avg reward: 0.61:  11%|█▏        | 53/468 [00:03<00:24, 17.26it/s]
    avg reward: 0.61:  11%|█▏        | 53/468 [00:03<00:24, 17.26it/s]
    avg reward: 0.66:  11%|█▏        | 53/468 [00:03<00:24, 17.26it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  12%|█▏        | 56/468 [00:03<00:26, 15.54it/s]
    avg reward: 0.66:  13%|█▎        | 61/468 [00:03<00:25, 16.14it/s]
    avg reward: 0.66:  13%|█▎        | 61/468 [00:03<00:25, 16.14it/s]
    avg reward: 0.66:  13%|█▎        | 61/468 [00:03<00:25, 16.14it/s]
    avg reward: 0.66:  13%|█▎        | 61/468 [00:03<00:25, 16.14it/s]
    avg reward: 0.66:  13%|█▎        | 61/468 [00:03<00:25, 16.14it/s]
    avg reward: 0.66:  14%|█▍        | 65/468 [00:03<00:20, 19.49it/s]
    avg reward: 0.72:  14%|█▍        | 65/468 [00:03<00:20, 19.49it/s]
    avg reward: 0.72:  14%|█▍        | 65/468 [00:04<00:20, 19.49it/s]
    avg reward: 0.72:  14%|█▍        | 65/468 [00:04<00:20, 19.49it/s]
    avg reward: 0.72:  15%|█▍        | 68/468 [00:04<00:23, 17.18it/s]
    avg reward: 0.72:  15%|█▍        | 68/468 [00:04<00:23, 17.18it/s]
    avg reward: 0.72:  15%|█▍        | 68/468 [00:04<00:23, 17.18it/s]
    avg reward: 0.72:  15%|█▍        | 68/468 [00:04<00:23, 17.18it/s]
    avg reward: 0.72:  15%|█▌        | 71/468 [00:04<00:25, 15.79it/s]
    avg reward: 0.72:  15%|█▌        | 71/468 [00:04<00:25, 15.79it/s]
    avg reward: 0.72:  15%|█▌        | 71/468 [00:04<00:25, 15.79it/s]
    avg reward: 0.72:  15%|█▌        | 71/468 [00:04<00:25, 15.79it/s]
    avg reward: 0.72:  15%|█▌        | 71/468 [00:04<00:25, 15.79it/s]
    avg reward: 0.72:  16%|█▌        | 75/468 [00:04<00:20, 19.22it/s]
    avg reward: 0.78:  16%|█▌        | 75/468 [00:04<00:20, 19.22it/s]
    avg reward: 0.78:  16%|█▌        | 75/468 [00:04<00:20, 19.22it/s]
    avg reward: 0.78:  16%|█▌        | 75/468 [00:04<00:20, 19.22it/s]
    avg reward: 0.78:  17%|█▋        | 78/468 [00:04<00:23, 16.59it/s]
    avg reward: 0.78:  17%|█▋        | 78/468 [00:04<00:23, 16.59it/s]
    avg reward: 0.78:  17%|█▋        | 78/468 [00:04<00:23, 16.59it/s]
    avg reward: 0.83:  17%|█▋        | 78/468 [00:04<00:23, 16.59it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:04<00:25, 15.37it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:04<00:25, 15.37it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:04<00:25, 15.37it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:04<00:25, 15.37it/s]
    avg reward: 0.83:  17%|█▋        | 81/468 [00:04<00:25, 15.37it/s]
    avg reward: 0.83:  18%|█▊        | 85/468 [00:04<00:20, 18.57it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:20, 18.57it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:20, 18.57it/s]
    avg reward: 0.86:  18%|█▊        | 85/468 [00:05<00:20, 18.57it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 16.14it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 16.14it/s]
    avg reward: 0.86:  19%|█▉        | 88/468 [00:05<00:23, 16.14it/s]
    avg reward: 0.87:  19%|█▉        | 88/468 [00:05<00:23, 16.14it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.87:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.86:  19%|█▉        | 91/468 [00:05<00:24, 15.24it/s]
    avg reward: 0.86:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.86:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.86:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.86:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.86:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.87:  21%|██        | 96/468 [00:05<00:22, 16.18it/s]
    avg reward: 0.87:  22%|██▏       | 101/468 [00:05<00:22, 16.48it/s]
    avg reward: 0.87:  22%|██▏       | 101/468 [00:05<00:22, 16.48it/s]
    avg reward: 0.87:  22%|██▏       | 101/468 [00:05<00:22, 16.48it/s]
    avg reward: 0.87:  22%|██▏       | 101/468 [00:06<00:22, 16.48it/s]
    avg reward: 0.87:  22%|██▏       | 101/468 [00:06<00:22, 16.48it/s]
    avg reward: 0.90:  22%|██▏       | 101/468 [00:06<00:22, 16.48it/s]
    avg reward: 0.90:  23%|██▎       | 106/468 [00:06<00:21, 16.91it/s]
    avg reward: 0.90:  23%|██▎       | 106/468 [00:06<00:21, 16.91it/s]
    avg reward: 0.90:  23%|██▎       | 106/468 [00:06<00:21, 16.91it/s]
    avg reward: 0.90:  23%|██▎       | 106/468 [00:06<00:21, 16.91it/s]
    avg reward: 0.90:  23%|██▎       | 106/468 [00:06<00:21, 16.91it/s]
    avg reward: 0.90:  24%|██▎       | 110/468 [00:06<00:17, 20.14it/s]
    avg reward: 0.92:  24%|██▎       | 110/468 [00:06<00:17, 20.14it/s]
    avg reward: 0.92:  24%|██▎       | 110/468 [00:06<00:17, 20.14it/s]
    avg reward: 0.92:  24%|██▎       | 110/468 [00:06<00:17, 20.14it/s]
    avg reward: 0.92:  24%|██▍       | 113/468 [00:06<00:20, 17.18it/s]
    avg reward: 0.92:  24%|██▍       | 113/468 [00:06<00:20, 17.18it/s]
    avg reward: 0.92:  24%|██▍       | 113/468 [00:06<00:20, 17.18it/s]
    avg reward: 0.93:  24%|██▍       | 113/468 [00:06<00:20, 17.18it/s]
    avg reward: 0.93:  25%|██▍       | 116/468 [00:06<00:21, 16.11it/s]
    avg reward: 0.93:  25%|██▍       | 116/468 [00:06<00:21, 16.11it/s]
    avg reward: 0.93:  25%|██▍       | 116/468 [00:06<00:21, 16.11it/s]
    avg reward: 0.93:  25%|██▍       | 116/468 [00:06<00:21, 16.11it/s]
    avg reward: 0.93:  25%|██▍       | 116/468 [00:06<00:21, 16.11it/s]
    avg reward: 0.90:  25%|██▍       | 116/468 [00:07<00:21, 16.11it/s]
    avg reward: 0.90:  26%|██▌       | 121/468 [00:07<00:20, 16.70it/s]
    avg reward: 0.90:  26%|██▌       | 121/468 [00:07<00:20, 16.70it/s]
    avg reward: 0.90:  26%|██▌       | 121/468 [00:07<00:20, 16.70it/s]
    avg reward: 0.90:  26%|██▌       | 121/468 [00:07<00:20, 16.70it/s]
    avg reward: 0.90:  26%|██▌       | 121/468 [00:07<00:20, 16.70it/s]
    avg reward: 0.90:  27%|██▋       | 125/468 [00:07<00:17, 19.76it/s]
    avg reward: 0.91:  27%|██▋       | 125/468 [00:07<00:17, 19.76it/s]
    avg reward: 0.91:  27%|██▋       | 125/468 [00:07<00:17, 19.76it/s]
    avg reward: 0.91:  27%|██▋       | 125/468 [00:07<00:17, 19.76it/s]
    avg reward: 0.91:  27%|██▋       | 128/468 [00:07<00:20, 16.31it/s]
    avg reward: 0.91:  27%|██▋       | 128/468 [00:07<00:20, 16.31it/s]
    avg reward: 0.91:  27%|██▋       | 128/468 [00:07<00:20, 16.31it/s]
    avg reward: 0.89:  27%|██▋       | 128/468 [00:07<00:20, 16.31it/s]
    avg reward: 0.89:  28%|██▊       | 131/468 [00:07<00:22, 14.98it/s]
    avg reward: 0.89:  28%|██▊       | 131/468 [00:07<00:22, 14.98it/s]
    avg reward: 0.89:  28%|██▊       | 131/468 [00:07<00:22, 14.98it/s]
    avg reward: 0.89:  28%|██▊       | 131/468 [00:07<00:22, 14.98it/s]
    avg reward: 0.89:  28%|██▊       | 131/468 [00:07<00:22, 14.98it/s]
    avg reward: 0.89:  29%|██▉       | 135/468 [00:07<00:18, 18.28it/s]
    avg reward: 0.92:  29%|██▉       | 135/468 [00:07<00:18, 18.28it/s]
    avg reward: 0.92:  29%|██▉       | 135/468 [00:07<00:18, 18.28it/s]
    avg reward: 0.92:  29%|██▉       | 135/468 [00:07<00:18, 18.28it/s]
    avg reward: 0.92:  29%|██▉       | 138/468 [00:08<00:20, 16.38it/s]
    avg reward: 0.92:  29%|██▉       | 138/468 [00:08<00:20, 16.38it/s]
    avg reward: 0.92:  29%|██▉       | 138/468 [00:08<00:20, 16.38it/s]
    avg reward: 0.95:  29%|██▉       | 138/468 [00:08<00:20, 16.38it/s]
    avg reward: 0.95:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.95:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.95:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.95:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.95:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.96:  30%|███       | 141/468 [00:08<00:21, 14.99it/s]
    avg reward: 0.96:  31%|███       | 146/468 [00:08<00:20, 15.74it/s]
    avg reward: 0.96:  31%|███       | 146/468 [00:08<00:20, 15.74it/s]
    avg reward: 0.96:  31%|███       | 146/468 [00:08<00:20, 15.74it/s]
    avg reward: 0.96:  31%|███       | 146/468 [00:08<00:20, 15.74it/s]
    avg reward: 0.96:  31%|███       | 146/468 [00:08<00:20, 15.74it/s]
    avg reward: 0.96:  32%|███▏      | 150/468 [00:08<00:16, 18.84it/s]
    avg reward: 0.95:  32%|███▏      | 150/468 [00:08<00:16, 18.84it/s]
    avg reward: 0.95:  32%|███▏      | 150/468 [00:08<00:16, 18.84it/s]
    avg reward: 0.95:  32%|███▏      | 150/468 [00:08<00:16, 18.84it/s]
    avg reward: 0.95:  33%|███▎      | 153/468 [00:08<00:19, 16.55it/s]
    avg reward: 0.95:  33%|███▎      | 153/468 [00:08<00:19, 16.55it/s]
    avg reward: 0.95:  33%|███▎      | 153/468 [00:08<00:19, 16.55it/s]
    avg reward: 0.94:  33%|███▎      | 153/468 [00:09<00:19, 16.55it/s]
    avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:20, 15.58it/s]
    avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:20, 15.58it/s]
    avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:20, 15.58it/s]
    avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:20, 15.58it/s]
    avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:20, 15.58it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:16, 19.04it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:16, 19.04it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:16, 19.04it/s]
    avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:16, 19.04it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.45it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.45it/s]
    avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:18, 16.45it/s]
    avg reward: 0.95:  35%|███▍      | 163/468 [00:09<00:18, 16.45it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:09<00:19, 15.31it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:09<00:19, 15.31it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:09<00:19, 15.31it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:09<00:19, 15.31it/s]
    avg reward: 0.95:  35%|███▌      | 166/468 [00:09<00:19, 15.31it/s]
    avg reward: 0.95:  36%|███▋      | 170/468 [00:09<00:15, 18.73it/s]
    avg reward: 0.97:  36%|███▋      | 170/468 [00:09<00:15, 18.73it/s]
    avg reward: 0.97:  36%|███▋      | 170/468 [00:09<00:15, 18.73it/s]
    avg reward: 0.97:  36%|███▋      | 170/468 [00:09<00:15, 18.73it/s]
    avg reward: 0.97:  37%|███▋      | 173/468 [00:09<00:18, 16.39it/s]
    avg reward: 0.97:  37%|███▋      | 173/468 [00:10<00:18, 16.39it/s]
    avg reward: 0.97:  37%|███▋      | 173/468 [00:10<00:18, 16.39it/s]
    avg reward: 1.01:  37%|███▋      | 173/468 [00:10<00:18, 16.39it/s]
    avg reward: 1.01:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 1.01:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 1.01:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 1.01:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 1.01:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 0.98:  38%|███▊      | 176/468 [00:10<00:18, 15.38it/s]
    avg reward: 0.98:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.98:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.98:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.98:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.98:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.97:  39%|███▊      | 181/468 [00:10<00:17, 16.07it/s]
    avg reward: 0.97:  40%|███▉      | 186/468 [00:10<00:17, 16.52it/s]
    avg reward: 0.97:  40%|███▉      | 186/468 [00:10<00:17, 16.52it/s]
    avg reward: 0.97:  40%|███▉      | 186/468 [00:10<00:17, 16.52it/s]
    avg reward: 0.97:  40%|███▉      | 186/468 [00:10<00:17, 16.52it/s]
    avg reward: 0.97:  40%|███▉      | 186/468 [00:10<00:17, 16.52it/s]
    avg reward: 0.97:  41%|████      | 190/468 [00:10<00:13, 19.95it/s]
    avg reward: 0.97:  41%|████      | 190/468 [00:11<00:13, 19.95it/s]
    avg reward: 0.97:  41%|████      | 190/468 [00:11<00:13, 19.95it/s]
    avg reward: 0.97:  41%|████      | 190/468 [00:11<00:13, 19.95it/s]
    avg reward: 0.97:  41%|████      | 193/468 [00:11<00:15, 17.22it/s]
    avg reward: 0.97:  41%|████      | 193/468 [00:11<00:15, 17.22it/s]
    avg reward: 0.97:  41%|████      | 193/468 [00:11<00:15, 17.22it/s]
    avg reward: 0.96:  41%|████      | 193/468 [00:11<00:15, 17.22it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:16, 16.28it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:16, 16.28it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:16, 16.28it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:16, 16.28it/s]
    avg reward: 0.96:  42%|████▏     | 196/468 [00:11<00:16, 16.28it/s]
    avg reward: 0.96:  43%|████▎     | 200/468 [00:11<00:13, 19.56it/s]
    avg reward: 0.97:  43%|████▎     | 200/468 [00:11<00:13, 19.56it/s]
    avg reward: 0.97:  43%|████▎     | 200/468 [00:11<00:13, 19.56it/s]
    avg reward: 0.97:  43%|████▎     | 200/468 [00:11<00:13, 19.56it/s]
    avg reward: 0.97:  43%|████▎     | 203/468 [00:11<00:15, 16.97it/s]
    avg reward: 0.97:  43%|████▎     | 203/468 [00:11<00:15, 16.97it/s]
    avg reward: 0.97:  43%|████▎     | 203/468 [00:11<00:15, 16.97it/s]
    avg reward: 0.97:  43%|████▎     | 203/468 [00:11<00:15, 16.97it/s]
    avg reward: 0.97:  44%|████▍     | 206/468 [00:11<00:16, 15.77it/s]
    avg reward: 0.97:  44%|████▍     | 206/468 [00:11<00:16, 15.77it/s]
    avg reward: 0.97:  44%|████▍     | 206/468 [00:11<00:16, 15.77it/s]
    avg reward: 0.97:  44%|████▍     | 206/468 [00:11<00:16, 15.77it/s]
    avg reward: 0.97:  44%|████▍     | 206/468 [00:11<00:16, 15.77it/s]
    avg reward: 0.97:  45%|████▍     | 210/468 [00:11<00:13, 18.95it/s]
    avg reward: 0.97:  45%|████▍     | 210/468 [00:12<00:13, 18.95it/s]
    avg reward: 0.97:  45%|████▍     | 210/468 [00:12<00:13, 18.95it/s]
    avg reward: 0.97:  45%|████▍     | 210/468 [00:12<00:13, 18.95it/s]
    avg reward: 0.97:  46%|████▌     | 213/468 [00:12<00:14, 17.12it/s]
    avg reward: 0.97:  46%|████▌     | 213/468 [00:12<00:14, 17.12it/s]
    avg reward: 0.97:  46%|████▌     | 213/468 [00:12<00:14, 17.12it/s]
    avg reward: 0.99:  46%|████▌     | 213/468 [00:12<00:14, 17.12it/s]
    avg reward: 0.99:  46%|████▌     | 216/468 [00:12<00:16, 15.59it/s]
    avg reward: 0.99:  46%|████▌     | 216/468 [00:12<00:16, 15.59it/s]
    avg reward: 0.99:  46%|████▌     | 216/468 [00:12<00:16, 15.59it/s]
    avg reward: 0.99:  46%|████▌     | 216/468 [00:12<00:16, 15.59it/s]
    avg reward: 0.99:  46%|████▌     | 216/468 [00:12<00:16, 15.59it/s]
    avg reward: 0.99:  47%|████▋     | 220/468 [00:12<00:13, 18.93it/s]
    avg reward: 1.00:  47%|████▋     | 220/468 [00:12<00:13, 18.93it/s]
    avg reward: 1.00:  47%|████▋     | 220/468 [00:12<00:13, 18.93it/s]
    avg reward: 1.00:  47%|████▋     | 220/468 [00:12<00:13, 18.93it/s]
    avg reward: 1.00:  48%|████▊     | 223/468 [00:12<00:14, 16.91it/s]
    avg reward: 1.00:  48%|████▊     | 223/468 [00:12<00:14, 16.91it/s]
    avg reward: 1.00:  48%|████▊     | 223/468 [00:12<00:14, 16.91it/s]
    avg reward: 1.00:  48%|████▊     | 223/468 [00:13<00:14, 16.91it/s]
    avg reward: 1.00:  48%|████▊     | 226/468 [00:13<00:15, 15.35it/s]
    avg reward: 1.00:  48%|████▊     | 226/468 [00:13<00:15, 15.35it/s]
    avg reward: 1.00:  48%|████▊     | 226/468 [00:13<00:15, 15.35it/s]
    avg reward: 1.00:  48%|████▊     | 226/468 [00:13<00:15, 15.35it/s]
    avg reward: 1.00:  48%|████▊     | 226/468 [00:13<00:15, 15.35it/s]
    avg reward: 1.00:  49%|████▉     | 230/468 [00:13<00:12, 18.57it/s]
    avg reward: 1.02:  49%|████▉     | 230/468 [00:13<00:12, 18.57it/s]
    avg reward: 1.02:  49%|████▉     | 230/468 [00:13<00:12, 18.57it/s]
    avg reward: 1.02:  49%|████▉     | 230/468 [00:13<00:12, 18.57it/s]
    avg reward: 1.02:  50%|████▉     | 233/468 [00:13<00:14, 16.36it/s]
    avg reward: 1.02:  50%|████▉     | 233/468 [00:13<00:14, 16.36it/s]
    avg reward: 1.02:  50%|████▉     | 233/468 [00:13<00:14, 16.36it/s]
    avg reward: 1.02:  50%|████▉     | 233/468 [00:13<00:14, 16.36it/s]
    avg reward: 1.02:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.02:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.02:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.02:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.02:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.05:  50%|█████     | 236/468 [00:13<00:14, 15.76it/s]
    avg reward: 1.05:  51%|█████▏    | 241/468 [00:13<00:14, 16.14it/s]
    avg reward: 1.05:  51%|█████▏    | 241/468 [00:13<00:14, 16.14it/s]
    avg reward: 1.05:  51%|█████▏    | 241/468 [00:13<00:14, 16.14it/s]
    avg reward: 1.05:  51%|█████▏    | 241/468 [00:13<00:14, 16.14it/s]
    avg reward: 1.05:  51%|█████▏    | 241/468 [00:13<00:14, 16.14it/s]
    avg reward: 1.05:  52%|█████▏    | 245/468 [00:13<00:11, 19.33it/s]
    avg reward: 1.08:  52%|█████▏    | 245/468 [00:14<00:11, 19.33it/s]
    avg reward: 1.08:  52%|█████▏    | 245/468 [00:14<00:11, 19.33it/s]
    avg reward: 1.08:  52%|█████▏    | 245/468 [00:14<00:11, 19.33it/s]
    avg reward: 1.08:  53%|█████▎    | 248/468 [00:14<00:13, 16.41it/s]
    avg reward: 1.08:  53%|█████▎    | 248/468 [00:14<00:13, 16.41it/s]
    avg reward: 1.08:  53%|█████▎    | 248/468 [00:14<00:13, 16.41it/s]
    avg reward: 1.06:  53%|█████▎    | 248/468 [00:14<00:13, 16.41it/s]
    avg reward: 1.06:  54%|█████▎    | 251/468 [00:14<00:14, 15.41it/s]
    avg reward: 1.06:  54%|█████▎    | 251/468 [00:14<00:14, 15.41it/s]
    avg reward: 1.06:  54%|█████▎    | 251/468 [00:14<00:14, 15.41it/s]
    avg reward: 1.06:  54%|█████▎    | 251/468 [00:14<00:14, 15.41it/s]
    avg reward: 1.06:  54%|█████▎    | 251/468 [00:14<00:14, 15.41it/s]
    avg reward: 1.06:  54%|█████▍    | 255/468 [00:14<00:11, 18.69it/s]
    avg reward: 1.08:  54%|█████▍    | 255/468 [00:14<00:11, 18.69it/s]
    avg reward: 1.08:  54%|█████▍    | 255/468 [00:14<00:11, 18.69it/s]
    avg reward: 1.08:  54%|█████▍    | 255/468 [00:14<00:11, 18.69it/s]
    avg reward: 1.08:  55%|█████▌    | 258/468 [00:14<00:12, 16.39it/s]
    avg reward: 1.08:  55%|█████▌    | 258/468 [00:14<00:12, 16.39it/s]
    avg reward: 1.08:  55%|█████▌    | 258/468 [00:14<00:12, 16.39it/s]
    avg reward: 1.10:  55%|█████▌    | 258/468 [00:15<00:12, 16.39it/s]
    avg reward: 1.10:  56%|█████▌    | 261/468 [00:15<00:13, 15.03it/s]
    avg reward: 1.10:  56%|█████▌    | 261/468 [00:15<00:13, 15.03it/s]
    avg reward: 1.10:  56%|█████▌    | 261/468 [00:15<00:13, 15.03it/s]
    avg reward: 1.10:  56%|█████▌    | 261/468 [00:15<00:13, 15.03it/s]
    avg reward: 1.10:  56%|█████▌    | 261/468 [00:15<00:13, 15.03it/s]
    avg reward: 1.10:  57%|█████▋    | 265/468 [00:15<00:11, 18.30it/s]
    avg reward: 1.13:  57%|█████▋    | 265/468 [00:15<00:11, 18.30it/s]
    avg reward: 1.13:  57%|█████▋    | 265/468 [00:15<00:11, 18.30it/s]
    avg reward: 1.13:  57%|█████▋    | 265/468 [00:15<00:11, 18.30it/s]
    avg reward: 1.13:  57%|█████▋    | 268/468 [00:15<00:12, 16.04it/s]
    avg reward: 1.13:  57%|█████▋    | 268/468 [00:15<00:12, 16.04it/s]
    avg reward: 1.13:  57%|█████▋    | 268/468 [00:15<00:12, 16.04it/s]
    avg reward: 1.16:  57%|█████▋    | 268/468 [00:15<00:12, 16.04it/s]
    avg reward: 1.16:  58%|█████▊    | 271/468 [00:15<00:13, 15.01it/s]
    avg reward: 1.16:  58%|█████▊    | 271/468 [00:15<00:13, 15.01it/s]
    avg reward: 1.16:  58%|█████▊    | 271/468 [00:15<00:13, 15.01it/s]
    avg reward: 1.16:  58%|█████▊    | 271/468 [00:15<00:13, 15.01it/s]
    avg reward: 1.16:  58%|█████▊    | 271/468 [00:15<00:13, 15.01it/s]
    avg reward: 1.16:  59%|█████▉    | 275/468 [00:15<00:10, 18.39it/s]
    avg reward: 1.18:  59%|█████▉    | 275/468 [00:15<00:10, 18.39it/s]
    avg reward: 1.18:  59%|█████▉    | 275/468 [00:15<00:10, 18.39it/s]
    avg reward: 1.18:  59%|█████▉    | 275/468 [00:15<00:10, 18.39it/s]
    avg reward: 1.18:  59%|█████▉    | 278/468 [00:15<00:11, 15.91it/s]
    avg reward: 1.18:  59%|█████▉    | 278/468 [00:15<00:11, 15.91it/s]
    avg reward: 1.18:  59%|█████▉    | 278/468 [00:15<00:11, 15.91it/s]
    avg reward: 1.21:  59%|█████▉    | 278/468 [00:16<00:11, 15.91it/s]
    avg reward: 1.21:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.21:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.21:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.21:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.21:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.23:  60%|██████    | 281/468 [00:16<00:12, 15.19it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  61%|██████    | 286/468 [00:16<00:11, 16.09it/s]
    avg reward: 1.23:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.23:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.23:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.23:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.23:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.24:  62%|██████▏   | 291/468 [00:16<00:10, 16.86it/s]
    avg reward: 1.24:  63%|██████▎   | 296/468 [00:16<00:09, 17.34it/s]
    avg reward: 1.24:  63%|██████▎   | 296/468 [00:16<00:09, 17.34it/s]
    avg reward: 1.24:  63%|██████▎   | 296/468 [00:17<00:09, 17.34it/s]
    avg reward: 1.24:  63%|██████▎   | 296/468 [00:17<00:09, 17.34it/s]
    avg reward: 1.24:  63%|██████▎   | 296/468 [00:17<00:09, 17.34it/s]
    avg reward: 1.24:  64%|██████▍   | 300/468 [00:17<00:08, 20.83it/s]
    avg reward: 1.25:  64%|██████▍   | 300/468 [00:17<00:08, 20.83it/s]
    avg reward: 1.25:  64%|██████▍   | 300/468 [00:17<00:08, 20.83it/s]
    avg reward: 1.25:  64%|██████▍   | 300/468 [00:17<00:08, 20.83it/s]
    avg reward: 1.25:  65%|██████▍   | 303/468 [00:17<00:09, 17.82it/s]
    avg reward: 1.25:  65%|██████▍   | 303/468 [00:17<00:09, 17.82it/s]
    avg reward: 1.25:  65%|██████▍   | 303/468 [00:17<00:09, 17.82it/s]
    avg reward: 1.26:  65%|██████▍   | 303/468 [00:17<00:09, 17.82it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  65%|██████▌   | 306/468 [00:17<00:10, 16.19it/s]
    avg reward: 1.26:  66%|██████▋   | 311/468 [00:17<00:09, 16.84it/s]
    avg reward: 1.26:  66%|██████▋   | 311/468 [00:17<00:09, 16.84it/s]
    avg reward: 1.26:  66%|██████▋   | 311/468 [00:17<00:09, 16.84it/s]
    avg reward: 1.26:  66%|██████▋   | 311/468 [00:17<00:09, 16.84it/s]
    avg reward: 1.26:  66%|██████▋   | 311/468 [00:17<00:09, 16.84it/s]
    avg reward: 1.26:  67%|██████▋   | 315/468 [00:17<00:07, 20.37it/s]
    avg reward: 1.26:  67%|██████▋   | 315/468 [00:18<00:07, 20.37it/s]
    avg reward: 1.26:  67%|██████▋   | 315/468 [00:18<00:07, 20.37it/s]
    avg reward: 1.26:  67%|██████▋   | 315/468 [00:18<00:07, 20.37it/s]
    avg reward: 1.26:  68%|██████▊   | 318/468 [00:18<00:08, 17.43it/s]
    avg reward: 1.26:  68%|██████▊   | 318/468 [00:18<00:08, 17.43it/s]
    avg reward: 1.26:  68%|██████▊   | 318/468 [00:18<00:08, 17.43it/s]
    avg reward: 1.26:  68%|██████▊   | 318/468 [00:18<00:08, 17.43it/s]
    avg reward: 1.26:  69%|██████▊   | 321/468 [00:18<00:09, 15.99it/s]
    avg reward: 1.26:  69%|██████▊   | 321/468 [00:18<00:09, 15.99it/s]
    avg reward: 1.26:  69%|██████▊   | 321/468 [00:18<00:09, 15.99it/s]
    avg reward: 1.26:  69%|██████▊   | 321/468 [00:18<00:09, 15.99it/s]
    avg reward: 1.26:  69%|██████▊   | 321/468 [00:18<00:09, 15.99it/s]
    avg reward: 1.26:  69%|██████▉   | 325/468 [00:18<00:07, 19.35it/s]
    avg reward: 1.25:  69%|██████▉   | 325/468 [00:18<00:07, 19.35it/s]
    avg reward: 1.25:  69%|██████▉   | 325/468 [00:18<00:07, 19.35it/s]
    avg reward: 1.25:  69%|██████▉   | 325/468 [00:18<00:07, 19.35it/s]
    avg reward: 1.25:  70%|███████   | 328/468 [00:18<00:08, 17.10it/s]
    avg reward: 1.25:  70%|███████   | 328/468 [00:18<00:08, 17.10it/s]
    avg reward: 1.25:  70%|███████   | 328/468 [00:18<00:08, 17.10it/s]
    avg reward: 1.25:  70%|███████   | 328/468 [00:18<00:08, 17.10it/s]
    avg reward: 1.25:  71%|███████   | 331/468 [00:18<00:08, 15.48it/s]
    avg reward: 1.25:  71%|███████   | 331/468 [00:18<00:08, 15.48it/s]
    avg reward: 1.25:  71%|███████   | 331/468 [00:18<00:08, 15.48it/s]
    avg reward: 1.25:  71%|███████   | 331/468 [00:18<00:08, 15.48it/s]
    avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 15.48it/s]
    avg reward: 1.25:  72%|███████▏  | 335/468 [00:19<00:07, 18.84it/s]
    avg reward: 1.25:  72%|███████▏  | 335/468 [00:19<00:07, 18.84it/s]
    avg reward: 1.25:  72%|███████▏  | 335/468 [00:19<00:07, 18.84it/s]
    avg reward: 1.25:  72%|███████▏  | 335/468 [00:19<00:07, 18.84it/s]
    avg reward: 1.25:  72%|███████▏  | 338/468 [00:19<00:08, 15.65it/s]
    avg reward: 1.25:  72%|███████▏  | 338/468 [00:19<00:08, 15.65it/s]
    avg reward: 1.25:  72%|███████▏  | 338/468 [00:19<00:08, 15.65it/s]
    avg reward: 1.25:  72%|███████▏  | 338/468 [00:19<00:08, 15.65it/s]
    avg reward: 1.25:  73%|███████▎  | 341/468 [00:19<00:08, 14.59it/s]
    avg reward: 1.25:  73%|███████▎  | 341/468 [00:19<00:08, 14.59it/s]
    avg reward: 1.25:  73%|███████▎  | 341/468 [00:19<00:08, 14.59it/s]
    avg reward: 1.25:  73%|███████▎  | 341/468 [00:19<00:08, 14.59it/s]
    avg reward: 1.25:  73%|███████▎  | 341/468 [00:19<00:08, 14.59it/s]
    avg reward: 1.25:  74%|███████▎  | 345/468 [00:19<00:06, 18.00it/s]
    avg reward: 1.25:  74%|███████▎  | 345/468 [00:19<00:06, 18.00it/s]
    avg reward: 1.25:  74%|███████▎  | 345/468 [00:19<00:06, 18.00it/s]
    avg reward: 1.25:  74%|███████▎  | 345/468 [00:19<00:06, 18.00it/s]
    avg reward: 1.25:  74%|███████▍  | 348/468 [00:19<00:07, 15.29it/s]
    avg reward: 1.25:  74%|███████▍  | 348/468 [00:19<00:07, 15.29it/s]
    avg reward: 1.25:  74%|███████▍  | 348/468 [00:19<00:07, 15.29it/s]
    avg reward: 1.25:  74%|███████▍  | 348/468 [00:20<00:07, 15.29it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  75%|███████▌  | 351/468 [00:20<00:07, 14.65it/s]
    avg reward: 1.25:  76%|███████▌  | 356/468 [00:20<00:07, 15.40it/s]
    avg reward: 1.25:  76%|███████▌  | 356/468 [00:20<00:07, 15.40it/s]
    avg reward: 1.25:  76%|███████▌  | 356/468 [00:20<00:07, 15.40it/s]
    avg reward: 1.25:  76%|███████▌  | 356/468 [00:20<00:07, 15.40it/s]
    avg reward: 1.25:  76%|███████▌  | 356/468 [00:20<00:07, 15.40it/s]
    avg reward: 1.25:  77%|███████▋  | 360/468 [00:20<00:05, 18.76it/s]
    avg reward: 1.25:  77%|███████▋  | 360/468 [00:20<00:05, 18.76it/s]
    avg reward: 1.25:  77%|███████▋  | 360/468 [00:20<00:05, 18.76it/s]
    avg reward: 1.25:  77%|███████▋  | 360/468 [00:20<00:05, 18.76it/s]
    avg reward: 1.25:  78%|███████▊  | 363/468 [00:20<00:06, 16.27it/s]
    avg reward: 1.25:  78%|███████▊  | 363/468 [00:20<00:06, 16.27it/s]
    avg reward: 1.25:  78%|███████▊  | 363/468 [00:20<00:06, 16.27it/s]
    avg reward: 1.26:  78%|███████▊  | 363/468 [00:20<00:06, 16.27it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:20<00:06, 15.57it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:20<00:06, 15.57it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:20<00:06, 15.57it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:06, 15.57it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:06, 15.57it/s]
    avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:06, 15.57it/s]
    avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 16.35it/s]
    avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 16.35it/s]
    avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 16.35it/s]
    avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 16.35it/s]
    avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 16.35it/s]
    avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 19.77it/s]
    avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 19.77it/s]
    avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 19.77it/s]
    avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 19.77it/s]
    avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:05, 16.36it/s]
    avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:05, 16.36it/s]
    avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:05, 16.36it/s]
    avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:05, 16.36it/s]
    avg reward: 1.26:  81%|████████▏ | 381/468 [00:21<00:05, 15.37it/s]
    avg reward: 1.26:  81%|████████▏ | 381/468 [00:21<00:05, 15.37it/s]
    avg reward: 1.26:  81%|████████▏ | 381/468 [00:21<00:05, 15.37it/s]
    avg reward: 1.26:  81%|████████▏ | 381/468 [00:21<00:05, 15.37it/s]
    avg reward: 1.26:  81%|████████▏ | 381/468 [00:21<00:05, 15.37it/s]
    avg reward: 1.26:  82%|████████▏ | 385/468 [00:21<00:04, 18.82it/s]
    avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 18.82it/s]
    avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 18.82it/s]
    avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 18.82it/s]
    avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 16.41it/s]
    avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 16.41it/s]
    avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 16.41it/s]
    avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 16.41it/s]
    avg reward: 1.26:  84%|████████▎ | 391/468 [00:22<00:04, 15.57it/s]
    avg reward: 1.26:  84%|████████▎ | 391/468 [00:22<00:04, 15.57it/s]
    avg reward: 1.26:  84%|████████▎ | 391/468 [00:22<00:04, 15.57it/s]
    avg reward: 1.26:  84%|████████▎ | 391/468 [00:22<00:04, 15.57it/s]
    avg reward: 1.26:  84%|████████▎ | 391/468 [00:22<00:04, 15.57it/s]
    avg reward: 1.26:  84%|████████▍ | 395/468 [00:22<00:03, 18.86it/s]
    avg reward: 1.26:  84%|████████▍ | 395/468 [00:22<00:03, 18.86it/s]
    avg reward: 1.26:  84%|████████▍ | 395/468 [00:22<00:03, 18.86it/s]
    avg reward: 1.26:  84%|████████▍ | 395/468 [00:22<00:03, 18.86it/s]
    avg reward: 1.26:  85%|████████▌ | 398/468 [00:22<00:04, 16.74it/s]
    avg reward: 1.26:  85%|████████▌ | 398/468 [00:22<00:04, 16.74it/s]
    avg reward: 1.26:  85%|████████▌ | 398/468 [00:22<00:04, 16.74it/s]
    avg reward: 1.26:  85%|████████▌ | 398/468 [00:22<00:04, 16.74it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:22<00:04, 15.29it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:22<00:04, 15.29it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:22<00:04, 15.29it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:22<00:04, 15.29it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.29it/s]
    avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.29it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  87%|████████▋ | 406/468 [00:23<00:03, 16.18it/s]
    avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.27:  88%|████████▊ | 411/468 [00:23<00:03, 16.77it/s]
    avg reward: 1.27:  89%|████████▉ | 416/468 [00:23<00:03, 17.03it/s]
    avg reward: 1.27:  89%|████████▉ | 416/468 [00:23<00:03, 17.03it/s]
    avg reward: 1.27:  89%|████████▉ | 416/468 [00:23<00:03, 17.03it/s]
    avg reward: 1.27:  89%|████████▉ | 416/468 [00:23<00:03, 17.03it/s]
    avg reward: 1.27:  89%|████████▉ | 416/468 [00:23<00:03, 17.03it/s]
    avg reward: 1.26:  89%|████████▉ | 416/468 [00:24<00:03, 17.03it/s]
    avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 17.53it/s]
    avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 17.53it/s]
    avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 17.53it/s]
    avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 17.53it/s]
    avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 17.53it/s]
    avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.87it/s]
    avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.87it/s]
    avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.87it/s]
    avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.87it/s]
    avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.72it/s]
    avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.72it/s]
    avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.72it/s]
    avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.72it/s]
    avg reward: 1.26:  92%|█████████▏| 431/468 [00:24<00:02, 16.13it/s]
    avg reward: 1.26:  92%|█████████▏| 431/468 [00:24<00:02, 16.13it/s]
    avg reward: 1.26:  92%|█████████▏| 431/468 [00:24<00:02, 16.13it/s]
    avg reward: 1.26:  92%|█████████▏| 431/468 [00:24<00:02, 16.13it/s]
    avg reward: 1.26:  92%|█████████▏| 431/468 [00:24<00:02, 16.13it/s]
    avg reward: 1.26:  93%|█████████▎| 435/468 [00:24<00:01, 19.57it/s]
    avg reward: 1.26:  93%|█████████▎| 435/468 [00:24<00:01, 19.57it/s]
    avg reward: 1.26:  93%|█████████▎| 435/468 [00:24<00:01, 19.57it/s]
    avg reward: 1.26:  93%|█████████▎| 435/468 [00:24<00:01, 19.57it/s]
    avg reward: 1.26:  94%|█████████▎| 438/468 [00:24<00:01, 17.02it/s]
    avg reward: 1.26:  94%|█████████▎| 438/468 [00:24<00:01, 17.02it/s]
    avg reward: 1.26:  94%|█████████▎| 438/468 [00:24<00:01, 17.02it/s]
    avg reward: 1.26:  94%|█████████▎| 438/468 [00:25<00:01, 17.02it/s]
    avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.66it/s]
    avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.66it/s]
    avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.66it/s]
    avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.66it/s]
    avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.66it/s]
    avg reward: 1.26:  95%|█████████▌| 445/468 [00:25<00:01, 19.03it/s]
    avg reward: 1.26:  95%|█████████▌| 445/468 [00:25<00:01, 19.03it/s]
    avg reward: 1.26:  95%|█████████▌| 445/468 [00:25<00:01, 19.03it/s]
    avg reward: 1.26:  95%|█████████▌| 445/468 [00:25<00:01, 19.03it/s]
    avg reward: 1.26:  96%|█████████▌| 448/468 [00:25<00:01, 16.79it/s]
    avg reward: 1.26:  96%|█████████▌| 448/468 [00:25<00:01, 16.79it/s]
    avg reward: 1.26:  96%|█████████▌| 448/468 [00:25<00:01, 16.79it/s]
    avg reward: 1.26:  96%|█████████▌| 448/468 [00:25<00:01, 16.79it/s]
    avg reward: 1.26:  96%|█████████▋| 451/468 [00:25<00:01, 15.81it/s]
    avg reward: 1.26:  96%|█████████▋| 451/468 [00:25<00:01, 15.81it/s]
    avg reward: 1.26:  96%|█████████▋| 451/468 [00:25<00:01, 15.81it/s]
    avg reward: 1.26:  96%|█████████▋| 451/468 [00:25<00:01, 15.81it/s]
    avg reward: 1.26:  96%|█████████▋| 451/468 [00:25<00:01, 15.81it/s]
    avg reward: 1.26:  97%|█████████▋| 455/468 [00:25<00:00, 19.07it/s]
    avg reward: 1.26:  97%|█████████▋| 455/468 [00:25<00:00, 19.07it/s]
    avg reward: 1.26:  97%|█████████▋| 455/468 [00:25<00:00, 19.07it/s]
    avg reward: 1.26:  97%|█████████▋| 455/468 [00:26<00:00, 19.07it/s]
    avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 16.92it/s]
    avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 16.92it/s]
    avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 16.92it/s]
    avg reward: 1.27:  98%|█████████▊| 458/468 [00:26<00:00, 16.92it/s]
    avg reward: 1.27:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.27:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.27:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.27:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.27:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 15.96it/s]
    avg reward: 1.26: 100%|█████████▉| 466/468 [00:26<00:00, 16.78it/s]
    avg reward: 1.26: 100%|█████████▉| 466/468 [00:26<00:00, 16.78it/s]
    avg reward: 1.26: 100%|██████████| 468/468 [00:26<00:00, 17.64it/s]
    
      0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.9609375:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.96484375:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.9739583333333334:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.96875:   0%|          | 0/79 [00:00<?, ?it/s]           
    Val Acc: 0.9703125:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.9713541666666666:   0%|          | 0/79 [00:00<?, ?it/s]
    Val Acc: 0.9713541666666666:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.9732142857142857:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.974609375:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]       
    Val Acc: 0.9774305555555556:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.97890625:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]        
    Val Acc: 0.9808238636363636:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.9817708333333334:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.9813701923076923:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]
    Val Acc: 0.98046875:   8%|▊         | 6/79 [00:00<00:01, 50.13it/s]        
    Val Acc: 0.98046875:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9807291666666667:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9814453125:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]      
    Val Acc: 0.9820772058823529:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9817708333333334:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9823190789473685:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.98203125:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]        
    Val Acc: 0.9825148809523809:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9829545454545454:  18%|█▊        | 14/79 [00:00<00:01, 55.17it/s]
    Val Acc: 0.9829545454545454:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.983016304347826:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s] 
    Val Acc: 0.9827473958333334:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.983125:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]          
    Val Acc: 0.9828725961538461:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.9826388888888888:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.9829799107142857:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.9832974137931034:  28%|██▊       | 22/79 [00:00<00:00, 58.73it/s]
    Val Acc: 0.9832974137931034:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.9838541666666667:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.9831149193548387:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.982421875:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]       
    Val Acc: 0.9824810606060606:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.9818474264705882:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.9819196428571428:  37%|███▋      | 29/79 [00:00<00:00, 61.46it/s]
    Val Acc: 0.9819196428571428:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9817708333333334:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9816300675675675:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9810855263157895:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9807692307692307:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.980859375:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]       
    Val Acc: 0.9811356707317073:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9813988095238095:  44%|████▍     | 35/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9813988095238095:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9809229651162791:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9806463068181818:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9803819444444445:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9806385869565217:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9805518617021277:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9803059895833334:  53%|█████▎    | 42/79 [00:00<00:00, 62.88it/s]
    Val Acc: 0.9803059895833334:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9805484693877551:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9803125:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]         
    Val Acc: 0.9800857843137255:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9795673076923077:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9796580188679245:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9797453703703703:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9799715909090909:  61%|██████    | 48/79 [00:00<00:00, 61.68it/s]
    Val Acc: 0.9799715909090909:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9803292410714286:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9804002192982456:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9800646551724138:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9801377118644068:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9802083333333333:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9802766393442623:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.9800907258064516:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s]
    Val Acc: 0.980406746031746:  70%|██████▉   | 55/79 [00:00<00:00, 60.85it/s] 
    Val Acc: 0.980406746031746:  80%|███████▉  | 63/79 [00:00<00:00, 62.37it/s]
    Val Acc: 0.9803466796875:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]  
    Val Acc: 0.9804086538461538:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]
    Val Acc: 0.98046875:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]        
    Val Acc: 0.980643656716418:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]
    Val Acc: 0.9805836397058824:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]
    Val Acc: 0.9805253623188406:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]
    Val Acc: 0.9803571428571428:  80%|███████▉  | 63/79 [00:01<00:00, 62.37it/s]
    Val Acc: 0.9803571428571428:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9803036971830986:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.98046875:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]        
    Val Acc: 0.9804152397260274:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9806798986486487:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9807291666666667:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9809827302631579:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9809253246753247:  89%|████████▊ | 70/79 [00:01<00:00, 63.93it/s]
    Val Acc: 0.9809253246753247:  97%|█████████▋| 77/79 [00:01<00:00, 64.35it/s]
    Val Acc: 0.9808693910256411:  97%|█████████▋| 77/79 [00:01<00:00, 64.35it/s]
    Val Acc: 0.9808: 100%|██████████| 79/79 [00:01<00:00, 65.08it/s]
    epoch 0, val_acc: 0.98, avg reward: 1.26: 100%|██████████| 1/1 [00:27<00:00, 27.80s/it]


The resulting architecture is:

.. code:: python

    mynet.graph




.. figure:: output_enas_proxylessnas_ad55b9_25_0.svg



Store the trained model as a static network
-------------------------------------------

The trained ENAS network can be saved to disk for future inferences.

.. code:: python

    mynet.export('enas')


.. parsed-literal::
    :class: output

    /var/lib/jenkins/miniconda3/envs/autogluon_docs-v0_0_15/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
      and should_run_async(code)


Load it back with mxnet:

.. code:: python

    mynet_static = mx.gluon.nn.SymbolBlock.imports("enas-symbol.json", ['data'], "enas.params")
    y = mynet_static(mx.nd.zeros((1, 1, 28, 28)))
    print(y.shape)


.. parsed-literal::
    :class: output

    (1, 10)


Reference
---------

[1] Efficient Neural Architecture Search via Parameter Sharing H Pham,
MY Guan, B Zoph, QV Le, J Dean *International Conference on Machine
Learning (ICML)*

[3] ProxylessNAS: Direct Neural Architecture Search on Target Task and
Hardware Han Cai, Ligeng Zhu, Song Han *International Conference on
Learning Representations (ICLR)*, 2019.