How to Use ENAS/ProxylessNAS in Ten Minutes¶

What is the Key Idea of ENAS and ProxylessNAS?¶

Traditional reinforcement learning-based neural architecture search learns an architecture controller by iteratively sampling the architecture and training the model to get final reward to update the controller. It is extremely expensive process due to training CNN.

https://raw.githubusercontent.com/zhanghang1989/AutoGluonWebdata/master/docs/tutorial/proxyless.png

Fig. 3 ProxylessNAS¶

Recent work of ENAS and ProxylessNAS construct an over-parameterized network (supernet) and share the weights across different architecture to speed up the search speed. The reward is calculated every few iterations instead of every training period.

Import MXNet and AutoGluon:

import autogluon.core as ag
import mxnet as mx
import mxnet.gluon.nn as nn

How to Construct a SuperNet¶

Basic NN blocks for CNN.

class Identity(mx.gluon.HybridBlock):
    def hybrid_forward(self, F, x):
        return x

class ConvBNReLU(mx.gluon.HybridBlock):
    def __init__(self, in_channels, channels, kernel, stride):
        super().__init__()
        padding = (kernel - 1) // 2
        self.conv = nn.Conv2D(channels, kernel, stride, padding, in_channels=in_channels)
        self.bn = nn.BatchNorm(in_channels=channels)
        self.relu = nn.Activation('relu')
    def hybrid_forward(self, F, x):
        return self.relu(self.bn(self.conv(x)))

AutoGluon ENAS Unit¶

from autogluon.extra.contrib.enas import *

@enas_unit()
class ResUnit(mx.gluon.HybridBlock):
    def __init__(self, in_channels, channels, hidden_channels, kernel, stride):
        super().__init__()
        self.conv1 = ConvBNReLU(in_channels, hidden_channels, kernel, stride)
        self.conv2 = ConvBNReLU(hidden_channels, channels, kernel, 1)
        if in_channels == channels and stride == 1:
            self.shortcut = Identity()
        else:
            self.shortcut = nn.Conv2D(channels, 1, stride, in_channels=in_channels)
    def hybrid_forward(self, F, x):
        return self.conv2(self.conv1(x)) + self.shortcut(x)

AutoGluon Sequntial¶

Creating a ENAS network using Sequential Block:

mynet = ENAS_Sequential(
    ResUnit(1, 8, hidden_channels=ag.space.Categorical(4, 8), kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(8, 8, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(8, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
    ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
    nn.GlobalAvgPool2D(),
    nn.Flatten(),
    nn.Activation('relu'),
    nn.Dense(10, in_units=16),
)

mynet.initialize()

#mynet.graph

Evaluate Network Latency and Define Reward Function¶

x = mx.nd.random.uniform(shape=(1, 1, 28, 28))
y = mynet.evaluate_latency(x)

Show the latencies:

print('Average latency is {:.2f} ms, latency of the current architecture is {:.2f} ms'.format(mynet.avg_latency, mynet.latency))

Average latency is 4.27 ms, latency of the current architecture is 4.77 ms

We also provide number of params

mynet.nparams

Define the reward function:

reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.1)

Start the Training¶

Construct experiment scheduler, which automatically creates an RL controller based on user-defined search space.

scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                           reward_fn=reward_fn, batch_size=128, num_gpus=1,
                           warmup_epochs=0, epochs=1, controller_lr=3e-3,
                           plot_frequency=10, update_arch_frequency=5)

Start the training:

scheduler.run()

  0%|          | 0/1 [00:00<?, ?it/s]
  0%|          | 0/468 [00:00<?, ?it/s][A
avg reward: 0.09:   0%|          | 0/468 [00:00<?, ?it/s][A
avg reward: 0.09:   0%|          | 1/468 [00:00<05:05,  1.53it/s][A
avg reward: 0.09:   0%|          | 1/468 [00:00<05:05,  1.53it/s][A
avg reward: 0.09:   0%|          | 1/468 [00:00<05:05,  1.53it/s][A
avg reward: 0.09:   0%|          | 1/468 [00:00<05:05,  1.53it/s][A
avg reward: 0.09:   0%|          | 1/468 [00:00<05:05,  1.53it/s][A
avg reward: 0.09:   1%|          | 5/468 [00:00<00:55,  8.31it/s][A
avg reward: 0.10:   1%|          | 5/468 [00:00<00:55,  8.31it/s][A
avg reward: 0.10:   1%|          | 5/468 [00:00<00:55,  8.31it/s][A
avg reward: 0.10:   1%|          | 5/468 [00:01<00:55,  8.31it/s][A
avg reward: 0.10:   2%|▏         | 8/468 [00:01<00:47,  9.75it/s][A
avg reward: 0.10:   2%|▏         | 8/468 [00:01<00:47,  9.75it/s][A
avg reward: 0.10:   2%|▏         | 8/468 [00:01<00:47,  9.75it/s][A
avg reward: 0.10:   2%|▏         | 8/468 [00:01<00:47,  9.75it/s][A
avg reward: 0.10:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.10:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.10:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.10:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.10:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.11:   2%|▏         | 11/468 [00:01<00:41, 11.04it/s][A
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:33, 13.68it/s][A
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:33, 13.68it/s][A
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:33, 13.68it/s][A
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:33, 13.68it/s][A
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:33, 13.68it/s][A
avg reward: 0.11:   4%|▍         | 20/468 [00:01<00:25, 17.67it/s][A
avg reward: 0.12:   4%|▍         | 20/468 [00:01<00:25, 17.67it/s][A
avg reward: 0.12:   4%|▍         | 20/468 [00:01<00:25, 17.67it/s][A
avg reward: 0.12:   4%|▍         | 20/468 [00:01<00:25, 17.67it/s][A
avg reward: 0.12:   5%|▍         | 23/468 [00:01<00:27, 15.91it/s][A
avg reward: 0.12:   5%|▍         | 23/468 [00:01<00:27, 15.91it/s][A
avg reward: 0.12:   5%|▍         | 23/468 [00:01<00:27, 15.91it/s][A
avg reward: 0.13:   5%|▍         | 23/468 [00:02<00:27, 15.91it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.81it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.81it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.81it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.81it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.81it/s][A
avg reward: 0.13:   6%|▋         | 30/468 [00:02<00:23, 18.83it/s][A
avg reward: 0.24:   6%|▋         | 30/468 [00:02<00:23, 18.83it/s][A
avg reward: 0.24:   6%|▋         | 30/468 [00:02<00:23, 18.83it/s][A
avg reward: 0.24:   6%|▋         | 30/468 [00:02<00:23, 18.83it/s][A
avg reward: 0.24:   7%|▋         | 33/468 [00:02<00:26, 16.46it/s][A
avg reward: 0.24:   7%|▋         | 33/468 [00:02<00:26, 16.46it/s][A
avg reward: 0.24:   7%|▋         | 33/468 [00:02<00:26, 16.46it/s][A
avg reward: 0.40:   7%|▋         | 33/468 [00:02<00:26, 16.46it/s][A
avg reward: 0.40:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.40:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.40:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.40:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.40:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.51:   8%|▊         | 36/468 [00:02<00:28, 15.38it/s][A
avg reward: 0.51:   9%|▉         | 41/468 [00:02<00:25, 16.66it/s][A
avg reward: 0.51:   9%|▉         | 41/468 [00:02<00:25, 16.66it/s][A
avg reward: 0.51:   9%|▉         | 41/468 [00:02<00:25, 16.66it/s][A
avg reward: 0.51:   9%|▉         | 41/468 [00:02<00:25, 16.66it/s][A
avg reward: 0.51:   9%|▉         | 41/468 [00:03<00:25, 16.66it/s][A
avg reward: 0.58:   9%|▉         | 41/468 [00:03<00:25, 16.66it/s][A
avg reward: 0.58:  10%|▉         | 46/468 [00:03<00:24, 17.14it/s][A
avg reward: 0.58:  10%|▉         | 46/468 [00:03<00:24, 17.14it/s][A
avg reward: 0.58:  10%|▉         | 46/468 [00:03<00:24, 17.14it/s][A
avg reward: 0.58:  10%|▉         | 46/468 [00:03<00:24, 17.14it/s][A
avg reward: 0.58:  10%|▉         | 46/468 [00:03<00:24, 17.14it/s][A
avg reward: 0.58:  11%|█         | 50/468 [00:03<00:20, 20.42it/s][A
avg reward: 0.60:  11%|█         | 50/468 [00:03<00:20, 20.42it/s][A
avg reward: 0.60:  11%|█         | 50/468 [00:03<00:20, 20.42it/s][A
avg reward: 0.60:  11%|█         | 50/468 [00:03<00:20, 20.42it/s][A
avg reward: 0.60:  11%|█▏        | 53/468 [00:03<00:23, 18.00it/s][A
avg reward: 0.60:  11%|█▏        | 53/468 [00:03<00:23, 18.00it/s][A
avg reward: 0.60:  11%|█▏        | 53/468 [00:03<00:23, 18.00it/s][A
avg reward: 0.58:  11%|█▏        | 53/468 [00:03<00:23, 18.00it/s][A
avg reward: 0.58:  12%|█▏        | 56/468 [00:03<00:24, 16.67it/s][A
avg reward: 0.58:  12%|█▏        | 56/468 [00:03<00:24, 16.67it/s][A
avg reward: 0.58:  12%|█▏        | 56/468 [00:03<00:24, 16.67it/s][A
avg reward: 0.58:  12%|█▏        | 56/468 [00:03<00:24, 16.67it/s][A
avg reward: 0.58:  12%|█▏        | 56/468 [00:03<00:24, 16.67it/s][A
avg reward: 0.58:  13%|█▎        | 60/468 [00:03<00:19, 20.43it/s][A
avg reward: 0.59:  13%|█▎        | 60/468 [00:04<00:19, 20.43it/s][A
avg reward: 0.59:  13%|█▎        | 60/468 [00:04<00:19, 20.43it/s][A
avg reward: 0.59:  13%|█▎        | 60/468 [00:04<00:19, 20.43it/s][A
avg reward: 0.59:  13%|█▎        | 63/468 [00:04<00:22, 18.06it/s][A
avg reward: 0.59:  13%|█▎        | 63/468 [00:04<00:22, 18.06it/s][A
avg reward: 0.59:  13%|█▎        | 63/468 [00:04<00:22, 18.06it/s][A
avg reward: 0.62:  13%|█▎        | 63/468 [00:04<00:22, 18.06it/s][A
avg reward: 0.62:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.62:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.62:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.62:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.62:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.67:  14%|█▍        | 66/468 [00:04<00:24, 16.58it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:04<00:22, 17.43it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:04<00:22, 17.43it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:04<00:22, 17.43it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:04<00:22, 17.43it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:04<00:22, 17.43it/s][A
avg reward: 0.67:  16%|█▌        | 75/468 [00:04<00:18, 21.00it/s][A
avg reward: 0.63:  16%|█▌        | 75/468 [00:04<00:18, 21.00it/s][A
avg reward: 0.63:  16%|█▌        | 75/468 [00:04<00:18, 21.00it/s][A
avg reward: 0.63:  16%|█▌        | 75/468 [00:04<00:18, 21.00it/s][A
avg reward: 0.63:  17%|█▋        | 78/468 [00:04<00:21, 18.02it/s][A
avg reward: 0.63:  17%|█▋        | 78/468 [00:04<00:21, 18.02it/s][A
avg reward: 0.63:  17%|█▋        | 78/468 [00:04<00:21, 18.02it/s][A
avg reward: 0.68:  17%|█▋        | 78/468 [00:05<00:21, 18.02it/s][A
avg reward: 0.68:  17%|█▋        | 81/468 [00:05<00:23, 16.68it/s][A
avg reward: 0.68:  17%|█▋        | 81/468 [00:05<00:23, 16.68it/s][A
avg reward: 0.68:  17%|█▋        | 81/468 [00:05<00:23, 16.68it/s][A
avg reward: 0.68:  17%|█▋        | 81/468 [00:05<00:23, 16.68it/s][A
avg reward: 0.68:  17%|█▋        | 81/468 [00:05<00:23, 16.68it/s][A
avg reward: 0.68:  18%|█▊        | 85/468 [00:05<00:18, 20.68it/s][A
avg reward: 0.73:  18%|█▊        | 85/468 [00:05<00:18, 20.68it/s][A
avg reward: 0.73:  18%|█▊        | 85/468 [00:05<00:18, 20.68it/s][A
avg reward: 0.73:  18%|█▊        | 85/468 [00:05<00:18, 20.68it/s][A
avg reward: 0.73:  19%|█▉        | 88/468 [00:05<00:21, 17.56it/s][A
avg reward: 0.73:  19%|█▉        | 88/468 [00:05<00:21, 17.56it/s][A
avg reward: 0.73:  19%|█▉        | 88/468 [00:05<00:21, 17.56it/s][A
avg reward: 0.80:  19%|█▉        | 88/468 [00:05<00:21, 17.56it/s][A
avg reward: 0.80:  19%|█▉        | 91/468 [00:05<00:23, 16.01it/s][A
avg reward: 0.80:  19%|█▉        | 91/468 [00:05<00:23, 16.01it/s][A
avg reward: 0.80:  19%|█▉        | 91/468 [00:05<00:23, 16.01it/s][A
avg reward: 0.80:  19%|█▉        | 91/468 [00:05<00:23, 16.01it/s][A
avg reward: 0.80:  19%|█▉        | 91/468 [00:05<00:23, 16.01it/s][A
avg reward: 0.80:  20%|██        | 95/468 [00:05<00:18, 20.08it/s][A
avg reward: 0.79:  20%|██        | 95/468 [00:05<00:18, 20.08it/s][A
avg reward: 0.79:  20%|██        | 95/468 [00:06<00:18, 20.08it/s][A
avg reward: 0.79:  20%|██        | 95/468 [00:06<00:18, 20.08it/s][A
avg reward: 0.79:  21%|██        | 98/468 [00:06<00:21, 17.53it/s][A
avg reward: 0.79:  21%|██        | 98/468 [00:06<00:21, 17.53it/s][A
avg reward: 0.79:  21%|██        | 98/468 [00:06<00:21, 17.53it/s][A
avg reward: 0.82:  21%|██        | 98/468 [00:06<00:21, 17.53it/s][A
avg reward: 0.82:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.82:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.82:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.82:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.82:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.84:  22%|██▏       | 101/468 [00:06<00:22, 16.29it/s][A
avg reward: 0.84:  23%|██▎       | 106/468 [00:06<00:21, 17.09it/s][A
avg reward: 0.84:  23%|██▎       | 106/468 [00:06<00:21, 17.09it/s][A
avg reward: 0.84:  23%|██▎       | 106/468 [00:06<00:21, 17.09it/s][A
avg reward: 0.84:  23%|██▎       | 106/468 [00:06<00:21, 17.09it/s][A
avg reward: 0.84:  23%|██▎       | 106/468 [00:06<00:21, 17.09it/s][A
avg reward: 0.84:  24%|██▎       | 110/468 [00:06<00:17, 20.55it/s][A
avg reward: 0.84:  24%|██▎       | 110/468 [00:06<00:17, 20.55it/s][A
avg reward: 0.84:  24%|██▎       | 110/468 [00:06<00:17, 20.55it/s][A
avg reward: 0.84:  24%|██▎       | 110/468 [00:06<00:17, 20.55it/s][A
avg reward: 0.84:  24%|██▍       | 113/468 [00:06<00:19, 18.05it/s][A
avg reward: 0.84:  24%|██▍       | 113/468 [00:06<00:19, 18.05it/s][A
avg reward: 0.84:  24%|██▍       | 113/468 [00:06<00:19, 18.05it/s][A
avg reward: 0.86:  24%|██▍       | 113/468 [00:07<00:19, 18.05it/s][A
avg reward: 0.86:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.86:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.86:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.86:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.86:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.88:  25%|██▍       | 116/468 [00:07<00:20, 17.06it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:19, 17.64it/s][A
avg reward: 0.88:  27%|██▋       | 126/468 [00:07<00:19, 17.53it/s][A
avg reward: 0.88:  27%|██▋       | 126/468 [00:07<00:19, 17.53it/s][A
avg reward: 0.88:  27%|██▋       | 126/468 [00:07<00:19, 17.53it/s][A
avg reward: 0.88:  27%|██▋       | 126/468 [00:07<00:19, 17.53it/s][A
avg reward: 0.88:  27%|██▋       | 126/468 [00:07<00:19, 17.53it/s][A
avg reward: 0.88:  28%|██▊       | 130/468 [00:07<00:16, 20.58it/s][A
avg reward: 0.90:  28%|██▊       | 130/468 [00:07<00:16, 20.58it/s][A
avg reward: 0.90:  28%|██▊       | 130/468 [00:07<00:16, 20.58it/s][A
avg reward: 0.90:  28%|██▊       | 130/468 [00:07<00:16, 20.58it/s][A
avg reward: 0.90:  28%|██▊       | 133/468 [00:07<00:18, 18.24it/s][A
avg reward: 0.90:  28%|██▊       | 133/468 [00:07<00:18, 18.24it/s][A
avg reward: 0.90:  28%|██▊       | 133/468 [00:08<00:18, 18.24it/s][A
avg reward: 0.90:  28%|██▊       | 133/468 [00:08<00:18, 18.24it/s][A
avg reward: 0.90:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.90:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.90:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.90:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.90:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.93:  29%|██▉       | 136/468 [00:08<00:19, 16.75it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  30%|███       | 141/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  31%|███       | 146/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  31%|███       | 146/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  31%|███       | 146/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  31%|███       | 146/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  31%|███       | 146/468 [00:08<00:18, 17.25it/s][A
avg reward: 0.93:  32%|███▏      | 150/468 [00:08<00:15, 20.59it/s][A
avg reward: 0.93:  32%|███▏      | 150/468 [00:09<00:15, 20.59it/s][A
avg reward: 0.93:  32%|███▏      | 150/468 [00:09<00:15, 20.59it/s][A
avg reward: 0.93:  32%|███▏      | 150/468 [00:09<00:15, 20.59it/s][A
avg reward: 0.93:  33%|███▎      | 153/468 [00:09<00:17, 18.35it/s][A
avg reward: 0.93:  33%|███▎      | 153/468 [00:09<00:17, 18.35it/s][A
avg reward: 0.93:  33%|███▎      | 153/468 [00:09<00:17, 18.35it/s][A
avg reward: 0.93:  33%|███▎      | 153/468 [00:09<00:17, 18.35it/s][A
avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.93:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.94:  33%|███▎      | 156/468 [00:09<00:18, 16.71it/s][A
avg reward: 0.94:  34%|███▍      | 161/468 [00:09<00:17, 17.25it/s][A
avg reward: 0.94:  34%|███▍      | 161/468 [00:09<00:17, 17.25it/s][A
avg reward: 0.94:  34%|███▍      | 161/468 [00:09<00:17, 17.25it/s][A
avg reward: 0.94:  34%|███▍      | 161/468 [00:09<00:17, 17.25it/s][A
avg reward: 0.94:  34%|███▍      | 161/468 [00:09<00:17, 17.25it/s][A
avg reward: 0.94:  35%|███▌      | 165/468 [00:09<00:14, 20.33it/s][A
avg reward: 0.93:  35%|███▌      | 165/468 [00:09<00:14, 20.33it/s][A
avg reward: 0.93:  35%|███▌      | 165/468 [00:09<00:14, 20.33it/s][A
avg reward: 0.93:  35%|███▌      | 165/468 [00:09<00:14, 20.33it/s][A
avg reward: 0.93:  36%|███▌      | 168/468 [00:09<00:16, 18.41it/s][A
avg reward: 0.93:  36%|███▌      | 168/468 [00:09<00:16, 18.41it/s][A
avg reward: 0.93:  36%|███▌      | 168/468 [00:09<00:16, 18.41it/s][A
avg reward: 0.94:  36%|███▌      | 168/468 [00:10<00:16, 18.41it/s][A
avg reward: 0.94:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.94:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.94:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.94:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.94:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.93:  37%|███▋      | 171/468 [00:10<00:18, 16.47it/s][A
avg reward: 0.93:  38%|███▊      | 176/468 [00:10<00:16, 17.57it/s][A
avg reward: 0.93:  38%|███▊      | 176/468 [00:10<00:16, 17.57it/s][A
avg reward: 0.93:  38%|███▊      | 176/468 [00:10<00:16, 17.57it/s][A
avg reward: 0.93:  38%|███▊      | 176/468 [00:10<00:16, 17.57it/s][A
avg reward: 0.93:  38%|███▊      | 176/468 [00:10<00:16, 17.57it/s][A
avg reward: 0.93:  38%|███▊      | 180/468 [00:10<00:13, 20.79it/s][A
avg reward: 0.94:  38%|███▊      | 180/468 [00:10<00:13, 20.79it/s][A
avg reward: 0.94:  38%|███▊      | 180/468 [00:10<00:13, 20.79it/s][A
avg reward: 0.94:  38%|███▊      | 180/468 [00:10<00:13, 20.79it/s][A
avg reward: 0.94:  39%|███▉      | 183/468 [00:10<00:15, 18.03it/s][A
avg reward: 0.94:  39%|███▉      | 183/468 [00:10<00:15, 18.03it/s][A
avg reward: 0.94:  39%|███▉      | 183/468 [00:10<00:15, 18.03it/s][A
avg reward: 0.94:  39%|███▉      | 183/468 [00:10<00:15, 18.03it/s][A
avg reward: 0.94:  40%|███▉      | 186/468 [00:10<00:16, 16.78it/s][A
avg reward: 0.94:  40%|███▉      | 186/468 [00:10<00:16, 16.78it/s][A
avg reward: 0.94:  40%|███▉      | 186/468 [00:11<00:16, 16.78it/s][A
avg reward: 0.94:  40%|███▉      | 186/468 [00:11<00:16, 16.78it/s][A
avg reward: 0.94:  40%|███▉      | 186/468 [00:11<00:16, 16.78it/s][A
avg reward: 0.94:  41%|████      | 190/468 [00:11<00:13, 20.68it/s][A
avg reward: 0.94:  41%|████      | 190/468 [00:11<00:13, 20.68it/s][A
avg reward: 0.94:  41%|████      | 190/468 [00:11<00:13, 20.68it/s][A
avg reward: 0.94:  41%|████      | 190/468 [00:11<00:13, 20.68it/s][A
avg reward: 0.94:  41%|████      | 193/468 [00:11<00:15, 17.58it/s][A
avg reward: 0.94:  41%|████      | 193/468 [00:11<00:15, 17.58it/s][A
avg reward: 0.94:  41%|████      | 193/468 [00:11<00:15, 17.58it/s][A
avg reward: 0.95:  41%|████      | 193/468 [00:11<00:15, 17.58it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  42%|████▏     | 196/468 [00:11<00:16, 16.24it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:15, 17.04it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:15, 17.04it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:15, 17.04it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:15, 17.04it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:11<00:15, 17.04it/s][A
avg reward: 0.95:  43%|████▎     | 201/468 [00:12<00:15, 17.04it/s][A
avg reward: 0.95:  44%|████▍     | 206/468 [00:12<00:14, 17.61it/s][A
avg reward: 0.95:  44%|████▍     | 206/468 [00:12<00:14, 17.61it/s][A
avg reward: 0.95:  44%|████▍     | 206/468 [00:12<00:14, 17.61it/s][A
avg reward: 0.95:  44%|████▍     | 206/468 [00:12<00:14, 17.61it/s][A
avg reward: 0.95:  44%|████▍     | 206/468 [00:12<00:14, 17.61it/s][A
avg reward: 0.95:  45%|████▍     | 210/468 [00:12<00:12, 20.76it/s][A
avg reward: 0.96:  45%|████▍     | 210/468 [00:12<00:12, 20.76it/s][A
avg reward: 0.96:  45%|████▍     | 210/468 [00:12<00:12, 20.76it/s][A
avg reward: 0.96:  45%|████▍     | 210/468 [00:12<00:12, 20.76it/s][A
avg reward: 0.96:  46%|████▌     | 213/468 [00:12<00:14, 18.01it/s][A
avg reward: 0.96:  46%|████▌     | 213/468 [00:12<00:14, 18.01it/s][A
avg reward: 0.96:  46%|████▌     | 213/468 [00:12<00:14, 18.01it/s][A
avg reward: 0.96:  46%|████▌     | 213/468 [00:12<00:14, 18.01it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  46%|████▌     | 216/468 [00:12<00:15, 16.63it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:12<00:14, 17.33it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:12<00:14, 17.33it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:12<00:14, 17.33it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:12<00:14, 17.33it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:12<00:14, 17.33it/s][A
avg reward: 0.96:  47%|████▋     | 221/468 [00:13<00:14, 17.33it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  48%|████▊     | 226/468 [00:13<00:13, 17.30it/s][A
avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.96:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.95:  49%|████▉     | 231/468 [00:13<00:13, 17.48it/s][A
avg reward: 0.95:  50%|█████     | 236/468 [00:13<00:12, 17.97it/s][A
avg reward: 0.95:  50%|█████     | 236/468 [00:13<00:12, 17.97it/s][A
avg reward: 0.95:  50%|█████     | 236/468 [00:13<00:12, 17.97it/s][A
avg reward: 0.95:  50%|█████     | 236/468 [00:13<00:12, 17.97it/s][A
avg reward: 0.95:  50%|█████     | 236/468 [00:13<00:12, 17.97it/s][A
avg reward: 0.95:  51%|█████▏    | 240/468 [00:13<00:10, 20.85it/s][A
avg reward: 0.95:  51%|█████▏    | 240/468 [00:14<00:10, 20.85it/s][A
avg reward: 0.95:  51%|█████▏    | 240/468 [00:14<00:10, 20.85it/s][A
avg reward: 0.95:  51%|█████▏    | 240/468 [00:14<00:10, 20.85it/s][A
avg reward: 0.95:  52%|█████▏    | 243/468 [00:14<00:12, 18.36it/s][A
avg reward: 0.95:  52%|█████▏    | 243/468 [00:14<00:12, 18.36it/s][A
avg reward: 0.95:  52%|█████▏    | 243/468 [00:14<00:12, 18.36it/s][A
avg reward: 0.96:  52%|█████▏    | 243/468 [00:14<00:12, 18.36it/s][A
avg reward: 0.96:  53%|█████▎    | 246/468 [00:14<00:13, 16.28it/s][A
avg reward: 0.96:  53%|█████▎    | 246/468 [00:14<00:13, 16.28it/s][A
avg reward: 0.96:  53%|█████▎    | 246/468 [00:14<00:13, 16.28it/s][A
avg reward: 0.96:  53%|█████▎    | 246/468 [00:14<00:13, 16.28it/s][A
avg reward: 0.96:  53%|█████▎    | 246/468 [00:14<00:13, 16.28it/s][A
avg reward: 0.96:  53%|█████▎    | 250/468 [00:14<00:11, 19.54it/s][A
avg reward: 0.96:  53%|█████▎    | 250/468 [00:14<00:11, 19.54it/s][A
avg reward: 0.96:  53%|█████▎    | 250/468 [00:14<00:11, 19.54it/s][A
avg reward: 0.96:  53%|█████▎    | 250/468 [00:14<00:11, 19.54it/s][A
avg reward: 0.96:  54%|█████▍    | 253/468 [00:14<00:12, 17.42it/s][A
avg reward: 0.96:  54%|█████▍    | 253/468 [00:14<00:12, 17.42it/s][A
avg reward: 0.96:  54%|█████▍    | 253/468 [00:14<00:12, 17.42it/s][A
avg reward: 0.97:  54%|█████▍    | 253/468 [00:14<00:12, 17.42it/s][A
avg reward: 0.97:  55%|█████▍    | 256/468 [00:14<00:13, 16.14it/s][A
avg reward: 0.97:  55%|█████▍    | 256/468 [00:14<00:13, 16.14it/s][A
avg reward: 0.97:  55%|█████▍    | 256/468 [00:14<00:13, 16.14it/s][A
avg reward: 0.97:  55%|█████▍    | 256/468 [00:14<00:13, 16.14it/s][A
avg reward: 0.97:  55%|█████▍    | 256/468 [00:14<00:13, 16.14it/s][A
avg reward: 0.97:  56%|█████▌    | 260/468 [00:14<00:10, 19.93it/s][A
avg reward: 0.96:  56%|█████▌    | 260/468 [00:15<00:10, 19.93it/s][A
avg reward: 0.96:  56%|█████▌    | 260/468 [00:15<00:10, 19.93it/s][A
avg reward: 0.96:  56%|█████▌    | 260/468 [00:15<00:10, 19.93it/s][A
avg reward: 0.96:  56%|█████▌    | 263/468 [00:15<00:11, 17.88it/s][A
avg reward: 0.96:  56%|█████▌    | 263/468 [00:15<00:11, 17.88it/s][A
avg reward: 0.96:  56%|█████▌    | 263/468 [00:15<00:11, 17.88it/s][A
avg reward: 0.96:  56%|█████▌    | 263/468 [00:15<00:11, 17.88it/s][A
avg reward: 0.96:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.96:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.96:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.96:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.96:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.97:  57%|█████▋    | 266/468 [00:15<00:12, 16.09it/s][A
avg reward: 0.97:  58%|█████▊    | 271/468 [00:15<00:11, 17.20it/s][A
avg reward: 0.97:  58%|█████▊    | 271/468 [00:15<00:11, 17.20it/s][A
avg reward: 0.97:  58%|█████▊    | 271/468 [00:15<00:11, 17.20it/s][A
avg reward: 0.97:  58%|█████▊    | 271/468 [00:15<00:11, 17.20it/s][A
avg reward: 0.97:  58%|█████▊    | 271/468 [00:15<00:11, 17.20it/s][A
avg reward: 0.97:  59%|█████▉    | 275/468 [00:15<00:09, 20.72it/s][A
avg reward: 0.98:  59%|█████▉    | 275/468 [00:15<00:09, 20.72it/s][A
avg reward: 0.98:  59%|█████▉    | 275/468 [00:16<00:09, 20.72it/s][A
avg reward: 0.98:  59%|█████▉    | 275/468 [00:16<00:09, 20.72it/s][A
avg reward: 0.98:  59%|█████▉    | 278/468 [00:16<00:10, 17.53it/s][A
avg reward: 0.98:  59%|█████▉    | 278/468 [00:16<00:10, 17.53it/s][A
avg reward: 0.98:  59%|█████▉    | 278/468 [00:16<00:10, 17.53it/s][A
avg reward: 0.97:  59%|█████▉    | 278/468 [00:16<00:10, 17.53it/s][A
avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:11, 16.60it/s][A
avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:11, 16.60it/s][A
avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:11, 16.60it/s][A
avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:11, 16.60it/s][A
avg reward: 0.97:  60%|██████    | 281/468 [00:16<00:11, 16.60it/s][A
avg reward: 0.97:  61%|██████    | 285/468 [00:16<00:08, 20.57it/s][A
avg reward: 0.97:  61%|██████    | 285/468 [00:16<00:08, 20.57it/s][A
avg reward: 0.97:  61%|██████    | 285/468 [00:16<00:08, 20.57it/s][A
avg reward: 0.97:  61%|██████    | 285/468 [00:16<00:08, 20.57it/s][A
avg reward: 0.97:  62%|██████▏   | 288/468 [00:16<00:10, 17.92it/s][A
avg reward: 0.97:  62%|██████▏   | 288/468 [00:16<00:10, 17.92it/s][A
avg reward: 0.97:  62%|██████▏   | 288/468 [00:16<00:10, 17.92it/s][A
avg reward: 0.97:  62%|██████▏   | 288/468 [00:16<00:10, 17.92it/s][A
avg reward: 0.97:  62%|██████▏   | 291/468 [00:16<00:10, 16.39it/s][A
avg reward: 0.97:  62%|██████▏   | 291/468 [00:16<00:10, 16.39it/s][A
avg reward: 0.97:  62%|██████▏   | 291/468 [00:16<00:10, 16.39it/s][A
avg reward: 0.97:  62%|██████▏   | 291/468 [00:16<00:10, 16.39it/s][A
avg reward: 0.97:  62%|██████▏   | 291/468 [00:16<00:10, 16.39it/s][A
avg reward: 0.97:  63%|██████▎   | 295/468 [00:16<00:08, 20.37it/s][A
avg reward: 0.97:  63%|██████▎   | 295/468 [00:17<00:08, 20.37it/s][A
avg reward: 0.97:  63%|██████▎   | 295/468 [00:17<00:08, 20.37it/s][A
avg reward: 0.97:  63%|██████▎   | 295/468 [00:17<00:08, 20.37it/s][A
avg reward: 0.97:  64%|██████▎   | 298/468 [00:17<00:09, 18.05it/s][A
avg reward: 0.97:  64%|██████▎   | 298/468 [00:17<00:09, 18.05it/s][A
avg reward: 0.97:  64%|██████▎   | 298/468 [00:17<00:09, 18.05it/s][A
avg reward: 0.98:  64%|██████▎   | 298/468 [00:17<00:09, 18.05it/s][A
avg reward: 0.98:  64%|██████▍   | 301/468 [00:17<00:10, 16.01it/s][A
avg reward: 0.98:  64%|██████▍   | 301/468 [00:17<00:10, 16.01it/s][A
avg reward: 0.98:  64%|██████▍   | 301/468 [00:17<00:10, 16.01it/s][A
avg reward: 0.98:  64%|██████▍   | 301/468 [00:17<00:10, 16.01it/s][A
avg reward: 0.98:  64%|██████▍   | 301/468 [00:17<00:10, 16.01it/s][A
avg reward: 0.98:  65%|██████▌   | 305/468 [00:17<00:08, 20.06it/s][A
avg reward: 0.98:  65%|██████▌   | 305/468 [00:17<00:08, 20.06it/s][A
avg reward: 0.98:  65%|██████▌   | 305/468 [00:17<00:08, 20.06it/s][A
avg reward: 0.98:  65%|██████▌   | 305/468 [00:17<00:08, 20.06it/s][A
avg reward: 0.98:  66%|██████▌   | 308/468 [00:17<00:09, 17.20it/s][A
avg reward: 0.98:  66%|██████▌   | 308/468 [00:17<00:09, 17.20it/s][A
avg reward: 0.98:  66%|██████▌   | 308/468 [00:17<00:09, 17.20it/s][A
avg reward: 0.98:  66%|██████▌   | 308/468 [00:17<00:09, 17.20it/s][A
avg reward: 0.98:  66%|██████▋   | 311/468 [00:17<00:09, 15.86it/s][A
avg reward: 0.98:  66%|██████▋   | 311/468 [00:17<00:09, 15.86it/s][A
avg reward: 0.98:  66%|██████▋   | 311/468 [00:18<00:09, 15.86it/s][A
avg reward: 0.98:  66%|██████▋   | 311/468 [00:18<00:09, 15.86it/s][A
avg reward: 0.98:  66%|██████▋   | 311/468 [00:18<00:09, 15.86it/s][A
avg reward: 0.97:  66%|██████▋   | 311/468 [00:18<00:09, 15.86it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  68%|██████▊   | 316/468 [00:18<00:09, 16.87it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  69%|██████▊   | 321/468 [00:18<00:08, 17.03it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:18<00:08, 17.61it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:18<00:08, 17.61it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:18<00:08, 17.61it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:18<00:08, 17.61it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:18<00:08, 17.61it/s][A
avg reward: 0.97:  70%|██████▉   | 326/468 [00:19<00:08, 17.61it/s][A
avg reward: 0.97:  71%|███████   | 331/468 [00:19<00:07, 17.72it/s][A
avg reward: 0.97:  71%|███████   | 331/468 [00:19<00:07, 17.72it/s][A
avg reward: 0.97:  71%|███████   | 331/468 [00:19<00:07, 17.72it/s][A
avg reward: 0.97:  71%|███████   | 331/468 [00:19<00:07, 17.72it/s][A
avg reward: 0.97:  71%|███████   | 331/468 [00:19<00:07, 17.72it/s][A
avg reward: 0.97:  72%|███████▏  | 335/468 [00:19<00:06, 20.46it/s][A
avg reward: 0.97:  72%|███████▏  | 335/468 [00:19<00:06, 20.46it/s][A
avg reward: 0.97:  72%|███████▏  | 335/468 [00:19<00:06, 20.46it/s][A
avg reward: 0.97:  72%|███████▏  | 335/468 [00:19<00:06, 20.46it/s][A
avg reward: 0.97:  72%|███████▏  | 338/468 [00:19<00:07, 17.44it/s][A
avg reward: 0.97:  72%|███████▏  | 338/468 [00:19<00:07, 17.44it/s][A
avg reward: 0.97:  72%|███████▏  | 338/468 [00:19<00:07, 17.44it/s][A
avg reward: 0.98:  72%|███████▏  | 338/468 [00:19<00:07, 17.44it/s][A
avg reward: 0.98:  73%|███████▎  | 341/468 [00:19<00:07, 16.18it/s][A
avg reward: 0.98:  73%|███████▎  | 341/468 [00:19<00:07, 16.18it/s][A
avg reward: 0.98:  73%|███████▎  | 341/468 [00:19<00:07, 16.18it/s][A
avg reward: 0.98:  73%|███████▎  | 341/468 [00:19<00:07, 16.18it/s][A
avg reward: 0.98:  73%|███████▎  | 341/468 [00:19<00:07, 16.18it/s][A
avg reward: 0.98:  74%|███████▎  | 345/468 [00:19<00:06, 19.63it/s][A
avg reward: 0.98:  74%|███████▎  | 345/468 [00:19<00:06, 19.63it/s][A
avg reward: 0.98:  74%|███████▎  | 345/468 [00:19<00:06, 19.63it/s][A
avg reward: 0.98:  74%|███████▎  | 345/468 [00:20<00:06, 19.63it/s][A
avg reward: 0.98:  74%|███████▍  | 348/468 [00:20<00:07, 16.96it/s][A
avg reward: 0.98:  74%|███████▍  | 348/468 [00:20<00:07, 16.96it/s][A
avg reward: 0.98:  74%|███████▍  | 348/468 [00:20<00:07, 16.96it/s][A
avg reward: 0.98:  74%|███████▍  | 348/468 [00:20<00:07, 16.96it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  75%|███████▌  | 351/468 [00:20<00:07, 15.86it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  76%|███████▌  | 356/468 [00:20<00:06, 16.56it/s][A
avg reward: 0.98:  77%|███████▋  | 361/468 [00:20<00:06, 17.02it/s][A
avg reward: 0.98:  77%|███████▋  | 361/468 [00:20<00:06, 17.02it/s][A
avg reward: 0.98:  77%|███████▋  | 361/468 [00:20<00:06, 17.02it/s][A
avg reward: 0.98:  77%|███████▋  | 361/468 [00:20<00:06, 17.02it/s][A
avg reward: 0.98:  77%|███████▋  | 361/468 [00:20<00:06, 17.02it/s][A
avg reward: 0.98:  78%|███████▊  | 365/468 [00:20<00:05, 20.26it/s][A
avg reward: 0.98:  78%|███████▊  | 365/468 [00:21<00:05, 20.26it/s][A
avg reward: 0.98:  78%|███████▊  | 365/468 [00:21<00:05, 20.26it/s][A
avg reward: 0.98:  78%|███████▊  | 365/468 [00:21<00:05, 20.26it/s][A
avg reward: 0.98:  79%|███████▊  | 368/468 [00:21<00:05, 17.99it/s][A
avg reward: 0.98:  79%|███████▊  | 368/468 [00:21<00:05, 17.99it/s][A
avg reward: 0.98:  79%|███████▊  | 368/468 [00:21<00:05, 17.99it/s][A
avg reward: 0.99:  79%|███████▊  | 368/468 [00:21<00:05, 17.99it/s][A
avg reward: 0.99:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.99:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.99:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.99:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.99:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.98:  79%|███████▉  | 371/468 [00:21<00:05, 16.45it/s][A
avg reward: 0.98:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.98:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.98:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.98:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.98:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.99:  80%|████████  | 376/468 [00:21<00:05, 16.80it/s][A
avg reward: 0.99:  81%|████████▏ | 381/468 [00:21<00:05, 17.25it/s][A
avg reward: 0.99:  81%|████████▏ | 381/468 [00:21<00:05, 17.25it/s][A
avg reward: 0.99:  81%|████████▏ | 381/468 [00:21<00:05, 17.25it/s][A
avg reward: 0.99:  81%|████████▏ | 381/468 [00:21<00:05, 17.25it/s][A
avg reward: 0.99:  81%|████████▏ | 381/468 [00:22<00:05, 17.25it/s][A
avg reward: 0.99:  82%|████████▏ | 385/468 [00:22<00:04, 20.49it/s][A
avg reward: 0.99:  82%|████████▏ | 385/468 [00:22<00:04, 20.49it/s][A
avg reward: 0.99:  82%|████████▏ | 385/468 [00:22<00:04, 20.49it/s][A
avg reward: 0.99:  82%|████████▏ | 385/468 [00:22<00:04, 20.49it/s][A
avg reward: 0.99:  83%|████████▎ | 388/468 [00:22<00:04, 17.85it/s][A
avg reward: 0.99:  83%|████████▎ | 388/468 [00:22<00:04, 17.85it/s][A
avg reward: 0.99:  83%|████████▎ | 388/468 [00:22<00:04, 17.85it/s][A
avg reward: 0.99:  83%|████████▎ | 388/468 [00:22<00:04, 17.85it/s][A
avg reward: 0.99:  84%|████████▎ | 391/468 [00:22<00:04, 16.74it/s][A
avg reward: 0.99:  84%|████████▎ | 391/468 [00:22<00:04, 16.74it/s][A
avg reward: 0.99:  84%|████████▎ | 391/468 [00:22<00:04, 16.74it/s][A
avg reward: 0.99:  84%|████████▎ | 391/468 [00:22<00:04, 16.74it/s][A
avg reward: 0.99:  84%|████████▎ | 391/468 [00:22<00:04, 16.74it/s][A
avg reward: 0.99:  84%|████████▍ | 395/468 [00:22<00:03, 20.59it/s][A
avg reward: 0.99:  84%|████████▍ | 395/468 [00:22<00:03, 20.59it/s][A
avg reward: 0.99:  84%|████████▍ | 395/468 [00:22<00:03, 20.59it/s][A
avg reward: 0.99:  84%|████████▍ | 395/468 [00:22<00:03, 20.59it/s][A
avg reward: 0.99:  85%|████████▌ | 398/468 [00:22<00:03, 17.83it/s][A
avg reward: 0.99:  85%|████████▌ | 398/468 [00:22<00:03, 17.83it/s][A
avg reward: 0.99:  85%|████████▌ | 398/468 [00:22<00:03, 17.83it/s][A
avg reward: 0.99:  85%|████████▌ | 398/468 [00:23<00:03, 17.83it/s][A
avg reward: 0.99:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 0.99:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 0.99:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 0.99:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 0.99:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 1.00:  86%|████████▌ | 401/468 [00:23<00:04, 16.14it/s][A
avg reward: 1.00:  87%|████████▋ | 406/468 [00:23<00:03, 16.84it/s][A
avg reward: 1.00:  87%|████████▋ | 406/468 [00:23<00:03, 16.84it/s][A
avg reward: 1.00:  87%|████████▋ | 406/468 [00:23<00:03, 16.84it/s][A
avg reward: 1.00:  87%|████████▋ | 406/468 [00:23<00:03, 16.84it/s][A
avg reward: 1.00:  87%|████████▋ | 406/468 [00:23<00:03, 16.84it/s][A
avg reward: 1.00:  88%|████████▊ | 410/468 [00:23<00:02, 20.18it/s][A
avg reward: 1.00:  88%|████████▊ | 410/468 [00:23<00:02, 20.18it/s][A
avg reward: 1.00:  88%|████████▊ | 410/468 [00:23<00:02, 20.18it/s][A
avg reward: 1.00:  88%|████████▊ | 410/468 [00:23<00:02, 20.18it/s][A
avg reward: 1.00:  88%|████████▊ | 413/468 [00:23<00:03, 17.48it/s][A
avg reward: 1.00:  88%|████████▊ | 413/468 [00:23<00:03, 17.48it/s][A
avg reward: 1.00:  88%|████████▊ | 413/468 [00:23<00:03, 17.48it/s][A
avg reward: 1.00:  88%|████████▊ | 413/468 [00:23<00:03, 17.48it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:23<00:03, 16.02it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:23<00:03, 16.02it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:23<00:03, 16.02it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:23<00:03, 16.02it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:23<00:03, 16.02it/s][A
avg reward: 1.00:  89%|████████▉ | 416/468 [00:24<00:03, 16.02it/s][A
avg reward: 1.00:  90%|████████▉ | 421/468 [00:24<00:02, 17.02it/s][A
avg reward: 1.00:  90%|████████▉ | 421/468 [00:24<00:02, 17.02it/s][A
avg reward: 1.00:  90%|████████▉ | 421/468 [00:24<00:02, 17.02it/s][A
avg reward: 1.00:  90%|████████▉ | 421/468 [00:24<00:02, 17.02it/s][A
avg reward: 1.00:  90%|████████▉ | 421/468 [00:24<00:02, 17.02it/s][A
avg reward: 1.00:  91%|█████████ | 425/468 [00:24<00:02, 20.41it/s][A
avg reward: 1.00:  91%|█████████ | 425/468 [00:24<00:02, 20.41it/s][A
avg reward: 1.00:  91%|█████████ | 425/468 [00:24<00:02, 20.41it/s][A
avg reward: 1.00:  91%|█████████ | 425/468 [00:24<00:02, 20.41it/s][A
avg reward: 1.00:  91%|█████████▏| 428/468 [00:24<00:02, 17.64it/s][A
avg reward: 1.00:  91%|█████████▏| 428/468 [00:24<00:02, 17.64it/s][A
avg reward: 1.00:  91%|█████████▏| 428/468 [00:24<00:02, 17.64it/s][A
avg reward: 1.00:  91%|█████████▏| 428/468 [00:24<00:02, 17.64it/s][A
avg reward: 1.00:  92%|█████████▏| 431/468 [00:24<00:02, 15.97it/s][A
avg reward: 1.00:  92%|█████████▏| 431/468 [00:24<00:02, 15.97it/s][A
avg reward: 1.00:  92%|█████████▏| 431/468 [00:24<00:02, 15.97it/s][A
avg reward: 1.00:  92%|█████████▏| 431/468 [00:24<00:02, 15.97it/s][A
avg reward: 1.00:  92%|█████████▏| 431/468 [00:24<00:02, 15.97it/s][A
avg reward: 1.00:  93%|█████████▎| 435/468 [00:24<00:01, 19.78it/s][A
avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 19.78it/s][A
avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 19.78it/s][A
avg reward: 1.01:  93%|█████████▎| 435/468 [00:25<00:01, 19.78it/s][A
avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.79it/s][A
avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.79it/s][A
avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.79it/s][A
avg reward: 1.01:  94%|█████████▎| 438/468 [00:25<00:01, 16.79it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  94%|█████████▍| 441/468 [00:25<00:01, 15.56it/s][A
avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.47it/s][A
avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.47it/s][A
avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.47it/s][A
avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.47it/s][A
avg reward: 1.01:  95%|█████████▌| 446/468 [00:25<00:01, 16.47it/s][A
avg reward: 1.01:  96%|█████████▌| 450/468 [00:25<00:00, 19.79it/s][A
avg reward: 1.01:  96%|█████████▌| 450/468 [00:25<00:00, 19.79it/s][A
avg reward: 1.01:  96%|█████████▌| 450/468 [00:25<00:00, 19.79it/s][A
avg reward: 1.01:  96%|█████████▌| 450/468 [00:25<00:00, 19.79it/s][A
avg reward: 1.01:  97%|█████████▋| 453/468 [00:25<00:00, 17.69it/s][A
avg reward: 1.01:  97%|█████████▋| 453/468 [00:25<00:00, 17.69it/s][A
avg reward: 1.01:  97%|█████████▋| 453/468 [00:25<00:00, 17.69it/s][A
avg reward: 1.01:  97%|█████████▋| 453/468 [00:26<00:00, 17.69it/s][A
avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 16.43it/s][A
avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 16.43it/s][A
avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 16.43it/s][A
avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 16.43it/s][A
avg reward: 1.01:  97%|█████████▋| 456/468 [00:26<00:00, 16.43it/s][A
avg reward: 1.01:  98%|█████████▊| 460/468 [00:26<00:00, 20.02it/s][A
avg reward: 1.01:  98%|█████████▊| 460/468 [00:26<00:00, 20.02it/s][A
avg reward: 1.01:  98%|█████████▊| 460/468 [00:26<00:00, 20.02it/s][A
avg reward: 1.01:  98%|█████████▊| 460/468 [00:26<00:00, 20.02it/s][A
avg reward: 1.01:  99%|█████████▉| 463/468 [00:26<00:00, 17.71it/s][A
avg reward: 1.01:  99%|█████████▉| 463/468 [00:26<00:00, 17.71it/s][A
avg reward: 1.01:  99%|█████████▉| 463/468 [00:26<00:00, 17.71it/s][A
avg reward: 1.01:  99%|█████████▉| 463/468 [00:26<00:00, 17.71it/s][A
avg reward: 1.01: 100%|█████████▉| 466/468 [00:26<00:00, 16.24it/s][A
avg reward: 1.01: 100%|█████████▉| 466/468 [00:26<00:00, 16.24it/s][A
avg reward: 1.01: 100%|██████████| 468/468 [00:26<00:00, 17.48it/s]

  0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9765625:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.98828125:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9817708333333334:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.974609375:   0%|          | 0/79 [00:00<?, ?it/s]       [A
Val Acc: 0.9765625:   0%|          | 0/79 [00:00<?, ?it/s]  [A
Val Acc: 0.9778645833333334:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9778645833333334:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9787946428571429:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9794921875:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s]      [A
Val Acc: 0.9791666666666666:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.98046875:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s]        [A
Val Acc: 0.9815340909090909:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9830729166666666:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9807692307692307:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9815848214285714:   8%|▊         | 6/79 [00:00<00:01, 56.03it/s][A
Val Acc: 0.9815848214285714:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.9807291666666667:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.98095703125:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s]     [A
Val Acc: 0.9811580882352942:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.9809027777777778:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.9794407894736842:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.979296875:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s]       [A
Val Acc: 0.9791666666666666:  18%|█▊        | 14/79 [00:00<00:00, 65.78it/s][A
Val Acc: 0.9791666666666666:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.9801136363636364:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.9802989130434783:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.98046875:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s]        [A
Val Acc: 0.98:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s]      [A
Val Acc: 0.9801682692307693:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.9806134259259259:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.9807477678571429:  27%|██▋       | 21/79 [00:00<00:00, 66.62it/s][A
Val Acc: 0.9807477678571429:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9806034482758621:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9802083333333333:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9798387096774194:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.979736328125:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s]    [A
Val Acc: 0.9798768939393939:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9793198529411765:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9787946428571429:  35%|███▌      | 28/79 [00:00<00:00, 66.23it/s][A
Val Acc: 0.9787946428571429:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.9793836805555556:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.979518581081081:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s] [A
Val Acc: 0.9796463815789473:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.9789663461538461:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.97890625:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s]        [A
Val Acc: 0.9784679878048781:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.9786086309523809:  44%|████▍     | 35/79 [00:00<00:00, 67.50it/s][A
Val Acc: 0.9786086309523809:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.9783793604651163:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.9783380681818182:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.978125:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s]          [A
Val Acc: 0.9782608695652174:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.9785571808510638:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.97900390625:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s]     [A
Val Acc: 0.9792729591836735:  53%|█████▎    | 42/79 [00:00<00:00, 66.65it/s][A
Val Acc: 0.9792729591836735:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.97921875:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s]        [A
Val Acc: 0.9787071078431373:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9786658653846154:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9787735849056604:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9788773148148148:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9786931818181818:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9789341517857143:  62%|██████▏   | 49/79 [00:00<00:00, 65.56it/s][A
Val Acc: 0.9789341517857143:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9787554824561403:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9787176724137931:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9788135593220338:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.978515625:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s]       [A
Val Acc: 0.9784836065573771:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9788306451612904:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9791666666666666:  71%|███████   | 56/79 [00:00<00:00, 66.78it/s][A
Val Acc: 0.9791666666666666:  80%|███████▉  | 63/79 [00:00<00:00, 67.59it/s][A
Val Acc: 0.9793701171875:  80%|███████▉  | 63/79 [00:00<00:00, 67.59it/s]   [A
Val Acc: 0.9794471153846154:  80%|███████▉  | 63/79 [00:00<00:00, 67.59it/s][A
Val Acc: 0.9796401515151515:  80%|███████▉  | 63/79 [00:00<00:00, 67.59it/s][A
Val Acc: 0.9795942164179104:  80%|███████▉  | 63/79 [00:01<00:00, 67.59it/s][A
Val Acc: 0.9796645220588235:  80%|███████▉  | 63/79 [00:01<00:00, 67.59it/s][A
Val Acc: 0.9797327898550725:  80%|███████▉  | 63/79 [00:01<00:00, 67.59it/s][A
Val Acc: 0.9797991071428571:  80%|███████▉  | 63/79 [00:01<00:00, 67.59it/s][A
Val Acc: 0.9797991071428571:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9799735915492958:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9801432291666666:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9799871575342466:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.980152027027027:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s] [A
Val Acc: 0.9798958333333333:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9796463815789473:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9795048701298701:  89%|████████▊ | 70/79 [00:01<00:00, 66.52it/s][A
Val Acc: 0.9795048701298701:  97%|█████████▋| 77/79 [00:01<00:00, 66.53it/s][A
Val Acc: 0.9797676282051282:  97%|█████████▋| 77/79 [00:01<00:00, 66.53it/s][A
Val Acc: 0.9796: 100%|██████████| 79/79 [00:01<00:00, 66.44it/s]
epoch 0, val_acc: 0.98, avg reward: 1.01: 100%|██████████| 1/1 [00:28<00:00, 28.01s/it]

The resulting architecture is:

mynet.graph

../../_images/output_enas_proxylessnas_ad55b9_21_0.svg

Change the reward trade-off:

reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.8)
mynet.initialize(force_reinit=True)
scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                           reward_fn=reward_fn, batch_size=128, num_gpus=1,
                           warmup_epochs=0, epochs=1, controller_lr=3e-3,
                           plot_frequency=10, update_arch_frequency=5)
scheduler.run()

  0%|          | 0/1 [00:00<?, ?it/s]
  0%|          | 0/468 [00:00<?, ?it/s][A
avg reward: 0.12:   0%|          | 0/468 [00:00<?, ?it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   0%|          | 1/468 [00:00<02:02,  3.80it/s][A
avg reward: 0.12:   1%|▏         | 6/468 [00:00<00:37, 12.47it/s][A
avg reward: 0.12:   1%|▏         | 6/468 [00:00<00:37, 12.47it/s][A
avg reward: 0.12:   1%|▏         | 6/468 [00:00<00:37, 12.47it/s][A
avg reward: 0.12:   1%|▏         | 6/468 [00:00<00:37, 12.47it/s][A
avg reward: 0.12:   1%|▏         | 6/468 [00:00<00:37, 12.47it/s][A
avg reward: 0.12:   2%|▏         | 10/468 [00:00<00:24, 18.81it/s][A
avg reward: 0.12:   2%|▏         | 10/468 [00:00<00:24, 18.81it/s][A
avg reward: 0.12:   2%|▏         | 10/468 [00:00<00:24, 18.81it/s][A
avg reward: 0.12:   2%|▏         | 10/468 [00:00<00:24, 18.81it/s][A
avg reward: 0.12:   3%|▎         | 13/468 [00:00<00:28, 15.79it/s][A
avg reward: 0.12:   3%|▎         | 13/468 [00:00<00:28, 15.79it/s][A
avg reward: 0.12:   3%|▎         | 13/468 [00:00<00:28, 15.79it/s][A
avg reward: 0.12:   3%|▎         | 13/468 [00:01<00:28, 15.79it/s][A
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:30, 14.77it/s][A
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:30, 14.77it/s][A
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:30, 14.77it/s][A
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:30, 14.77it/s][A
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:30, 14.77it/s][A
avg reward: 0.12:   4%|▍         | 20/468 [00:01<00:23, 19.45it/s][A
avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:23, 19.45it/s][A
avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:23, 19.45it/s][A
avg reward: 0.13:   4%|▍         | 20/468 [00:01<00:23, 19.45it/s][A
avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:27, 16.33it/s][A
avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:27, 16.33it/s][A
avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:27, 16.33it/s][A
avg reward: 0.13:   5%|▍         | 23/468 [00:01<00:27, 16.33it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:29, 14.93it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:01<00:29, 14.93it/s][A
avg reward: 0.13:   6%|▌         | 26/468 [00:02<00:29, 14.93it/s][A
avg reward: 0.13:   6%|▌         | 28/468 [00:02<00:37, 11.74it/s][A
avg reward: 0.13:   6%|▌         | 28/468 [00:02<00:37, 11.74it/s][A
avg reward: 0.13:   6%|▌         | 28/468 [00:02<00:37, 11.74it/s][A
avg reward: 0.15:   6%|▌         | 28/468 [00:02<00:37, 11.74it/s][A
avg reward: 0.15:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.15:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.15:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.15:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.15:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.28:   7%|▋         | 31/468 [00:02<00:45,  9.53it/s][A
avg reward: 0.28:   8%|▊         | 36/468 [00:02<00:35, 12.02it/s][A
avg reward: 0.28:   8%|▊         | 36/468 [00:02<00:35, 12.02it/s][A
avg reward: 0.28:   8%|▊         | 36/468 [00:02<00:35, 12.02it/s][A
avg reward: 0.28:   8%|▊         | 36/468 [00:02<00:35, 12.02it/s][A
avg reward: 0.28:   8%|▊         | 36/468 [00:02<00:35, 12.02it/s][A
avg reward: 0.45:   8%|▊         | 36/468 [00:03<00:35, 12.02it/s][A
avg reward: 0.45:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.45:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.45:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.45:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.45:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.52:   9%|▉         | 41/468 [00:03<00:31, 13.74it/s][A
avg reward: 0.52:  10%|▉         | 46/468 [00:03<00:28, 14.74it/s][A
avg reward: 0.52:  10%|▉         | 46/468 [00:03<00:28, 14.74it/s][A
avg reward: 0.52:  10%|▉         | 46/468 [00:03<00:28, 14.74it/s][A
avg reward: 0.52:  10%|▉         | 46/468 [00:03<00:28, 14.74it/s][A
avg reward: 0.52:  10%|▉         | 46/468 [00:03<00:28, 14.74it/s][A
avg reward: 0.52:  11%|█         | 50/468 [00:03<00:23, 17.96it/s][A
avg reward: 0.59:  11%|█         | 50/468 [00:03<00:23, 17.96it/s][A
avg reward: 0.59:  11%|█         | 50/468 [00:03<00:23, 17.96it/s][A
avg reward: 0.59:  11%|█         | 50/468 [00:03<00:23, 17.96it/s][A
avg reward: 0.59:  11%|█▏        | 53/468 [00:03<00:25, 16.09it/s][A
avg reward: 0.59:  11%|█▏        | 53/468 [00:03<00:25, 16.09it/s][A
avg reward: 0.59:  11%|█▏        | 53/468 [00:03<00:25, 16.09it/s][A
avg reward: 0.63:  11%|█▏        | 53/468 [00:03<00:25, 16.09it/s][A
avg reward: 0.63:  12%|█▏        | 56/468 [00:03<00:28, 14.23it/s][A
avg reward: 0.63:  12%|█▏        | 56/468 [00:03<00:28, 14.23it/s][A
avg reward: 0.63:  12%|█▏        | 56/468 [00:03<00:28, 14.23it/s][A
avg reward: 0.63:  12%|█▏        | 56/468 [00:04<00:28, 14.23it/s][A
avg reward: 0.63:  12%|█▏        | 56/468 [00:04<00:28, 14.23it/s][A
avg reward: 0.63:  13%|█▎        | 60/468 [00:04<00:22, 17.95it/s][A
avg reward: 0.65:  13%|█▎        | 60/468 [00:04<00:22, 17.95it/s][A
avg reward: 0.65:  13%|█▎        | 60/468 [00:04<00:22, 17.95it/s][A
avg reward: 0.65:  13%|█▎        | 60/468 [00:04<00:22, 17.95it/s][A
avg reward: 0.65:  13%|█▎        | 63/468 [00:04<00:24, 16.50it/s][A
avg reward: 0.65:  13%|█▎        | 63/468 [00:04<00:24, 16.50it/s][A
avg reward: 0.65:  13%|█▎        | 63/468 [00:04<00:24, 16.50it/s][A
avg reward: 0.61:  13%|█▎        | 63/468 [00:04<00:24, 16.50it/s][A
avg reward: 0.61:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.61:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.61:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.61:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.61:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.66:  14%|█▍        | 66/468 [00:04<00:25, 15.54it/s][A
avg reward: 0.66:  15%|█▌        | 71/468 [00:04<00:24, 16.54it/s][A
avg reward: 0.66:  15%|█▌        | 71/468 [00:04<00:24, 16.54it/s][A
avg reward: 0.66:  15%|█▌        | 71/468 [00:04<00:24, 16.54it/s][A
avg reward: 0.66:  15%|█▌        | 71/468 [00:04<00:24, 16.54it/s][A
avg reward: 0.66:  15%|█▌        | 71/468 [00:04<00:24, 16.54it/s][A
avg reward: 0.67:  15%|█▌        | 71/468 [00:05<00:24, 16.54it/s][A
avg reward: 0.67:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.67:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.67:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.67:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.67:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.74:  16%|█▌        | 76/468 [00:05<00:23, 16.98it/s][A
avg reward: 0.74:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.74:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.74:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.74:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.74:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.81:  17%|█▋        | 81/468 [00:05<00:21, 17.70it/s][A
avg reward: 0.81:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.81:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.81:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.81:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.81:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.73:  18%|█▊        | 86/468 [00:05<00:21, 17.80it/s][A
avg reward: 0.73:  19%|█▉        | 91/468 [00:05<00:20, 18.03it/s][A
avg reward: 0.73:  19%|█▉        | 91/468 [00:05<00:20, 18.03it/s][A
avg reward: 0.73:  19%|█▉        | 91/468 [00:05<00:20, 18.03it/s][A
avg reward: 0.73:  19%|█▉        | 91/468 [00:05<00:20, 18.03it/s][A
avg reward: 0.73:  19%|█▉        | 91/468 [00:05<00:20, 18.03it/s][A
avg reward: 0.77:  19%|█▉        | 91/468 [00:06<00:20, 18.03it/s][A
avg reward: 0.77:  21%|██        | 96/468 [00:06<00:20, 17.94it/s][A
avg reward: 0.77:  21%|██        | 96/468 [00:06<00:20, 17.94it/s][A
avg reward: 0.77:  21%|██        | 96/468 [00:06<00:20, 17.94it/s][A
avg reward: 0.77:  21%|██        | 96/468 [00:06<00:20, 17.94it/s][A
avg reward: 0.77:  21%|██        | 96/468 [00:06<00:20, 17.94it/s][A
avg reward: 0.77:  21%|██▏       | 100/468 [00:06<00:17, 20.64it/s][A
avg reward: 0.83:  21%|██▏       | 100/468 [00:06<00:17, 20.64it/s][A
avg reward: 0.83:  21%|██▏       | 100/468 [00:06<00:17, 20.64it/s][A
avg reward: 0.83:  21%|██▏       | 100/468 [00:06<00:17, 20.64it/s][A
avg reward: 0.83:  22%|██▏       | 103/468 [00:06<00:19, 18.48it/s][A
avg reward: 0.83:  22%|██▏       | 103/468 [00:06<00:19, 18.48it/s][A
avg reward: 0.83:  22%|██▏       | 103/468 [00:06<00:19, 18.48it/s][A
avg reward: 0.87:  22%|██▏       | 103/468 [00:06<00:19, 18.48it/s][A
avg reward: 0.87:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.87:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.87:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.87:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.87:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.88:  23%|██▎       | 106/468 [00:06<00:20, 17.26it/s][A
avg reward: 0.88:  24%|██▎       | 111/468 [00:06<00:20, 17.74it/s][A
avg reward: 0.88:  24%|██▎       | 111/468 [00:06<00:20, 17.74it/s][A
avg reward: 0.88:  24%|██▎       | 111/468 [00:06<00:20, 17.74it/s][A
avg reward: 0.88:  24%|██▎       | 111/468 [00:07<00:20, 17.74it/s][A
avg reward: 0.88:  24%|██▎       | 111/468 [00:07<00:20, 17.74it/s][A
avg reward: 0.88:  25%|██▍       | 115/468 [00:07<00:16, 20.84it/s][A
avg reward: 0.87:  25%|██▍       | 115/468 [00:07<00:16, 20.84it/s][A
avg reward: 0.87:  25%|██▍       | 115/468 [00:07<00:16, 20.84it/s][A
avg reward: 0.87:  25%|██▍       | 115/468 [00:07<00:16, 20.84it/s][A
avg reward: 0.87:  25%|██▌       | 118/468 [00:07<00:19, 18.30it/s][A
avg reward: 0.87:  25%|██▌       | 118/468 [00:07<00:19, 18.30it/s][A
avg reward: 0.87:  25%|██▌       | 118/468 [00:07<00:19, 18.30it/s][A
avg reward: 0.88:  25%|██▌       | 118/468 [00:07<00:19, 18.30it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.88:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.89:  26%|██▌       | 121/468 [00:07<00:20, 17.05it/s][A
avg reward: 0.89:  27%|██▋       | 126/468 [00:07<00:19, 17.28it/s][A
avg reward: 0.89:  27%|██▋       | 126/468 [00:07<00:19, 17.28it/s][A
avg reward: 0.89:  27%|██▋       | 126/468 [00:07<00:19, 17.28it/s][A
avg reward: 0.89:  27%|██▋       | 126/468 [00:07<00:19, 17.28it/s][A
avg reward: 0.89:  27%|██▋       | 126/468 [00:07<00:19, 17.28it/s][A
avg reward: 0.92:  27%|██▋       | 126/468 [00:08<00:19, 17.28it/s][A
avg reward: 0.92:  28%|██▊       | 131/468 [00:08<00:19, 17.69it/s][A
avg reward: 0.92:  28%|██▊       | 131/468 [00:08<00:19, 17.69it/s][A
avg reward: 0.92:  28%|██▊       | 131/468 [00:08<00:19, 17.69it/s][A
avg reward: 0.92:  28%|██▊       | 131/468 [00:08<00:19, 17.69it/s][A
avg reward: 0.92:  28%|██▊       | 131/468 [00:08<00:19, 17.69it/s][A
avg reward: 0.92:  29%|██▉       | 135/468 [00:08<00:16, 20.78it/s][A
avg reward: 0.94:  29%|██▉       | 135/468 [00:08<00:16, 20.78it/s][A
avg reward: 0.94:  29%|██▉       | 135/468 [00:08<00:16, 20.78it/s][A
avg reward: 0.94:  29%|██▉       | 135/468 [00:08<00:16, 20.78it/s][A
avg reward: 0.94:  29%|██▉       | 138/468 [00:08<00:18, 18.09it/s][A
avg reward: 0.94:  29%|██▉       | 138/468 [00:08<00:18, 18.09it/s][A
avg reward: 0.94:  29%|██▉       | 138/468 [00:08<00:18, 18.09it/s][A
avg reward: 0.97:  29%|██▉       | 138/468 [00:08<00:18, 18.09it/s][A
avg reward: 0.97:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.97:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.97:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.97:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.97:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.99:  30%|███       | 141/468 [00:08<00:19, 16.50it/s][A
avg reward: 0.99:  31%|███       | 146/468 [00:08<00:19, 16.85it/s][A
avg reward: 0.99:  31%|███       | 146/468 [00:08<00:19, 16.85it/s][A
avg reward: 0.99:  31%|███       | 146/468 [00:08<00:19, 16.85it/s][A
avg reward: 0.99:  31%|███       | 146/468 [00:08<00:19, 16.85it/s][A
avg reward: 0.99:  31%|███       | 146/468 [00:09<00:19, 16.85it/s][A
avg reward: 0.99:  32%|███▏      | 150/468 [00:09<00:16, 19.84it/s][A
avg reward: 0.95:  32%|███▏      | 150/468 [00:09<00:16, 19.84it/s][A
avg reward: 0.95:  32%|███▏      | 150/468 [00:09<00:16, 19.84it/s][A
avg reward: 0.95:  32%|███▏      | 150/468 [00:09<00:16, 19.84it/s][A
avg reward: 0.95:  33%|███▎      | 153/468 [00:09<00:17, 17.84it/s][A
avg reward: 0.95:  33%|███▎      | 153/468 [00:09<00:17, 17.84it/s][A
avg reward: 0.95:  33%|███▎      | 153/468 [00:09<00:17, 17.84it/s][A
avg reward: 0.95:  33%|███▎      | 153/468 [00:09<00:17, 17.84it/s][A
avg reward: 0.95:  33%|███▎      | 156/468 [00:09<00:18, 16.47it/s][A
avg reward: 0.95:  33%|███▎      | 156/468 [00:09<00:18, 16.47it/s][A
avg reward: 0.95:  33%|███▎      | 156/468 [00:09<00:18, 16.47it/s][A
avg reward: 0.95:  33%|███▎      | 156/468 [00:09<00:18, 16.47it/s][A
avg reward: 0.95:  33%|███▎      | 156/468 [00:09<00:18, 16.47it/s][A
avg reward: 0.95:  34%|███▍      | 160/468 [00:09<00:15, 20.23it/s][A
avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 20.23it/s][A
avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 20.23it/s][A
avg reward: 0.94:  34%|███▍      | 160/468 [00:09<00:15, 20.23it/s][A
avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:17, 17.70it/s][A
avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:17, 17.70it/s][A
avg reward: 0.94:  35%|███▍      | 163/468 [00:09<00:17, 17.70it/s][A
avg reward: 0.95:  35%|███▍      | 163/468 [00:10<00:17, 17.70it/s][A
avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 16.50it/s][A
avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 16.50it/s][A
avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 16.50it/s][A
avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 16.50it/s][A
avg reward: 0.95:  35%|███▌      | 166/468 [00:10<00:18, 16.50it/s][A
avg reward: 0.95:  36%|███▋      | 170/468 [00:10<00:14, 20.63it/s][A
avg reward: 0.97:  36%|███▋      | 170/468 [00:10<00:14, 20.63it/s][A
avg reward: 0.97:  36%|███▋      | 170/468 [00:10<00:14, 20.63it/s][A
avg reward: 0.97:  36%|███▋      | 170/468 [00:10<00:14, 20.63it/s][A
avg reward: 0.97:  37%|███▋      | 173/468 [00:10<00:16, 17.99it/s][A
avg reward: 0.97:  37%|███▋      | 173/468 [00:10<00:16, 17.99it/s][A
avg reward: 0.97:  37%|███▋      | 173/468 [00:10<00:16, 17.99it/s][A
avg reward: 0.99:  37%|███▋      | 173/468 [00:10<00:16, 17.99it/s][A
avg reward: 0.99:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 0.99:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 0.99:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 0.99:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 0.99:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 1.00:  38%|███▊      | 176/468 [00:10<00:17, 16.64it/s][A
avg reward: 1.00:  39%|███▊      | 181/468 [00:10<00:16, 17.07it/s][A
avg reward: 1.00:  39%|███▊      | 181/468 [00:10<00:16, 17.07it/s][A
avg reward: 1.00:  39%|███▊      | 181/468 [00:10<00:16, 17.07it/s][A
avg reward: 1.00:  39%|███▊      | 181/468 [00:10<00:16, 17.07it/s][A
avg reward: 1.00:  39%|███▊      | 181/468 [00:10<00:16, 17.07it/s][A
avg reward: 1.00:  40%|███▉      | 185/468 [00:10<00:13, 20.72it/s][A
avg reward: 0.99:  40%|███▉      | 185/468 [00:11<00:13, 20.72it/s][A
avg reward: 0.99:  40%|███▉      | 185/468 [00:11<00:13, 20.72it/s][A
avg reward: 0.99:  40%|███▉      | 185/468 [00:11<00:13, 20.72it/s][A
avg reward: 0.99:  40%|████      | 188/468 [00:11<00:15, 17.84it/s][A
avg reward: 0.99:  40%|████      | 188/468 [00:11<00:15, 17.84it/s][A
avg reward: 0.99:  40%|████      | 188/468 [00:11<00:15, 17.84it/s][A
avg reward: 0.96:  40%|████      | 188/468 [00:11<00:15, 17.84it/s][A
avg reward: 0.96:  41%|████      | 191/468 [00:11<00:16, 16.38it/s][A
avg reward: 0.96:  41%|████      | 191/468 [00:11<00:16, 16.38it/s][A
avg reward: 0.96:  41%|████      | 191/468 [00:11<00:16, 16.38it/s][A
avg reward: 0.96:  41%|████      | 191/468 [00:11<00:16, 16.38it/s][A
avg reward: 0.96:  41%|████      | 191/468 [00:11<00:16, 16.38it/s][A
avg reward: 0.96:  42%|████▏     | 195/468 [00:11<00:13, 20.06it/s][A
avg reward: 0.98:  42%|████▏     | 195/468 [00:11<00:13, 20.06it/s][A
avg reward: 0.98:  42%|████▏     | 195/468 [00:11<00:13, 20.06it/s][A
avg reward: 0.98:  42%|████▏     | 195/468 [00:11<00:13, 20.06it/s][A
avg reward: 0.98:  42%|████▏     | 198/468 [00:11<00:15, 17.47it/s][A
avg reward: 0.98:  42%|████▏     | 198/468 [00:11<00:15, 17.47it/s][A
avg reward: 0.98:  42%|████▏     | 198/468 [00:11<00:15, 17.47it/s][A
avg reward: 0.97:  42%|████▏     | 198/468 [00:11<00:15, 17.47it/s][A
avg reward: 0.97:  43%|████▎     | 201/468 [00:11<00:16, 15.83it/s][A
avg reward: 0.97:  43%|████▎     | 201/468 [00:12<00:16, 15.83it/s][A
avg reward: 0.97:  43%|████▎     | 201/468 [00:12<00:16, 15.83it/s][A
avg reward: 0.97:  43%|████▎     | 201/468 [00:12<00:16, 15.83it/s][A
avg reward: 0.97:  43%|████▎     | 201/468 [00:12<00:16, 15.83it/s][A
avg reward: 0.96:  43%|████▎     | 201/468 [00:12<00:16, 15.83it/s][A
avg reward: 0.96:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.96:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.96:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.96:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.96:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.98:  44%|████▍     | 206/468 [00:12<00:15, 16.94it/s][A
avg reward: 0.98:  45%|████▌     | 211/468 [00:12<00:14, 17.21it/s][A
avg reward: 0.98:  45%|████▌     | 211/468 [00:12<00:14, 17.21it/s][A
avg reward: 0.98:  45%|████▌     | 211/468 [00:12<00:14, 17.21it/s][A
avg reward: 0.98:  45%|████▌     | 211/468 [00:12<00:14, 17.21it/s][A
avg reward: 0.98:  45%|████▌     | 211/468 [00:12<00:14, 17.21it/s][A
avg reward: 0.98:  46%|████▌     | 215/468 [00:12<00:12, 20.57it/s][A
avg reward: 1.01:  46%|████▌     | 215/468 [00:12<00:12, 20.57it/s][A
avg reward: 1.01:  46%|████▌     | 215/468 [00:12<00:12, 20.57it/s][A
avg reward: 1.01:  46%|████▌     | 215/468 [00:12<00:12, 20.57it/s][A
avg reward: 1.01:  47%|████▋     | 218/468 [00:12<00:13, 18.36it/s][A
avg reward: 1.01:  47%|████▋     | 218/468 [00:12<00:13, 18.36it/s][A
avg reward: 1.01:  47%|████▋     | 218/468 [00:12<00:13, 18.36it/s][A
avg reward: 1.03:  47%|████▋     | 218/468 [00:13<00:13, 18.36it/s][A
avg reward: 1.03:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.03:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.03:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.03:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.03:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.04:  47%|████▋     | 221/468 [00:13<00:15, 15.99it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  48%|████▊     | 226/468 [00:13<00:14, 16.80it/s][A
avg reward: 1.04:  49%|████▉     | 231/468 [00:13<00:13, 16.98it/s][A
avg reward: 1.04:  49%|████▉     | 231/468 [00:13<00:13, 16.98it/s][A
avg reward: 1.04:  49%|████▉     | 231/468 [00:13<00:13, 16.98it/s][A
avg reward: 1.04:  49%|████▉     | 231/468 [00:13<00:13, 16.98it/s][A
avg reward: 1.04:  49%|████▉     | 231/468 [00:13<00:13, 16.98it/s][A
avg reward: 1.04:  50%|█████     | 235/468 [00:13<00:11, 20.24it/s][A
avg reward: 1.05:  50%|█████     | 235/468 [00:13<00:11, 20.24it/s][A
avg reward: 1.05:  50%|█████     | 235/468 [00:13<00:11, 20.24it/s][A
avg reward: 1.05:  50%|█████     | 235/468 [00:14<00:11, 20.24it/s][A
avg reward: 1.05:  51%|█████     | 238/468 [00:14<00:12, 17.86it/s][A
avg reward: 1.05:  51%|█████     | 238/468 [00:14<00:12, 17.86it/s][A
avg reward: 1.05:  51%|█████     | 238/468 [00:14<00:12, 17.86it/s][A
avg reward: 1.07:  51%|█████     | 238/468 [00:14<00:12, 17.86it/s][A
avg reward: 1.07:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.07:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.07:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.07:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.07:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.08:  51%|█████▏    | 241/468 [00:14<00:13, 16.31it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  53%|█████▎    | 246/468 [00:14<00:13, 16.77it/s][A
avg reward: 1.08:  54%|█████▎    | 251/468 [00:14<00:12, 17.32it/s][A
avg reward: 1.08:  54%|█████▎    | 251/468 [00:14<00:12, 17.32it/s][A
avg reward: 1.08:  54%|█████▎    | 251/468 [00:14<00:12, 17.32it/s][A
avg reward: 1.08:  54%|█████▎    | 251/468 [00:14<00:12, 17.32it/s][A
avg reward: 1.08:  54%|█████▎    | 251/468 [00:14<00:12, 17.32it/s][A
avg reward: 1.09:  54%|█████▎    | 251/468 [00:15<00:12, 17.32it/s][A
avg reward: 1.09:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.09:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.09:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.09:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.09:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.08:  55%|█████▍    | 256/468 [00:15<00:12, 17.36it/s][A
avg reward: 1.08:  56%|█████▌    | 261/468 [00:15<00:11, 17.61it/s][A
avg reward: 1.08:  56%|█████▌    | 261/468 [00:15<00:11, 17.61it/s][A
avg reward: 1.08:  56%|█████▌    | 261/468 [00:15<00:11, 17.61it/s][A
avg reward: 1.08:  56%|█████▌    | 261/468 [00:15<00:11, 17.61it/s][A
avg reward: 1.08:  56%|█████▌    | 261/468 [00:15<00:11, 17.61it/s][A
avg reward: 1.08:  57%|█████▋    | 265/468 [00:15<00:09, 20.66it/s][A
avg reward: 1.09:  57%|█████▋    | 265/468 [00:15<00:09, 20.66it/s][A
avg reward: 1.09:  57%|█████▋    | 265/468 [00:15<00:09, 20.66it/s][A
avg reward: 1.09:  57%|█████▋    | 265/468 [00:15<00:09, 20.66it/s][A
avg reward: 1.09:  57%|█████▋    | 268/468 [00:15<00:11, 17.73it/s][A
avg reward: 1.09:  57%|█████▋    | 268/468 [00:15<00:11, 17.73it/s][A
avg reward: 1.09:  57%|█████▋    | 268/468 [00:15<00:11, 17.73it/s][A
avg reward: 1.13:  57%|█████▋    | 268/468 [00:15<00:11, 17.73it/s][A
avg reward: 1.13:  58%|█████▊    | 271/468 [00:15<00:12, 16.11it/s][A
avg reward: 1.13:  58%|█████▊    | 271/468 [00:15<00:12, 16.11it/s][A
avg reward: 1.13:  58%|█████▊    | 271/468 [00:16<00:12, 16.11it/s][A
avg reward: 1.13:  58%|█████▊    | 271/468 [00:16<00:12, 16.11it/s][A
avg reward: 1.13:  58%|█████▊    | 271/468 [00:16<00:12, 16.11it/s][A
avg reward: 1.13:  59%|█████▉    | 275/468 [00:16<00:09, 19.47it/s][A
avg reward: 1.16:  59%|█████▉    | 275/468 [00:16<00:09, 19.47it/s][A
avg reward: 1.16:  59%|█████▉    | 275/468 [00:16<00:09, 19.47it/s][A
avg reward: 1.16:  59%|█████▉    | 275/468 [00:16<00:09, 19.47it/s][A
avg reward: 1.16:  59%|█████▉    | 278/468 [00:16<00:11, 16.82it/s][A
avg reward: 1.16:  59%|█████▉    | 278/468 [00:16<00:11, 16.82it/s][A
avg reward: 1.16:  59%|█████▉    | 278/468 [00:16<00:11, 16.82it/s][A
avg reward: 1.18:  59%|█████▉    | 278/468 [00:16<00:11, 16.82it/s][A
avg reward: 1.18:  60%|██████    | 281/468 [00:16<00:11, 16.15it/s][A
avg reward: 1.18:  60%|██████    | 281/468 [00:16<00:11, 16.15it/s][A
avg reward: 1.18:  60%|██████    | 281/468 [00:16<00:11, 16.15it/s][A
avg reward: 1.18:  60%|██████    | 281/468 [00:16<00:11, 16.15it/s][A
avg reward: 1.18:  60%|██████    | 281/468 [00:16<00:11, 16.15it/s][A
avg reward: 1.18:  61%|██████    | 285/468 [00:16<00:09, 20.01it/s][A
avg reward: 1.19:  61%|██████    | 285/468 [00:16<00:09, 20.01it/s][A
avg reward: 1.19:  61%|██████    | 285/468 [00:16<00:09, 20.01it/s][A
avg reward: 1.19:  61%|██████    | 285/468 [00:16<00:09, 20.01it/s][A
avg reward: 1.19:  62%|██████▏   | 288/468 [00:16<00:10, 17.40it/s][A
avg reward: 1.19:  62%|██████▏   | 288/468 [00:16<00:10, 17.40it/s][A
avg reward: 1.19:  62%|██████▏   | 288/468 [00:16<00:10, 17.40it/s][A
avg reward: 1.21:  62%|██████▏   | 288/468 [00:17<00:10, 17.40it/s][A
avg reward: 1.21:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.21:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.21:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.21:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.21:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.22:  62%|██████▏   | 291/468 [00:17<00:11, 15.70it/s][A
avg reward: 1.22:  63%|██████▎   | 296/468 [00:17<00:10, 16.22it/s][A
avg reward: 1.22:  63%|██████▎   | 296/468 [00:17<00:10, 16.22it/s][A
avg reward: 1.22:  63%|██████▎   | 296/468 [00:17<00:10, 16.22it/s][A
avg reward: 1.22:  63%|██████▎   | 296/468 [00:17<00:10, 16.22it/s][A
avg reward: 1.22:  64%|██████▍   | 299/468 [00:17<00:09, 18.30it/s][A
avg reward: 1.22:  64%|██████▍   | 299/468 [00:17<00:09, 18.30it/s][A
avg reward: 1.23:  64%|██████▍   | 299/468 [00:17<00:09, 18.30it/s][A
avg reward: 1.23:  64%|██████▍   | 299/468 [00:17<00:09, 18.30it/s][A
avg reward: 1.23:  65%|██████▍   | 302/468 [00:17<00:09, 16.64it/s][A
avg reward: 1.23:  65%|██████▍   | 302/468 [00:17<00:09, 16.64it/s][A
avg reward: 1.23:  65%|██████▍   | 302/468 [00:17<00:09, 16.64it/s][A
avg reward: 1.23:  65%|██████▍   | 302/468 [00:17<00:09, 16.64it/s][A
avg reward: 1.24:  65%|██████▍   | 302/468 [00:17<00:09, 16.64it/s][A
avg reward: 1.24:  65%|██████▌   | 306/468 [00:17<00:10, 16.01it/s][A
avg reward: 1.24:  65%|██████▌   | 306/468 [00:18<00:10, 16.01it/s][A
avg reward: 1.24:  65%|██████▌   | 306/468 [00:18<00:10, 16.01it/s][A
avg reward: 1.24:  65%|██████▌   | 306/468 [00:18<00:10, 16.01it/s][A
avg reward: 1.24:  65%|██████▌   | 306/468 [00:18<00:10, 16.01it/s][A
avg reward: 1.24:  66%|██████▌   | 310/468 [00:18<00:07, 19.80it/s][A
avg reward: 1.24:  66%|██████▌   | 310/468 [00:18<00:07, 19.80it/s][A
avg reward: 1.24:  66%|██████▌   | 310/468 [00:18<00:07, 19.80it/s][A
avg reward: 1.24:  66%|██████▌   | 310/468 [00:18<00:07, 19.80it/s][A
avg reward: 1.24:  67%|██████▋   | 313/468 [00:18<00:08, 17.39it/s][A
avg reward: 1.24:  67%|██████▋   | 313/468 [00:18<00:08, 17.39it/s][A
avg reward: 1.24:  67%|██████▋   | 313/468 [00:18<00:08, 17.39it/s][A
avg reward: 1.24:  67%|██████▋   | 313/468 [00:18<00:08, 17.39it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  68%|██████▊   | 316/468 [00:18<00:09, 16.09it/s][A
avg reward: 1.24:  69%|██████▊   | 321/468 [00:18<00:08, 17.04it/s][A
avg reward: 1.24:  69%|██████▊   | 321/468 [00:18<00:08, 17.04it/s][A
avg reward: 1.24:  69%|██████▊   | 321/468 [00:18<00:08, 17.04it/s][A
avg reward: 1.24:  69%|██████▊   | 321/468 [00:18<00:08, 17.04it/s][A
avg reward: 1.24:  69%|██████▊   | 321/468 [00:18<00:08, 17.04it/s][A
avg reward: 1.24:  69%|██████▉   | 325/468 [00:18<00:06, 20.52it/s][A
avg reward: 1.25:  69%|██████▉   | 325/468 [00:19<00:06, 20.52it/s][A
avg reward: 1.25:  69%|██████▉   | 325/468 [00:19<00:06, 20.52it/s][A
avg reward: 1.25:  69%|██████▉   | 325/468 [00:19<00:06, 20.52it/s][A
avg reward: 1.25:  70%|███████   | 328/468 [00:19<00:07, 18.09it/s][A
avg reward: 1.25:  70%|███████   | 328/468 [00:19<00:07, 18.09it/s][A
avg reward: 1.25:  70%|███████   | 328/468 [00:19<00:07, 18.09it/s][A
avg reward: 1.25:  70%|███████   | 328/468 [00:19<00:07, 18.09it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  71%|███████   | 331/468 [00:19<00:08, 16.42it/s][A
avg reward: 1.25:  72%|███████▏  | 336/468 [00:19<00:07, 17.49it/s][A
avg reward: 1.25:  72%|███████▏  | 336/468 [00:19<00:07, 17.49it/s][A
avg reward: 1.25:  72%|███████▏  | 336/468 [00:19<00:07, 17.49it/s][A
avg reward: 1.25:  72%|███████▏  | 336/468 [00:19<00:07, 17.49it/s][A
avg reward: 1.25:  72%|███████▏  | 336/468 [00:19<00:07, 17.49it/s][A
avg reward: 1.25:  73%|███████▎  | 340/468 [00:19<00:06, 20.83it/s][A
avg reward: 1.26:  73%|███████▎  | 340/468 [00:19<00:06, 20.83it/s][A
avg reward: 1.26:  73%|███████▎  | 340/468 [00:19<00:06, 20.83it/s][A
avg reward: 1.26:  73%|███████▎  | 340/468 [00:19<00:06, 20.83it/s][A
avg reward: 1.26:  73%|███████▎  | 343/468 [00:19<00:06, 18.26it/s][A
avg reward: 1.26:  73%|███████▎  | 343/468 [00:19<00:06, 18.26it/s][A
avg reward: 1.26:  73%|███████▎  | 343/468 [00:20<00:06, 18.26it/s][A
avg reward: 1.25:  73%|███████▎  | 343/468 [00:20<00:06, 18.26it/s][A
avg reward: 1.25:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.25:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.25:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.25:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.25:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.26:  74%|███████▍  | 346/468 [00:20<00:07, 16.90it/s][A
avg reward: 1.26:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.26:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.26:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.26:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.26:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.27:  75%|███████▌  | 351/468 [00:20<00:06, 16.78it/s][A
avg reward: 1.27:  76%|███████▌  | 356/468 [00:20<00:06, 16.97it/s][A
avg reward: 1.27:  76%|███████▌  | 356/468 [00:20<00:06, 16.97it/s][A
avg reward: 1.27:  76%|███████▌  | 356/468 [00:20<00:06, 16.97it/s][A
avg reward: 1.27:  76%|███████▌  | 356/468 [00:20<00:06, 16.97it/s][A
avg reward: 1.27:  76%|███████▌  | 356/468 [00:20<00:06, 16.97it/s][A
avg reward: 1.26:  76%|███████▌  | 356/468 [00:21<00:06, 16.97it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  77%|███████▋  | 361/468 [00:21<00:06, 17.41it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  78%|███████▊  | 366/468 [00:21<00:05, 17.62it/s][A
avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 18.03it/s][A
avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 18.03it/s][A
avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 18.03it/s][A
avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 18.03it/s][A
avg reward: 1.26:  79%|███████▉  | 371/468 [00:21<00:05, 18.03it/s][A
avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 21.08it/s][A
avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 21.08it/s][A
avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 21.08it/s][A
avg reward: 1.26:  80%|████████  | 375/468 [00:21<00:04, 21.08it/s][A
avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:04, 18.40it/s][A
avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:04, 18.40it/s][A
avg reward: 1.26:  81%|████████  | 378/468 [00:21<00:04, 18.40it/s][A
avg reward: 1.27:  81%|████████  | 378/468 [00:22<00:04, 18.40it/s][A
avg reward: 1.27:  81%|████████▏ | 381/468 [00:22<00:05, 16.71it/s][A
avg reward: 1.27:  81%|████████▏ | 381/468 [00:22<00:05, 16.71it/s][A
avg reward: 1.27:  81%|████████▏ | 381/468 [00:22<00:05, 16.71it/s][A
avg reward: 1.27:  81%|████████▏ | 381/468 [00:22<00:05, 16.71it/s][A
avg reward: 1.27:  81%|████████▏ | 381/468 [00:22<00:05, 16.71it/s][A
avg reward: 1.27:  82%|████████▏ | 385/468 [00:22<00:04, 20.03it/s][A
avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 20.03it/s][A
avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 20.03it/s][A
avg reward: 1.26:  82%|████████▏ | 385/468 [00:22<00:04, 20.03it/s][A
avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 18.01it/s][A
avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 18.01it/s][A
avg reward: 1.26:  83%|████████▎ | 388/468 [00:22<00:04, 18.01it/s][A
avg reward: 1.27:  83%|████████▎ | 388/468 [00:22<00:04, 18.01it/s][A
avg reward: 1.27:  84%|████████▎ | 391/468 [00:22<00:04, 16.19it/s][A
avg reward: 1.27:  84%|████████▎ | 391/468 [00:22<00:04, 16.19it/s][A
avg reward: 1.27:  84%|████████▎ | 391/468 [00:22<00:04, 16.19it/s][A
avg reward: 1.27:  84%|████████▎ | 391/468 [00:22<00:04, 16.19it/s][A
avg reward: 1.27:  84%|████████▎ | 391/468 [00:22<00:04, 16.19it/s][A
avg reward: 1.27:  84%|████████▍ | 395/468 [00:22<00:03, 20.20it/s][A
avg reward: 1.27:  84%|████████▍ | 395/468 [00:23<00:03, 20.20it/s][A
avg reward: 1.27:  84%|████████▍ | 395/468 [00:23<00:03, 20.20it/s][A
avg reward: 1.27:  84%|████████▍ | 395/468 [00:23<00:03, 20.20it/s][A
avg reward: 1.27:  85%|████████▌ | 398/468 [00:23<00:04, 17.12it/s][A
avg reward: 1.27:  85%|████████▌ | 398/468 [00:23<00:04, 17.12it/s][A
avg reward: 1.27:  85%|████████▌ | 398/468 [00:23<00:04, 17.12it/s][A
avg reward: 1.26:  85%|████████▌ | 398/468 [00:23<00:04, 17.12it/s][A
avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.82it/s][A
avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.82it/s][A
avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.82it/s][A
avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.82it/s][A
avg reward: 1.26:  86%|████████▌ | 401/468 [00:23<00:04, 15.82it/s][A
avg reward: 1.26:  87%|████████▋ | 405/468 [00:23<00:03, 19.82it/s][A
avg reward: 1.26:  87%|████████▋ | 405/468 [00:23<00:03, 19.82it/s][A
avg reward: 1.26:  87%|████████▋ | 405/468 [00:23<00:03, 19.82it/s][A
avg reward: 1.26:  87%|████████▋ | 405/468 [00:23<00:03, 19.82it/s][A
avg reward: 1.26:  87%|████████▋ | 408/468 [00:23<00:03, 17.29it/s][A
avg reward: 1.26:  87%|████████▋ | 408/468 [00:23<00:03, 17.29it/s][A
avg reward: 1.26:  87%|████████▋ | 408/468 [00:23<00:03, 17.29it/s][A
avg reward: 1.26:  87%|████████▋ | 408/468 [00:23<00:03, 17.29it/s][A
avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.11it/s][A
avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.11it/s][A
avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.11it/s][A
avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.11it/s][A
avg reward: 1.26:  88%|████████▊ | 411/468 [00:23<00:03, 16.11it/s][A
avg reward: 1.26:  89%|████████▊ | 415/468 [00:23<00:02, 19.87it/s][A
avg reward: 1.26:  89%|████████▊ | 415/468 [00:24<00:02, 19.87it/s][A
avg reward: 1.26:  89%|████████▊ | 415/468 [00:24<00:02, 19.87it/s][A
avg reward: 1.26:  89%|████████▊ | 415/468 [00:24<00:02, 19.87it/s][A
avg reward: 1.26:  89%|████████▉ | 418/468 [00:24<00:02, 17.63it/s][A
avg reward: 1.26:  89%|████████▉ | 418/468 [00:24<00:02, 17.63it/s][A
avg reward: 1.26:  89%|████████▉ | 418/468 [00:24<00:02, 17.63it/s][A
avg reward: 1.26:  89%|████████▉ | 418/468 [00:24<00:02, 17.63it/s][A
avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 16.01it/s][A
avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 16.01it/s][A
avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 16.01it/s][A
avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 16.01it/s][A
avg reward: 1.26:  90%|████████▉ | 421/468 [00:24<00:02, 16.01it/s][A
avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.08it/s][A
avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.08it/s][A
avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.08it/s][A
avg reward: 1.26:  91%|█████████ | 425/468 [00:24<00:02, 20.08it/s][A
avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.23it/s][A
avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.23it/s][A
avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.23it/s][A
avg reward: 1.26:  91%|█████████▏| 428/468 [00:24<00:02, 17.23it/s][A
avg reward: 1.26:  92%|█████████▏| 431/468 [00:25<00:02, 15.68it/s][A
avg reward: 1.26:  92%|█████████▏| 431/468 [00:25<00:02, 15.68it/s][A
avg reward: 1.26:  92%|█████████▏| 431/468 [00:25<00:02, 15.68it/s][A
avg reward: 1.26:  92%|█████████▏| 431/468 [00:25<00:02, 15.68it/s][A
avg reward: 1.26:  92%|█████████▏| 431/468 [00:25<00:02, 15.68it/s][A
avg reward: 1.26:  93%|█████████▎| 435/468 [00:25<00:01, 19.47it/s][A
avg reward: 1.25:  93%|█████████▎| 435/468 [00:25<00:01, 19.47it/s][A
avg reward: 1.25:  93%|█████████▎| 435/468 [00:25<00:01, 19.47it/s][A
avg reward: 1.25:  93%|█████████▎| 435/468 [00:25<00:01, 19.47it/s][A
avg reward: 1.25:  94%|█████████▎| 438/468 [00:25<00:01, 17.29it/s][A
avg reward: 1.25:  94%|█████████▎| 438/468 [00:25<00:01, 17.29it/s][A
avg reward: 1.25:  94%|█████████▎| 438/468 [00:25<00:01, 17.29it/s][A
avg reward: 1.26:  94%|█████████▎| 438/468 [00:25<00:01, 17.29it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  94%|█████████▍| 441/468 [00:25<00:01, 15.63it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:25<00:01, 16.57it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:25<00:01, 16.57it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:25<00:01, 16.57it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:25<00:01, 16.57it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:25<00:01, 16.57it/s][A
avg reward: 1.26:  95%|█████████▌| 446/468 [00:26<00:01, 16.57it/s][A
avg reward: 1.26:  96%|█████████▋| 451/468 [00:26<00:00, 17.20it/s][A
avg reward: 1.26:  96%|█████████▋| 451/468 [00:26<00:00, 17.20it/s][A
avg reward: 1.26:  96%|█████████▋| 451/468 [00:26<00:00, 17.20it/s][A
avg reward: 1.26:  96%|█████████▋| 451/468 [00:26<00:00, 17.20it/s][A
avg reward: 1.26:  96%|█████████▋| 451/468 [00:26<00:00, 17.20it/s][A
avg reward: 1.26:  97%|█████████▋| 455/468 [00:26<00:00, 20.42it/s][A
avg reward: 1.26:  97%|█████████▋| 455/468 [00:26<00:00, 20.42it/s][A
avg reward: 1.26:  97%|█████████▋| 455/468 [00:26<00:00, 20.42it/s][A
avg reward: 1.26:  97%|█████████▋| 455/468 [00:26<00:00, 20.42it/s][A
avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 17.93it/s][A
avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 17.93it/s][A
avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 17.93it/s][A
avg reward: 1.26:  98%|█████████▊| 458/468 [00:26<00:00, 17.93it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26:  99%|█████████▊| 461/468 [00:26<00:00, 16.53it/s][A
avg reward: 1.26: 100%|█████████▉| 466/468 [00:26<00:00, 17.26it/s][A
avg reward: 1.26: 100%|█████████▉| 466/468 [00:26<00:00, 17.26it/s][A
avg reward: 1.26: 100%|██████████| 468/468 [00:27<00:00, 17.33it/s]

  0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9765625:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.984375:   0%|          | 0/79 [00:00<?, ?it/s] [A
Val Acc: 0.984375:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.986328125:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.984375:   0%|          | 0/79 [00:00<?, ?it/s]   [A
Val Acc: 0.9830729166666666:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9799107142857143:   0%|          | 0/79 [00:00<?, ?it/s][A
Val Acc: 0.9799107142857143:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.9794921875:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s]      [A
Val Acc: 0.9791666666666666:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.97890625:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s]        [A
Val Acc: 0.9808238636363636:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.9817708333333334:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.9819711538461539:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.9810267857142857:   9%|▉         | 7/79 [00:00<00:01, 63.26it/s][A
Val Acc: 0.9810267857142857:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.9796875:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s]         [A
Val Acc: 0.9794921875:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.9806985294117647:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.9791666666666666:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.9802631578947368:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.980859375:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s]       [A
Val Acc: 0.9810267857142857:  18%|█▊        | 14/79 [00:00<00:01, 63.93it/s][A
Val Acc: 0.9810267857142857:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9815340909090909:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9816576086956522:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9817708333333334:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9821875:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s]         [A
Val Acc: 0.9813701923076923:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9809027777777778:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9807477678571429:  27%|██▋       | 21/79 [00:00<00:00, 62.75it/s][A
Val Acc: 0.9807477678571429:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9808728448275862:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9807291666666667:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9808467741935484:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.97998046875:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s]     [A
Val Acc: 0.9796401515151515:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9795496323529411:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9799107142857143:  35%|███▌      | 28/79 [00:00<00:00, 64.79it/s][A
Val Acc: 0.9799107142857143:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9796006944444444:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9797297297297297:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9796463815789473:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9791666666666666:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.979296875:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s]       [A
Val Acc: 0.9786585365853658:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9778645833333334:  44%|████▍     | 35/79 [00:00<00:00, 66.32it/s][A
Val Acc: 0.9778645833333334:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9778343023255814:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9783380681818182:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9774305555555556:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9772418478260869:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9767287234042553:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9767252604166666:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9770408163265306:  53%|█████▎    | 42/79 [00:00<00:00, 66.69it/s][A
Val Acc: 0.9770408163265306:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9775:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s]            [A
Val Acc: 0.9776348039215687:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9779146634615384:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9780365566037735:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9781539351851852:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9784090909090909:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9787946428571429:  62%|██████▏   | 49/79 [00:00<00:00, 65.44it/s][A
Val Acc: 0.9787946428571429:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9786184210526315:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.978448275862069:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s] [A
Val Acc: 0.9786811440677966:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9787760416666667:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9788678278688525:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9789566532258065:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9789186507936508:  71%|███████   | 56/79 [00:00<00:00, 66.07it/s][A
Val Acc: 0.9789186507936508:  80%|███████▉  | 63/79 [00:00<00:00, 66.16it/s][A
Val Acc: 0.97900390625:  80%|███████▉  | 63/79 [00:00<00:00, 66.16it/s]     [A
Val Acc: 0.9789663461538461:  80%|███████▉  | 63/79 [00:00<00:00, 66.16it/s][A
Val Acc: 0.978811553030303:  80%|███████▉  | 63/79 [00:01<00:00, 66.16it/s] [A
Val Acc: 0.9788945895522388:  80%|███████▉  | 63/79 [00:01<00:00, 66.16it/s][A
Val Acc: 0.9786305147058824:  80%|███████▉  | 63/79 [00:01<00:00, 66.16it/s][A
Val Acc: 0.978713768115942:  80%|███████▉  | 63/79 [00:01<00:00, 66.16it/s] [A
Val Acc: 0.9787946428571429:  80%|███████▉  | 63/79 [00:01<00:00, 66.16it/s][A
Val Acc: 0.9787946428571429:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9784330985915493:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9786241319444444:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9789169520547946:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9789907094594594:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9786458333333333:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.978515625:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s]       [A
Val Acc: 0.9784902597402597:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9783653846153846:  89%|████████▊ | 70/79 [00:01<00:00, 64.64it/s][A
Val Acc: 0.9783653846153846:  99%|█████████▊| 78/79 [00:01<00:00, 66.35it/s][A
Val Acc: 0.9784: 100%|██████████| 79/79 [00:01<00:00, 66.11it/s]
epoch 0, val_acc: 0.98, avg reward: 1.26: 100%|██████████| 1/1 [00:28<00:00, 28.25s/it]

The resulting architecture is:

mynet.graph

../../_images/output_enas_proxylessnas_ad55b9_25_0.svg

Store the trained model as a static network¶

The trained ENAS network can be saved to disk for future inferences.

mynet.export('enas')

Load it back with mxnet:

mynet_static = mx.gluon.nn.SymbolBlock.imports("enas-symbol.json", ['data'], "enas.params")
y = mynet_static(mx.nd.zeros((1, 1, 28, 28)))
print(y.shape)

(1, 10)

Reference¶

[1] Efficient Neural Architecture Search via Parameter Sharing H Pham, MY Guan, B Zoph, QV Le, J Dean International Conference on Machine Learning (ICML)

[3] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai, Ligeng Zhu, Song Han International Conference on Learning Representations (ICLR), 2019.