.. _course_alg: Search Algorithms ================= AutoGluon System Implementation Logic ------------------------------------- .. figure:: https://raw.githubusercontent.com/zhanghang1989/AutoGluonWebdata/master/doc/api/autogluon_system.png Important components of the AutoGluon system include the Searcher, Scheduler and Resource Manager: - The Searcher suggests hyperparameter configurations for the next training job. - The Scheduler runs the training job when computation resources become available. In this tutorial, we illustrate how various search algorithms work and compare their performance via toy experiments. FIFO Scheduling vs. Early Stopping ---------------------------------- In this section, we compare the different behaviors of a sequential First In, First Out (FIFO) scheduler using :class:`autogluon.core.scheduler.FIFOScheduler` vs. a preemptive scheduling algorithm :class:`autogluon.core.scheduler.HyperbandScheduler` that early-terminates certain training jobs that do not appear promising during their early stages. Create a Dummy Training Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python import numpy as np import autogluon.core as ag @ag.args( lr=ag.space.Real(1e-3, 1e-2, log=True), wd=ag.space.Real(1e-3, 1e-2), epochs=10) def train_fn(args, reporter): for e in range(args.epochs): dummy_accuracy = 1 - np.power(1.8, -np.random.uniform(e, 2*e)) reporter(epoch=e+1, accuracy=dummy_accuracy, lr=args.lr, wd=args.wd) FIFO Scheduler ~~~~~~~~~~~~~~ This scheduler runs training trials in order. When there are more resources available than required for a single training job, multiple training jobs may be run in parallel. .. code:: python scheduler = ag.scheduler.FIFOScheduler(train_fn, resource={'num_cpus': 2, 'num_gpus': 0}, num_trials=20, reward_attr='accuracy', time_attr='epoch') scheduler.run() scheduler.join_jobs() .. parsed-literal:: :class: output 0%| | 0/20 [00:00, ]