autogluon.space¶
Search space of possible hyperparameter values to consider.
Example
Define a dummy training function with searchable spaces for hyperparameters lr and wd:
>>> import numpy as np
>>> import autogluon as ag
>>> @ag.args(
>>> lr=ag.space.Real(1e-3, 1e-2, log=True),
... wd=ag.space.Real(1e-3, 1e-2),
... epochs=10)
>>> def train_fn(args, reporter):
... print('lr: {}, wd: {}'.format(args.lr, args.wd))
... for e in range(args.epochs):
... dummy_accuracy = 1 - np.power(1.8, -np.random.uniform(e, 2*e))
... reporter(epoch=e+1, accuracy=dummy_accuracy, lr=args.lr, wd=args.wd)
Create a scheduler to manage training jobs and begin hyperparameter tuning with the provided search space:
>>> scheduler = ag.scheduler.HyperbandScheduler(train_fn,
>>> resource={'num_cpus': 2, 'num_gpus': 0},
>>> num_trials=10,
>>> reward_attr='accuracy',
>>> time_attr='epoch',
>>> grace_period=1)
>>> scheduler.run()
>>> scheduler.join_jobs()
Visualize the results:
>>> scheduler.get_training_curves(plot=True)
Search Space¶
Search space for numeric hyperparameter that takes continuous values. |
|
Search space for numeric hyperparameter that takes integer values. |
|
Search space for hyperparameter that is either True or False. |
|
Nested search space for hyperparameters which are categorical. |
|
Nested search space corresponding to an ordered list of hyperparameters. |
|
Nested search space for dictionary containing multiple hyperparameters. |
|
Searchable objects, created by decorating a custom Python class or function using the |
Real¶
-
class
autogluon.space.
Real
(lower, upper, default=None, log=False)¶ Search space for numeric hyperparameter that takes continuous values.
- Parameters
- lowerfloat
the lower bound of the search space
- upperfloat
the upper bound of the search space
- defaultfloat (optional)
default value
- log(True/False)
Whether to search the values on a logarithmic rather than linear scale. This is useful for numeric hyperparameters (such as learning rates) whose search space spans many orders of magnitude.
Examples
>>> learning_rate = ag.Real(0.01, 0.1, log=True)
- Attributes
default
Return default value of hyperparameter corresponding to this search space.
hp
Return hyperparameter corresponding to this search space.
rand
Return randomly sampled (but valid) value from this search space.
Methods
get_hp
(name)Fetch particular hyperparameter based on its name.
-
get_hp
(name)¶ Fetch particular hyperparameter based on its name.
Int¶
-
class
autogluon.space.
Int
(lower, upper, default=None)¶ Search space for numeric hyperparameter that takes integer values.
- Parameters
- lowerint
The lower bound of the search space
- upperint
The upper bound of the search space
- defaultint (optional)
Default value
Examples
>>> range = ag.space.Int(0, 100)
- Attributes
default
Return default value of hyperparameter corresponding to this search space.
hp
Return hyperparameter corresponding to this search space.
rand
Return randomly sampled (but valid) value from this search space.
Methods
get_hp
(name)Fetch particular hyperparameter based on its name.
-
get_hp
(name)¶ Fetch particular hyperparameter based on its name.
Bool¶
-
class
autogluon.space.
Bool
¶ - Search space for hyperparameter that is either True or False.
ag.Bool() serves as shorthand for: ag.space.Categorical(True, False)
Examples
pretrained = ag.space.Bool()
- Attributes
default
Return default value of hyperparameter corresponding to this search space.
hp
Return hyperparameter corresponding to this search space.
rand
Return randomly sampled (but valid) value from this search space.
Methods
get_hp
(name)Fetch particular hyperparameter based on its name.
Categorical¶
-
class
autogluon.space.
Categorical
(*data)¶ Nested search space for hyperparameters which are categorical. Such a hyperparameter takes one value out of the discrete set of provided options.
- Parameters
- dataSpace or python built-in objects
the choice candidates
Examples
a = ag.space.Categorical(‘a’, ‘b’, ‘c’, ‘d’) b = ag.space.Categorical(‘resnet50’, autogluon_obj())
- Attributes
Methods
sample
(**config)Sample a configuration from this search space.
-
property
cs
¶ ConfigSpace representation of this search space.
-
property
kwspaces
¶ OrderedDict representation of this search space.
-
sample
(**config)¶ Sample a configuration from this search space.
List¶
-
class
autogluon.space.
List
(*args)¶ Nested search space corresponding to an ordered list of hyperparameters.
- Parameters
- argslist
a list of search spaces.
Examples
>>> sequence = ag.List( >>> ag.space.Categorical('conv3x3', 'conv5x5', 'conv7x7'), >>> ag.space.Categorical('BatchNorm', 'InstanceNorm'), >>> ag.space.Categorical('relu', 'sigmoid'), >>> )
- Attributes
Methods
sample
(**config)Sample a configuration from this search space.
-
property
cs
¶ ConfigSpace representation of this search space.
-
property
kwspaces
¶ OrderedDict representation of this search space.
-
sample
(**config)¶ Sample a configuration from this search space.
Dict¶
-
class
autogluon.space.
Dict
(**kwargs)¶ Nested search space for dictionary containing multiple hyperparameters.
Examples
>>> g = ag.space.Dict( >>> hyperparam1 = ag.space.Categorical('alpha', 'beta'), >>> hyperparam2 = ag.space.Int(0, 3) >>> ) >>> print(g)
- Attributes
Methods
sample
(**config)Sample a configuration from this search space.
-
property
cs
¶ ConfigSpace representation of this search space.
-
property
kwspaces
¶ OrderedDict representation of this search space.
-
sample
(**config)¶ Sample a configuration from this search space.
AutoGluonObject¶
-
class
autogluon.space.
AutoGluonObject
¶ Searchable objects, created by decorating a custom Python class or function using the
autogluon.obj()
orautogluon.func()
decorators.- Attributes
cs
ConfigSpace representation of this search space.
default
Return default value for hyperparameter corresponding to this search space.
rand
Randomly sample configuration from this nested search space.
Methods
__call__
(*args, **kwargs)Convenience method for interacting with AutoGluonObject.
init
()Instantiate an actual instance of this AutoGluonObject.
sample
()Sample a configuration from this search space.
-
property
cs
¶ ConfigSpace representation of this search space.
-
init
()¶ Instantiate an actual instance of this AutoGluonObject. In order to interact with such an object, you must always first call: object.init().
-
sample
()¶ Sample a configuration from this search space.