Open In Colab


Tune forecaster on single node#


In this guidance, we demonstrate how to tune forecaster on single node. In tuning process, forecaster will find the best hyperparameter combination among user-defined search space, which is a common process if users pursue a forecaster with higher accuracy.

Chronos support forecasting model’s hyperparameter tuning in 2 sepaerated APIs (i.e. Forecaster.tune and AutoTSEstimator) for users with different demands:



Single Node



Performance-awared Tuning


Feature Selection


Customized Model


Forecaster.tune provides easier and more stright-forward API for users who are familiar with Chronos forecasters, it is recommened to try this method first.

We will take AutoformerForecaster and nyc_taxi dataset as an example in this guide.


Before we begin, we need to install chronos if it isn’t already available, we choose to use pytorch as deep learning backend.

[ ]:
!pip install --pre --upgrade bigdl-chronos[pytorch,automl]
!pip uninstall -y torchtext # uninstall torchtext to avoid version conflict

Data preparation#

First, we load the nyc taxi dataset.

Currently, tune func only support Numpy Ndarray input.

[ ]:
from sklearn.preprocessing import StandardScaler
from import get_public_dataset

def get_tsdata():
    name = 'nyc_taxi'
    tsdata_train, tsdata_valid, _ = get_public_dataset(name)
    stand_scaler = StandardScaler()
    for tsdata in [tsdata_train, tsdata_valid]:
              .scale(stand_scaler, fit=(tsdata is tsdata_train))
    return tsdata_train, tsdata_valid

tsdata_train, tsdata_valid = get_tsdata()

input_feature_num = 1
output_feature_num = 1
lookback = 20
horizon = 1
label_len = 10

train_data = tsdata_train.roll(lookback=lookback, horizon=horizon, label_len=label_len, time_enc=True).to_numpy()
val_data = tsdata_valid.roll(lookback=lookback, horizon=horizon, label_len=label_len,time_enc=True).to_numpy()

train_data and val_data is compose of (x, y, x_enc, y_enc) as we set time_enc=True which is only necessary for Autoformer.


The first step of tuning a forecaster is to define forecaster with space parameters.

There are several common space choices:

space.Categorical : search space for hyperparameters which are categorical, e.g. a = space.Categorical(‘a’, ‘b’, ‘c’, ‘d’)

space.Real : search space for numeric hyperparameter that takes continuous values, e.g. learning_rate = space.Real(0.01, 0.1, log=True)

space.Int : search space for numeric hyperparameter that takes integer values, e.g. range = space.Int(0, 100)

How to change these hyperparameters might be tricky and highly based on experience, but lr, d_model, d_ff and layers or similar parameters usually has a great impact on performance.

[ ]:
import as space
from bigdl.chronos.forecaster.autoformer_forecaster import AutoformerForecaster

autoformer = AutoformerForecaster(input_feature_num=input_feature_num,
                                  metrics=['mae', 'mse', 'mape'],
                                  lr = space.Real(0.0001, 0.1, log=True),
                                  d_model=space.Categorical(32, 64, 128, 256),
                                  d_ff=space.Categorical(32, 64, 128, 256),

Then just call tune on the training data and validation data!

In addition to data, there are three parameters which need to be specified : n_trials, target_metric and direction(or directions for multi-objective HPO).

n_trials: number of trials to run. The more trials, the longer the running time, the better results.

target_metric: the target metric to optimize, a string or an instance of torchmetrics.metric.Metric, default to ‘mse’. If you want to try a multi-objective HPO, you need to pass in a list, for example [‘mse’, ‘latency’] in which latency is a built-in metric for performance.

direction: in which direction to optimize the target metric, “maximize” or “minimize”, default to “minimize”. If you want to try a multi-objective HPO, you need to set direction=None, and specify directions which is a list containing direction for each metric, for example [‘minimize’, ‘minimize’].

there are other two parameters which you may change their default values : epochs and batch_size.

[ ]:
autoformer.tune(train_data, validation_data=val_data,
                n_trials=10, target_metric='mse', direction="minimize")

Then, you can see the whole trial history by calling search_summary().

[ ]:

After tune, the model parameters of autoformer is initialized according to the best trial parameters. You need to fit the model again.

[ ]:, epochs=4, batch_size=32)
# evaluate on val set
evaluate = autoformer.evaluate(val_data)

Save and load(Optional)#

After tuning and fitting, you can save your model by calling save with a filename.

[ ]:"best.ckpt")

Then, when you need to load the model weights, just call load() with corresponding filename.

[ ]:

Or if there is a new session, just define a new forecaster with six necessary parameters: input_feature_num, output_feature_num, past_seq_len, future_seq_len, label_len, and freq, then load with corresponding filename.

[ ]:
new_autoformer = AutoformerForecaster(input_feature_num=input_feature_num,