Pierce Lamb
Nov 17, 2023

--

Hi Mimansa, I believe `_tuning_objective_metric` is a value that is placed into the hyperparameters object when a tuning job is initiated in sagemaker.

In my training framework, in the settings.ini file, I set these two values for tuning jobs:

```

METRIC_NAME=objective_metric_rouge

METRIC_REGEX=.*objective_metric_rouge=(.*?);

```

Which get parsed when the setting.ini file is read:

```

self._metric_name = config('METRIC_NAME')

self._metric_regex = config('METRIC_REGEX')

@property

def metric_name(self) -> str:

return self._metric_name

@property

def metric_regex(self) -> str:

return self._metric_regex

```

These values are then passed to the `HyperparameterTuner()` object and sagemaker places `_tuning_objective_metric` within the return hyperparameters:

```

metric_definitions = [{"Name": config.metric_name, "Regex": config.metric_regex}]

objective_metric_name = config.metric_name

tuner = HyperparameterTuner(

estimator,

objective_metric_name,

tunable_hyperparams,

metric_definitions,

objective_type=config.objective_type,

max_jobs=config.max_jobs,

max_parallel_jobs=config.max_parallel_jobs

)

tuner.fit(inputs=config.training_data_uri)

```

And one can read the hyperparameters with:

```

hyperparams = environment.read_hyperparameters()

is_tuning_job = '_tuning_objective_metric' in hyperparams

```

--

--

Pierce Lamb
Pierce Lamb

Written by Pierce Lamb

Data & Machine Learning Engineer at a Security startup

No responses yet