etna.metrics.Coverage#

class Coverage(quantiles: Tuple[float, float] | None = None, mode: str = 'per-segment', upper_name: str | None = None, lower_name: str | None = None, missing_mode: str = 'error', **kwargs)[source]#

Bases: BaseIntervalsMetricWithMissingHandling

Coverage metric for prediction intervals - precenteage of samples in the interval [lower quantile, upper quantile].

\[Coverage(y\_true, y\_pred) = \frac{\sum_{i=1}^{n}{[ y\_true_i \ge y\_pred_i^{lower\_quantile}] * [y\_true_i \le y\_pred_i^{upper\_quantile}] }}{n}\]

This metric can handle missing values with parameter missing_mode. If there are too many of them in ignore mode, the result will be None.

Notes

Works just if quantiles presented in y_pred

When quantiles, upper_name and lower_name all set to None then 0.025 and 0.975 quantiles will be used.

Init metric.

Parameters:

  • quantiles (Tuple[float, float] | None) – lower and upper quantiles

  • mode (str) –

    “macro” or “per-segment”, way to aggregate metric values over segments:

    • if “macro” computes average value

    • if “per-segment” – does not aggregate metrics

    See MetricAggregationMode.

  • upper_name (str | None) – name of column with upper border of the interval

  • lower_name (str | None) – name of column with lower border of the interval

  • missing_mode (str) – mode of handling missing values (see MetricMissingMode)

  • kwargs – metric’s computation arguments

Methods

set_params(**params)

Return new object instance with modified parameters.

to_dict()

Collect all information about etna object in dict.

__call__(y_true, y_pred)

Compute metric's value with y_true and y_pred.

Attributes

This class stores its __init__ parameters as attributes.

greater_is_better

Whether higher metric value is better.

name

Name of the metric for representation.
__call__(y_true: TSDataset, y_pred: TSDataset) → float | None | Dict[str, float | None][source]#

Compute metric’s value with y_true and y_pred.

Notes

Note that if y_true and y_pred are not sorted Metric will sort it anyway

Parameters:

  • y_true (TSDataset) – dataset with true time series values

  • y_pred (TSDataset) – dataset with predicted time series values

Return type:

metric’s value aggregated over segments or not (depends on mode)

set_params(**params: dict) → Self[source]#

Return new object instance with modified parameters.

Method also allows to change parameters of nested objects within the current object. For example, it is possible to change parameters of a model in a Pipeline.

Nested parameters are expected to be in a <component_1>.<...>.<parameter> form, where components are separated by a dot.

Parameters:

**params (dict) – Estimator parameters

Returns:

New instance with changed parameters

Return type:

Self

Examples

>>> from etna.pipeline import Pipeline
>>> from etna.models import NaiveModel
>>> from etna.transforms import AddConstTransform
>>> model = NaiveModel(lag=1)
>>> transforms = [AddConstTransform(in_column="target", value=1)]
>>> pipeline = Pipeline(model, transforms=transforms, horizon=3)
>>> pipeline.set_params(**{"model.lag": 3, "transforms.0.value": 2})
Pipeline(model = NaiveModel(lag = 3, ), transforms = [AddConstTransform(in_column = 'target', value = 2, inplace = True, out_column = None, )], horizon = 3, )
to_dict()[source]#

Collect all information about etna object in dict.

property greater_is_better: None[source]#

Whether higher metric value is better.

property name: str[source]#

Name of the metric for representation.