simet.metrics.downstream_task.sample.sample_tstr

simet.metrics.downstream_task.sample.sample_tstr

SampleTSTR

Bases: SampleDownstreamTask

Toy downstream task using Train on Synth, Test on Real.

Trains the underlying model on loader.synth_downstream_dataloader and evaluates on loader.real_downstream_dataloader, returning the final accuracy reported by the parent implementation.

Requirements

• DatasetLoader must have been constructed with a downstream_transform so that synth_downstream_dataloader and real_downstream_dataloader are available. Otherwise, an AttributeError will be raised when accessing these attributes.

Example

task = SampleTSTR() score = task.compute(loader) # uses synth for train, real for test 0.0 <= score <= 1.0 True

name property

name

Human-readable task name.

compute

compute(loader)

Train on synthetic, test on real, and return accuracy.

Parameters:

Name	Type	Description	Default
loader	DatasetLoader	Must expose synth_downstream_dataloader and real_downstream_dataloader (present only if a downstream_transform was provided).	required

Returns:

Name	Type	Description
float	float	Final test accuracy on the real set in [0.0, 1.0].

Source code in simet/metrics/downstream_task/sample/sample_tstr.py

@override
def compute(self, loader: DatasetLoader) -> float:
    """Train on synthetic, test on real, and return accuracy.

    Args:
        loader (DatasetLoader): Must expose
            `synth_downstream_dataloader` and `real_downstream_dataloader`
            (present only if a `downstream_transform` was provided).

    Returns:
        float: Final test accuracy on the real set in `[0.0, 1.0]`.
    """
    return self._compute(
        train_set=loader.synth_downstream_dataloader,
        test_set=loader.real_downstream_dataloader,
    )