Trainer package

Package for files related to model training.

Trainer.Model_Trainer

class Trainer.ModelTrainer.ModelTrainer(model_creation_function, data_source_paths: list, save_path, load_datasets_path=None, cache_path=None, batch_size: int = 1, train_print_frequency: int = 10, epochs: int = 10, dummy_epoch=True, produce_torch_datasets_only=False, num_workers: int = 10, num_validation_samples: int = 10, num_test_samples: int = 10, data_root: pathlib.Path = PosixPath('/cfs/share/data/RTM/Leoben/sim_output'), data_processing_function=None, data_gather_function=None, looping_strategy=None, cache_mode=<CachingMode.FileList: 2>, loss_criterion=<sphinx.ext.autodoc.importer._MockObject object>, optimizer_function=<function ModelTrainer.<lambda>>, lr_scheduler_function=None, optimizer_path=None, classification_evaluator_function=None, checkpointing_strategy=<CheckpointingStrategy.Best: 1>, run_eval_step_before_training=False, dont_care_num_samples=False, use_mixed_precision=False, sampler=None, caching_torch=True, demo_path=None, resize_label_to=(0, 0), load_test_set_in_training_mode=False)

Bases: object

Unified class for training a model.

Parameters:
  • model_creation_function – Pytorch model to be trained. Passed as a lambda call, e.g. lambda: YourModel()
  • data_source_paths – List of file paths containing the files for training.
  • save_path – Path for saving outputs.
  • load_datasets_path – Path containing dedidacted Datasets in a pickled format.
  • cache_path – Path containing cached objects.
  • batch_size – Batch size for training.
  • train_print_frequency – Frequency (in steps) in which infos about the current loss gets logged.
  • epochs – Number of epochs for training.
  • dummy_epoch – If set to True, a dummy epoch will be fetched before training. This results in better shuffling for the first epoch
  • produce_torch_datasets_only – Only write cache data, useful for producing and publishing certain datasets. Output directory is the caching directory, that comes out of the handle_torch_caching method. The root dir is saved in Ressources.training.datasets_dryspots_torch
  • num_workers – Number of processes for processing data.
  • num_validation_samples – Number of samples for the validation set.
  • num_test_samples – Number of samples for the test set.
  • data_root – The root directory for the training data.
  • data_processing_function – function object used by the data generator for transforming paths into desired data.
  • data_gather_function – function object used by the data generator for gathering the paths to the single files.
  • cache_mode – From enum CachingMode in Pipeline.TorchDataGeneratorUtils.torch_internal.py. Specifies if and how caching is done.
  • loss_criterion – Loss criterion for training.
  • optimizer_function – Object of a Torch optimizer. Passed as a lambda call, e.g. lambda: params: torch.optim.Adam(params, lr=0.0001)
  • classification_evaluator – Classification Evaluator for evaluating the models performance.
  • checkpointing_strategy – From enum CheckpointingStrategy in Pipeline.TorchDataGeneratorUtils.torch_internal.py. Specifies which checkpoints are stored during training.
create_optimizer_and_lr_scheduler()
inference_on_test_set(output_path: pathlib.Path = None, checkpoint_path: pathlib.Path = None, classification_evaluator_function=None)

Start evaluation on a dedicated test set. :param output_path: Directory for test outputs. :param checkpoint_path: … :param classification_evaluator_function: lambda with Evaluator object that should be used for the test run.

resize_label_if_necessary(label)

Resize the label: saves online storage by making it possible to use the bigger image labels of 1140 sensors also for 80 and 20 sensors :param label: :return:

start_training()

Sets up training and logging and starts train loop

Trainer.evaluation

Evaluation classes must provide three functions even if not all of them have functionality:

  • commit(output, label): updates the evaluation class with a new pair of a single prediction and a single label
  • print_metrics(): prints a set of application-specific print_metrics
  • reset: Resets the internal metrics of an evaluator, e.g. after a evaluation loop is finished.
class Trainer.evaluation.BinaryClassificationEvaluator(save_path: pathlib.Path = None, skip_images=True, with_text_overlay=False, summary_writer=None, advanced_eval=False)

Bases: Trainer.evaluation.Evaluator

Evaluator specifically for binary classification. Calculates common metrics and a confusion matrix.

commit(net_output, label, inputs, aux, *args)

Updates the confusion matrix.

Parameters:
  • net_output – predictions of the model
  • label – associated labels
print_metrics(step_count=0)

Prints the counts of True/False Positives and True/False Negatives, Accuracy, Precision, Recall, Specificity and the confusion matrix.

reset()

Resets the internal counters for the next evaluation loop.

class Trainer.evaluation.Evaluator

Bases: object

commit(*args, **kwargs)
print_metrics(*args, **kwargs)
reset(*args, **kwargs)
class Trainer.evaluation.SensorToFlowfrontEvaluator(save_path: pathlib.Path = None, sensors_shape=(38, 30), skip_images=True, summary_writer=None, print_n_images=-1)

Bases: Trainer.evaluation.Evaluator

commit(net_output, label, inputs, aux, *args)
print_metrics(step_count)
reset()
Trainer.evaluation.pixel_wise_loss_multi_input_single_label(input, target)
Trainer.evaluation.plot_predictions_and_label(input, target, _str)
Trainer.evaluation.save_img(path, _str, x, index)