Model

Definition

A model is used to encapsulate a PyTorch module. It is mainly used to train, evaluate and test a neural network.

The API is similar to sklearn or tensorflow ones.

By using this template, you will be able to use the Tuner class and fit method.

from sentarget.nn import Model


class MyModel(Model):
    def __init__(self, *args):
        super(Model, self).__init__()
        # initialize your module as usual

    def forward(*args):
        # one forward step
        pass

    def run(train_iterator, criterion, optimizer):
        # train one single time the network on train data
        pass

    def evaluate(eval_iterator, criterion):
        # evaluate one single time the network on eval / valid data
        pass

    # Optional
    def predict(test_iterator):
        # test one single time the network on test data
        pass

Example

# Run and train the model
model = MyModel()
# By using the 'Model' template, you can use the pre-defined ``fit`` method,
# used to train and evaluate the model epochs time.
model.fit(epochs, train_iterator, eval_iterator, criterion, optimizer)

Solver

Definition

If you prefer to separate the training functions from your model, you can use the Solver class.

A Solver is an object used for training, evaluating and testing a model. The performance is stored in a dictionary, both for training and testing. In addition, the best model occurred during training is stored, as well as it’s checkpoint to re-load a model at a specific epoch.

from sentarget.nn import Solver


class SolverBiLSTM(Solver):
    # Performer for the bidirectional LSTM model

    def __init__(self, model=None, criterion=None, optimizer=None):
        super().__init__(model, criterion, optimizer)

    def train(self, iterator):
        # train one single time the network on train data

        # Train mode
        self.model.train()
        for (idx, batch) in enumerate(iterator):
            self.optimizer.zero_grad()
            # Do stuff

        # Store the loss, accuracy and metrics in a dictionary
        results_train = {"loss": epoch_loss / len(iterator),
                         "accuracy": epoch_acc / len(iterator),
                         # some other metrics...
                         }

        return results_train

    def evaluate(self, iterator):
        # evaluate one single time the network on eval / valid data

        # Eval mode
        self.model.eval()
        with torch.no_grad():
            for (idx, batch) in enumerate(iterator):
                # One forward step
                # Do stuff

        # Store the loss, accuracy and metrics in a dictionary
        results_eval = {"loss": epoch_loss / len(iterator),
                        "accuracy": epoch_acc / len(iterator),
                         # some other metrics...
                        }

        return results_eval

Example

import torch.nn as nn
import torch.optim as optim

model = nn.Sequential(nn.Linear(10, 100), nn.Sigmoid(), nn.Linear(100, 5), nn.ReLU())
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss(ignore_index = LABEL_PAD_IDX)

solver = BiLSTMSolver(model, optimizer=optimizer, criterion=criterion)

# epochs = number of training loops
# train_iterator = Iterator, DataLoader... Training data
# eval_iterator = Iterator, DataLoader... Eval data
solver.fit(train_iterator, eval_iterator, epochs=epochs)