02 Classes for the Blob Model

[dehlong]

Hello,
I am currently at time 13:07 of the course and one thing is not clear to me. When we are building the blob model, it looks like this:

from torch import nn
class BlobModel(nn.Module):
    def __init__(self, input_features, output_features, hidden_units=8):
        """Initializes multi-class classification model.
        
        Args:
            input_features (int): Number of input features to the model
            output_features (int): Number of output features to the model (number of output classes)
            hidden_units (int): Number of hidden units between layers, default 8

        Returns:

        Example:
        """
        super().__init__()
        self.liner_layer_stack = nn.Sequential(
            nn.Linear(in_features = input_features, out_features=hidden_units),
            nn.ReLU(),
            nn.Linear(in_features=hidden_units, out_features=hidden_units),
            nn.ReLU(),
            nn.Linear(in_features=hidden_units, out_features=output_features)
        )

    def forward(self, x):
        return self.linear_layer_stack(x)
    
# Create an instance of Blob model and send it to the target device
model_4 = BlobModel(input_features=2,
                    output_features=4,    # QUESTION IS RELATED TO THIS
                    hidden_units=8).to(device)
model_4

My question is: Why are we setting the output_features to 4? Shouldn't it be 1? Yes, I know we have 4 classes but we want the model to output 1 number (0-3 based on which class does it belong to). Similarly, when we were building the binary classification model, we set the final out_features=1 and not 2, even though we had 2 categories because we wanted the model to output 1 number. Can anyone explain what is happening here as these seem like 2 contradicting statements?

[LuluW8071]

@dehlong
In binary classification, there r only two values to predict: either 1 or 0 (positive or negative). The raw logits r passed through a sigmoid function for binary classification, which produces a single output representing the probability of the positive class.

In multiclass classification, there r more than two classes to predict. We set n outputs, where each element in that output representing the score or probability for each class when passed through the softmax function. The class with the highest probability is chosen as the prediction.

[mrdbourke]

This is correct.

Binary classification

For binary classification, it's slightly different to multi-class classification, as there only needs to be 1 output which gets passed to the sigmoid function (torch.sigmoid).

Multi-class classification

Where as for multi-class classification, one output per class is required (e.g. if there are 4 classes, your output layer should have 4 output features), these are then passed to the softmax function (torch.softmax) to be turned into prediction probabilities (values between 0 & 1). The index with the highest value is the most probable class according to the model (though this does not mean it will be correct).

Note: To add to the confusion, you can also reframe a multi-class classification problem as many binary classification problems (e.g. one class vs rest or one class vs all). In essence, for a problem with N classes, create N binary classifiers. However, this approach can often be a bit more complicated than the above.

[dehlong]

Yes guys, thank you for explanation. This makes sense.
So basically for binary classification we we can say that p is a probability that it belongs to one class whilst 1-p is the probability it belongs to the other class.

Thank you for the answer.