Resolving RuntimeError: expected scalar type Long but found Float in PyTorch

Problem Background and Error Analysis

In PyTorch deep learning development, data type mismatches are a common source of errors. In the user's provided code example, a RuntimeError: expected scalar type Long but found Float error occurs during the calculation of negative log-likelihood loss (NLLLoss). The error stack trace clearly indicates that the problem arises at the line criterion(output, labels), where criterion is an instance of nn.NLLLoss().

Root Cause Analysis

PyTorch's nn.NLLLoss function has strict requirements for input tensor data types. For classification tasks, target labels must be integer types, which correspond to torch.LongTensor in PyTorch. However, the labels tensor in the user's code is likely of floating-point type (torch.FloatTensor), causing the data type mismatch error.

From the Q&A data, we can see that the user attempted multiple approaches without success:

1. Attempted to adjust tensor data types, but encountered conflicts where the model expected float types while the loss function expected long types
2. Using the Variable wrapper did not resolve the issue
3. Uncertainty about the proper usage of data loaders

Detailed Explanation of PyTorch Data Type System

PyTorch defines nine CPU tensor types and corresponding nine GPU tensor types, each with specific application scenarios:

╔══════════════════════════╦═══════════════════════════════╦════════════════════╦═════════════════════════╗
║        Data type         ║             dtype             ║     CPU tensor     ║       GPU tensor        ║
╠══════════════════════════╬═══════════════════════════════╬════════════════════╬═════════════════════════╣
║ 32-bit floating point    ║ torch.float32 or torch.float  ║ torch.FloatTensor  ║ torch.cuda.FloatTensor  ║
║ 64-bit floating point    ║ torch.float64 or torch.double ║ torch.DoubleTensor ║ torch.cuda.DoubleTensor ║
║ 16-bit floating point    ║ torch.float16 or torch.half   ║ torch.HalfTensor   ║ torch.cuda.HalfTensor   ║
║ 8-bit integer (unsigned) ║ torch.uint8                   ║ torch.ByteTensor   ║ torch.cuda.ByteTensor   ║
║ 8-bit integer (signed)   ║ torch.int8                    ║ torch.CharTensor   ║ torch.cuda.CharTensor   ║
║ 16-bit integer (signed)  ║ torch.int16 or torch.short    ║ torch.ShortTensor  ║ torch.cuda.ShortTensor  ║
║ 32-bit integer (signed)  ║ torch.int32 or torch.int      ║ torch.IntTensor    ║ torch.cuda.IntTensor    ║
║ 64-bit integer (signed)  ║ torch.int64 or torch.long     ║ torch.LongTensor   ║ torch.cuda.LongTensor   ║
║ Boolean                  ║ torch.bool                    ║ torch.BoolTensor   ║ torch.cuda.BoolTensor   ║
╚══════════════════════════╩═══════════════════════════════╩════════════════════╩═════════════════════════╝

In classification tasks, target labels must use torch.LongTensor (corresponding to torch.int64) because classification indices require precise integer representation, while floating-point numbers may introduce rounding errors.

Complete Solution

Based on the best answer from the Q&A data, the core method to resolve this error is to ensure target label tensors have the correct data type:

# Convert target labels to LongTensor
target_tensor = target_tensor.type(torch.LongTensor)
# Or use more concise syntax
target_tensor = target_tensor.long()

In the user's specific case, type conversion should be applied to Yt_train during the data preprocessing stage:

# Convert data type before creating data loader
Yt_train = Yt_train.type(torch.LongTensor)
# Or
Yt_train = Yt_train.long()

# Then create data loader
dataloaders_test = torch.utils.data.DataLoader(Yt_train, batch_size=64)

Code Refactoring and Best Practices

Beyond data type conversion, the user's code contains other areas for improvement:

# 1. Proper usage of data loaders
# The original code creates two separate data loaders, but typically Dataset should wrap the data
from torch.utils.data import TensorDataset

train_dataset = TensorDataset(Xt_train, Yt_train)
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=64, shuffle=True)

# 2. Consistency in model definition
# The user's code defines two different models - consistency should be maintained
model = nn.Sequential(
    nn.Linear(784, 28),
    nn.ReLU(),
    nn.Linear(28, 64),
    nn.ReLU(),
    nn.Linear(64, 10),
    nn.LogSoftmax(dim=1)  # NLLLoss requires LogSoftmax instead of Softmax
)

# 3. Training loop optimization
for epoch in range(epochs):
    running_loss = 0
    for images, labels in train_loader:
        # Ensure image data is float type and labels are long type
        images = images.view(images.shape[0], -1).float()
        labels = labels.long()
        
        optimizer.zero_grad()
        output = model(images)
        loss = criterion(output, labels)
        loss.backward()
        optimizer.step()
        
        running_loss += loss.item()
    
    print(f"Epoch {epoch+1}/{epochs}, Loss: {running_loss/len(train_loader)}")

Understanding Data Type Requirements

Different PyTorch loss functions have varying requirements for input data types:

1. Classification loss functions: Such as nn.NLLLoss, nn.CrossEntropyLoss require target labels to be LongTensor
2. Regression loss functions: Such as nn.MSELoss, nn.L1Loss typically accept floating-point inputs
3. Binary classification loss: Such as nn.BCELoss requires target values between 0 and 1, typically floating-point types

Understanding these requirements is crucial for avoiding data type errors. During model development, always check the data types of input and output tensors:

# Check data types during debugging
print(f"Images dtype: {images.dtype}")
print(f"Labels dtype: {labels.dtype}")
print(f"Output dtype: {output.dtype}")

Preventive Measures and Debugging Techniques

To avoid similar data type errors, the following preventive measures are recommended:

1. Standardized data preprocessing: Ensure correct data types during the data loading stage
2. Use type assertions: Add data type checks at critical code locations
3. Documentation review: Carefully read PyTorch official documentation regarding loss function input requirements
4. Unit testing: Write test cases for data loading and preprocessing code

When encountering data type errors, follow these debugging steps:

1. Examine the error stack trace to identify the exact location of the error
2. Print data types and shapes of relevant tensors
3. Consult documentation for related functions or classes to understand input requirements
4. Add data type conversions at key nodes in the data flow

Conclusion

The RuntimeError: expected scalar type Long but found Float error is a common issue in PyTorch development, rooted in the requirement for target labels to use integer types (LongTensor) in classification tasks. By understanding PyTorch's data type system, correctly applying data type conversion methods, and following best practices, developers can effectively avoid and resolve such issues. In deep learning development, maintaining awareness of data types is an essential prerequisite for ensuring proper model training.