View the runnable example on GitHub

Accelerate Computer Vision Data Processing Pipeline#

You can use transforms and datasets from, which takes advantage of OpenCV and libjpeg-turbo, as a drop-in replacement of torchvision.transforms and torchvision.datasets to easily accelerate your computer vision data processing pipeline in PyTorch or PyTorch Lightning applications.

Suppose that you would like to preprocess OxfordIIITPet dataset. To accelerate this preocess, you could simply import BigDL-Nano transforms and datasets to replace torchvision.transforms and torchvision.datasets:

[ ]:
# from torchvision import transforms
# from torchvision.datasets import OxfordIIITPet
from import transforms
from import OxfordIIITPet

# Data processing steps are the same as using torchvision
train_transform = transforms.Compose([transforms.Resize(256),
                                        transforms.ColorJitter(brightness=.5, hue=.3),
                                        transforms.Normalize([0.485, 0.456, 0.406],
                                                            [0.229, 0.224, 0.225])])
val_transform = transforms.Compose([transforms.Resize(256),
                                    transforms.Normalize([0.485, 0.456, 0.406],
                                                            [0.229, 0.224, 0.225])])

train_dataset = OxfordIIITPet(root="/tmp/data", transform=train_transform, download=True)
val_dataset = OxfordIIITPet(root="/tmp/data", transform=val_transform)

You could then create train/validate dataloader as normal:

[ ]:
# obtain training indices that will be used for validation
import torch

indices = torch.randperm(len(train_dataset))
val_size = len(train_dataset) // 4
train_dataset =, indices[:-val_size])
val_dataset =, indices[-val_size:])

# create dataloaders
from import DataLoader

train_dataloader = DataLoader(train_dataset, batch_size=32)
val_dataloader = DataLoader(val_dataset, batch_size=32)

📚 Related Readings