PixeltablePixeltable Logo

Export data for ML training

Convert Pixeltable data to PyTorch DataLoader format for model training.

Data Import/Export · Notebook · Updated

Open in ColabRead in the docsView on GitHub

Colab runs it in your browser. If it calls a model provider, it asks for that provider's API key in a cell.

How it works

You use query.to_pytorch_dataset() to create an iterable dataset compatible with PyTorch DataLoader.

What it covers

  • Convert query results to PyTorch Dataset
  • Use with DataLoader for batch training
  • Export to Parquet for external tools

What it uses

Install
terminal
pip install pixeltable torch torchvision
Pixeltable calls
pxt.create_table()insert()add_computed_column()to_pytorch_dataset()

A look at the code

data-export-pytorch.ipynb
# Add a resize step to ensure all images have the same size
training_data.add_computed_column(
    image_resized=training_data.image.resize((224, 224))
)


# Convert to PyTorch dataset
# 'pt' format returns images as CxHxW tensors with values in [0,1]
pytorch_dataset = training_data.select(
    training_data.image_resized, training_data.label
).to_pytorch_dataset(image_format='pt')

One cell of the notebook, copied as it is. Open it in Colab for the rest.

More in Data Import/Export

Browse all 103