RT Avi Chawla (@_avichawla)PyTorch dataloader has 2 terrible default settings.Fixing them gave me ~5x speedup.When you train a PyTorch model on a GPU:- .to(device) transfers the data to the GPU.- Everything after this executes on the GPU.This means when the GPU is working, the CPU is idle, and when the CPU is working, the GPU is idle.Memory pinning optimizes this as follows:- When the model is trained on the 1st mini-batch, the CPU can transfer the 2nd mini-batch to the GPU.- This ensures that the GPU does ...

PyTorch dataloader has 2 terrible default settings.Fixing them gave me ~5x speedup.When you train a PyTorch model on a GPU:- .to(device) transfers the data to the GPU.- Everything after this executes on the GPU.This means when the GPU is working, the CPU is idle, and when the CPU is working, the GPU is idle.Memory pinning optimizes this as follows:- When the model is trained on the 1st mini-batch, the CPU can transfer the 2nd mini-batch to the GPU.- This ensures that the GPU does not have to wait for the ne ...