Here's a hidden detail about Dropout that many people don't know.Assume that:- There are 100 neurons in a layer, and all activation values are 1.- The weight from 100 neurons to a neuron ‘A’ in the next layer is 1.- Dropout rate = 50%Computing the input of neuron ‘A’:- During training → Approx. 50 (since ~50% of values will be dropped).- During inference → 100 (since we don't use Dropout during inference).So essentially, during training, the average neuron input is significantly lower than that during infer ...

RT Avi Chawla (@_avichawla)PyTorch dataloader has 2 terrible default settings.Fixing them gave me ~5x speedup.When you train a PyTorch model on a GPU:- .to(device) transfers the data to the GPU.- Everything after this executes on the GPU.This means when the GPU is working, the CPU is idle, and when the CPU is working, the GPU is idle.Memory pinning optimizes this as follows:- When the model is trained on the 1st mini-batch, the CPU can transfer the 2nd mini-batch to the GPU.- This ensures that the GPU does ...

PyTorch dataloader has 2 terrible default settings.Fixing them gave me ~5x speedup.When you train a PyTorch model on a GPU:- .to(device) transfers the data to the GPU.- Everything after this executes on the GPU.This means when the GPU is working, the CPU is idle, and when the CPU is working, the GPU is idle.Memory pinning optimizes this as follows:- When the model is trained on the 1st mini-batch, the CPU can transfer the 2nd mini-batch to the GPU.- This ensures that the GPU does not have to wait for the ne ...

Technology Breakthroughs - True technology breakthroughs are rare, the hype around KANs serves as a reminder [1] - Shifts like the success of Transformers only come once in a decade or more [1] Industry Dynamics - Transformers aligned with hardware, data, and economics, proving to be a significant breakthrough [1]

The ultimate Full-stack AI Engineering roadmap to go from 0 to 100.This is the exact mapped-out path on what it actually takes to go from Beginner → Full-Stack AI Engineer.> Start with Coding Fundamentals.> Learn Python, Bash, Git, and testing.> Every strong AI engineer starts with fundamentals.> Learn how to interact with models by understanding LLM APIs.> This will teach you structured outputs, caching, system prompts, etc.> APIs are great, but raw LLMs still need the latest info to be effective.> Learn h ...

A free 5-step roadmap to learn Python (even if you have no experience): https://t.co/QyUHPDoH2RAvi Chawla (@_avichawla):I've been coding in Python for 9 years now.If I were to start over today, here's a complete roadmap: ...

If you found it insightful, reshare it with your network.Find me → @_avichawlaEvery day, I share tutorials and insights on DS, ML, LLMs, and RAGs.Avi Chawla (@_avichawla):I've been coding in Python for 9 years now.If I were to start over today, here's a complete roadmap: ...

Links:- FreeCodeCamp: https://t.co/qS3FMKJ4t7- Deep Learning: https://t.co/2Pi0rUehre- Harvard: https://t.co/8bHwz8hW4l- Corey: https://t.co/HfiIUvo6fk- Project learning: https://t.co/mDIIQtJOJCLearn MCPs from scratch (with 11 projects): https://t.co/yzmieK4Z0c ...

I've been coding in Python for 9 years now.If I were to start over today, here's a complete roadmap: ...

RT Avi Chawla (@_avichawla)This is the best way to understand how ML models actually work!Use Drawdata to draw a 2D dataset in Jupyter. Use it to actively pick data from the widget and update the model as the data is being drawn!Fully interactive, real-time, and open-source! https://t.co/u4zUbSTZ3B ...