If the following image doesn’t make sense to you, there’s a good chance you haven’t fully grasped positional encodings in transformer architecture. Please stick around as I’ll try to explain them in a way that actually feels intuitive. The 128-dimensional positional encoding for a sentence Context This blog is a part of my quest to properly understand the transformer architecture. Previously, I tried to explain self-attention in my own way here....
We instinctively know that “bank” means something completely different in “river bank” versus “money bank”, but how do you teach that distinction to a computer? We keep hearing that “Attention is All You Need”, but why? The answer lies in a deceptively simple mechanism that eventually allowed models to stop looking at words in isolation and start understanding the relationships between them. Context This post is inspired by a YouTube series that finally made the attention mechanism click for me....
This post is an inner monologue about something I’ve been thinking a lot about lately—the thin line between engineering and hacking, and how AI is blurring that line faster than ever. When I say “engineering,” I’m talking about building scalable, reliable systems with thoughtful design. “Hacking,” on the other hand, is about quick, scrappy solutions that just work, even if they’re held together with duct tape and a prayer. But with AI advancing at an insane pace, I can’t help but feel like the distinction between the two is fading....
TL;DR: Always use functools.wraps for your decorators. Decorators in python are wonderful. They let you modify or extend the behavior of functions or methods without permanently modifying their source code. I’ve used them in several places in my code to add features to my functions. But recently, I came across a weird, or rather interesting, bug that I felt was worth sharing. The Issue So we had a Flask API that was responsible for ML inference....
What is it and why is it needed? Last week, I spent some time writing prompts for classifying documents into a fixed set of classes based on their OCR text. I spent some time tweaking and experimenting with the prompt for the model to behave exactly how I wanted it to behave. But prompt tuning, for those who have tried it, knows how painfully daunting it is to make these models behave exactly how you’d like them to behave… that also consistently....
Hello to Everyone Reading This, It’s been quite a while since I last penned anything down. The last memory I have of writing was during my GRE’s Analytical Writing Assessment (AWA) — ah, the good ol’ days! But let’s set the jokes aside for a moment. I used to write articles and courses for The Click Reader for over a year. However, after moving on from Click Reader, my writing habits took an unexpected hiatus....