Teaching Llama 3 to be Polite
Spent ~$3 on A100 (80GB) instance on RunPod
Feb 25, 20268 min read
Command Palette
Search for a command to run...
Latest articles
I Pretrained a 360M LLaMA-Style Language Model from Scratch on 6B FineWeb Tokens (Single H100)
Pretraining an LLM from scratch usually sounds like “big-lab-only” territory. I wanted to test how far a smaller, practical setup can go while keeping the process transparent and reproducible. This post documents an end-to-end run of training a ~360M...
Feb 8, 20265 min readFrom Karpathy's micrograd to smoltorch: Understanding Autograd from First Principles
Why I Built My Own Deep Learning Framework? After watching Andrej Karpathy’s micrograd lecture, I had a realization: I’d been using PyTorch for months, but I didn’t really understand how autograd worked. Sure, I could call .backward() and get gradien...
Nov 18, 20257 min readI Built a Tiny Vector Database (and Pointed It at FAISS)
Vector DBs are everywhere these days: Pinecone, Weaviate, Qdrant, Chroma, FAISS … you name it! Most of them are full-featured systems with servers, APIs, dashboards, the works. Sometimes the best way to demystify hype is to build it yourself. Here’s ...
Aug 28, 20254 min readThe instability of a softmax function
The softmax, as we know, is numerically unstable when applied to vectors containing very small or very large numbers because of the exponential function involved in its computation. The softmax formula is: \(\text{softmax}(x_{i}) = \frac{e^{x_{i}}}{\...
Mar 5, 20252 min readIntroduction to CUDA Programming: A Beginner's Guide
This is an attempt to start a new series on CUDA programming! If you're curious about accelerating your computing tasks using GPUs, CUDA (Compute Unified Device Architecture) is an incredibly powerful tool to explore. Whether you're a developer, rese...
Jun 22, 202414 min read1. Introduction to Deep Learning - McCulloch-Pitts Neuron & Perceptron
Source: CS7015/CS6910 of IIT Madras, Prof. Mitesh M. Khapra In the era of computer's infancy, came a brilliant idea that would change the course of history in a very radical way. Let's dive back in history, imagine a world where machines could mimic...
Oct 4, 20236 min read
How ImageMagick helped me create perfect product mockups
Product mockups are a great way to showcase your designs, branding and products in a very realistic, natural and visually appealing way. Mockups help us visualize the design concepts and make decisions on colour schemes, layouts and overall aesthetic...
May 2, 202313 min read
Setup VS Code for remote development
Well in order to setup a remote environment, you need a DigitalOcean account, you can use my referral code below to get $100 for 60 days just to try these things out 👇 After you're done creating an account, let's move to the next step Creating a Dr...
Sep 15, 20223 min read