Dev Log
Exploring the frontiers of Artificial Intelligence, from Transformers to Reasoning Models.
The $100 Intelligence: Deconstructing Nanochat
I’ve spent the last few weeks digging through the ~8,000 lines of Python, Rust, and Shell that make up Karpathy's nanochat. A masterclass in minimalist engineering.
The Rise of Reasoning Models
I've been experimenting with Chain-of-Thought prompting recently, and it's wild how it shifts LLMs from just pattern matching to actual logical deduction. Here's what I found...
Optimizing RAG Pipelines
Spent the last week debugging hallucinations in my production RAG system. Turns out, hybrid search strategies and proper sharding were the missing pieces. A deep dive into what worked.
Fine-Tuning Llama 3 on Consumer Hardware
Can you actually finetune a state-of-the-art model on a single 3090? I tried it using QLoRA and quantization. Here's my practical guide and the results I got.