Instruction Finetuning GPT-2 with PyTorch
Aug 30, 2025•13 min read
Literally coding every function from scratch.
Blog
Pretraining Language Model with PyTorch
Aug 29, 2025•16 min read
Literally coding every function from scratch.
GPT-oss Fundamentals
Aug 20, 2025•4 min read
A dive into OpenAI's first ever open-weight model after 5 years.
TopGPT
Aug 10, 2025•18 min read
A practical workflow for adapting GPT-2 to emulate a specific influencer’s tone: continue pretraining on long-form transcripts, align with supervised fine-tuning on crafted Q&A, then polish with Direct Preference Optimization (DPO) to balance authenticity and safety.
Inherent Constraints of Deep Learning
Jul 16, 2025•21 min read
Fundamental constraints through the history of deep learning summarized in a somewhat length post.
Finetuning Llama 3.2 3B Instruct
Jul 2, 2025•9 min read
Supervised finetuning Llama model for better at customer servicing.
GPT2 Distributed Training
Jun 6, 2025•19 min read
Engineering considerations when pre-training, finetuning a language model.
Simulated Cafe Ordering System
May 3, 2025•18 min read
A sandbox showcasing the basic capabilities of an AI agent.
Breaking Down the Transformer Architecture
Apr 23, 2025•19 min read
A lot of theory about trasnformer and its inner working.