Michael Min Wah Leung

Notes on post-training, sequence modelling, and the occasional brain. Code-first, results-honest, written by an ML engineer with a research background in neuroscience.

Writing

Readable shallow, steerable deep: the same split in three transformers

interpretability

steering

probing

transformers

A direction I can read a composed sentiment concept off of is nearly perpendicular to the direction that steers it, and it is a worse lever at matched dose. The same split holds on three transformers and two normalization families, and a logit-lens diagnostic offers a candidate account of why the useful site is deep.

Does a single direction mediate refusal? A small reproduction

interpretability

safety

steering

transformers

A reproduction of Arditi et al. 2024 on a 1.5B chat model: ablating one difference-of-means direction lowered held-out harmful refusal where a norm-matched random direction did not, and adding it raised harmless refusal. A first model could not be tested at all, which is part of what the exercise taught me.

A probe at layer 0 is a lie detector for your experiment

interpretability

probing

transformers

Two linear probes on Gemma 3 1B hit 100% accuracy and taught opposite lessons. The number is not the finding; the shape of the accuracy-by-layer curve is.

Why SFT learned the words but GRPO learned the rules

post-training

GRPO

RLHF

LLMs

Teaching a 14B model a proprietary equipment-naming taxonomy with a hand-tuned reward function, and why ~250 lines of reward code and a quarter-epoch of GRPO closed the gap that more SFT couldn’t.

From consuming a pretrained model to training my own

seq2seq

sign-language

MLX

hybrid-inference

Building a continuous-sign-language Copilot: a Transformer Seq2Seq trained from scratch on How2Sign, two training backends, and a hybrid runtime that reaches 93.6% sentence-level recognition.

Patient-specific filters as biomarkers

neuroscience

signal-processing

interpretability

ICA, FOOOF, and CSP for noisy EEG, and what spatial filters taught me about feature extraction in transformers.