These are replications of some results from prominent papers in mechanistic interpretability, as well as some other mechanistic interpretability exercises
This replicates some of the results in the paper Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small by Wang et al
This replicates some of the results in Anthropic's paper Toy Models of Superposition by Elhage et al
This replicates some of the results in the paper Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task by Li et al
It utilizes some work directly from their accompanying library Othello World
This completes some of the exercises from from Nanda's A Mechanistic Interpretability Analysis of Grokking and thereby replicates some of the work in Progress measures for grokking via mechanistic interpretability
This reverse engineers one of the circuits a simple transformer uses for bracket classification, or taking a string like "(())()" and predicting whether the parentheses are balanced or not.
This creates a basic transformer from scratch
These were completed during the summer of 2023 following the ARENA curriculum. I completed the exercises by implementing the functions within the main.py files. The skeletons/outlines of the main.py scripts and the supplementary files are the work of the ARENA team and not mine.
These exercises rely heavily on the TransformerLens library