Circuits: A Living Document
Introduction
In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there has been some skepticism about this view, but what happens if you take it seriously?
The Circuits Thread
The Circuits thread is a collection of short articles, experiments, and critical commentary around a narrow or unusual research topic, along with a slack channel for real-time discussion and collaboration. It is intended to be an earlier stage than a full Distill paper, allowing for more fluid publishing, feedback, and discussion.
Articles and Comments
The natural unit of publication for investigating circuits seems to be short papers on individual circuits or small families of features. Compared to normal machine learning papers, this is a small and unusual topic for a paper.
Zoom In: An Introduction to Circuits
Does it make sense to treat individual neurons and the connections between them as a serious object of study? This essay proposes three claims which, if true, might justify serious inquiry into them: the existence of meaningful features, the existence of meaningful circuits between features, and the universality of those features and circuits.
An Overview of Early Vision in InceptionV1
An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of "neuron groups." This article sets the stage for future deep dives into particular aspects of early vision.
Curve Detectors
Every vision model we’ve explored in detail contains neurons which detect curves. Curve detectors is the first in a series of three articles exploring this neuron family in detail.
Naturally Occurring Equivariance in Neural Networks
Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights.
High-Low Frequency Detectors
A family of early-vision neurons reacting to directional transitions from high to low spatial frequency.
Curve Circuits
We reverse-engineer a non-trivial learned algorithm from the weights of a neural network and use its core ideas to craft an artificial neural network from scratch that reimplements it.
Visualizing Weights
We present techniques for visualizing, contextualizing, and understanding neural network weights.
Branch Specialization
When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings.
Weight Banding
Weights in the final layer of common visual models appear as horizontal bands. We investigate how and why.
Get Involved
The Circuits thread is open to articles exploring individual features, circuits, and their organization within neural networks. Critical commentary and discussion of existing articles is also welcome. The thread is organized through the open #circuits channel on the Distill slack.
About the Thread Format
Part of Distill’s mandate is to experiment with new forms of scientific publishing. We believe that reconciling faster and more continuous approaches to publication with review and discussion is an important open problem in scientific publishing.
Citation Information
If you wish to cite this thread as a whole, citation information can be found below. The author order is all participants in the thread in alphabetical order. Since this is a living document, the citation may add additional authors as it evolves. You can also cite individual articles using the citation information provided at the bottom of the corresponding article.
Updates and Corrections
If you see mistakes or want to suggest changes, please create an issue on GitHub.
Reuse
Diagrams and text are licensed under Creative Commons Attribution CC-BY 4.0 with the source available on GitHub, unless noted otherwise. The figures that have been reused from other sources don’t fall under this license and can be recognized by a note in their caption: "Figure from …".
Citation
For attribution in academic contexts, please cite this work as:
Cammarata, et al., "Thread: Circuits", Distill, 2020.
BibTeX citation:
@article{cammarata2020thread:,
author = {Cammarata, Nick and Carter, Shan and Goh, Gabriel and Olah, Chris and Petrov, Michael and Schubert, Ludwig and Voss, Chelsea and Egan, Ben and Lim, Swee Kiat},
title = {Thread: Circuits},
journal = {Distill},
year = {2020},
note = {https://distill.pub/2020/circuits},
doi = {10.23915/distill.00024}
}
