exploring the loss landscape, one epoch at a time
indie ml researcher. recovering pytorch enthusiast. interested in attention mechanisms, sparse transformers, and the intersection of anime and AI. previously: big tech, now based & building in the open.
Experiments with sparse attention patterns and their effects on transformer architectures. Currently exploring efficient implementations and visualization techniques.
Tools for visualizing high-dimensional loss landscapes in 3D. Because sometimes you need to see where your model is going (or where it's stuck).