Title: On the emergent geometry within certain AI models
Abstract: In this talk we take a peek into some geometry that emerge within Natural Language Processing (NLP) models based on the transformer architecture. First we will introduce the setting and describe, in a functional format, what the related systems do and where the geometry emerges. We'll then turn to some possibilities and limitations afforded to us by the mathematics of high-dimensional geometry. Finally we finish by looking several colorful visualizations of some real data. This talk is partially based on my recent master's thesis (https://jyx.jyu.fi/handle/123456789/97810) and a small survey paper that I hope to finish before summer.
Warning: We analyze various abstract objects from a mathematician's point of view and point at some related mathematical theorems but will not give actual proofs of anything. Even worse, we actually study things in the "real world" i.e. outside the Platonic ideal universe of mathematics. One might benefit from thinking of this talk as an invitation rather than as a report. Caveat mathematicus purus.
