Playing with LLMs

 In the past few days I got to play with machine learning a bit. My idea of using complex numbers didn’t seem to be super effective as I expected. For example models didn’t end up encoding polarity in phase rotation of embeddings and model found an easier way to go around it.

But this concept of rotation seemed to be useful in general. Complex numbers were one way of performing rotations but rotation itself can be done by itself in embeddings. 

The goal here is to find some natural structure that the math sits on it without a need to use super generalized calculations.

This is an interesting concept but the main burden is on attention and transformers. If that can be fit into a better structure that gives the same performance smaller amount of computations, it can lead to a lot of time saving and power saving for a particular task.

The reason I was working on embedding was not to save power or computation, although that is welcome but my goal was to be able to see if a complex model which can integrate over many paths around the complex plane can be helpful to understand the nature of paradoxes and poems. It’s great if an underlying nature of language pops out that can show us the elementary particles of language. A kind of standard model of language.