Workstream Proposal - Interpretability

Interpretability may give us insights to improve and control our models. I believe Ditto is the first model we should focus on due to the intractability of GPT.

GPT is Intractable

LLM tokens are simply words, or components of words. It's thus quite obvious what the model is thinking about each token, and we can correlate the activations with the token. For our gpt tokens are completely inscrutable. Worse, there's 13 of them each output, with different time delays, all entangled in superposition.

Our model is also brittle. In LLM's people have been successful in injecting activations to produce things like golden gate claude. This likely won't work in audio due to our high bitrate output. Injecting anything OOD causes decreased fidelity and rapid deterioration to noise. This has been observed with fp8 quantization, off by 1 bugs, and all attempts to be clever with attention masks. Even if we are able to identify some control vector, we won't be able to apply it.

Ditto is Tractable

Ditto produces a single embedding to focus our efforts on. This embedding can be used to prompt GPT, making any insights useful production features. This embedding lives in both the audio and text space, opening research directions like this paper.

Advancements in ditto interpretability are useful for search, explore, and generation. If we use ditto as semantic, we may understand what semantic is doing and pave the way for gpt interpretability.

Roadmap

• Try text based decomposition of ditto embeddings
• Try fine grained segmentation of text tags
• Train a SAE on the ditto embedding - this should stack with text based decomposition
  • try interpreting features with retrieval
  • try interpreting with ablated generations
  • figure out which features are a single activation
    • volume, key, genre, artist, singer, bpm?
• Scale up labelling if needed

What Can Be Outsourced?

Ideally we provide the ditto model and training dataset. This should be everything needed from our side.