Workstream Proposal - Personalization

Music is special among modalities in that everyone has a personal taste in music. For the first time, we have the ability to create music catering to everyone's individual tastes. I believe our current DPO formulation forces the model to learn the average taste, which ends up being generic pop. How much better can music be when its not targeting the greatest common denominator?

Ignoring Cold Starts

The most challenging part of the problem is handling users we don't know much about, which is most of them. I propose we validate personalization on our power users first, then extend it to everyone via interpretability or clustering.

Personalization for Power Users

First filter preference data to users with at least 100 labelled pairs. During DPO, prepend a low dimensional user embedding to the prompt. Freeze the weights of the original model, only train the embedding. In this way we can prevent memorization and learn more robust taste embeddings.

We should see the reward model accuracy increase after prepending user ids, indicating the model is extracting useful taste vectors.

We can now already AB test these user prompts on the users.

Personalization for All

Since the taste embeddings are low dimensional, we can easily visualize them with UMAP and cluster them. We can let users select another users taste profile, similar to personas.

We should be able to ablate each dimension by setting extreme values and listening to the resulting music. Hopefully this is interpretable. We could train a SAE if we need to, or make it a sparse embedding layer from the start.

Making it interpretable is useful for cold starts. If we can ask users what they like, this can quickly bootstrap their taste embedding.