Mango's Blog

Why Role Labels Matter More Than You Think and Why ChatML's assistant role is kinda bad

FYI, Writing at night and I haven't gotten this reviewed by friends

So it's been a while since I've thought about chat formats. It's not really the biggest thing I think about when training LLMs, but as I've had to deal with more and more weird-ass chat templates, I've given a bit of thought to what someone in my circle said:

"ChatML having assistant role tags can introduce biases when training de-aligned models, where 'Assistant' to the model means that it must act harmless and helpful, as well as harming writing performance since the association between assistant writing can harm prose writing and creativity and affect the topics that an LLM might bring up, considering it will be constrained to safe topics when not applicable."

I used to think this was kinda horseshit, till I started learning that words... MEAN WORDS. Certain words trigger certain basins contained within models that affect the final output. That's not truly a novel concept in itself, but this doesn't just apply to creative writing tasks and un-aligned models.

Rather, it applies to all sorts of tasks that a downstream model might be used for. Take for example judging: LLM-as-a-judge has been a pretty popular task, and for SLMs that are trained with rating in mind, you should probably avoid assistant tags because... instruct-tuned models have been trained to be "helpful," and unless you are a very large model, this means you won't have the capacity to distinguish between "feedback" and "not responding to harmful requests." Instead, you will treat each request as a sort of careful exercise and not exert force to criticize the user, and this all falls back to "how do we align models?" But I think a very simple fix is to just move from chat formats with specific biases.

Now, despite my HATRED of the fact that Mistral decides to change chat formats almost every other second, I will give it to them that the Mistral V7 Tekken format is one of their top things right now:

<s>[SYSTEM_PROMPT]<system>[/SYSTEM_PROMPT][INST]<message>[/INST]<response></s>[INST]<user message>[/INST]

Let's see here:

Pretty much perfect, to be honest. Now, one can argue:

"But... but... if you train the model with assistant data, won't that embed assistant bias to the chat format?"

And you would be right! I totally agree with you. However, it's a lot easier to burn out this assistant label when you are fine-tuning the model as it IS NOT LABELED AS AN ASSISTANT, THUS YOU CAN FIT IT FOR ANY ROLE.

In any case, I think 90% of the time ChatML, etc. is fine, but I hope more people use alternate roles for ChatML or something akin to that, What i was originally thinking was ChatML but alternate roles for specific domains eg: instead of just and assistant role, we have a writer role, a Scientist role, an Agent role, and ideally you'd train each dataset in with the different chat format and that'd trigger specific embeddings related to the training data that would help "bring out" that knowledge per say...

For one thing, The different writer role has been tried: https://huggingface.co/dreamgen/lucid-v1-nemo shoutout to my boy DreamJinn for this, but he introduces a new writer role that works alongside an "assistant" role.

so you could have turns like

System -> Explains prose style

Writer Character Jake -> Character-1 and acts out the sequence

Assistant -> Ask for advice on how to push plot forward

Writer Character Amy -> Character-2 and acts out the sequence.

Overall this is pretty much the "best" way to do, I know it takes alot more work but there is clear benefit and reasoning as to why just having an assistant role is bad.