Evaluating LLMs with WeightWatcher Part III: The Magic of Mistral, a Story of Dragon Kings
calculatedcontent.com
Recently, the Mistral models have taken the LLM world by storm. The Mistral Mixture of Experts (MOE) 8x7b model outperforms other models in it’s weight class such as LLamA 2 70B and GPT 3.5. …