**AI Systems Still Writing Like Robots, But New Technique Promises to Help**
Researchers have found that Large Language Models (LLMs) have a bad habit of relying on familiar, formulaic language when generating writing, often overusing certain tokens or phrases. This limits their ability to express themselves in a more natural, human-like way.
The problem lies in the way LLMs are trained, particularly when it comes to **Distribution Fine Tuning (DFT)**. A recent technical report suggests that models trained using this approach fail to match the distribution of the training data, which can lead to stilted writing. This is because DFT focuses on optimizing a specific aspect of the model’s performance, rather than fine-tuning its overall language distribution.
**How DFT Falls Short**
Traditional fine-tuning, or SFT, involves adjusting a model’s parameters to fit a specific task or dataset. However, SFT doesn’t always capture the nuances of language distribution, leading to the formulaic writing that plagues many LLMs. In contrast, DFT aims to improve the model’s performance by adjusting its distribution of tokens or phrases. But as the technical report shows, this approach can actually make things worse.
**A More Effective Approach: Maximum Likelihood Estimation**
The researchers behind the report propose an alternative approach called **Maximum Likelihood Estimation (MLE)**. MLE involves training the model to generate text that is most likely to occur in the training data, rather than relying on a specific distribution of tokens or phrases. This can help LLMs to produce more natural, varied writing that is less prone to formulaic language.
**What this means**
For those who work with LLMs, or are interested in improving their writing abilities, this development offers hope. By using MLE to fine-tune language models, it may be possible to create systems that are more capable of generating high-quality, human-like writing. This could have significant implications for applications like content generation, chatbots, and language translation.
**The Road Ahead**
While this research is promising, it’s still early days for DFT and MLE. More work needs to be done to refine these approaches and understand their limitations. Nevertheless, the potential benefits of more natural, varied writing from LLMs are too great to ignore. As research in this area continues, we can expect to see significant improvements in the quality of language generation from AI systems.
