Research Dispatch Neural network compression
A simpler rule beat the hierarchy on GPT-2. Here’s what that means.
A controlled experiment rewarded a sophisticated compression hierarchy. A small real-model test did not. The more useful result was the one that complicated the original story.
Read the dispatch