New study finds language models memorize about 3.6 bits per parameter
Researchers unveil a method to measure LLM memorization capacity, finding GPT-style models store roughly 3.6 bits of information per parameter, with implications for grokking and scaling.
llmmemorizationmachine-learningscaling-laws