Pretraining frontier-scale LLMs in FP8 is now standard practice, but moving to 4-bit floating point has remained an…
I’ve been using different LLMs a lot lately and I’ve noticed the newer versions of ChatGPT and Claude…
Last quarter I needed to back up about 40 specific ChatGPT conversations before deleting the rest of my…
I set out to test whether AAVE-coded (African American English Vernacular) prompts cause MoE language models to route,…
CPU is just a secondhand 10900x. Using 128k context, unquantized kv cache. Model is at q8_0 to mitigate…
In preparation for a lightning talk I’m giving at PyCon US this afternoon I decided to figure out…
Source Read original →
CPU is just a secondhand 10900x. Using 128k context, unquantized kv cache. Model is at q8_0 to mitigate…
Tested a bit on my XTX, a bit share hope helpful, thanks to Lucebox! Lucebox DFlash + PFlash…
Sorry if the title is confusing. What I’m trying to say is that since coding agents can write…
