ylai to LocalLLaMA@sh.itjust.worksEnglish · 7 months agoMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comexternal-linkmessage-square3fedilinkarrow-up131arrow-down11
arrow-up130arrow-down1external-linkMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comylai to LocalLLaMA@sh.itjust.worksEnglish · 7 months agomessage-square3fedilink
minus-squarexcjs@programming.devlinkfedilinkEnglisharrow-up1·edit-25 months agoI just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/
I just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/