High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs

github.com

cross-posted to:
localllama@sh.itjust.works
hackernews@derp.foo

High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs

github.com

bot@lemmy.smeargle.fansMB to Hacker News@lemmy.smeargle.fans · 11 months ago

cross-posted to:
localllama@sh.itjust.works
hackernews@derp.foo

GitHub - SJTU-IPADS/PowerInfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

github.com

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs - GitHub - SJTU-IPADS/PowerInfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

HN Discussion

You must log in or # to comment.

Chat