ijeff@lemdro.idM to

AI Stuff@lemdro.idEnglish · 1 year ago

Large Language Models up to 4x Faster on RTX With TensorRT-LLM for Windows

blogs.nvidia.com

1

cross-posted to:
technology@lemmy.world

6

Large Language Models up to 4x Faster on RTX With TensorRT-LLM for Windows

blogs.nvidia.com

ijeff@lemdro.idM to

AI Stuff@lemdro.idEnglish · 1 year ago

1

cross-posted to:
technology@lemmy.world

Striking Performance: Large Language Models up to 4x Faster on RTX With TensorRT-LLM for Windows

blogs.nvidia.com

Generative AI on PC is getting up to 4x faster via TensorRT-LLM for Windows, an open-source library that accelerates inference performance.

You must log in or register to comment.

Chat

ijeff@lemdro.idOPM
link
fedilink
English
arrow-up
1·
edit-2
1 year ago
Their inference prowess has been keeping me on Nvidia. Really wish AMD would step up its development in this area.

AI Stuff@lemdro.id

aistuff@lemdro.id

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !aistuff@lemdro.id

A place for all things artificial intelligence

Stay up-to-date with the latest news, reviews, and insightful discussions about artificial intelligence. Whether you’re interested in machine learning, neural networks, natural language processing, or AI applications, this is the place to be!

Subscribe: !aistuff@lemdro.id

Quick Links

Subscribe Links

Rules

1. Stay on topic

All posts should be directly related to artificial intelligence. This includes discussions, news, research, tutorials, applications, and anything else specifically about AI.

2. No reposts/rehosted content

Submit original sources, unless the content is not available in English. Reposts about the same AI-related content are not allowed.

3. No self-promotional spam

Only active members of the community can post their AI-related apps, projects, or resources, and they must actively participate in discussions. Please avoid posting self-promotional content that does not contribute to the AI community.

4. No editorializing titles

When sharing AI-related articles or content, refrain from changing the original titles. You may add the author’s name if relevant.

5. No offensive/low-effort content

Avoid posting offensive, irrelevant, or low-effort content that does not contribute positively to the AI community.

6. No unauthorized polls/bots/giveaways

Do not create unauthorized polls, use bots to generate content, or organize giveaways related to AI without proper authorization.

7. No affiliate links

Posting AI-related affiliate links is not allowed.

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

2 users / day
2 users / week
2 users / month
19 users / 6 months
13 local subscribers
299 subscribers
59 Posts
32 Comments
Modlog

mods:
ijeff@lemdro.id