Tenstorrent Shows Off Single-User LLM Speed For Workstation

SANTA CLARA, Calif. – EE Times got an exclusive demo of Tenstorrent’s Loud Box workstation running Llama3.1-70B (BF8 precision) at 15 tokens per second per user for 32 concurrent users.

Speeds above 10

Source link
lol