开源AI加速器OpenTPU的性能与硬件集成痛点
问题概述
用户希望在模型推理的成本和延迟上获得显著提升，但现有的开源加速器在运行大模型时受限于内存带宽和算力，缺乏将模型直接烧录进芯片的经济可行方案；同时，用户对AI自动设计或优化硬件以实现更高效资源利用抱有期待。
痛点标签
- 推理性能不足，吞吐量低
- 成本高，缺乏经济可行的硬件集成方案
- 内存带宽瓶颈限制大模型运行
- 希望AI能自动设计或优化硬件架构以提升性能
待验证问题
- 您是否在实际项目中遇到过模型推理性能瓶颈？
- 您是否愿意为此类加速器付费？
- 您是否希望AI能自动设计或优化硬件架构以提升性能？
证据
原话 · @匿名
Really dumb question from a software guy. Why aren't the labs burning their frontier models into chips already? Seems like the performance gains and cost per request would be worth it. That said, I understand neither the economics nor the physical challenges to doing this.
原话 · @匿名
After using AI to develop risc-v CPU cores, the same technique was used for developing openTPU. An open source AI inference engine. It's able to run most of the modern models like Qwen 3.5, Gemma 4, and many others. The TPU started able to produce only a few tokens per second and trough a recursive self improvement loop got to 80+ tok/sec on the smallers models.
原话 · @匿名
Recursive self-improvement will kill us all!"Also: Here is our recursive self-improvement hard at work...
原话 · @匿名
I haven't really dug into the results yet, but my guess is that a SOTA model has been able to produce an accelerator that runs a model since around December. The obvious next step is to get enough memory throughput to run that SOTA model itself so that it develop its own hardware. But perhaps the more interesting question is this: Can an AI be given a big FPGA and design a model architecture that takes advantage of the fabric being reconfigurable.
免责声明：本卡片由 AI 从社区讨论中提炼，属于「痛点证据」，是需求分析与验证的起点，不等同于「已验证的市场需求」。AI 存在分析局限，关键决策前请打开上方原始讨论人工复核。