高效推理硬件中的权重存储与数值表示创新
问题概述
当前 AI 推理芯片在存储只读权重时普遍使用可变电压的 RAM，导致显著的功耗和面积浪费；同时乘法电路的功耗和面积大于加法电路，缺乏利用对数表示等新颖数值编码来降低功耗的实现；此外，关于推理性能提升的基准仍不明确。
痛点标签
- 只读权重存储在 RAM 中导致浪费，可通过 ROM 或高带宽闪存（HBF）降低成本
- 乘法电路功耗大、面积大，需要更高效的加法替代方案
- 对数数值表示的潜力尚未被充分利用，转换开销和实际收益不明确
- 缺乏统一的推理性能基准，导致创新收益难以量化
待验证问题
- 是否有现成的高带宽闪存（HBF）技术可以在短期内集成到推理芯片中？
- 对数表示的转换开销是否真的可以忽略，实际功耗节省是否可观？
- 当前的推理性能基准在哪些方面缺乏标准化，以便评估上述创新的实际收益？
证据
原话 · @匿名
I am missing a mention of ROM in the article. Keeping read-only weights in RAM is rather wasteful, as ROM can be implemented more cheaply. Approaches like High Bandwidth Flash (HBF) are relevant here, and should come to market in a few years. Further optimization may be possible by tailoring for sequential access, since inference of a particular model is very predictable.
原话 · @匿名
Tensordyne is expected to accelerate AI inference with a logarithmic number system that leans on a property of logarithms: The log of A times B equals the log of A plus the log of B. So, storing numbers as their exponents lets the chip add where it would otherwise multiply. That matters in silicon because multiplier circuits draw more power and use more die area than adders do. Tensordyne says its rack-scale hardware, called Napier, can produce up to 1,300 tokens per second per user, and can do so while using less than a tenth as much power as comparable Nvidia hardware.
事实 · @匿名
Excellent article. I believe the majority of benchmark performance gains moving forward will come from this side of the stack enabling faster iteration/recursion.
免责声明：本卡片由 AI 从社区讨论中提炼，属于「痛点证据」，是需求分析与验证的起点，不等同于「已验证的市场需求」。AI 存在分析局限，关键决策前请打开上方原始讨论人工复核。