SMEPilot: Characterizing and Optimizing LLM Inference with Scalable Matrix Extensions
来自IPADS的文章。这篇文章把ARM里的矩阵加速器SME用在CPU LLM Serving上。SME是ARM提出的一个加速装置,功能类似Intel的AMX加速器(AMX类似GPU里的TensorCore) Modern CPUs increasingly integrate matrix extensions, such as Arm Scalable Matrix Extension (SME
