| 引用本文: | 熊志民,陈云华,冯忍,陈平华.基于速率编码的极低延迟深度脉冲神经网络研究[J].控制理论与应用,2025,42(3):531~540.[点击复制] |
| XIONG Zhi-min,CHEN Yun-hua,FENG Ren,CHEN Ping-hua.Ultra-low-latency deep spiking neural network based on rate coding[J].Control Theory and Technology,2025,42(3):531~540.[点击复制] |
|
| 基于速率编码的极低延迟深度脉冲神经网络研究 |
| Ultra-low-latency deep spiking neural network based on rate coding |
| 摘要点击 20 全文点击 0 投稿时间：2023-01-11 修订日期：2025-01-04 |
| 查看全文 查看/发表评论 下载PDF阅读器 |
| DOI编号 10.7641/CTA.2024.30012 |
| 2025,42(3):531-540 |
| 中文关键词 脉冲神经网络 ANN-SNN转化 速率编码 |
| 英文关键词 spiking neural network ANN-SNN conversion rate coding |
| 基金项目 广东省自然科学基金项目(2021A1515012233, 2016A030313713)资助. |
|
| 中文摘要 |
| 脉冲神经网络(SNN)具有强大的时空信息表征、异步事件处理能力,但由于脉冲发放过程不具有连续可微性,其训练是一个难题.人工神经网络(ANN)转SNN的方法,能够获得较高推理精度的深度SNN,但却存在SNN网络延迟和功耗过高的问题.为了降低网络延迟和功耗,本文从脉冲信息传递的异步特性入手,分析了极低延迟下SNN精度损失的主要原因,提出残余膜电位误差(RMPE)的概念,并对其进行分析与推导,建立残余膜电位与初始膜电位和权重之间的关系模型.基于所建立的残余膜电位模型,提出一种初始膜电位和权重的分层校准算法,减少残余膜电位误差,从而解决脉冲输入序列均匀分布假设与真实分布不一致的问题.提出一种ANN-SNN的双阶段转化框架,在第1阶段,采用带有可训练分层阈值的量化截断激活函数对ANN进行二次训练,以实现量化误差与截断误差的最优化;在第2阶段,对SNN进行微调训练,以进一步缩小残余膜电位误差,使得在极低延迟下的ANN-SNN转化也能获得较高的精度.实验结果表明,本文方法在推理延迟和功耗方面都优于现有的方法. |
| 英文摘要 |
| Spiking neural network (SNN) possesses robust capabilities for spatiotemporal information representation and asynchronous event processing. However, training SNN is challenging due to the non-differentiable nature of the spiking process. Converting artificial neural network (ANN) to SNN can yield deep SNN with high inference accuracy, but this approach often results in increased latency and power consumption in SNN. To mitigate network latency and power consumption, we have analyzed the primary reasons behind the loss of SNN accuracy at ultra-low latency, focusing on the asynchronous transfer characteristics of spikes. We introduce the concept of residual membrane potential error (RMPE) to address these issues. We have analyzed and derived the relationship between residual membrane potential and both the initial membrane potential and weights. Based on this understanding, we propose a layer-by-layer calibration algorithm for adjusting initial membrane potential and weights, aiming to reduce residual membrane potential error. This approach resolves the discrepancy between the assumption of a uniform distribution of spike input trains and the actual distribution. We propose a two-stage conversion framework for ANN-to-SNN conversion. In the first stage, we employ a quantization clipping activation function with a trainable stratification threshold. This allows us to train the ANN twice, optimizing both quantization error and clipping error. In the second stage, we further fine-tune the SNN to reduce residual membrane potential errors at ultra-low latency. Experimental results demonstrate that our proposed method outperforms existing methods in terms of inference latency and power consumption. |
|
|
|
|
|