Publications

2D Histogram based Region Proposal (H2RP) Algorithm for Neuromorphic Vision Sensors

Published in International Conference on Neuromorphic Systems 2021 (ICONS 2021) , 2021

In this paper, we propose a novel approach called H2RP (2D Histogram based Region Proposal) for efficient region proposal on AER (Address Event Representation) data generated by commercial neuromorphic vision sensors on a stationary setup. The proposed H2RP technique is based upon 2D histogram event-count images. We show that proposed H2RP has low computational footprint and no memory overhead compared to other state of the art region proposal techniques for neuromorphic vision data. Proposed technique derives its high computational efficiency by eliminating explicit requirement of dedicated noise filtering. To validate the proposed technique we benchmark it on two different real world AER datasets generated from two different commercial event sensors. Datasets used for this study constitute the (i) DAVIS-traffic surveillance dataset, and (ii) specially recorded moving object dataset. Our pipeline including H2RP along with an overlap tracker/kalman filter shows a gain of ∼87× in terms of computational footprint compared to other state of the art algorithms. Further, we incorporate a light CNN for object recognition. We also evaluate the proposed pipeline on two different edge-hardware (Jetson Nano, Raspberry Pi 3B+) platforms.

Recommended citation: Medya, R., Bezugam, S.S., Bane, D. and Suri, M., 2021, July. 2D Histogram based Region Proposal (H2RP) Algorithm for Neuromorphic Vision Sensors. In International Conference on Neuromorphic Systems 2021 (pp. 1-6). , doi: 10.1145/3477145.3477263. https://doi.org/10.1145/3477145.3477263

Fully Unsupervised Spike-Rate-Dependent Plasticity Learning With Oxide Based Memory Devices

Published in IEEE Transactions on Electron Devices , 2021

Artificial synapses are fundamental building blocks for neuromorphic computing hardware. In this work, we demonstrate fully unsupervised neuromorphic learning using analog weight change properties of bilayer nonfilamentry oxide-based resistive memory (OxRAM) devices as synaptic elements. Essential functions of a biological synapse, such as potentiation, depression, and spike-rate-dependent plasticity (SRDP), are experimentally demonstrated. Furthermore, we show the tune-ability of SRDP with preneuron/postneuron spike-train parameters (frequency and amplitude). Through simulations, we show a two-layer fully connected neuromorphic network powered by SRDP and OxRAM synapses. Our network achieves state-of-the-art best classification accuracy results compared to the literature (for SRDP-based purely unsupervised spiking neural networks (SNNs), without the use of any convolutional layer) on full Modified National Institute of Standards and Technology (MNIST) dataset: 92.07% (training) and 90.76% (test). Detailed impact of device properties, such as variability, retention, and endurance, is analyzed. The impact of introducing sparsity during training and weight pruning during inference was also investigated.

Recommended citation: M. Kumar, S. S. Bezugam, S. Khan and M. Suri, "Fully Unsupervised Spike-Rate-Dependent Plasticity Learning With Oxide- Based Memory Devices," in IEEE Transactions on Electron Devices, vol. 68, no. 7, pp. 3346-3352, July 2021, doi: 10.1109/TED.2021.3077346. https://doi.org/10.1109/TED.2021.3077346

Published in , 1900

An adaptive threshold neuron for recurrent spiking neural networks with nanodevice hardware implementation

Published in Nature Communications , 2021

We propose a Double EXponential Adaptive Threshold (DEXAT) neuron model that improves the performance of neuromorphic Recurrent Spiking Neural Networks (RSNNs) by providing faster convergence, higher accuracy and a flexible long short-term memory. We present a hardware efficient methodology to realize the DEXAT neurons using tightly coupled circuit-device interactions and experimentally demonstrate the DEXAT neuron block using oxide based non-filamentary resistive switching devices. Using experimentally extracted parameters we simulate a full RSNN that achieves a classification accuracy of 96.1% on SMNIST dataset and 91% on Google Speech Commands (GSC) dataset. We also demonstrate full end-to-end real-time inference for speech recognition using real fabricated resistive memory circuit based DEXAT neurons. Finally, we investigate the impact of nanodevice variability and endurance illustrating the robustness of DEXAT based RSNNs.

Recommended citation: Shaban, A., Bezugam, S.S. & Suri, M. An adaptive threshold neuron for recurrent spiking neural networks with nanodevice hardware implementation. Nat Commun 12, 4234 (2021). https://doi.org/10.1038/s41467-021-24427-8 https://rdcu.be/cn6jE

Multi-Class Classification of Blood Cells – End to End Computer Vision based diagnosis case study

Published in Arxiv, 2021

The diagnosis of blood-based diseases often involves identifying and characterizing patient blood samples. Automated methods to detect and classify blood cell subtypes have important medical applications. Automated medical image processing and analysis offers a powerful tool for medical diagnosis. In this work we tackle the problem of white blood cell classification based on the morphological characteristics of their outer contour, color. The work we would explore a set of preprocessing and segmentation (Color-based segmentation, Morphological processing, contouring) algorithms along with a set of features extraction methods (Corner detection algorithms and Histogram of Gradients(HOG)), dimensionality reduction algorithms (Principal Component Analysis(PCA)) that are able to recognize and classify through various Unsupervised(k-nearest neighbors) and Supervised (Support Vector Machine, Decision Trees, Linear Discriminant Analysis, Quadratic Discriminant Analysis, Naive Bayes) algorithms different categories of white blood cells to Eosinophil, Lymphocyte, Monocyte, and Neutrophil. We even take a step forwards to explore various Deep Convolutional Neural network architecture (Sqeezent, MobilenetV1,MobilenetV2, InceptionNet etc.) without preprocessing/segmentation and with preprocessing. We would like to explore many algorithms to identify the robust algorithm with least time complexity and low resource requirement. The outcome of this work can be a cue to selection of algorithms as per requirement for automated blood cell classification.

Recommended citation: Bezugam, S.S., 2021. Multi-Class Classification of Blood Cells--End to End Computer Vision based diagnosis case study. arXiv preprint arXiv:2106.12548. https://arxiv.org/abs/2106.12548

Efficient Video Summarization Framework using EEG and Eye-tracking Signals

Published in Arxiv , 2021

This paper proposes an efficient video summarization framework that will give a gist of the entire video in a few key-frames or video skims. Existing video summarization frameworks are based on algorithms that utilize computer vision low-level feature extraction or high-level domain level extraction. However, being the ultimate user of the summarized video, humans remain the most neglected aspect. Therefore, the proposed paper considers human’s role in summarization and introduces human visual attention-based summarization techniques. To understand human attention behavior, we have designed and performed experiments with human participants using electroencephalogram (EEG) and eye-tracking technology. The EEG and eye-tracking data obtained from the experimentation are processed simultaneously and used to segment frames containing useful information from a considerable video volume. Thus, the frame segmentation primarily relies on the cognitive judgments of human beings. Using our approach, a video is summarized by 96.5% while maintaining higher precision and high recall factors. The comparison with the state-of-the-art techniques demonstrates that the proposed approach yields ceiling-level performance with reduced computational cost in summarising the videos.

Recommended citation: Efficient Video Summarization Framework using EEG and Eye-tracking Signals,SS Bezugam, S Majumdar, C Ralekar, TK Gandhi - arXiv preprint arXiv:2101.11249, 2021 https://arxiv.org/abs/2101.11249

Exploiting analogue OxRAM conductance modulation for contrast enhancement application

Published in IET Electronics Letters , 2020

The authors present a unique application of analogue oxide-based resistive memory (OxRAM) device for sensor-level information storage and computation. They show that quality of low-contrast images in low-light can be improved by carefully exploiting OxRAM conductance modulation from specific bi-layer OxRAM material stacks. The proposed methodology involves conversion of light intensity to pulse frequency followed by resistance encoding as different non-volatile OxRAM resistance states.

Recommended citation: Kumar, A., Bezugam, S.S., Hudec, B., Hou, T.-H. and Suri, M. (2020), Exploiting analogue OxRAM conductance modulation for contrast enhancement application. Electron. Lett., 56: 594-597. https://doi.org/10.1049/el.2020.0106 https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/el.2020.0106