U.S. flag

An official website of the United States government, Department of Justice.

CSLM: Convertible Short-Term and Long-Term Memory in Differential Neural Computers

NCJ Number
305071
Journal
IEEE Transactions on Neural Networks and Learning Systems Volume: 32 Issue: 9 Dated: Sept 2021 Pages: 4026-4038
Author(s)
Date Published
September 2021
Length
13 pages
Annotation

This article introduces a variation of differential neural computer (DNC) architecture with a convertible short-term and long-term memory, named CSLM-DNC.

Abstract

External memory-based neural networks, such as differentiable neural computers (DNCs), have recently gained importance and popularity to solve complex sequential learning tasks that pose challenges to conventional neural networks; however, a trained DNC usually has a low-memory utilization efficiency. Unlike the memory architecture of the original DNC, the new scheme of short-term and long-term memories offers different importance of memory locations for read and write, and they can be converted over time. This is mainly motivated by the human brain, where short-term memory stores large amounts of noisy and unimportant information and decays rapidly, while long-term memory stores important information and lasts for a long time. The conversion of these two types of memory is allowed and is able to be learned according to their reading and writing frequency. The authors quantitatively and qualitatively evaluate the proposed CSLM-DNC architecture on the tasks of question answering, copy and repeat copy, showing that it can significantly improve memory efficiency and learning performance. (Publisher abstract provided)

Date Published: September 1, 2021