Symposium EP06 : Materials, Devices and Systems for Machine Learning and Neuromorphic Computing

Current approaches to brain-inspired computing rely on neural networks fashioned out of conventional von Neumann architectures. These tend to be energy-inefficient compared to biological systems on account of bottlenecks to memory where the weights for the neural network are stored, modified and frequently accessed. This work relies on two innovations: one is a device called the Charge Trap Transistor (CTT) – a high-k-metal-gate CMOS transistor that can be used in an analog memory array to store and update weights, as well as perform matrix multiplication; the other is a method to scale such a neuromorphic system to brain-like scales with billions of intimately interconnected neurons.
In this talk, we describe the physics of charge trapping in the high-k bulk FinFET as well as partially depleted SOI devices and describe ways in which this effect can be used as an analog memory. The analog memory device possesses adequate resolution for supervised and unsupervised learning as well as an inference engine similar to a memristor-based one but with the advantage that it is a very conventional CMOS transistor and can be combined intimately with CMOS logic. Such array units can be cascaded into multiple levels of a deep neural network and this may be scaled up in two dimensions using a Silicon Interconnect Fabric (Si-IF) or in three dimensions using 3D Wafer-Scale Integration (3DWSI).
We will describe the technologies as well as the processing and material challenges of these methods.

A major barrier to realizing neuromorphic hardware has been the development of analog memory with a linear and symmetric programmability required for neural algorithms and the energy efficiency to compete with conventional hardware. On one hand, CMOS-based floating gate memory (FGM) such as NAND Flash has a large programming voltage (>8V) that limits array-level energy efficiency. On the other hand, memristors, i.e. phase change memory and filament-forming metal oxides, suffer from non-linear weight updates and device-to-device variability that limits neural network accuracy.

To address these shortcomings, we introduce FGM based on ion insertion electrodes called ionic floating-gate memory (IFG). Similar to CMOS-based FGM, IFG is programmed via voltage pulses to a control gate that act to modulate the conductance of a semiconducting channel. However, for IFG, conductance switching occurs as ions are reversibly exchanged between the floating gate and channel through the process of ion insertion/extraction. Here, the floating gate acts as a reservoir of ions that is separated from the channel by an electronically-insulating but ionically-conductive solid electrolyte. After insertion/extraction through the electrolyte, ions diffusive throughout the channel volume in order to modulate the bulk doping.

IFG can provide benefits over CMOS which relies on hot carrier injection or Fowler-Nordheim tunneling in order to trap charges in oxides. First, bulk ion-insertion, also known as psuedocapacitance, significantly increases the charge-density per analog level, which relaxes the strict requirements for retention and supports many more levels for nanoscaled devices. Second, pseudocapacitance can act to reduce the floating gate potential to <100 mV, increasing programming linearity and allowing for control gates with low voltage thresholds (<1V). Here, we demonstrate an IFG device consisting of a newly-developed diffusive memristor (acting as a control gate) coupled to a non-volatile redox transistor (acting as a floating gate and channel) [1]. Our IFG devices can be tuned to >50 resistance levels with a linear and symmetric programming response that is suitable for neural algorithms. Neural network simulations of the device performance are found to reach ideal accuracy when classifying MNIST hand-written digits.

[1] van de Burgt, Y. et al. Nature Materials 16, 414–418 (2017)

Charge transport through resistive memory devices has been an arena of intense research. Here we explore temperature dependent charge transport in a memristive film of a Ru-complex of azo-aromatic ligand. In-situ Raman and UV-Vis spectroscopic measurements establish that the switching in film conductance is controlled by the ligand redox states of the film molecules, while the counter ions account for the hysteresis[1]. At temperature values lower than 145K, the hysteresis in J(V) gradually decays till ~5K where it completely quenches. In an intermediate temperature range of 135K to 110K, we observe well-resolved conductance plateaus at different applied bias with a sharp transition in between them. The number of plateaus N in the J(V) follows the empirical rule of N= d[nm]/5 where d is the film thickness in nm. The conductance plateaus correspond to a correlation of [J₀]^N, with J₀ = current of the first plateau. Each of the plateaus is characterized by in-situ Raman spectroscopy as well as photoluminescence (PL) measurement, both of which exhibit sharp transitions corresponding to the switching observed in J(V). PL and Raman peak intensities scale as N x I₀, where I₀ is the PL/Raman intensity of the first plateau. These observations indicate a layer by layer electron doping in the film where layers of around 5 nm get doped sequentially. The intermediate temperature range provides an optimal activation energy for counterion motion in the film leading to the formation of electronic phases with sharp boundaries. By reducing the van der Wall radius of the counterion this step can be reduced to a single layer ~ 1.5nm! This is an unprecedented result in an amorphous film. This multi-step memory has the potential for neuromorphic computing.

[1] Goswami, Sreetosh, et al. "Robust resistive memory devices using solution-processable metal-coordinated azo aromatics." Nature Materials (2017), DOI: 10.1038/NMAT5009

Hardware artificial neural network (ANN) system with high density synapse devices can perform massive parallel computing for pattern recognition with low power consumption. To implement neuromorphic system with on-chip learning capability, we need to develop ideal synapse device with various device requirements such as scalability, MLC characteristics, low power operation, data retention, and symmetric and linear conductance change under potentiation/depression modes. Although various devices such as ReRAM, PRAM, and MRAM were proposed for synapse applications, these devices have limitation for neuromorphic system application.

In this talk, I will cover various ReRAM synapse devices such as filamentary switching ReRAM (HfO_x, TaO_x, Cu-CBRAM) with MLC characteristics, interface switching ReRAM (Pr_0.7Ca_0.3MnO₃, TiO_x) with analog memory characteristics, and HfZrO_x ferroelectric device. By optimizing forming and potentiation/depression conditions, we could improve conductance linearity and MLC characteristics of filamentary synapse device. Interface ReRAM has better MLC characteristics with limited retention and conductance linearity. By controlling the reactivity of metal electrode and oxygen concentration in oxide, we can modulate the synapse characteristics. Ferroelectric device exhibits good retention characteristics but it requires 3-terminal device.

To overcome the limitation of conventional CMOS neuron, we have investigated NbO₂-IMT device for oscillator neuron applications. We have confirmed feasibility of pattern recognition using IMT oscillator device. Based on various synapse device characteristics, we have estimated the pattern recognition accuracy of MNIST handwritten digits and CIFAR-10 dataset. We have confirmed that synapse device characteristics directly affect pattern recognition accuracy.

In recent years, huge efforts have been made in the research community to understand and control the Resistive Switching (RS) phenomena for applications ranging from Non-volatile memories to Neuromorphic computing. For instance, nonvolatile memories based on the RS effect, where two (or more) reversible resistance states can be induced upon application of an electric field, have emerged with excellent performance. On the other hand, RS phenomena have been used to emulate neurological functions of the brain and so it appears very appealing to correlate the RS mechanisms to materials properties in order to achieve full control of the electronic functionality thus enabling the design of new devices.
The RS phenomenon has been observed in many oxide systems, in particular in perovskite oxides, which are materials showing outstanding properties associated to the strong electronic correlation, such as metal-insulator transitions (MIT). This is the case of the metallic perovskite La_1-xSr_xMnO₃ (LSMO), RENiO₃ (RE=rare earth) and YBa₂Cu₃O₇ (YBCO) family compounds, which are able to display Volume RS effects induced by the MIT and therefore, small changes in carrier concentration can induce huge resistance changes. The mechanism underlying this phenomenon is still unclear although oxygen vacancies mobility plays an key role in the mechanism underneath this phenomenon.
In this presentation we will discuss our studies on bipolar resistive switching of the mentioned perovskite oxides which were grown by Chemical Solution Deposition. Switching characteristics have been evaluated by C-SPM and I(V) curves with metal electrodes. Scanning tunneling microscopy and spectroscopy (STM/S), transport and resistive measurements were performed to gain insight into the local density of states of the material for different resistance states. Large resistance rations (10²-10⁴), over 100 cycle experiments, multilevel switching and switching dynamics have been evaluated. In addition, we have confirmed the strong influence of different atmospheres on the resistive switching properties of bare LSMO thin films. Remarkably, this influence disappears when the LSMO layer is capped with a CeO₂ ultrathin layer grown by Atomic Layer Deposition, which acts as an oxygen reservoir, making this bilayer a proper material choice for encapsulation. A 3-terminal proof of principle device based on Ag/CeO₂/LSMO/CeO₂/Ag will be presented.
We believe that these results contribute to a better understanding of the physical mechanism behind the robust RS effect of these metallic perovskite oxides films which can be now considered for neuromorphic computing applications.

Conductive bridge memory, in which the switching results from the electrochemical growth and dissolution of a metallic filament (typically Cu or Ag), has been shown to emulate basic neuromorphic computing functions, such as spike-timing dependent plasticity and short and long-term potentiation [1-2]. However, the integration of these devices in high-density neural networks requires low power consumption, with stable bipolar switching at currents of less than 10μA. This has proved to be a key challenge in the development of solid-state synaptic devices as operation in low current regimes has been shown to lead to large resistance fluctuations and reduced device performance [3]. A more detailed understanding of the filamentary resistive switching mechanisms is required for the realisation of functional devices.
The devices employed in this study consisted of insulator-metal (MIM) capacitor structures, comprising: Pt (25 nm)/Ag(30 nm) /HfO₂ (20 nm)/Pt 50 nm. The evolution of switching behaviour was examined as a function of compliance current ranging from 1 μA to 5 mA. A transition from volatile to bipolar memory switching was observed for devices formed with compliance currents in the order of 100 μA. These results are shown to be consistent with a model in which low compliance currents lead to thin, unstable filaments, and high compliance currents lead to thicker, more stable filaments, with the transition from volatile to non-volatile behaviour determined by the relative thermal stability of the filament (i.e. the Rayleigh instability criteria).
The applicability of an analytical model of non-volatile switching presented in [4] is also extended to HfO₂ based memory devices. This analytical model calculates the evolution of a tunnelling gap between the filament and active electrode during a triangular voltage sweep. A quantitative prediction of device variation between the high and low resistance states is important for the realisation power efficient spike-timing dependent plasticity. These findings provide an important insight into the influence of the electroforming process on switching behaviours and device stabilisation for emerging applications in solid state synapses.

References

[1] Li, Yi, et al. "Ultrafast synaptic events in a chalcogenide memristor." Scientific reports 3 (2013).
[2] S Yu et al. "An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation." IEEE Transactions on Electron Devices 58.8, pp 2729-2737 (2011).
[3] B Chen et al. "A novel operation scheme for oxide-based resistive-switching memory devices to achieve controlled switching behaviors." IEEE Electron Device Letters 32.3, pp 282-284 (2011).
[4] S Menzel, and R Waser. "Analytical analysis of the generic SET and RESET characteristics of electrochemical metallization memory cells." Nanoscale 5.22,pp 11003-11010 (2013)

Neural networks are the state of the art for artificial intelligence. Advanced deep neural networks have even been shown to outperform humans in challenging image recognition tasks. However, these networks are usually trained in supervised settings using large amounts of well-structured and labeled datasets. Training, which lasts from days to weeks on large-scale computing platforms, consume energy on the order of hundreds of kilowatts. In contrast, the biological brain is accurate and almost effortless in recognizing images with incredible computational efficiency in an unsupervised fashion. To close this gap, new methods for implementing unsupervised neural network learning with energy efficient hardware are crucial. In this work, we investigate hardware implementations of spiking neural networks using resistive synaptic devices. We discuss device-level and network-level approaches to improve energy efficiency in network training. We show that accurate modeling of synaptic devices is very important for assessing network performance on hardware implementations. We develop compact models for device variations based on experimental data and investigate their impact on unsupervised learning performance. Finally, we discuss methodologies to redesign the network training algorithm using realistic characteristics of devices in order to compensate for variations, demonstrating high recognition accuracy and energy efficiency during network training in hardware.

While machine-learning approaches have done tremendous progresses these last years, more is expected with the third generation of neural networks that should sustain this evolution. In addition to unsupervised learning and spike-based computing capability, this new generation of computing machines will be intrinsically dynamical systems that will shift our conception of electronic. In this context, investigating new material implementation of neuromorphic concept seems a very attracting direction. In this presentation, I will present our recent efforts toward the development of neuromorphic synapses that present attractive features for both spike-based computing and unsupervised learning. From their basic physics, I will show how their dynamics can be used to implement time-dependent computing functions. I will also extend this idea of dynamical computing to the case of reservoir computing based on organic sensors in order to show how neuromorphic concepts can be applied to a large class of dynamical problems.

New platforms specialized for artificial intelligence (AI) are necessary to reduce power consumption of deep neural network operations. One potential hardware alternative is a neuromorphic computing system, which utilizes a crossbar structure comprised of artificial synapses at crosspoints for fast synaptic weight update and low-power vector-matrix multiplications. Each array crosspoint must be capable of accessing many analog conductance states by symmetric potentiation and depression in response to uniform voltage pulses.

We have developed epitaxial random-access memory (epiRAM) as a hardware platform for neuromorphic computing. As a switching medium, single-crystalline Silicon (Si) prevents reduction of Silver ions (Ag⁺) within the lattice, which prohibits conductive bridging at typical operating temperatures. However, we have found that epitaxial Silicon-Germanium (Si-Ge) is a suitable solid electrolyte for Ag⁺ migration and Ag conduction channel evolution through threading dislocation pathways. Heteroepitaxial Si-Ge films are grown on Si in the kinetically-limited metastable regime to suppress dislocation glide. Threading pathways are widened using defect-selective etching. Within artificial synapses, metallic conduction pathways form through widened threading dislocations in the Si-Ge layer. This channel evolution allows artificial synapses to have tunable conductance states. Using metastable Si-Ge, we demonstrate that analog resistive switching can occur through devices with electrode area smaller than 25 x 25 nm with similar Current-Voltage (I-V) characteristics to bigger (5 µm x 5 µm) devices. This result suggests channel evolution is dominated by localized conductive regions comprised of dense threading dislocation arrays.

Synaptic weights in neural networks can be updated and stored as conductance states of epiRAM devices at array crosspoints. Conduction channel evolution is confined for linear and gradual conductance change during neural network training. Switching threshold voltage and read current tuning is accomplished by adjusting the Schottky barrier at the p+ Si/ Ag conduction channel interface. Semiconductor doping can rectify sneak currents to enable large-scale arrays. EpiRAM in passive crossbars could be trained for MNIST handwriting digit recognition with up to 95.1% accuracy, only 2% lower than the binary Multilayer Perceptron (MLP) software baseline. Thus, epitaxial Si-Ge artificial synapses are a promising new hardware platform for AI.

New computational platforms such as oscillator based neuromorphic computing models [1] can overcome bottlenecks of traditional von Neumann architectures. Recently, a novel concept of oscillator based on ferroelectric field-effect-transistor (FEFET) has been proposed [2]. A coupled oscillator network, which mimics a biological brain [3], [4], has demonstrated its great efficacy in solving computationally hard problems such as associative memory, pattern matching and recognition, and combinatorial optimization [5], [6]. Ferroelectrics, an emerging material, is well-suited for new computing paradigms due to its built-in hysteresis and non-volatility [7]. In this talk, we will analyze ferroelectric oscillators and their synchronization dynamics in a coupled network. Coupled oscillators and computing models based on emerging devices such as metal-insulator-transition (MIT) devices [8] and spin transfer nano-oscillators (STNO) [9], [10] have been reported. FEFET oscillators have unique advantages over them. First, FEFET are voltage driven and hence are significantly more power efficient compared to STNOs which require high critical current for oscillations to set in. Secondly, MIT materials are temperature sensitive thus uniform operation over a reasonable range of operating temperature can be challenging. On the other hand, FEFET is significantly less temperature sensitive, and hence stable oscillation over a reasonable wide temperature range is possible in these circuits. Thirdly, owing to being a three-terminal device, the FEFET oscillator is much more diverse in terms of functionalities and circuit configurability. Both gates of the top FEFET and the bottom MOSFET can be used to control the dynamics of the couple network leading to a vast design space and innovative interconnections where voltage swings and D.C. levels of the oscillations as well as the overall power can be dynamically tuned by nearest neighbor interactions, a global input, or a feedback from the output.
In summary, FEFET oscillators can serve as an efficient physical platform for alternative and neuromorphic computing models.

Reference:
[1] Hoppensteadt, Frank C., et al. Phys. Rev. Lett., vol. 82, no. 14, May 1999, pp. 2983–2986.
[2] Wang, Zheng, et al. IEEE Electron Device. Lett., vol. 38, no. 11, 2017, pp. 1614–1617.
[3] Schemmel, Johannes, et al. 2008 IEEE International Joint Conference on Neural Networks, 2008
[4] Merolla, P. A., et al. Science, vol. 345, no. 6197, July 2014, pp. 668–673.
[5] J. J. Hopfield, et al. Biol. Cybern., vol. 52, no. 3, pp. 141–152, 1985.
[6] Traversa, F. L., et al. Science Advances, vol. 1, no. 6, Mar. 2015.
[7] Muller, J., et al. ECS Transactions, vol. 64, no. 8, Sept. 2014, pp. 159–168.
[8] Shukla, N., et al. 2014 IEEE International Electron Devices Meeting, 2014
[9] Nikonov, Dmitri E., et al. IEEE J. Explor. Solid-State Computat. Devices and Circuits, vol. 1, 2015, pp. 85–93.
[10] Yogendra, Karthik, et al. IEEE Trans. Magn., vol. 51, no. 10, 2015, pp. 1–9.

Spiking Neural Networks propose to mimic nature’s way of recognizing patterns and making decisions in a fuzzy manner. To develop such networks in hardware – two requirements are key. First, extremely large networks are needed in order to emulate the brain of advanced organisms e.g. human brain consists of 100 billion neurons each connected to 10000 synapses. This level of integration is challenging even for silicon-based VLSI Technology. Second, another challenge is to mimic the extreme energy efficiency of the biological brain. This requires highly scaled and energy efficient components like neurons with the ability to enable Very Large Scale Integration (VLSI). We have proposed a silicon-based leaky integrate and fire (LIF) neuron that is based on conventional silicon-on-insulator (SOI) technology [1]. The floating body effect of the partially depleted SOI transistor is used to store “holes” generated by impact ionization in the floating body, which performs the “integrate” function. Recombination or equivalent hole loss mimics the “leak” functions. The “hole” storage reduces the source barrier to increase the transistor current. Upon reaching a threshold current level, an external circuit records a “firing” event and resets the SOI MOSFET by draining all the stored holes. While the Leaky Integrate and Fire operation are mimicked well, which is equivalent to regular or fast spiking, there are other spiking patterns that biological neurons are able to drum out different spiking patterns e.g. intrinsically bursting or chattering dynamics. This three-terminal device is able to mimic these dynamics of biological spiking patterns. We will explore the design space with TCAD simulations. In terms of application, the neuron is able to show classification problems with reasonable accuracy, as well as some navigation problems. The neuron is compared with the other approaches in terms of areal and energy efficiency.

Reference:
1. S. Dutta, V. Kumar, A. Shukla, N. R. Mohapatra, and U. Ganguly, “Leaky Integrate and Fire Neuron by Charge-Discharge Dynamics in Floating-Body MOSFET” Scientific Reports, Article No. 8257, 2017

In this study, we investigated the synaptic transistors with single- and double-gate structures with Pt/HfOx/n-IGZO MOS stack. Similar to the two-terminal memristor-based synaptic device, the three- and four-terminal transistors exhibited the synaptic motion with drain current modulation through memcapacitance and channel conductance change. The memcapacitance, i.e. a memorized capacitance change, in Pt/HfOx/n-IGZO was analog, polarity-dependent, reversible and nonvolatile so that the drain current with this gate stack was altered as repeatedly applying drain and gate voltages. In addition, the channel conductance change resulting from the ionic interaction between HfOx and IGZO was clearly observed. The modulated drain current in transistor with this MOS stack corresponds to the synaptic weight modulation particularly with analog potentiation, depression, pulse amplitude, width, number, time interval dependent characteristics, which emulate various synaptic motions. In single-gate transistor, the drain current was gradually increased by up to three orders of magnitude as repeating application of gate voltage. Also, its increase was proportional to the positive voltage pulse amplitude and width. Reversibly, the decrease of current was also proportional to the negative voltage pulse amplitude and width. In addition, the current change was changed from volatile to nonvolatile feature as repeating pulses with high amplitude and short interval between pulsing, referring the short-term to long-term transition. Besides the single-gate synaptic transistor altering and measuring the synaptic weight modulation with the same gate, the double-gate synaptic transistor was fabricated and its synaptic motions were demonstrated. Even if the single-gate transistor could modulate the channel conductance; it might have disturbance of synapse state as operating signal processing and learning by the same single gate voltage. From this point, it is beneficial to use double gates; one for learning and the other for signal processing or reading, at the sacrifice of little more complexity in device structure. The channel conductance was altered by applying the voltage in Pt/HfOx/n-IGZO stack and it was measured with Al/YSZ/SiO₂/n-IGZO stack that did not alter the conductance but just measured it for reading operation. It verified the separate synaptic weight modulation and reading without disturbance, providing wider synaptic operation schemes with double-gate transistor. The various synaptic motions with single- and double-gate synaptic transistor will be discussed in detail.

An organic electronic transistor has been developed that possesses the combined long- and short-term potentiation and depression. The principle of operation of the transistor resides at the boundary of field-effect operation and electrochemical switching and has been built up using a thiophene-based trimer as the semiconductor. The trimer can undergo switching of two kinds, a combined feature that represent the fundamental change in synaptic weight at the short and long terms. The switching and modulation of device parameters will be reported in the context of neuromorphic and evolvable computing, especially targeting decision-tree architectures.

Organic/inorganic hybrid perovskite (OHP) has been recently highlighted as a next-generation material of electronic devices such as light emitting diodes, and solar cells and memory devices [1]. In particular, significant hysteresis caused by huge ion migration of OHP is a very valuable property for memory devices such as neuromorphic synaptic memory [2]. In recent study, by using methylammonium lead bromide perovskite (MAPbBr₃), two-terminal artificial synapse was demonstrated, and it showed typical synaptic properties caused by ion migration in the perovskite layer [3]. However, previously reported perovskite artificial synapse still needs to be improved in terms of memory retention time and cyclic operation stability. To overcome above issues, here, we report artificial synaptic devices based on polymer-blended perovskite layer. Substitutional polymers in the perovskite structure have an important role for improving ion migration as well as ion trapping in the trap sites such as grain boundaries, interface of layers and so on. In this way, ions can be easily moved when voltage was applied, and then, they cannot return to the original position when the voltage was removed; this mechanism induced long current retention time of perovskite-based artificial synapse memory [4]. Furthermore, passivating effect caused by substitutional polymers can suppress degradation of electrical property of perovskite layer. We expect that our approach would expand the research field of neuromorphic electronics using perovskite materials.

Reference
[1] A. Kojima, K. Teshima, Y. Shirai, T. Miyasaka, J. Am. Chem. Soc. 2009, 131, 6050
[2] Z. Xiao, Y. Yuan, Y. Shao, Q. Wang, Q. Dong, C. Bi, P. Sharma, A. Gruverman, J. Huang, Nat. Mater. 2015, 14, 193.
[3] W. Xu, H. Cho, Y. -H. Kim, Y. T. Kim, C. Wolf, C. G. Park, T.-W. Lee, Adv. Mater. 2016, 28, 5916
[4] S. Meloni, T. Moeh, W. Tress, M. Franckevicius, M. Saliba, Y. H. Lee, P. Gao, M. K. Nazeeruddin, S. M. Zakeeruddin, U. Rothlisberger, M. Graetzel, Nat. Commun. 2015, 7, 10334.

Conducting polymers have been investigated for use as ionically-controlled memory and decision-making neuromorphic devices [1,2]. Unfortunately, there has been little activity on the direct simulation of the operation of such devices that is necessary for the design of more complex circuits and systems. This work creates a numerical simulation scheme for a coupled ionic-electronic structure and uses it to simulate a memristor-like device based on the organic conductor poly(3,4 ethylenedioxythiophene):polystyrenesulfonate (PEDOT:PSS). The modeled device consists of a thin PEDOT:PSS strip with a metal contact on both sides covered with an electrolyte solution containing lithium and perchlorate ions. The conductivity of the device changes when the lithium ions in the electrolyte dedope the PEDOT:PSS by bonding with PSS polymers. A numerical drift diffusion and a Poisson solver was implemented with special features to model the physical properties of the memristor device. The simulator uses a novel method for modelling the ion movement where the saturation in the PEDOT:PSS was implemented by making the drift velocity and the diffusivity an inverse function of local ion concentration using a sigmoid function. The only scaling factors are used to adjust for the material drift and diffusion coefficients and the relative carrier density. The developed simulation algorithm was tested using analytical solutions to the drift diffusion equations and Poisson’s equation. 1-D and 2-D simulations were able to capture the essential physical effects. The comparison of 2-D simulations and the experimental results showed that proposed model worked as expected and produced results that were consistent with the operation of an actual fabricated device. This work can be used to further predict the operation of devices with varying geometries and more complex multi-device circuits and systems. The software is very flexible and additional effects can be added through enhanced modelling of the charge transport mechanisms.

[1] V. Erokhin, “Organic memristors : Basic principles”, Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 5–8 (2010).
[2] E. Barrera Ramirez, S. McGarry, “Geometric dependence in the electric response of electro-ionic polymer devices”, Organic Electronics, Vol. 15, Iss. 6, pp. 1131-1137 (2014).

Cyber security has become a serious concern as the internet penetrates every corner of our life over the last two decades. The rapidly developing human-machine interfacing calls for an effective and continuous authentication solution. Herein, we developed a two-factor, pressure-enhanced keystroke-dynamics-based security system that is capable of authenticating and even identifying users through their unique typing behavior, with an accuracy up to 98.7%. The system consists of a rationally designed triboelectric keystroke device that converts typing motions into analog electrical signals, and a support vector machine (SVM) algorithm based software platform for user classification. Our active sensing hardware based on triboelectrification is superior to conventional pressure-sensor-based keyboards in terms of system integrity and to previous triboelectric devices in terms of signal quality. Our device has a unique touch-proof feature, i.e. capable of distinguishing inadvertent touch from genuine typing, and thus has an improved signal-to-interference-plus-noise ratio from 2 dB to 10 dB thanks to a specifically designed shield structure. This unconventional silicone-based keystroke device is self-powered, stretchable and water/dust proof, which makes it highly mobile and applicable to versatile working environments. A customized SVM-based software platform was developed and integrated with the triboelectric keystroke device to construct the two-factor authentication/identification system. In the training process, normalized feature vectors are extracted from active typing signals to build user profile models via supervised learning with the help of principle component analysis and SVM. For the implementation of the authentication and identification, a similar process is carried out and the test user profile is cross-referenced with the existing profile database for decision making through a pre-trained LibSVM-based classifier, with a classical binary one for authentication and a multi-class one for identification. For the first time, moreover, the benefits of additional typing features based on signal magnitudes to keystroke dynamics have been quantified using our customized difference score scheme. The promising application of this novel system in the financial and computing industry can push cyber security to the next level, where leaked passwords would possibly be of no concern.
Ref: C. Wu, W. Ding, R. Liu, Z.L. Wang, et al. Keystroke dynamics enabled authentication and identification using triboelectric nanogenerator array. [In submission]

The concept of memcapacitance, defined as a memorized capacitance change, has been suggested and demonstrated with several structures. In particular, the memcapacitance in MOS capacitor structure is advantageous in that its capacitance change can modulate the drain current in MOS-based transistor. The nonvolatile capacitance change in MOS structure can be applied to nonvolatile memory by altering the drain current as well as the threshold voltage. In addition, it can be adopted in artificial synaptic device mimicking human brain motions, where the drain current corresponding to the synaptic weight can be altered by capacitance change in MOS as adaptive motion of synapse. In this study, the memcapacitance characteristics of Al/SiO₂/TaOx/n-IGZO MOS structure were investigated for the application to synaptic transistor. The MOS structure exhibited analog, polarity-dependent, and reversible memcapacitance through the redistribution of oxygen ions between TaOx and n-IGZO. When a positive bias was applied to the Al top electrode, an accumulation capacitance increased gradually, implying increased permittivity of TaOx as a possible result of the migration of oxygen ions from n-IGZO to TaOx. Consequently, the depletion capacitance increased owing to the increased oxygen vacancy concentration in the n-IGZO layer. The capacitances could be restored by applying a negative potential to repel oxygen ions from TaOx back to n-IGZO. Using this phenomenon, this MOS structure could be applied to nonvolatile memory and synaptic transistor with analog increase and decrease of capacitance leading to the change of memory state and the synaptic potential and depression behavior, respectively. The detailed memcapacitance characteristics of Al/SiO₂/TaOx/n-IGZO with respect to the thickness of each layer and voltage application condition, and so on.

Threshold switching or current-controlled negative differential resistance (CC-NDR) in strongly correlated oxides is of considerable technological interest for applications such as a memory selector elements and voltage-controlled oscillators [1]. A particular focus in recent years has been the development of coupled relaxation oscillators for neuromorphic computing applications, based on the fact that these have been shown to emulate the functionality of computational neurons [2]. Threshold switching in NbO_x thin films has been attributed to two main mechanisms, namely: a thermally induced metal-insulator phase transformation (MIT) [3] or Poole-Frenkel (PF) conduction [4], with more recent studies showing that both mechanisms can play a role [5].

In this study we fabricated simple metal-insulator-metal (MIM) capacitor structures, comprising: TiN (50 nm) /NbO_x (70 nm)/Pt (50 nm) heterostructures, with top Pt contacts of 15-150 µm diameter. The electroformed devices were found to exhibit a combination of volatile threshold switching and non-volatile bipolar resistive switching, and therefore comprised a selector/memory (1S1M) structure. The threshold and hold voltages were found to be independent of niobium oxide thickness and device area, suggesting that the switching volume is localized both laterally and vertically. This was confirmed by finite element modelling using both PF and MIT switching mechanisms. Specifically, the models predicted a significant increase in threshold voltage with increasing film thickness when switching occurred along the full length of the filamentary conduction path. Only by limiting the switching region to a fixed volume near the metal/oxide interface could the experimental results be reproduced. Experiments also showed that the threshold current decreased with increasing film thickness. Both finite element and equivalent circuit models suggest that this arises from a parallel resistance, most likely associated with a halo-region surrounding the metallic filament. These experimental results and model predictions are used to develop a comprehensive model of filamentary threshold switching in NbOx films.


References

1. S. Li, X. Liu, S. K. Nandi, D. K. Venkatachalam, and R. G. Elliman, Appl. Phys. Lett. 106, 212902 (2015).
2. M. D. Pickett, G. Medeiros-Ribeiro, and R. S. Williams, Nat. Mater., 12, 114–117 (2013).
3. M.D. Pickett, and R.S. Williams, Nanotechnology, 23(21): 215202. (2012)
4. S. Slesazeck, H. Mähne, H. Wylezich, A. Wachowiak, J. Radhakrishnan, A. Ascoli, R. Tetzlaff, and T. Mikolajick, RSC Advances, 5, 102318 (2015)
5. S. Kumar, Z. Wang, N. Davila, N. Kumari, K.J. Norris, X. Huang, J.P. Strachan, D. Vine, A.L.D. Kilcoyne, Y. Nishi, and R.S. Williams, 8, 658 (2017)

The biological synapses are functional links between neurons, through which 'information' transmitted in the neuron network. The information can be stored and processed simultaneously in the same synapse through tuning synaptic weight, which is defined as the strength of the correlation between two neighboring neurons, and the operation is collective and adaptive. Although silicon-based complementary metal-oxide-semiconductor circuits have been developed to emulate synaptic behaviors, it is still facing significant challenges in large-scale integrations and huge energy consumption. Memristive devices, in which the conductance can be retained according to the history of applied voltage and current, provide a more promising way to emulate synapses by substantial reduction in complexity and energy consumption. Recently, ionic/electronic hybrid three-terminal memristive devices have been introduced. It gives a more flexible operation for the signal processing and learning in synaptic circuits. In this work, we investigated a lateral three terminal memristive device based on α-phase molybdenum oxide (α-MoO₃), a typical two-dimensional (2D) transition metal oxides, in ambient atmosphere, and capitalized on the nanoscale device to mimic a biological synapse. Ionic liquid was selected as gate terminal, serving as pre-synaptic neuron to generate neurotransmitters. The ultrathin α-MoO₃ single crystal flake serves as post-synaptic neuron, whose conductance can be modulated. The excitatory post-synaptic current, depression and potentiation of synaptic weight, paired-pulse facilitation, the transition of short-term plasticity to long-term potentiation have been demonstrated in the three terminal devices. These results provide an insight into the potential application of two-dimentional α-MoO₃ for synaptic devices with high scaling ability, low energy consumption, and high processing efficiency.

References
[1]C. S. Yang, D. S. Shang, Y. S. Chai, et al., Phys. Chem. Chem. Phys. 18, 12466 (2016).
[2]C. S. Yang, D. S. Shang, Y. S. Chai, et al., Phys. Chem. Chem. Phys. 19, 4190 (2016).
[3]C. S. Yang, D. S. Shang, N. Liu, et al., Adv. Mater. 29, 1700906 (2017).

Memristive synaptic devices which can be reliably programmed to a continuum of resistance states at low energy cost (<100pJ per write operation is competitive with CMOS) are highly desirable for neuromorphic computing. Nevertheless, most memristors to date can be programmed to only a few resistance states (typically 2-3). In addition, despite recent progress in demonstrating memristor-based neuromorphic arrays, no architecture to date can operate with the projected energy efficiency while maintaining high accuracy. For example, filament forming metal-oxide (FFMO) and phase-change memristors (PCM) suffer from excessive write noise and non-linearities, mostly due to their stochastic switching mechanism.

Here, we demonstrate polymer-based synaptic devices based on organic mixed-ionic-electronic-conductors (MIECs) where the resistance of a transistor-like channel can be continuously tuned (over >100 distinct states) by low-voltage (<1V) potentiation/depotentiation pulses, enabling low-power operation (~pJ range). These devices show extremely low write noise (<1%) and close-to-symmetric resistance tunning in response to pulsed inputs. Reversible switching relies on controlled insertion/extraction of ions which couple to conjugated polymer chains. The ions dope/de-dope the transistor-like channel, altering its resistance.

High resistances are instrumental for building large neuromorphic arrays, and we demonstrate polymer-based synaptic devices which operate at resistances of the order ~10MΩ when scaled down to ~10µm dimensions. We show that the resistance switching mechanism above is general and can be exploited in several organic MIECs. As such, polymer-based synaptic devices offer a promising and yet to be fully exploited alternative to conventional memristive devices.

Deep Neural Networks (DNNs) are very large artificial neural networks trained using very large datasets, typically using the supervised learning technique known as backpropagation. Currently, CPUs and GPUs are used for these computations. Over the next few years, we can expect special-purpose hardware accelerators based on conventional digital-design techniques to optimize the GPU framework for these DNN computations. Here there are opportunities to increase speed and reduce power for two distinct but related tasks: training and forward-inference. During training, the weights of a DNN are adjusted to improve network performance through repeated exposure to the labelled data-examples of a large dataset. Often this involves a distributed network of chips working together in the cloud. During forward-inference, already trained networks are used to analyze new data-examples, sometimes in a latency-constrained cloud environment and sometimes in a power-constrained environment (sensors, mobile phones, “edge-of-network” devices, etc.).

Even after the improved computational performance and efficiency that is expected from these special-purpose digital accelerators, there would still be an opportunity for even higher performance and even better energy-efficiency from neuromorphic computation based on analog memories.

In this presentation, we discuss the origin of this opportunity as well as the challenges inherent in delivering on it, with a particular focus on memory materials. We review our work towards neuromorphic chips for the hardware acceleration of training and inference of Fully-Connected DNNs [1-4]. We use arrays of emerging non-volatile memories (NVM), such as Phase Change Memory, to implement the synaptic weights connecting layers of neurons. We will discuss the impact of real device characteristics – such as non-linearity, variability, asymmetry, and stochasticity – on performance, and describe how these effects determine the desired specifications for the analog resistive memories needed for this application. We present some novel solutions to finesse some of these issues in the near-term, and describe some challenges in designing and implementing the CMOS circuitry around the NVM array. We will end with an outlook on the prospects for analog memory-based DNN hardware accelerators.

[1] G. W. Burr et al., IEDM Tech. Digest, 29.5 (2014).
[2] G. W. Burr et al., IEEE Trans. Elec. Dev, 62(11), pp. 3498 (2015).
[3] G. W. Burr et al., IEDM Tech. Digest, 4.4 (2015).
[4] P. Narayanan et al., IBM J. Res. Dev., 61(4/5), 11:1-11 (2017).

The ultimate goal of A.I. research is to build machines that exceed the flexibility and dynamism of the human brain. Currently, the predominant approach in A.I. is to use unlimited data to solve narrowly defined problems. To progress towards human-like intelligence, A.I. benchmarks will need to be extended to focus more on data efficiency, flexibility of reasoning, and transfer of knowledge between tasks -- the constraints on a solution to a problem are as important as the problem itself. In this talk, I will describe the challenges and successes in making these ideas operational. At Vicarious, we use the language of probabilistic graphical models as the representational framework. Compared to neural networks, graphical models have several advantages such as the ability to incorporate prior knowledge, the ability to answer arbitrary probabilistic queries, and the ability to deal with uncertainty. However, one of the downsides of probabilistic graphical models is that inference can be intractable. By incorporating several insights that were originally discovered in neuroscience, we were able to create probabilistic models on which accurate inference can be performed using messremeage passing algorithms that are similar to the computations in a neural network. This allowed us to crack text-based CAPTCHAs with high data efficiency and to beat a text parsing benchmark with 300-fold efficiency compared to deep learning. Recently, we also showed progress in general game playing where we demonstrated vastly superior zero-shot generalization compared to deep reinforcement learning. I'll describe the opportunities in robotics that we are currently exploring and conclude with a description of the challenging problems that remain to be solved.

Using nanodevices is an exciting path for hardware implementation of compact and low power forms of computing. A critical challenge, however, is that at their smallest sizes, nanodevices tend to be stochastic and highly variable. The brain can be a source of inspiration for solving this issue as it computes using very low energy with components that are also stochastic and highly variable. Population coding is one method that seems to enable the brain to function in such conditions. In this paradigm, information (about an incoming stimulus for instance) is encoded in an assembly of neurons rather than a single neuron.
In this work, we draw an analogy between stochastic neurons and stochastic nanodevices, superparamagnetic tunnel junctions. We show that these devices can be used as artificial neurons in population-coding computing schemes.
Superparamagnetic tunnel junctions oscillate stochastically between two stable states with a given rate which depends on the electrical stimulus applied to the device. First, we demonstrate experimentally how the rates of a small population of superparamagnetic tunnel junctions can form a basis set of functions. This means that linear combinations of the individual rates can produce arbitrary non-linear functions, which is a powerful building block for computations. Then we use numerical simulations to show how two interconnected populations constitute a computing unit. The connecting synaptic weights are learned by trial and error, thus defining the transformation of information from the input to output populations. We show that this system is resilient to device variability and failure. We propose to use stable magnetic tunnel junctions to store the synaptic weights. Finally, we design the full architecture of the system, including the nanodevices and the CMOs circuits to interface them. We use standard circuit simulation tools to show that our system has low energy and low area consumption.

Emulation of biological synapses is necessary for future brain-inspired neuromorphic computational systems that could look beyond the standard von Neuman architectures. Here, artificial synapses based on ionic-electronic hybrid oxide-based transistors are demonstrated on rigid and flexible substrates. The flexible transistors reported here depict a high field-effect mobility of ≈9 cm² V⁻¹ s⁻¹ with good endurance and mechanical stability. Comprehensive learning abilities/synaptic rules like paired-pulse facilitation, excitatory and inhibitory postsynaptic currents, spike-time dependent plasticity, consolidation, superlinear amplification, and dynamic logic are successfully established depicting concurrent processing and memory functionalities with spatiotemporal correlation. The results present a fully solution processable approach to fabricate artificial synapses for next-generation printable transparent neural circuits.

We recently designed an example of “materials that compute”, where the material and computer are one and the same entity, and demonstrated the ability of these systems to perform pattern recognition. In our design, the system is composed of units, which encompass a self-oscillating polymer gel that undergoes the Belousov-Zhabotinsky (BZ) reaction and an overlaying piezoelectric (PZ) cantilever. Driven by the BZ oscillating chemical reaction, each gel periodically swells and shrinks in volume and thus rhythmically deflects the overlaying PZ cantilever. The periodic deflection of the PZ plate generates an oscillating electrical voltage, which is transmitted to other units through the electrical wires. As a result, the chemo-mechanical oscillation of one BZ gel affects the oscillations of all the other gels. Due to the interaction, the oscillations of the BZ-PZ units achieve in-phase or anti-phase synchronization with each other. These modes of synchronization can be used to represent binary information and enable the BZ-PZ oscillator network to store and to recognize patterns. Here, we show that introducing capacitors into the system allows us to increase the amount of information that can be stored in a given BZ-PZ network. The capacitors modify the system dynamics and create additional stable synchronization modes, which can be used for storing additional information. We then demonstrate how this extended information storage enhances the functionality of the BZ-PZ networks in performing computational tasks.

The future acceleration of many computational workloads is expected to depend on novel architectures, circuits, and devices. I describe an effort utilizing the analog nature of memristor crossbar arrays to accelerate vector-matrix multiplication, which underpins many applications in image and signal processing, neural networks, and scientific computations. Significant improvement over CPUs, GPUs, and custom ASICs is anticipated using such systems. I describe our work spanning atomic understanding and engineering of memristors, integration with CMOS circuits, fine programming control over memristors, and experimental implementations of neural networks, image and time-series data processing, error-robust operations.

Additionally, I will describe other applications of memristor technology for mimicking the neuronic behavior in neuromorphic systems, in addition to the synaptic functions. This work begins with our investigations of niobium oxide-based systems, including x-ray based physical characterization and emulation of neuron-like spiking behaviors. As devices were further scaled down to below 100 nm in lateral dimension, interesting dynamics was observed that we described as positive feedback coupling to thermal fluctuations. Chaotic behavior was observed and this device behavior was utilized for the construction of a system to solve optimization problems such as the traveling salesmen problem.

Large attention has been recently given to a novel technology named memristor, for having the potential of becoming the new electronic device standard. Memristors are dynamic nanoscale electron devices that are nowadays regarded as a promising solution for establishing next-generation memory and computation, owing to their potential of achieving “more” (functionality/information storage) for “less” (power and physical dimensions). Most interestingly, it has been envisioned that mimicking the functionality of biological brain systems could fulfil its potential. During this talk, I will present how memristors can be exploited in practical applications, with particular emphasis in the areas of memory and computation. I shall highlight the opportunities that this emerging technology brings for addressing the needs of modern massively parallel computing and will present several practical applications where metal-oxide memristors can be used for processing effectively and efficiently big data.

Since the preliminary investigation of neuromorphic computing, numerous advanced materials, devices, and systems were used or proposed to emulate neural network in human brain aimed at lower power consumption, higher fault-tolerant, and better ability of dynamic learning than conventional computing. Commercial level neuromorphic circuits based on the complementary metal-oxide-semiconductor (CMOS) are already accessible, which consists of 6 to 12 transistors each unit depending on the specific functionality and robustness of the design. However, the higher energy consumption and physical area have led researchers to look for architectures based on single device and novel materials.
In our work, we demonstrate a fully scalable fabrication of Indium phosphide (InP) channel transistors from templated films of InP grown directly on Si/SiO₂ wafer using thin-film vapor-liquid-solid growth. Several significant synaptic characteristics such as elasticity, short- and long-term plasticity, metaplasticiy, spike number dependent plasticity and spike timing dependent plasticity have been mimicked, by modeling gate electrode as the pre-synaptic axon terminal, the drain electrode as the post-synaptic dendrite, and the gate oxide-semiconductor channel as the synapse, in which FET channel conductance was interpreted as the synaptic weight.
Controlling the charging and discharging of interfacial traps in the MOS structure allows us to engineer hysteresis of the channel conductance to customize the synapse behavior and modify the synapse weight non-linearly. It underpins optimal selectivity of signal transduction and satisfies the key neuromorphic architecture characteristic--changing the properties of the device depending on its history, which is also relating to the procedure of “learn”.
Manipulating the hysteresis in a family of transfer characteristics, in spike timing dependent plasticity (STDP) emulation, we attain maximum potentiation (depression) for the minimum positive (negative) interval time, which gradually decays down to give elasticity, as we expected. This buttresses the scalable InP channel transistors on silicon as promising devices and platform for neuromorphic computation.

Despite a relatively small number of neurons (around 100,000 for the case of Drosophila melanogaster), insects exhibit an impressive array of abilities, including short term and long term memory, sensory integration, associative and self-learning capabilities, and extremely fine motor skills. Our research uses the insect brain as a model system to understand how to design neuromorphic architectures that can act as smart sensors capable to register, adapt, and respond to changes in the environment.

In this talk we will focus on two fundamental aspects of insect’s brain anatomy and their implication both at the architecture and materials levels: the implementation of context-selective plasticity in order to allow for reinforced learning, and the need to explore heterogeneous architectures integrating components that respond at different time scales in order to create stable feedback loops. From a materials standpoint, the ability to control the volatility of memristive elements and the way charge is stored, released, and dissipated, are key aspects to enable this functionality. We exemplify this by emulating a system inspired on the mushroom body of the insect brain.

Network connectivity is one of the main determinants for whether a neural network can be efficiently implemented in neuromorphic hardware. Connectivity demands will become even more severe in future applications of deep learning when the number of neurons in layers will increase, causing a quadratic growth in the number of connections between them. Two approaches may be pursued to tackle the connectivity bottleneck: (a) utilize new materials for the implementation of large-scale arrays of nanoscale synaptic connections, and (b) reduce the number of connections within a deep neural network.

When using nanoscale synaptic connections for training a deep neural network (approach a), one has to deal with the fact that parameter updates in these synapses are noisy. That is, the direction of each parameter update will consist of two parts, an intended update (usually given by the negative gradient of an error function), and a noise term. I will discuss a theoretical framework – termed synaptic sampling – that can be used to analyze the implications of noisy network training theoretically. In particular, our analysis shows that noisy learning rules implement a process that samples network parameter configurations from the posterior distribution of configurations, that is, the distribution that combines task performance and the parameter constraints in a Bayes optimal manner. Hence, noisy parameter updates can be utilized for training deep neural networks.

Furthermore, the same theoretical framework can be used for training very sparsely connected deep neural networks (approach b). I will discuss a novel algorithm, deep rewiring (DEEP R), that enables us to train directly a sparsely connected neural network. DEEP R automatically rewires the network during supervised training so that connections are established where they are needed most for the task, while its total number is all the time strictly bounded. We demonstrate that DEEP R can be used to train very sparse feedforward and recurrent neural networks on standard benchmark tasks with just a minor loss in performance. DEEP R is based on the synaptic sampling framework, thus inheriting its theoretical properties. In particular, we can view DEEP R as stochastic sampling of network configurations (that is, configurations of network weights and network connections) from a posterior distribution over configurations. DEEP R may thus provide the basis for future neuromorphic training algorithms since it allows to train large deep networks directly on hardware while keeping the total number of connections strictly bounded at any time during training.

The performance of current information processors are predominantly based on complementary metal-oxide-semiconductor (CMOS) transistors. However, CMOS scaling have started to face significant challenges and besides the physical limits, the conventional computing paradigm based on binary logic and Von Neumann architecture is becoming increasingly inefficient with onset of big data revolution and growing complexity of computation. Neuromorphic computing is the state-of-the-art research trend in the field of memory and logic devices where the goal is to build a versatile computer that is efficient in terms of energy and space, homogeneously scalable to large networks of neurons and synapses, and flexible enough to run complex behavioral models of the neocortex as well as networks inspired by neural architectures. Memristors, with their gradually modified conductivity level can mimic the biological synapses. Low energy consumption, ultrafast operation and small dimensions are the most essential requirements for a memristor to perform tasks similar to a synapse and become as efficient as human brain. A ferroelectric tunnel junction (FTJ), where gradual modulation of conductance can be achieved by controlled rotation of ferroelectric domains can act very efficiently as a synapse. Also the non-volatility of the stored information in the ferroelectric memories make them even more attractive as potential candidates for future neuromorphic computing building blocks. Here, we report on the performance of FTJs with a spin-coated organic ferroelectric P(VDF-TrFE) tunnel barrier. We have measured up to 10⁷% tunneling electroresistance (TER) effect in these FTJs on a semiconducting Nb-doped STO bottom electrode at room temperature that persists until the ferroelectric Curie point of P(VDF-TrFE) [1]. Also these junctions show very clear and reproducible memristive behavior based on variable amplitude and duration of the applied voltage pulses, fast switching, long data retention of the high, low and different intermediate states, which is extremely promising for neuromorphic applications.
[1] S. Majumdar, B. Chen, Q. Qin, H. S. Majumdar, S. van Dijken, Adv. Func. Mater. (in press).

Different from the von Neumann architecture in computers, our brain has more powerful properties including fault tolerance, ability to develop and learn as well as low power consumption. These characteristics make brain-like computation a very interesting topic to investigate. Varies kind of devices such as resistive memory, organic field effect transistor (OFET) with proton transfer dielectric or organic electrical chemical transistor (OECT) has demonstrated their potentials to mimic the signal transfer pattern at the synapse between neurons. In the current presentation, I will focus on the integration of the OECT devices to develop non- von Neumann circuits to mimic the processing system in the human brain under different external stimulations. The PEDOT with different doping levels of poly(tetrahydrofuran) (PTHF) will be used to regulate their paired-pulse facilitation and the synaptic plasticity of each OECT devices in the circuit. Compared with the transistors based on undoped PEDOT:PSS, the decay time constants of the excitatory post-synaptic current (EPSC) show orders of magnitude increase. Different OECT sensors will be connected to the OECT neuromorphic circuit as the triggering input to mimic the receptors on our skin. The proposed device structure can be employed as a standard configuration for neuromorphic computing with external stimulation inputs.

Coming generations of information technology will process unprecedented amounts of loosely-structured data, including streaming video and audio, natural languages, real-time sensor readings, contextual environments, or even brain signals. The computation demands of these abundant-data applications far exceed the capabilities of today’s electronics, and cannot be met by isolated improvements in transistor technologies, memories, or integrated circuit (IC) architectures alone. Transformative nanosystems, which leverage the unique properties of emerging nanotechnologies to create new IC architectures, are required to deliver unprecedented performance and energy efficiency. However, emerging nanomaterials and nanodevices face major obstacles such as inherent imperfections and variations. Thus, realizing working circuits, let alone transformative nanosystems, has been infeasible.

The N3XT (Nano-Engineered Computing Systems Technology) approach overcomes these challenges through recent advances across the computing stack: (a) new logic devices using nanomaterials such as one-dimensional carbon nanotubes (and two-dimensional semiconductors) for high performance and energy efficiency; (b) high-density non-volatile resistive and magnetic memories; (c) ultra-dense (e.g., monolithic) three-dimensional integration of logic and memory with fine-grained connectivity; (d) new IC architectures for computation immersed in memory; and, (e) new materials technologies and their integration for efficient heat removal.

N3XT hardware prototypes represent leading examples of transforming scientifically-interesting nanomaterials and nanodevices into actual nanosystems. Compared to conventional approaches, N3XT architectures promise to improve the energy efficiency of abundant-data applications significantly, in the range of three orders of magnitude. Such massive benefits enable new frontiers of applications for a wide range of computing systems, from embedded systems to the cloud.

Resistive RAM (ReRAM) elements based on transition-metal oxide layers are rapidly becoming viable options for nonvolatile multi-level information storage as well as for the implementation of memristive synaptic functions in neuromorphic operations, allowing easy integration with conventional CMOS technologies. In this talk, we will review the ongoing research at EPFL on the realization of various ReRAM elements based on TiOx, TaOx, WOx and HfOx layers tailored for low voltage operation, as well as the design and co-integration of the CMOS peripheral circuitry for the read/write operations. The fabrication and characterization of ReRAM devices are explored using wafer-scale post-processing as well as using individual samples. In particular, the chip embedding platform enabling post-processing of diced samples for fabrication of memristive elements will be discussed, and examples will be provided for potential neuromorphic functions such as spike-timing-dependent-plasticity (STDP) and back-propagation algorithms implemented on cross-bar arrays.

The next generation of information memories and neuromorphic computing in electronics rely largely on solving fundamental questions of mass and charge transport of oxygen ionic defects in materials and their structures. Here, understanding the defect kinetics in the solid state material building blocks and their interfaces with respect to lattice, charge carrier types and interfacial strains are the prerequisite to design new material properties beyond classic doping. Through this presentation basic theory¹ and model experiments for solid state oxides their impedances and memristance², electro-chemo-mechanics and lattice strain^3-5 modulations, and extrinsic doping strategy⁸ is being discussed as a new route for tuning material and properties in ionic conducting oxide film structures up to new device prototypes based on resistive switching. Central are the making of new oxide film materials components, and manipulation of the charge carrier transfer and defect chemistry (based on ionic, electronic and protonic carriers)^{1-2, 5-6}, which alter directly the resistive switching property and future computing performances. A careful study on the influence of microstructure and defect states vs. the materials` diffusion characteristics is in focus. For this, we suggest novel oxide heterostructure building blocks and show in-situ spectroscopic and microscopic techniques coupled with electrochemical micro-measurements to probe near order structural bond strength changes relative to ionic and electronic diffusion kinetics and the materials integration to new optimized device architectures and computing operation schemes.

¹⁾Memristor Kinetics and Diffusion Characteristics for Mixed Anionic-Electronic SrTiO_3-δ: The Memristor-based Cottrell Analysis Connecting Material to Device Performance
F Messerschmitt, M Kubicek, S Schweiger, JLM Rupp
Advanced Functional Materials, 24, 47, 7448 (2014)
²⁾Uncovering Two Competing Switching Mechanisms for Epitaxial and Ultra-Thin Strontium Titanate-based Resistive Switching Bits
M Kubicek, R Schmitt, F Messerschmitt, JLM Rupp
ACS Nano 9, 11, 10737 (2015)
³⁾Designing Strained Ionic Heterostructures for Resistive Swicthing Devices
S Schweiger, R Pfenninger, W Bowman, U Aschauer, JLM Rupp
Advanced Materials, 1605049 (2017)
⁴⁾ The Effect of Mechanical Twisting on Oxygen Ionic Transport in Solid State Energy Conversion Membranes
Y Shi, AH Bork, S Schweiger, JLM Rupp
Nature Materials, 14, 721 (2015)
⁵⁾ A Micro-Dot Multilayer Oxide Device: Let’s Tune the Strain-Ionic Transport Interaction
S. Schweiger, M. Kubicek, F. Messerschmitt, C. Murer, J.L.M. Rupp
ACS Nano, 8, 5, 5032 (2014)
⁶⁾ How does Moisture affect the Physical propert of Memristance for Anionic-Electronic Resistive Switching Memories?
F Messerschmitt, M Kubicek, JLM Rupp
Advanced Functional Materials, 25, 32, 5117 (2015)
⁷⁾ Design of Oxygen Vacancy Configuration for Memristive Systems
R. Schmitt, J. Spring, R. Korobko, J.L.M Rupp
ACS Nano, in press (2017)

As promising emerging technologies, transition metal oxide based resistive random-access memory (RRAM) and ferroelectric (FeRAM) are currently actively explored for multiple applications as on-chip memories, 3D-FPGA and neuromorphic computing systems. For their large scale integration, however it is imperative that their advanced functionalities be complemented with superior control of device characteristics in the desired design space. To achieve this goal, device level challenges as switching-parameter variability, cycling endurance, and data retention will need significant improvements. In order to cope with these challenges, by optimally altering the ionic migration based switching mechanism, possible improvement pathways have been recently proposed to enhance performance and data retention in RRAM and stabilize the orthorhombic ferroelectric phase in FeRAM devices. By employing a systematic evaluation of dopants effects on the electronic and kinetic properties and identifying the major factors affecting the switching characteristics of doped transition metal oxides, the opportunities and challenges of achieving desired device performance are addressed.

Memristive systems represent a large class of emerging nanoscale devices which exploit various physical mechanisms to achieve a controlled state-dependent and persistent conductance variation upon electrical stimuli. Industry compatible memristive devices are today of large interest as new building blocks for brain-inspired computing architectures where memory and computational units are co-localized. In this framework, resistance switching memories (RRAM) based on redox reactions and electrochemical phenomena in oxides [1] have been proposed as synaptic elements in spiking neural networks for hybrid CMOS/RRAM hardware in view of big data applications as well as for real time and low power computation systems [2].

In this talk, we will present our work toward the development of analog HfO₂-based electronic synapses. We investigate the dynamics of conductance evolution in a wide space of pulse time widths and voltages. Our results show that in memristive devices based on filamentary switching it is possible to tune the device conductance in an analog way upon application of identical electrical pulses [2-3], by using pulses with duration close to the intrinsic switching time. The use of weak programming conditions leads to an enhanced variability and a reduced memory window with respect the one for digital switching. Salient characteristics of the analog conductance control are the state dependent weight update as a function of the number of incoming pulses and the slow approach to conductance-end saturation value, thus realizing a softly bounded conductance update. This behavior has been recognized in literature [4] as a feature able to improve memory capacity. In order to analyze consistently a large set of data, we use a behavioral model implementing a multiplicative update rule with a weight-dependent weight update which describes the observed slow approach to boundary conductance values. We also discuss various non-idealities such as asymmetric processes for conductance increase (potentiation) and decrease (depression), pulse-to-pulse and cycle-to-cycle variability, to depict a complete picture of analog weighting in HfO₂-based electronic synapses. Finally, the experimental data sets are used to simulate a fully connected winner-take-all spiking neural network with leaky integrate and fire neurons equipped with circuitry emulating the temporal dynamics of biological synapses. The network is benchmarked against the classification of MNIST digits by implementing a semi-supervised spike-based learning protocol based on a generalization of spike time and rate dependent plasticity [5]. The effect on the recognition rate of the number of available analog levels is discussed.
1] S. Brivio et al., Nanotechnology 25, 385705 (2014)
[2] E. Covi et al., Front. Neurosci., 10, 482 (2016)
[3]S. Brivio et al , Appl. Phys. Lett. 109 133504E (2016)
[4] Fusi & Abbott, Nat. Neurosci. 10, 485 (2007)
[5] J. Brader et al., Neural Computation, 19, 2881 (2007)

The analog nature of neural connections (i.e. synaptic weights) as well as the neural network’s massive parallelism are partly why human brains (~20 W) are much better at complex tasks such as pattern recognition than even the most powerful computers (~1 MW) with significantly better energy efficiency. Currently, synapses are still being implemented by a dozens of Si transistors in today’s artificial neural networks, which are power-intensive.
In this talk, we present, for the first time, a low-power, electrochemically-tunable graphene synapse. Through electrochemical intercalation – inserting Li ions in between the layers of graphene, we can precisely and reversibly modulate the conductance of the graphene to emulate the synaptic plasticity in a neural network. For our electrochemical synapse, we adopt a planar nanobattery device configuration where exfoliated graphene and lithium iron phosphate (LFP) are the working and counter electrodes, with a polymer electrolyte (LiClO₄ in PEO) to facilitate ionic exchange.
Upon applying a 50-pA, 100-ms presynaptic pulse to the graphene device, Li ions are intercalated into the graphene through the electrolyte while electrons flow from LFP to graphene in the external circuit. This results in a 1% decrease (30 Ω) in graphene’s resistance R, mimicking the behavior of an excitatory synapse. This behavior is reversed during de-intercalation, mimicking an inhibitory synapse. We also demonstrate long term potentiation and depression (increase and decrease in synaptic weight) by applying a series of intercalating and de-intercalating pulses. We can repeatedly and reliably program the graphene synapse into 100 discrete states over the operating range. To achieve spike-timing-dependent plasticity (STDP), we varied the timing difference Δt between two pre-synaptic pulses. By increasing Δt from 1 to 16 ms, the change in synaptic conductance decreases from 8% to 1%.
In summary, we develop an electrochemical graphene synapse with low energy efficiency (5 pJ/synaptic event), precise control over synaptic conductance (1% ΔR/synaptic event), and good cycling performance. We demonstrate basic neuromorphic functionalities such as short-term and long-term plasticity, as well as spike-timing-dependent plasticity (STDP). This work can lead to the low-power hardware implementation of neural networks for neuromorphic computing.

Resistive switching (RS) phenomena in memristor have attracted a great deal of attention for the application of neuromorphic computing. Transition metal oxide can often function as a conductive or resistive layer in the memristor depending on the applied voltage or the injected current to the device. Such RS phenomena have been well explained by the generation and redistribution of oxygen vacancies caused by the application of external voltage. As-fabricated memristors typically have high resistivity and often require an electroforming process in which a conductive filament is preformed to lower the resistivity locally in the transition metal oxide. Oxygen vacancies play a key role in modulating the conductive filaments and their distribution is believed to determine high or low resistance states of the device.
Reduced titanium dioxide (TiO_2-x) is one of typical RS materials for the memristor. The filamentary mechanism is arguably the most commonly accepted explanation for the RS phenomena in TiO_2-x, which is closely related to the local phase transformation creating oxygen-deficient compounds. However, it is frequently argued that the formation of such conductive filaments is rather stochastic, of less controllability, and, more or less, accompanied by other physical changes in the device structure, such as blow-off or massive redistribution of electrodes, thereby posing a serious problem in the practical operation of devices. Thus non-filamentary-type RS operation based on the bulk carrier conduction modulation is a critical aspect of developing promising memristive devices with high stability, reliability, and controllability.
In this study, we have demonstrated the precise control of oxygen vacancy distribution and related bulk carrier conduction properties in non-filamentary-type memristive devices with a four terminal structure, using reduced TiO_2-x single crystals. In our device, a pair of diagonally opposite electrodes are dedicated to modify the oxygen vacancy distribution in the electrically active zone in between another pair of diagonally opposite electrodes. Here we have visually observed the oxygen vacancy distribution through the electrocoloring phenomena. This allows us to trace the motion of oxygen vacancies under a DC electric field during the device operation. A contrast caused by the electrocoloring well reflected the oxygen vacancy concentration in the device, which was able to be controlled by varying external voltages applied to the electrodes. We confirmed that the current flowing in the device greatly depends on the controlled morphology of oxygen vacancy distribution. In addition, such distribution and the resultant resistance in the device were found to be switchable when the polarization of the applied external voltage was reversed. Mechanisms behind such RS caused by the voltage-driven control of oxygen vacancy distribution in the four terminal memristive device are discussed.

A new generation of nanoelectronic computing technology beyond the traditional von Neumann architecture, which can significantly reduce power consumption for performing tasks such as recognition and learning, is highly desirable for many emerging applications demanding distributed intelligence. New material properties and novel electronic device concepts beyond transistors are essential for realizing the full potential of these new systems. 2D materials with its unique physical properties not found in other semiconductor materials offer promising new opportunities for such applications. In this talk, I will discuss our work on developing two-dimensional materials based electronic devices that can operate at biological level of voltage and current for low power neuromorphic electronics. This includes a record low power memristive device taking advantage of the atomically-thin geometry of 2D materials, black phosphorus based synaptic devices that can emulate the heterogeneity in synaptic connections, and novel low power transistor concepts based on new physical mechanisms suitable for building the peripheral circuits at ultralow voltage levels. I will conclude with remarks on the promising future research directions of 2D material based electronic devices and how their novel properties is expected to benefit the next-generation computation technologies.

Memristor has drawn extensive attention in recent years for its potential as next-generation non-volatile memory. In this paper, learning functions such as (short-term plasticity) STP and STP to (long-term plasticity) LTP transition, are achieved using a single device based on an Al (top electrode)/Y2O3/n-Si structure grown by dual ion beam sputtering (DIBS) technique..
To start with, a 80 nm thick Y2O3 (yttria) layer (switching layer) is deposited over bottom electrode n-Si, at a substrate temperature of 300 °C, with DIBS background pressure of 1 × 10^-8 mBar and Ar:O₂ (2:3) (flow rate in sccm), respectively. Next, circular Al top electrode (TE) of 100 nm is deposited on the surface of switching layer. The current-voltage (I-V) characteristics of the device are measured using Keithley 2612A sourcemeter and Everbeing probe-station. n-Si(BE)/ Y₂O₃ interface has been observed by cross-sectional high-resolution transmission electron microscopy (HR-TEM) using HR-TEM: JEOL JEM-2010.To study the current–voltage (I-V) characteristics consecutive triangular voltage waveforms with peak voltages of -5 to 5 V is applied to the device.Here it is important to note that our amorphous yttria-based device shows a pinched hysteresis loop (a fingerprint of the memristive system and Resistive Switching (RS). Contrary to highly ordered and crystalline structures, amorphous films do not have long-range order and contain defects which may be a more favorable condition for RS. Temporal characteristics of learning behavior (LB) in our device is understood through the following procedure. current–voltage (I-V) characteristics shows that the synaptic weight of the device progressively increases when the device is stimulated with 40 consecutive voltage pulses. It is interesting to observe that when the voltage pulses are removed, synaptic weight decays at very fast rate at the initial stage and slows down and stabilizes at a value of ~28% of its initial value after ~20 s. Such a variation trend is similar to the “forgetting (or retention) curve” of human memory. When the re-stimulation process is initiated from the previous state, it takes less number (15) of pulses, to recover to ~100% of the memory of device. It bears a striking resemblance to the learning behavior of biological systems, which would allow re-learning of the elapsed information to be at a much faster rate. We study the STP-to-LTP transition by applying different numbers of input excitation pulses to the device. We have demonstrated decay time for curves increases if the number of input excitation pulses increases from 40 (t ~ 7.38 s) to 120 (t ~ 37.78 s). The decay time increases from few seconds to tens of seconds as the number of input pulses increase, which indicates a decreasing forgetting rate.
In conclusion, We believe our work will be very useful for further research in understanding mechanisms in transmitting information between the electrical synapse in an artificial neural network.

Emulation of brain-like signal processing is the foundation for development of efficient learning circuitry, but few devices offer the dynamic range of tunable conductance necessary for mimicking synapses. A hybrid semiconductor which couples electronic and ionic conduction would enable low-power dynamic tuning of metastable resistance states and unlock novel device architectures. Here, we utilize hybrid organic-inorganic perovskite semiconductors to emulate intracellular ion/neurotransmitter flux dynamics responsible for temporal plasticity in chemical synapses. Our devices display a multimodal response with synaptic signatures observable in multiple outputs. This facilitates pattern recognition and image de-noising using a two-layer neural network with energy consumption four orders of magnitude lower than digital signal-processors. Injection barrier in the films define the degree of synaptic plasticity, more notable for larger organic cations.

The realization of healthcare products that can be wearable, attachable, and implantable to human body based on electronic and optical devices has become a paradigm of the new era. Furthermore, the real-time utilization and therapy through biometric information will create great worth by combining with information technology and enrich our daily life in the near future. However, conventional healthcare electronics are commonly fabricated by using inorganic materials, so that it is not easily applicable to flexible electronics because of many problems such as mechanical instability, fragility, and high fabrication costs. In this regard, studies related to flexible electronics using organic materials such as small molecules, polymers, and metal-doped composites have been steadily emerging. However, conventional polymeric organic materials involve complex chemical synthesis and processing, which consume a lot of energy and produce toxic byproducts that cause environmental pollution. For these reasons, biologically reliable biomaterials are attracting a lot of attention due to many advantages such as high flexibility, simple processability, non-toxic characteristic, and biodegradability.
In this study, we successfully fabricated resistive random access memory (RRAM) which is key component of data storage system, and analyzed nonvolatile switching behavior by utilizing a novel natural biomaterial, glucose, as a switching layer of RRAM without additional purification or extraction. We could confirm that solution processed glucose-based RRAM devices showed reasonable switching performance with ~10³ memory window, 100 cycles endurance, and 10⁴ seconds data retention. In addition, we were able to evaluate switching characteristics of the RRAM devices fabricated on PI film as being attached to a transparent vial to realize flexible and wearable applications. Finally, we demonstrated the possibility for the completely transient memory devices that are consisted of cross-bar array structure on glass and rice paper substrate with Mg electrode. The goal of this research is to provide the paths and insights for the production of biologically benign and environmentally friendly storage applications, as well as biocompatible electronic systems to fulfil the future demand.

Memristor as proposed in 1971 as fourth fundamental element besides capacitor (C) inductor (L) and resistor (R) has gained status as a physical reality after HP scientists demonstrated a TiO₂ based memristor in 2008. Two-terminal Resistive random access memories (RRAM) devices categorized as "memristors" find application as non-volatile memory applications.Emulation of biological synapses with memristors for neuromorphic applications is of primary interest in this work. To fabricate our device, first of all, a 60 nm ZnO thin film is deposited over Al/SiO₂/Si substrate at a substrate temperature of 100 °C, with DIBS background pressure of 1 × 10^-8 mBar and Ar:O₂ (2:3) (flow rate in sccm), respectively. Further, metal masking is used to make circular Al electrodes of 500 µm to be deposited on the surface of ZnO thin film. To verify the memristive properties of device, I-V characteristics of device is evaluated by sweeping a DC voltage in sequence of 0-(+7 V)- 0-(-7 V)-0 in steps of 0.5 V at a particular ramp rate (Voltage/second) for a compliance current of 1 mA. Further, increasing the ramp rate, we observe pinched hysteresis with zero–crossing said to be the fingerprints of memristor. Later, pulsed bias mode is employed for short-term plasticity (STP), long-term potentiation (LTP), spike-timing-dependent-potentiation (STDP), and conductance modulation measurements with a current compliance of 1.0mA. When a continuous sweep positive pulse voltage from 0 to 7 V is applied to the device, the conductivity decreases continuously with several easily recognized states and when a continuous sweep negative pulse voltage from 0 to −7 V is applied to the device, the conductivity increase continuously. The relative change of the synaptic weights (ΔW) defined as (I₂– I₁)/I₁ (I₁ and I₂ are currents at different voltage levels) demonstrates STDP learning rule in memristive device in terms of the relative change of the memristor synaptic weight (ΔW) with the relative spike timing (Δt). In order to observe the transformation from STP to LTP during a fixed time interval of 78 s, the memory retention curves of memristive device by using different pulse numbers (N= 10, 20, 40, 80, 120) are observed. The results show that the synaptic weights begin to deteriorate after the applied pulse is removed. As we start, the decay rate as calculated is relatively faster, corresponding to the relaxation of the STP. Later, the decay rate slows down, with the relaxation of the LTP process. Similar transition rule is observed with the memory forgetting curve of the human brain. Our memristor system is observed to be functioning as the synaptic device, because of its nonlinear transfer characteristics similar to the neural synapse, the smallest unit of learning and memory of the human brain. We hope that our work would be very crucial in realization of similar memristive devices with neuromorphic properties to be emulated as biological synapse.