Artificial stimulus-response system capable of conscious response

INTRODUCTION

Humans can recognize various stimuli and have a sophisticated reaction system to respond to them. A design of an artificial stimulus-response system and a device for using it have recently received research attention because such a system can contribute to improving the lives of humans with impaired nervous systems (18). Developments have been made to mimic the human stimulus-response mechanism and to apply them to living organisms (915). Recently, Lee group (9) invented an artificial afferent nervous system comprising pressure sensors, an organic ring oscillator, and a synaptic transistor. They demonstrated that the artificial nerve could be made compatible with living beings by being connected to the nerve of a cockroach and subsequent actuation of a leg muscle by stimulation. For artificial nervous systems mimicking the biological nervous system to be applicable to humans, they must have the characteristics of human responses (1622). Human responses to external stimuli are classified into unconscious and conscious responses depending on, respectively, the absence or presence of awareness and ability to control the response. Unconscious responses such as knee reflexes or Pavlov’s dog experiment do not involve a conscious decision (23). In contrast, conscious responses are those that require a learning process by the cerebral cortex, such as an athlete’s starting reaction or catching a flying ball. This type of response can be controlled by learning from repeated stimuli and responses, and this learning modulates the synaptic connections and thereby optimizes the response (1822, 24). In this context, many researchers have dedicated efforts to the realization of artificial synapses as electronic devices (2532). Recently, advanced artificial neural systems combined with stimulus sensors and actuators have been reported (13, 914, 3335). For example, Lee group (11) successfully realized light-interactive actuation using an optoelectronic sensorimotor synapse and a neuromuscular system. However, these systems performed signal transmission based on unconscious response, but they did not completely mimic the biological stimulus-response system with conscious response. Simulation of the human stimulus-response system requires integration of neurons capable of conscious responses, stimulus detection, actuation, and control of synaptic connection strengthened by a learning process.

In this study, we developed an artificial stimulus-response system capable of conscious response. The developed system consists of a quantum dot (QD) photodiode, a retentive electric double layer (EDL) transistor, complementary metal-oxide semiconductor (CMOS)–based artificial neuron (AN) circuits, and a robot hand, which correspond to the visual receptor, synapse, neuron, and muscle, respectively. A vertical graphene/indium gallium zinc oxide (IGZO)/indium tin oxide (ITO) heterostructure was used as the base layer for both the artificial visual receptor (AVR) and the artificial synapse (AS). The photoresponse behavior of the AVR was induced via the photogating effect of the InP QD layer deposited onto graphene. The synaptic function of the system was successfully implemented by the modulation and retention of the graphene work function through control of ion movement in the retentive EDL. On the same subject : Sapphire Substrates Market 2021 Is Booming Across the Globe by Share, Size, Growth, Segments and Forecast to 2027 | Top Players Analysis. Furthermore, signal transmission was achieved using the retentive EDL in the AS and the CMOS neuron circuit. Last, artificial conscious response was generated by the integration of the robot hand with the above-listed components of the artificial nervous system, and the response time of the system was successfully controlled with far less circuit complexity than conventional all-CMOS–based system. In addition, because of the pulse-based operation, the lower energy consumption was achieved compared to all-CMOS–based circuit. On the basis of the findings of the present study, we expect that the realization of an artificial stimulus-response system will open a new chapter in research on organs with artificial intelligence that would be useful for the treatment of patients with neurological disorders.

To see also :
This fabrication capability, the 90-nanometer fully depleted silicon-on-insulator (FDSOI) complementary metal-oxide-semiconductor (CMOS)…

RESULTS

Artificial stimulus-response system

Figure 1A shows a schematic illustration of the biological stimulus-response system, which is composed of a retina, neurons, a synapse, and a muscle. First, the retina detects an external light stimulus and converts it into an electrical signal. Next, the sensory neuron sends an action potential to the synapse, which facilitates or depresses the signal. The firing neuron then analyzes and interprets the signal and delivers the integrated signal to the muscle. Last, the muscle performs an action as a response. This response process is analogous to that in our proposed artificial stimulus-response system, which consists of an AVR, AN circuits, an AS, and a robot hand. In this system, the AVR converts an external light stimulus into an electrical signal. Next, the sensory AN circuit transmits the electrical signal from the AVR to the AS, which performs integration of the sensory input. Then, the firing AN circuit generates appropriate responses to operate the robot hand. Figure 1 (B and C) illustrates the vertically stacked structures of the InP QD–based AVR and retentive EDL–based AS, respectively. The AVR and AS both have the graphene/IGZO/ITO heterostructure as the base layer, which serves as a platform for signal transmission (fig. S1). Note that the vertical structure provides a higher integration density than conventional lateral devices. The InP QD layer in the AVR is coated onto graphene, and this layer converts a light stimulus into an electrical signal by the photogating effect (36). For fabrication of the AS, poly(vinylidene fluoride-trifluoroethylene) [P(VDF-TrFE)] and an ion gel {a mixture of poly(vinylidene fluoride-co-hexafluoropropylene) [P(VDF-HFP)] and 1-ethyl-3-methylimidazolium bis(trifluoromethylsulfonyl)imide ([EMIM][TFSI]) ionic liquid} are coated to form the retentive EDL, which controls the synaptic connections through a learning process. Synaptic connections in a conscious response can be facilitated using the above described AVR, AS circuit, AN circuits, and robot hand. Figure 1D depicts the learning process, wherein through learning, a human individual is able to successfully catch a ball that starts to fall immediately after provision of light illumination as a stimulus. This successful catching is attributed to enhanced synaptic connections after learning and consequently, a faster response. Figure 1E shows the neural signals before and after the learning process. The decrease in the time interval between the start of illumination and the generation of the pulse train indicates a reduced response time, which is in agreement with the result depicted in Fig. 1D.

Fig. 1 Artificial stimulus-response system.

(A) Schematic illustration of biological stimulus-response system composed of retina, neurons, synapse, and muscle. On the same subject : 2021 Latest Report on Radio Frequency Front-End Module Market Size, Demand Status. (B and C) Schematic device structure of (B) AVR with InP QD layer and (C) AS with retentive EDL. (D) Schematic illustration of a human individual catching a ball successfully through learning. (E) Neural signals before and after learning process.

Electrical properties of AS with retentive EDL

The retentive EDL–based AS was inspired by the biological synapse in the stimulus-response system (Fig. 2A). The biological synapse transfers an action potential from the presynaptic terminal to the postsynaptic terminal. Synaptic connections in conscious response can be modulated by the presynaptic spike input (conscious decision), which causes a change in the postsynaptic signal (20). The postsynaptic signals show different trends depending on the degree to which synaptic connections are maintained. Short-term plasticity (STP) is achieved through either a temporary increase or a temporary decrease in the synaptic connections by the application of a single spike or infrequent spikes. In contrast, application of repeated presynaptic spikes causes a long-term change in the synaptic connections. A long-lasting increase in synaptic connections is referred to as long-term potentiation (LTP), and a long-lasting decrease is referred to as long-term depression (LTD).

Fig. 2 Electrical properties of AS with retentive EDL.

(A) Schematic of biological synapse in stimulus-response system. This may interest you : IEEE MTT-S International Microwave Symposium (IMS2020) Kicks Off Next Week. (B) Cross-sectional schematic of AS based on vertically stacked graphene/IGZO/ITO heterostructure and retentive EDL. (C) Schematic of Al/retentive EDL/graphene heterostructure, describing the working mechanism of AS. (D) Nyquist plot of retentive EDL at various voltages. (E) Schematic of Kelvin probe force microscopy (KPFM) measurement setup. (F) KPFM images and histograms of Wgr at various voltages. (G) Retention properties of Wgr as determined by KPFM at various voltages. (H) Comparison of postsynaptic current (PSC) behaviors with two different gate dielectric layers (ion gel only and retentive EDL). (I) PSC triggered by two Vpre pulses applied with time interval of Δt. Ipost1 and Ipost2 are the increased PSC value at 2 s after application of the first pulse and the second (consecutive) pulse, respectively. (J) Plot of Ipost2/Ipost1 as a function of Δt.

Figure 2B shows a cross-sectional schematic of the vertically stacked AS with the graphene/IGZO/ITO heterostructure. First, the ITO drain electrode was sputtered onto a glass substrate, which was followed by the deposition of the IGZO layer. Then, monolayer graphene (fig. S2), which served as a postsynaptic terminal, was transferred onto the IGZO layer; this resulted in the formation of a Schottky barrier (SB) at the IGZO-graphene interface. Next, an ion gel/P(VDF-TrFE) gate dielectric layer (i.e., the retentive EDL) was spin-coated onto graphene. In the retentive EDL, the P(VDF-TrFE) layer served as an ion transport–mitigating layer. Last, an Al presynaptic terminal was deposited onto the retentive EDL (see the transfer and output characteristics of the resulting artificial synaptic device in fig. S3). Briefly, presynaptic voltage (Vpre) was applied at the Al presynaptic terminal, and postsynaptic current (PSC) (Ipost) was transmitted to the graphene postsynaptic terminal after signal processing by the retentive EDL. Here, Vpre and Ipost correspond to presynaptic spike and postsynaptic signal in Fig. 2A. The detailed working mechanism of the AS is depicted in Fig. 2C. At low voltages (Vpre < 3 V), the transport of [EMIM]+ and [TFSI] was mostly limited to within the P(VDF-HFP) matrix, and their transport through the adjacent P(VDF-TrFE) layer (i.e., the ion transport–mitigating layer) was restricted. In contrast, at high voltages (Vpre ≥ 3 V), the electric field was high enough for [EMIM]+ and [TFSI] to readily penetrate into the P(VDF-TrFE) layer. Because the as-penetrated ions could be released only from the P(VDF-TrFE) layer over time, the formation/deformation behavior of the EDL was retentive. To verify this operation mechanism, we used a testbed device comprising the ion gel and ion transport–mitigating layers sandwiched between Al and ITO electrodes (fig. S4). Electrochemical impedance spectroscopy (EIS) analysis was performed on the testbed device, as shown in Fig. 2D. It should be noted that the ITO electrode was selected, which is vulnerable to faradic reactions under contact with ionic liquids. The Nyquist plot revealed distinct behaviors at different operation voltages. At low voltages below 3 V, a tilted straight line was observed in the plot, indicating that the testbed device behaved as a capacitor with some degree of nonideality. Because the transport of [EMIM]+ and [TFSI] was considered to occur within the P(VDF-HFP) matrix only and not through the adjacent P(VDF-TrFE) layer at these voltages, the retentive EDL could be simply considered as two capacitors connected in series: an EDL capacitor (the ion gel layer) and a dielectric capacitor (the ion-free ion transport–mitigating layer). At voltages above 3 V, a semicircle was observed in the Nyquist plot, indicating that the testbed device could be considered to be equivalent to a capacitor and a resistor connected in parallel. The appearance of the resistor component (i.e., the charge transfer resistance) under these voltage conditions is attributed to the ions that penetrated into the P(VDF-TrFE) layer, which ended up making direct contact with the ITO electrode. As mentioned above, ITO is vulnerable to electrochemical degradation when in contact with [EMIM][TFSI]. As the voltage increased, more ions could penetrate into the P(VDF-TrFE) layer, which led to a smaller charge transfer resistance with a smaller semicircle in the Nyquist plot.

These ion transport properties in the retentive EDL influence the work function of graphene (Wgr). To visualize the modulation and retention of Wgr by ion movement in the retentive EDL, we performed Kelvin probe force microscopy (KPFM) measurements, as shown in Fig. 2E. The contact potential difference (VCPD) between graphene and the KPFM tip was measured at various voltages (0 to 5 V in 1-V steps) applied to the ITO electrode. Wgr was calculated as Wgr = WtipeVCPD, where Wtip is the work function of the tip (4.2 eV). Figure 2F shows the KPFM images and corresponding Wgr values at various voltages. The graphene work function on the P(VDF-TrFE) layer was estimated to be 4.58 eV at Vpre = 0 V, and it was found to be lower at a higher Vpre (4.36 eV at Vpre = 5 V). The electric field generated by ion movement in the retentive EDL influenced the charge density of graphene. As a result, the variation in charge density caused a notable change in the work function and modulation of the SB height between graphene and the IGZO layer. (fig. S5). The ion movement in the ion transport–mitigating layer also affected the retention property of Wgr. To confirm this, Wgr was monitored as a function of time after application of 3-, 4-, and 5-V voltage pulses at the ITO electrode. Figure 2G shows the changes in Wgr with time (also see fig. S6), which reveals that Wgr decreased gradually with time. The time constant (τ) of the decay curves increased with an increase in the applied Vpre (17.9, 20.1, and 26.1 s at 3, 4, and 5 V, respectively). A larger τ implies that the ion transport–mitigating layer maintained the decrease in the work function for a long time and thereby enhanced the synaptic connections for prolonged periods of time. It should be noted that the work function did not return to its initial value (4.58 eV) even after 60 s.

On the basis of the above-described synapse facilitation mechanism of the retentive EDL, the LTP behavior of the PSC (denoted by Ipost) was examined as a function of the electrical pulse input (3, 4, and 5 V under λpre = 25 ms) (Fig. 2H). When the Vpre pulse was applied to the AS, Ipost increased abruptly and then decayed, which is analogous to STP. After the decay of the Ipost, LTP characteristics showed retention of Ipost. This behavior became prominent at higher Vpre. A higher Ipost was measured with at higher Vpre because of the enhanced synaptic connections (fig. S7). For comparison, the LTP behavior of a device with only the ion gel, i.e., without the P(VDF-TrFE) layer, was also analyzed to confirm the role of this layer as the ion transport–mitigating layer. When Vpre of 3 V was applied to this device, a rapid decrease occurred in Ipost, which resulted in negligible LTP characteristics. Therefore, it is obvious that the ion transport–mitigating layer plays a key role in the generation of LTP characteristics. The LTP characteristics could also be controlled by tuning of the pulse width (λpre) of presynaptic spikes (fig. S8). To achieve depression of Ipost, we applied negative Vpre (−3, −4, and −5 V), which induced STD and LTD behaviors as shown in fig. S9. The effect of the ion transport–mitigating layer could also be observed from the enhanced retention of Ipost under the application of consecutive pulses. Figure 2I and fig. S10 show the application of paired pulses within a short time interval (Δt) and the subsequent increase in Ipost. The increased Ipost value at 2 s after application of the first pulse and the second (consecutive) pulse is denoted as Ipost1 and Ipost2, respectively. Figure 2J shows the ratio of Ipost2 to Ipost1 (Ipost2/Ipost1) as a function of Δt, where this ratio is an indicator of enhancement of synaptic connections. Ipost2/Ipost1 was high at small values of Δt because a strengthened synaptic connection by the first pulse was still dominant. At larger values of Δt, Ipost2/Ipost1 decreased but remained at a level above a certain value owing to the weaker strengthening effect of the first pulse. Meanwhile, the ion gel–only device had an Ipost2/Ipost1 value close to one regardless of Δt, which supported our observation that the ion transport–mitigating layer contributed notably to maintaining synaptic connections (fig. S11).

Artificial nervous system with AS circuit and AN circuit

A firing component capable of processing the strengthened signal in the AS is required for complete realization of the signal transmission system. Therefore, an AN circuit capable of analyzing and interpreting this signal was designed. Figure 3A shows the incorporation of the AS and AN circuits into the artificial nervous system. The AS circuit, which is aimed at amplifying a weak AS signal, is composed of the AS and a noninverting amplifier (×103). The AN circuit is composed of a resistor-capacitor (RC) circuit and a noninverting comparator to mimic the leaky integrate-and-fire (LIF) neuron model. The LIF model states that the neuronal action potential (VNO) is generated by leaky accumulation of the synaptic output voltage (VSO). In the AN circuit, leaky accumulation occurs through the RC circuit and leads to the generation of VNO by the comparator. Figure 3B shows the signal transmission process in detail. First, a Vpre pulse train is applied to the AS, which is subsequently converted into Ipost. Next, Ipost flows into the noninverting amplifier to magnify the input signal and is converted into VSO. In the AN circuit, VSO is converted into a neuron input voltage (VNI) through charging and discharging of the RC circuit. It should be noted that VNI shows leaky integration with a peak and trough owing to the STP and LTP behaviors of the AS. Last, VNI is delivered to the comparator to determine whether it exceeds the threshold voltage. When the accumulated VNI is lower than the set voltage (Vset), the comparator outputs −VCC as VNO. Conversely, if VNI is higher than Vset, then the comparator outputs +VCC as VNO. Thus, it is possible to generate a VNO pulse train by setting Vset to an appropriate value, which is referred to as the firing process in the LIF model.

Fig. 3 Artificial nervous system with AS circuit and AN circuit.

(A) Circuit diagram of artificial nervous system incorporated with AS circuit and AN circuit. (B) Signal transmission process in artificial nervous system. (C) LTP/LTD characteristics of AS induced by application of consecutive Vpre pulses. (D) Learning process of AS circuit, which involves application of positive Vpre pulses (i.e., learning periods) alternating with forgetting periods (10 s each). L1, L2, and L3 are defined as the VSO increments from its value in the initial state to those after each forgetting period of 10 s. (E) Iterative learning process of AS circuit. (F) Plots of VNI and peak generation time for different learning processes (i.e., without learning, after first learning process, and after second learning process).

To verify the signal flow described in Fig. 3B, we measured the output signals of the AS and AN circuits. First, 100 potentiation pulses (+4 V) followed by 100 depression pulses (−3 V) were consecutively applied as Vpre at a frequency of 9.6 Hz and a pulse width of 25 ms (Fig. 3C, top). The LTP and LTD properties were observed through the application of the consecutive pulses, as shown in Fig. 3C (bottom) and fig. S12 (stability test). During LTP, application of repeated potentiation pulses to the retentive EDL–based AS strengthened the synaptic connections. In contrast, application of repeated depression pulses led to a decrease in Ipost and its return to the initial state, which is characterized as LTD. Figure 3D shows the process of learning in the AS by the utilization of the LTP characteristics. The top panel shows three Vpre pulse trains for learning, wherein learning periods were alternated with forgetting periods (10 s each). Each Vpre pulse train corresponds to the first 50 potentiation pulses depicted in Fig. 3C. Application of the first pulse train led to an increase in VSO, which resulted in facilitation of synaptic connections. During the forgetting period, VSO decreased gradually with time, which indicates weakened synaptic connections. Here, L is defined as the increase in VSO from its value in the initial state to that after a forgetting time of 10 s, the value of L increased from 4.3 V (after the first learning process) to 5.6 V (after the third learning process), as shown in fig. S13. That is, the increase in VSO was maintained over the long term, despite the forgetting periods in between the learning periods. Furthermore, the unlearning process in the artificial nervous system is demonstrated, as shown in fig. S14. By applying negative Vpre at the end of the second and third learning processes, the L1 value was maintained after the second and third learning processes. Figure 3E depicts an iterative learning process with VNI and VNO over time. VNO was generated when VNI exceeded Vset (7.3 V). Before the learning process, the peak generation time of VNO was 6.3 s. The peak generation time decreased to 1.7 s after the second learning process (Fig. 3F). In this manner, verification of the signal flow in the signal transmission system provided a clear validation of the learning ability of the artificial nervous system.

Artificial stimulus-response system emulating human conscious response

Last, an artificial stimulus-response system was constructed as a proof of concept of its emulation of human conscious response. The stimulus and its response were set as depicted in Fig. 1D: The learning process causes the robot hand to promptly perform grabbing action after recognition of a visual signal. Figure 4A shows a schematic of the constructed system, which includes the AVR, sensory AN circuit, AS circuit, firing AN circuit, and actuator. The stimulus-response process begins with the absorption of visible light and the conversion of this stimulus into an electrical pulse by the AVR. Here, the incident visible light and the AVR are considered to correspond to, respectively, a visual stimulus and sensory receptor in the biological stimulus-response system. Figure 4B depicts the cross-sectional structure of the AVR. Here, the graphene/IGZO/ITO heterostructure functions as an SB diode. Under illumination, the InP QD layer absorbs light (fig. S15) to induce a positive photogating effect on graphene. The photogating effect was visualized through the KPFM measurement as shown in fig. S16. The photo-generated electrons were transferred to graphene while photo-generated holes remained in InP QDs. Because of the charge separation, the work function of the InP QD layer was increased, and the SB at the graphene-IGZO junction was reduced, which eventually led to an increase in the diode current as shown in fig. S17 (band diagram). The diode characteristics under light illumination are shown in Fig. 4C and fig. S18. The light stimulus is converted to Vpre through application of a voltage pulse input (Vpulse) with an amplitude of 2 V (−1 and 1 V) and a frequency of 9.6 Hz to the AVR, which then enables control of the Vpre pulse train. As a result, the photocurrent change (from 8 to 100 μA at 1 V) is converted to a voltage change (from 8 to 100 mV, respectively). A signal exceeding the threshold value (60 mV) leads to fire operation of the sensory AN circuit (fig. S19). Figure 4D shows the dynamics of Vpre generated by the AVR and the sensory AN circuit under dark and illumination conditions. The sensory AN circuit, triggered by a signal exceeding the threshold value, generates Vpre, which is then transmitted to the AS; this consequently results in the LTP characteristic of the AS. That is, the signal is transmitted to the AS only under light illumination, which induces a learning effect by strengthening the synaptic connections. After the signal processing by the AS circuit and firing AN circuit as depicted in Fig. 3, the transmitted signal lastly passes through the transistor and enters the robot hand.

Fig. 4 Artificial stimulus-response system emulating human conscious response.

(A) Circuit diagram of artificial stimulus-response system with AVR, sensory AN circuit, AS circuit, firing AN circuit, and actuator. (B) Cross-sectional schematic of AVR with vertically stacked graphene/IGZO/ITO heterostructure. (C) Current-voltage characteristics of AVR under dark and illumination conditions (50 μW, 520 nm). (D) Dynamic response of Vpre generated by AVR and sensory AN circuit under dark and illumination conditions. (E) Real-time photographic images of robot hand and VNO signals fired from firing AN circuit before and after learning. Photo credit: S. Kim [SKKU Advanced Institute of Nanotechnology (SAINT), Sungkyunkwan University].

Figure 4E shows the signals fired from the firing AN circuit and the consequent activation of the robot hand. The black and red lines indicate VNO before and after learning, respectively. Here, the learning process was repeated three times; each learning process included a forgetting time of 10 s after exposure to the input light stimulus for 10 s. Initially, the artificial stimuli-response system with the robot hand was kept in the dark. After 1.0 s, light stimulus was applied to the robot hand both before and after learning. Before learning, the time required for activation of the robot hand was 2.56 s. However, after learning, the time interval between the application of the external signal input and the activation of the robot hand decreased notably to 0.23 s. This result confirms that our artificial neural system is able to emulate a conscious response and that it markedly improves the response time after learning from repeated light stimuli. Note that the pulse-based operation of our system enabled the lower energy consumption compared to all-CMOS–based circuit (table S1).

Acknowledgments: Funding: This research was supported by Creative Materials Discovery Program through the National Research Foundation (NRF) of Korea (NRF) funded by Ministry of Science and ICT (NRF-2019M3D1A1078299), the Basic Science Program (NRF-2020R1A2C2007819) through the NRF of Korea funded by the Ministry of Science and ICT, and the Korea Medical Device Development Fund grant funded by the Korea government (the Ministry of Science and ICT) (project number: KMDF202012B02-02). Author contributions: J.H.C. initiated the research and designed all the experiments. S.K. and D.G.R. carried out most of the experimental work and data analysis. S.K. conducted the fabrication and analysis of the device. D.G.R. fabricated the AN circuit for the artificial nervous system. Y.Y.C. and Y.C. assisted the data analysis. H.W., J.I.L., M.S.K., and Y.J.S. characterized the retentive EDL. J.P. and S.J. synthesized QDs. S.B.J. contributed to the AVR characterization. All authors discussed the results and contributed to write the paper. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the corresponding authors.

To see also :
Analog Devices (NASDAQ:ADI) will announce its earnings results before the market opens…

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top
Malcare WordPress Security