Issue

Vol. 17 (2025)

RGB Color-Discriminable Photonic Synapse for Neuromorphic Vision System

https://doi.org/10.1007/s40820-024-01579-y
Bum Ho Jeong
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea
Jaewon Lee
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea
Miju Ku
Department of Mechanical Engineering, Hanyang University, Seoul 04763, Korea
Jongmin Lee
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea
Dohyung Kim
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea
Seokhyun Ham
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea
Kyu‑Tae Lee
Department of Physics, Inha University, Incheon 22212, Korea
Young‑Beom Kim
Department of Mechanical Engineering, Hanyang University, Seoul 04763, Korea
Hui Joon Park
Department of Organic and Nano Engineering & Human‑Tech Convergence Program, Hanyang University, Seoul 04763, Korea

Corresponding Author: Hui Joon Park

huijoon@hanyang.ac.kr

Nano-Micro Letters, Vol. 17 (2025), Article Number: 78

Abstract

To emulate the functionality of the human retina and achieve a neuromorphic visual system, the development of a photonic synapse capable of multispectral color discrimination is of paramount importance. However, attaining robust color discrimination across a wide intensity range, even irrespective of medium limitations in the channel layer, poses a significant challenge. Here, we propose an approach that can bestow the color-discriminating synaptic functionality upon a three-terminal transistor flash memory even with enhanced discriminating capabilities. By incorporating the strong induced dipole moment effect at the excitation, modulated by the wavelength of the incident light, into the floating gate, we achieve outstanding RGB color-discriminating synaptic functionality within a remarkable intensity range spanning from 0.05 to 40 mW cm−2. This approach is not restricted to a specific medium in the channel layer, thereby enhancing its applicability. The effectiveness of this color-discriminating synaptic functionality is demonstrated through visual pre-processing of a photonic synapse array, involving the differentiation of RGB channels and the enhancement of image contrast with noise reduction. Consequently, a convolutional neural network can achieve an impressive inference accuracy of over 94% for Canadian-Institute-For-Advanced-Research-10 colorful image recognition task after the pre-processing. Our proposed approach offers a promising solution for achieving robust and versatile RGB color discrimination in photonic synapses, enabling significant advancements in artificial visual systems.


Highlights:
1 Photonic synapse capable of multispectral color discrimination is demonstrated.
2 Strong excited-state dipoles enable remarkable discrimination intensity (0.05–40 mW cm-2).
3 This approach is not restricted to a specific medium in the channel layer, and convolutional neural network with synapses array achieves over 94% inference accuracy for Canadian-Institute-For-Advanced-Research-10 images.

Keywords

Organic field-effect transistor Photonic synapse Excited-state dipole moment RGB color discrimination Neuromorphic visual system
Jeong, Bum Ho, Jaewon Lee, Miju Ku, Jongmin Lee, Dohyung Kim, Seokhyun Ham, Kyu‑Tae Lee, Young‑Beom Kim, and Hui Joon Park. 2024. “RGB Color-Discriminable Photonic Synapse for Neuromorphic Vision System”. Nano-Micro Letters 17 (November):78. https://doi.org/10.1007/s40820-024-01579-y.
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. I.P. Howard, Perceiving in Depth, Vol. 1. (Oxford Univ. Press, 2012), Ch.5
  2. L. Gu, S. Poddar, Y. Lin, Z. Long, D. Zhang et al., A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature 581, 278–282 (2020). https://doi.org/10.1038/s41586-020-2285-x
  3. K.C. Eldred, S.E. Hadyniak, K.A. Hussey, B. Brenerman, P.W. Zhang et al., Thyroid hormone signaling specifies cone subtypes in human retinal organoids. Science 362, eaau6348 (2018). https://doi.org/10.1126/science.aau6348
  4. G. Hong, T.M. Fu, M. Qiao, R.D. Viveros, X. Yang et al., A method for single-neuron chronic recording from the retina in awake mice. Science 360, 1447–1451 (2018). https://doi.org/10.1126/science.aas9160
  5. M.D. Fairchild, Color appearance models. (John Wiley & Sons, Ltd, Chicheester, UK, 2013).
  6. T. Gollisch, M. Meister, Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65, 150–164 (2010). https://doi.org/10.1016/j.neuron.2009.12.009
  7. H. Wang, Q. Zhao, Z. Ni, Q. Li, H. Liu et al., A ferroelectric/electrochemical modulated organic synapse for ultraflexible, artificial visual-perception system. Adv. Mater. 30, e1803961 (2018). https://doi.org/10.1002/adma.201803961
  8. S. Chen, Z. Lou, D. Chen, G. Shen, An artificial flexible visual memory system based on an UV-motivated memristor. Adv. Mater. 30, 1705400 (2018). https://doi.org/10.1002/adma.201705400
  9. C. Jo, J. Kim, J.Y. Kwak, S.M. Kwon, J.B. Park et al., Retina-inspired color-cognitive learning via chromatically controllable mixed quantum dot synaptic transistor arrays. Adv. Mater. 34, e2108979 (2022). https://doi.org/10.1002/adma.202108979
  10. J. Xing, C. Zhao, Y. Zou, W. Kong, Z. Yu et al., Modulating the optical and electrical properties of MAPbBr3 single crystals via voltage regulation engineering and application in memristors. Light Sci. Appl. 9, 111 (2020). https://doi.org/10.1038/s41377-020-00349-w
  11. Y.-X. Hou, Y. Li, Z.-C. Zhang, J.-Q. Li, D.-H. Qi et al., Large-scale and flexible optical synapses for neuromorphic computing and integrated visible information sensing memory processing. ACS Nano 15, 1497–1508 (2021). https://doi.org/10.1021/acsnano.0c08921
  12. S. Seo, J.J. Lee, R.G. Lee, T.H. Kim, S. Park et al., An optogenetics-inspired flexible van der waals optoelectronic synapse and its application to a convolutional neural network. Adv. Mater. 33, e2102980 (2021). https://doi.org/10.1002/adma.202102980
  13. F. Ma, Y. Zhu, Z. Xu, Y. Liu, X. Zheng et al., Optoelectronic perovskite synapses for neuromorphic computing. Adv. Funct. Mater. 30, 1908901 (2020). https://doi.org/10.1002/adfm.201908901
  14. Z. Lv, M. Chen, F. Qian, V. Roy, W. Ye et al., Mimicking neuroplasticity in a hybrid biopolymer transistor by dual modes modulation. Adv. Funct. Mater. 29, 1902374 (2019). https://doi.org/10.1002/adfm.201902374
  15. G. Moon, S.Y. Min, C. Han, S.H. Lee, H. Ahn et al., Atomically thin synapse networks on van der waals photo-memtransistors. Adv. Mater. 35, e2203481 (2023). https://doi.org/10.1002/adma.202203481
  16. F. Zhou, Z. Zhou, J. Chen, T.H. Choy, J. Wang et al., Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019). https://doi.org/10.1038/s41565-019-0501-3
  17. L. Zhu, S. Li, K. Shu, S. Ke, X. Wan et al., Visible-light responsive CdS-QDs modified InGaZnO synapse for biologically plausible color-to-gray conversion. Appl. Phys. Lett. 125, 033502 (2024). https://doi.org/10.1063/5.0219210
  18. X. Wang, C. Chen, L. Zhu, K. Shi, B. Peng et al., Vertically integrated spiking cone photoreceptor arrays for color perception. Nat. Commun. 14, 3444 (2023). https://doi.org/10.1038/s41467-023-39143-8
  19. J. Zhang, P. Guo, Z. Guo, L. Li, T. Sun et al., Retina-inspired artificial synapses with ultraviolet to near-infrared broadband responses for energy-efficient neuromorphic visual systems. Adv. Funct. Mater. 33, 2370197 (2023). https://doi.org/10.1002/adfm.202370197
  20. Y. Wang, Z. Lv, J. Chen, Z. Wang, Y. Zhou et al., Photonic synapses based on inorganic perovskite quantum dots for neuromorphic computing. Adv. Mater. 30, 1802883 (2018). https://doi.org/10.1002/adma.201802883
  21. R. Wang, P. Chen, D. Hao, J. Zhang, Q. Shi et al., Artificial synapses based on lead-free perovskite floating-gate organic field-effect transistors for supervised and unsupervised learning. ACS Appl. Mater. Interfaces 13, 43144–43154 (2021). https://doi.org/10.1021/acsami.1c08424
  22. K. Lee, H. Han, Y. Kim, J. Park, S. Jang et al., Retina-inspired structurally tunable synaptic perovskite nanocones. Adv. Funct. Mater. 31, 2170382 (2021). https://doi.org/10.1002/adfm.202170382
  23. T. Chen, X. Wang, D. Hao, S. Dai, Q. Ou et al., Photonic synapses with ultra-low energy consumption based on vertical organic field-effect transistors. Adv. Optical Mater. 9, 2002030 (2021). https://doi.org/10.1002/adom.202002030
  24. X. Hou, C. Liu, Y. Ding, L. Liu, S. Wang et al., A logic-memory transistor with the integration of visible information sensing-memory-processing. Adv. Sci. 7, 2002072 (2020). https://doi.org/10.1002/advs.202002072
  25. P.-Y. Chen, X. Peng, S. Yu, NeuroSim: a circuit-level macro model for benchmarking neuro-inspired architectures in online learning. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 37, 3067–3080 (2018). https://doi.org/10.1109/TCAD.2018.2789723
  26. A. Tsirigotis, G. Sarantoglou, S. Deligiannidis, E. Sanchez, A. Gutierrez et al. Photonic neuromorphic accelerator for convolutional neural networks based on an integrated reconfigurable mesh. arXiv 2405, 06434 (2024). https://doi.org/10.48550/arXiv.2405.06434
  27. Y. Yu, Q. Ma, H. Ling, W. Li, R. Ju et al., Small-molecule-based organic field-effect transistor for nonvolatile memory and artificial synapse. Adv. Funct. Mater. 29, 1904602 (2019). https://doi.org/10.1002/adfm.201904602
  28. J.-H. Lee, S. Kim, H. Kim, J. Lee, Solvent-dependent performance of solution-processed small-molecule organic field-effect transistors. Org. Electron. 52, 184–189 (2018). https://doi.org/10.1016/j.orgel.2017.10.026
  29. M. Małachowski, J. Żmija, Organic field-effect transistors. Opto-Electron. Rev. 18, 121–136 (2010). https://doi.org/10.2478/s11772-010-0008-9
  30. T. Li, J.W. Balk, P.P. Ruden, I.H. Campbell, D.L. Smith, Channel formation in organic field-effect transistors. J. Appl. Phys. 91, 4312–4318 (2002). https://doi.org/10.1063/1.1453509
  31. Y.-C. Lin, W.-C. Yang, Y.-C. Chiang, W.-C. Chen, Recent advances in organic phototransistors: nonvolatile memory, artificial synapses, and photodetectors. Small Sci. 2, 2100109 (2022). https://doi.org/10.1002/smsc.202100109
  32. S.A. Ok, B. Jo, S. Somasundaram, H.J. Woo, D.W. Lee et al., Management of transition dipoles in organic hole-transporting materials under solar irradiation for perovskite solar cells. Nat. Commun. 9, 4537 (2018). https://doi.org/10.1038/s41467-018-06998-1
  33. B. Jo, H. Park, E. Kamaraj, S. Lee, B. Jung et al., Synergistic effect of excited state property and aggregation characteristic of organic semiconductor on efficient hole-transportation in perovskite device. Adv. Funct. Mater. 31, 2007180 (2021). https://doi.org/10.1002/adfm.202007180
  34. J. Lee, B.H. Jeong, E. Kamaraj, D. Kim, H. Kim et al., Light-enhanced molecular polarity enabling multispectral color-cognitive memristor for neuromorphic visual system. Nat. Commun. 14, 5775 (2023). https://doi.org/10.1038/s41467-023-41419-y
  35. J.R. Lakowicz, Principles of Fluorescence Spectroscopy, 3rd edn. (Springer, MA, 2006)
  36. H. Imahori, K. Hosomizu, Y. Mori, T. Sato, T.K. Ahn et al., Substituent effects of porphyrin monolayers on the structure and photoelectrochemical properties of self-assembled monolayers of porphyrin on Indium−Tin oxide electrode. J. Phys. Chem. B 108, 5018–5025 (2004). https://doi.org/10.1021/jp037625e
  37. G. Ulrich, F. Nastasi, P. Retailleau, F. Puntoriero, R. Ziessel et al., Luminescent excited-state intramolecular proton-transfer (ESIPT) dyes based on 4-alkyne-functionalized[2, 2’-bipyridine]-3, 3’-diol dyes. Chemistry 14, 4381–4392 (2008). https://doi.org/10.1002/chem.200701803
  38. S. Lochbrunner, K. Stock, E. Riedle, Direct observation of the nuclear motion during ultrafast intramolecular proton transfer. J. Mol. Struct. 700, 13–18 (2004). https://doi.org/10.1016/j.molstruc.2004.01.038
  39. Y. Zhang, H. Yang, H. Ma, G. Bian, Q. Zang et al., Excitation wavelength dependent fluorescence of an ESIPT triazole derivative for amine sensing and anti-counterfeiting applications. Angew. Chem. Int. Ed. 58, 8773–8778 (2019). https://doi.org/10.1002/anie.201902890
  40. B. Li, Y. Liu, C. Wan, Z. Liu, M. Wang et al., Mediating short-term plasticity in an artificial memristive synapse by the orientation of silica mesopores. Adv. Mater. 30, e1706395 (2018). https://doi.org/10.1002/adma.201706395
  41. W. Xu, H. Cho, Y.-H. Kim, Y.-T. Kim, C. Wolf et al., Organometal halide perovskite artificial synapses. Adv. Mater. 28, 5916–5922 (2016). https://doi.org/10.1002/adma.201506363
  42. G.Q. Bi, M.M. Poo, Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472 (1998). https://doi.org/10.1523/JNEUROSCI.18-24-10464.1998
  43. M.P. Nusbaum, D.M. Blitz, A.M. Swensen, D. Wood, E. Marder, The roles of co-transmission in neural network modulation. Trends Neurosci. 24, 146–154 (2001). https://doi.org/10.1016/s0166-2236(00)01723-9
  44. L.Q. Zhu, C.J. Wan, L.Q. Guo, Y. Shi, Q. Wan, Artificial synapse network on inorganic proton conductor for neuromorphic systems. Nat. Commun. 5, 3158 (2014). https://doi.org/10.1038/ncomms4158
  45. R.A. Bear, D.A. Rintoul, B.A. Snyder, M. Smith-Caldas, C.D. Herren et al., Principles of Biology (New Prairie Press, Kansas State University Libraries, KS, USA, 2016)
  46. D. Kim, H. Bang, H.W. Baac, J. Lee, P.L. Truong et al., Room-temperature-processable highly reliable resistive switching memory with reconfigurability for neuromorphic computing and ultrasonic tissue classification. Adv. Funct. Mater. 33, 2213064 (2023). https://doi.org/10.1002/adfm.202213064
  47. R.A. John, N. Yantara, Y.F. Ng, G. Narasimman, E. Mosconi et al., Ionotronic halide perovskite drift-diffusive synapses for low-power neuromorphic computation. Adv. Mater. 30, e1805454 (2018). https://doi.org/10.1002/adma.201805454
  48. T. Ahmed, S. Kuriakose, E.L.H. Mayes, R. Ramanathan, V. Bansal et al., Optically stimulated artificial synapse based on layered black phosphorus. Small 15, e1900966 (2019). https://doi.org/10.1002/smll.201900966
  49. B. Yan, M. Vakulenko, S.-H. Min, W.W. Hauswirth, S. Nirenberg, Maintaining ocular safety with light exposure, focusing on devices for optogenetic stimulation. Vision. Res. 121, 57–71 (2016). https://doi.org/10.1016/j.visres.2016.01.006
  50. P. Soderberg, R. Matthes, M. Feychting, A. Ahlbom, E. Breitbart et al., Guidelines on limits of exposure to laser radiation of wavelengths between 180 nm and 1,000 µm. Health Phys. 105, 74–96 (2013). https://doi.org/10.1097/HP.0b013e3182983fd4
  51. D. Holmes, Reconstructing the retina. Nature 561, S2–S3 (2018). https://doi.org/10.1038/d41586-018-06111-y
  52. X. Peng, S. Huang, H. Jiang, A. Lu, S. Yu, DNN NeuroSim V2.0: an end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 40, 2306–2319 (2021). https://doi.org/10.1109/TCAD.2020.3043731
  53. S. Albawi, T.A. Mohammed, S. Al-Zawi, IEEE. In: 2017 International Conference on Engineering and Technology (ICET), Understanding of a Convolutional Neural Network, Antalya Turkey, (2017)
  54. F. Alibart, E. Zamanidoost, D.B. Strukov, Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013). https://doi.org/10.1038/ncomms3072
  55. X. Zhuge, J. Wang, F. Zhuge, Photonic synapses for ultrahigh-speed neuromorphic computing. Phys. Status Solidi RRL 13, 1900082 (2019). https://doi.org/10.1002/pssr.201900082
  56. C. Qian, S. Oh, Y. Choi, J.-H. Kim, J. Sun et al., Solar-stimulated optoelectronic synapse based on organic heterojunction with linearly potentiated synaptic weight for neuromorphic computing. Nano Energy 66, 104095 (2019). https://doi.org/10.1016/j.nanoen.2019.104095
Read More
Author biographies are not available.
Download this PDF file
PDF SUPPLEMENTARY FILE
Statistic
Read Counter : 1530 Download : 1145