Issue

Vol. 14 (2022)

Full-Fiber Auxetic-Interlaced Yarn Sensor for Sign-Language Translation Glove Assisted by Artificial Neural Network

https://doi.org/10.1007/s40820-022-00887-5
Ronghui Wu
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST), 50 UNIST‑Gil, Ulsan 44919, Republic of Korea
Sangjin Seo
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST), 50 UNIST‑Gil, Ulsan 44919, Republic of Korea
Liyun Ma
College of Physical Science and Technology, Xiamen University, Xiamen 361005, People’s Republic of China
Juyeol Bae
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST), 50 UNIST‑Gil, Ulsan 44919, Republic of Korea
Taesung Kim
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST), 50 UNIST‑Gil, Ulsan 44919, Republic of Korea

Corresponding Author: Taesung Kim

tskim@unist.ac.kr

Nano-Micro Letters, Vol. 14 (2022), Article Number: 139

Abstract

Yarn sensors have shown promising application prospects in wearable electronics owing to their shape adaptability, good flexibility, and weavability. However, it is still a critical challenge to develop simultaneously structure stable, fast response, body conformal, mechanical robust yarn sensor using full microfibers in an industrial-scalable manner. Herein, a full-fiber auxetic-interlaced yarn sensor (AIYS) with negative Poisson’s ratio is designed and fabricated using a continuous, mass-producible, structure-programmable, and low-cost spinning technology. Based on the unique microfiber interlaced architecture, AIYS simultaneously achieves a Poisson’s ratio of−1.5, a robust mechanical property (0.6 cN/dtex), and a fast train-resistance responsiveness (0.025 s), which enhances conformality with the human body and quickly transduce human joint bending and/or stretching into electrical signals. Moreover, AIYS shows good flexibility, washability, weavability, and high repeatability. Furtherly, with the AIYS array, an ultrafast full-letter sign-language translation glove is developed using artificial neural network. The sign-language translation glove achieves an accuracy of 99.8% for all letters of the English alphabet within a short time of 0.25 s. Furthermore, owing to excellent full letter-recognition ability, real-time translation of daily dialogues and complex sentences is also demonstrated. The smart glove exhibits a remarkable potential in eliminating the communication barriers between signers and non-signers.


Highlights:


1 Full-fiber auxetic-interlaced yarn sensor was fabricated by a continuous and mass-producible computerized wrapping spinning technology.
2 Auxetic-interlaced yarn sensor shows a Poisson’s ratio of − 1.5, a robust mechanical property (0.6 cN/dtex), and a fast train-resistance responsiveness (0.025 s).
3 A novel sign-language translation glove was developed to recognize the full English alphabet and translate the wearer’s sign language to text.

Keywords

Negative Poisson’s ratio yarns Interlaced yarn sensors Smart glove Deep learning Sign-language translation
Wu, Ronghui, Sangjin Seo, Liyun Ma, Juyeol Bae, and Taesung Kim. 2022. “Full-Fiber Auxetic-Interlaced Yarn Sensor for Sign-Language Translation Glove Assisted by Artificial Neural Network”. Nano-Micro Letters 14 (July):139. https://doi.org/10.1007/s40820-022-00887-5.
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. M.A. Ahmed, B.B. Zaidan, A.A. Zaidan, M.M. Salih, M.M.B. Lakulu et al., A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors 18(7), 2208 (2018). https://doi.org/10.3390/s18072208
  2. M.J. Cheok, Z. Omar, M.H. Jaward, A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cyber. 10(1), 131–153 (2019). https://doi.org/10.1007/s13042-017-0705-5
  3. S. Ameen, S. Vadera, A convolutional neural network to classify American sign language fingerspelling from depth and colour images. Expert Syst. 34(3), e12197 (2017). https://doi.org/10.1111/exsy.12197
  4. J. Jang, Y.S. Jun, H. Seo, M. Kim, J.U. Park, Motion detection using tactile sensors based on pressure-sensitive transistor arrays. Sensors 20(13), 3624 (2020). https://doi.org/10.3390/s20133624
  5. R. Ambar, C.K. Fai, M.H.A. Wahab, M.M.A. Jamil, A.A. Ma’radzi, Development of a wearable device for sign language recognition. J. Phys. Conf. Ser. 1019(1), 012017 (2017). https://doi.org/10.1088/1742-6596/1019/1/012017
  6. A. Moin, A. Zhou, A. Rahimi, A. Menon, S. Benatti et al., A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nat. Electron. 4(1), 54–63 (2021). https://doi.org/10.1038/s41928-020-00510-8
  7. J. Wu, L. Sun, R. Jafari, A wearable system for recognizing American sign language in real-time using IMU and surface EMG sensors. IEEE J. Biomed. Health Inform. 20(5), 1281–1290 (2016). https://doi.org/10.1109/Jbhi.2016.2598302
  8. V.E. Kosmidou, L.J. Hadjileontiadis, Sign language recognition using intrinsic-mode sample entropy on sEMG and accelerometer data. IEEE Trans. Biomed. Eng. 56(12), 2879–2890 (2009). https://doi.org/10.1109/Tbme.2009.2013200
  9. B.G. Lee, S.M. Lee, Smart wearable hand device for sign language interpretation system with sensors fusion. IEEE Sens. J. 18(3), 1224–1232 (2018). https://doi.org/10.1109/Jsen.2017.2779466
  10. J.J. Zhao, S. Han, Y. Yang, R.P. Fu, Y. Ming et al., Passive and space-discriminative ionic sensors based on durable nanocomposite electrodes toward sign language recognition. ACS Nano 11(9), 8590–8599 (2017). https://doi.org/10.1021/acsnano.7b02767
  11. M. Wang, Z. Yan, T. Wang, P.Q. Cai, S.Y. Gao et al., Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nat. Electron. 3(9), 563–570 (2020). https://doi.org/10.1038/s41928-020-0422-z
  12. S. He, Research of a sign language translation system based on deep learning. 2019 International conference on artificial intelligence and advanced manufacturing (AIAM), 392–396 (2019). https://doi.org/10.1109/AIAM48774.2019.00083
  13. M. Rivera-Acosta, S. Ortega-Cisneros, J. Rivera, F. Sandoval-Ibarra, American sign language alphabet recognition using a neuromorphic sensor and an artificial neural network. Sensors 17(10), 2176 (2017). https://doi.org/10.3390/s17102176
  14. Z.H. Zhou, K. Chen, X.S. Li, S.L. Zhang, Y.F. Wu et al., Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat. Electron. 3(9), 571–578 (2020). https://doi.org/10.1038/s41928-020-0428-6
  15. Y.Y. Luo, Y.Z. Li, P. Sharma, W. Shou, K. Wu et al., Learning human-environment interactions using conformal tactile textiles. Nat. Electron. 4(3), 193–201 (2021). https://doi.org/10.1038/s41928-021-00558-0
  16. J. Lee, S.J. Ihle, G.S. Pellegrino, H. Kim, J. Yea et al., Stretchable and suturable fibre sensors for wireless monitoring of connective tissue strain. Nat. Electron. 4, 291–301 (2021). https://doi.org/10.1038/s41928-021-00557-1
  17. L.Y. Ma, R.H. Wu, H. Miao, X.W. Fan, L.Q. Kong et al., All-in-one fibrous capacitive humidity sensor for human breath monitoring. Text. Res. J. 91(3–4), 398–405 (2021). https://doi.org/10.1177/0040517520944495
  18. T.J. Mun, S.H. Kim, J.W. Park, J.H. Moon, Y. Jang et al., Wearable energy generating and storing textile based on carbon nanotube yarns. Adv. Funct. Mater. 30(23), 2000411 (2020). https://doi.org/10.1002/adfm.202000411
  19. S. Shen, J. Yi, R.W. Cheng, L.Y. Ma, F.F. Sheng et al., Electromagnetic shielding triboelectric yarns for human-machine interacting. Adv. Electron. Mater. 8(2), 2101130 (2022). https://doi.org/10.1002/aelm.202101130
  20. Y.Y. Zheng, X. Han, J.W. Yang, Y.Y. Jing, X.Y. Chen et al., Durable, stretchable and washable inorganic-based woven thermoelectric textiles for power generation and solid-state cooling. Energy Environ. Sci. (2022). https://doi.org/10.1039/D1EE03633E
  21. L. Ma, R. Wu, S. Liu, A. Patil, H. Gong et al., A machine-fabricated 3D honeycomb-structured flame-retardant triboelectric fabric for fire escape and rescue. Adv. Mater. 32(28), 2003897 (2020). https://doi.org/10.1002/adma.202003897
  22. Z.Y. Chen, R.H. Wu, S.H. Guo, X.Y. Liu, H.B. Fu et al., 3D upper body reconstruction with sparse soft sensors. Soft Robot. 8(2), 226–239 (2021). https://doi.org/10.1089/soro.2019.0187
  23. Z.H. Zhou, S. Padgett, Z.X. Cai, G. Conta, Y.F. Wu et al., Single-layered ultra-soft washable smart textiles for all-around ballistocardiograph, respiration, and posture monitoring during sleep. Biosens. Bioelectron. 155, 112064 (2020). https://doi.org/10.1016/j.bios.2020.112064
  24. H.J. Zhang, W.Q. Han, K. Xu, H.J. Lin, Y.F. Lu et al., Stretchable and ultrasensitive intelligent sensors for wireless human-machine manipulation. Adv. Funct. Mater. 31(15), 2009466 (2021). https://doi.org/10.1002/adfm.202009466
  25. H.J. Zhang, W.Q. Han, K. Xu, Y. Zhang, Y.F. Lu et al., Metallic sandwiched-aerogel hybrids enabling flexible and stretchable intelligent sensor. Nano Lett. 20(5), 3449–3458 (2020). https://doi.org/10.1021/acs.nanolett.0c00372
  26. X.H. Liu, J.L. Miao, Q. Fan, W.X. Zhang, X.W. Zuo et al., Recent progress on smart fiber and textile based wearable strain sensors: materials, fabrications and applications. Adv. Fiber Mater. 4, 361–389 (2022). https://doi.org/10.1007/s42765-021-00126-3
  27. Y. Zheng, H. Liu, X. Chen, Y. Qiu, K.J.O.E. Zhang, Wearable thermoelectric-powered textile-based temperature and pressure dual-mode sensor arrays. Organ. Electron. 106, 106535 (2022). https://doi.org/10.1016/j.orgel.2022.106535
  28. H.D. Liu, H.J. Zhang, W.Q. Han, H.J. Lin, R.Z. Li et al., 3D printed flexible strain sensors: from printing to devices and signals. Adv. Mater. 33(8), 2004782 (2021). https://doi.org/10.1002/adma.202004782
  29. X.Y. Dong, Q. Liu, S. Liu, R.H. Wu, L.Y. Ma, Silk fibroin based conductive film for multifunctional sensing and energy harvesting. Adv. Fiber Mater. (2022). https://doi.org/10.1007/s42765-022-00152-9
  30. D.L. Wen, Y.X. Pang, P. Huang, Y.L. Wang, X.R. Zhang et al., Silk fibroin-based wearable all-fiber multifunctional sensor for smart clothing. Adv. Fiber Mater. (2022). https://doi.org/10.1007/s42765-022-00150-x
  31. R. Wu, L. Ma, A. Patil, Z. Meng, S. Liu et al., Graphene decorated carbonized cellulose fabric for physiological signal monitoring and energy harvesting. J. Mater. Chem. A 8(25), 12665–12673 (2020). https://doi.org/10.1039/d0ta02221g
  32. R. Wu, L. Ma, C. Hou, Z. Meng, W. Guo et al., Silk composite electronic textile sensor for high space precision 2D combo temperature-pressure sensing. Small 15(31), 1901558 (2019). https://doi.org/10.1002/smll.201901558
  33. K.Y. Meng, S.L. Zhao, Y.H. Zhou, Y.F. Wu, S.L. Zhang et al., A wireless textile-based sensor system for self-powered personalized health care. Matter 2(4), 896–907 (2020). https://doi.org/10.1016/j.matt.2019.12.025
  34. N. Nan, J.X. He, X.L. You, X.Q. Sun, Y.M. Zhou et al., A stretchable, highly sensitive, and multimodal mechanical fabric sensor based on electrospun conductive nanofiber yarn for wearable electronics. Adv. Mater. Technol. 4(3), 1800338 (2019). https://doi.org/10.1002/admt.201800338
  35. H. Li, J.Q. Cao, J.L. Chen, X. Liu, Y.W. Shao et al., Highly sensitive MXene helical yarn/fabric tactile sensors enabling full scale movement detection of human motions. Adv. Electron. Mater. 8(4), 2100890 (2021). https://doi.org/10.1002/aelm.202100890
  36. M. Zhao, D.W. Li, J.Y. Huang, D. Wang, A. Mensah et al., A multifunctional and highly stretchable electronic device based on silver nanowire/wrap yarn composite for a wearable strain sensor and heater. J. Mater. Chem. C 7(43), 13468–13476 (2019). https://doi.org/10.1039/C9TC04252K
  37. Z.F. Liu, S. Fang, F.A. Moura, J.N. Ding, N. Jiang et al., Hierarchically buckled sheath-core fibers for superelastic electronics, sensors, and muscles. Science 349(6246), 400–404 (2015). https://doi.org/10.1126/science.aaa7952
  38. Z. Wang, Y. Huang, J. Sun, Y. Huang, H. Hu et al., Polyurethane/cotton/carbon nanotubes core-spun yarn as high reliability stretchable strain sensor for human motion detection. ACS Appl. Mater. Interfaces 8(37), 24837–24843 (2016). https://doi.org/10.1021/acsami.6b08207
  39. P.W. Wang, J.J. Zhou, B.J. Xu, C. Lu, Q.A. Meng et al., Bioinspired anti-plateau-rayleigh-instability on dual parallel fibers. Adv. Mater. 32(45), 2003453 (2020). https://doi.org/10.1002/adma.202003453
  40. L. Ma, R. Wu, A. Patil, J. Yi, D. Liu et al., Acid and alkali-resistant textile triboelectric nanogenerator as a smart protective suit for liquid energy harvesting and self-powered monitoring in high-risk environments. Adv. Funct. Mater. 31(35), 2102963 (2021). https://doi.org/10.1002/adfm.202102963
  41. T. Yan, H. Zhou, H.T. Niu, H. Shao, H.X. Wang et al., Highly sensitive detection of subtle movement using a flexible strain sensor from helically wrapped carbon yarns. J. Mater. Chem. C 7(32), 10049–10058 (2019). https://doi.org/10.1039/C9TC03065D
  42. Z.Y. Wang, H. Hu, 3D auxetic warp-knitted spacer fabrics. Phys. Status Solidi B 251(2), 281–288 (2014). https://doi.org/10.1002/pssb.201384239
  43. Y. Jiang, Z.Y. Liu, N. Matsuhisa, D.P. Qi, W.R. Leow et al., Auxetic mechanical metamaterials to enhance sensitivity of stretchable strain sensors. Adv. Mater. 30(12), 1706589 (2018). https://doi.org/10.1002/adma.201706589
  44. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury et al., Pytorch: an imperative style, high-performance deep learning library. arXiv: 1912.01703 (2019). https://doi.org/10.48550/arXiv.1912.01703
  45. J. Ge, L. Sun, F.R. Zhang, Y. Zhang, L.A. Shi et al., A stretchable electronic fabric artificial skin with pressure-, lateral strain-, and flexion-sensitive properties. Adv. Mater. 28(4), 722–728 (2016). https://doi.org/10.1002/adma.201504239
  46. L.V. Maaten, G. Hinton, Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)
Read More
Author biographies are not available.
Statistic
Read Counter : 4737 Download : 3377

Most read articles by the same author(s)