|
1
|
Mai C, Chen H, Zeng L, Li Z, Liu G, Qiao Z, Qu Y, Li L, Li L. A Smart Cane Based on 2D LiDAR and RGB-D Camera Sensor-Realizing Navigation and Obstacle Recognition. SENSORS (BASEL, SWITZERLAND) 2024; 24:870. [PMID: 38339588 PMCID: PMC10856969 DOI: 10.3390/s24030870] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 12/27/2023] [Accepted: 01/19/2024] [Indexed: 02/12/2024]
Abstract
In this paper, an intelligent blind guide system based on 2D LiDAR and RGB-D camera sensing is proposed, and the system is mounted on a smart cane. The intelligent guide system relies on 2D LiDAR, an RGB-D camera, IMU, GPS, Jetson nano B01, STM32, and other hardware. The main advantage of the intelligent guide system proposed by us is that the distance between the smart cane and obstacles can be measured by 2D LiDAR based on the cartographer algorithm, thus achieving simultaneous localization and mapping (SLAM). At the same time, through the improved YOLOv5 algorithm, pedestrians, vehicles, pedestrian crosswalks, traffic lights, warning posts, stone piers, tactile paving, and other objects in front of the visually impaired can be quickly and effectively identified. Laser SLAM and improved YOLOv5 obstacle identification tests were carried out inside a teaching building on the campus of Hainan Normal University and on a pedestrian crossing on Longkun South Road in Haikou City, Hainan Province. The results show that the intelligent guide system developed by us can drive the omnidirectional wheels at the bottom of the smart cane and provide the smart cane with a self-leading blind guide function, like a "guide dog", which can effectively guide the visually impaired to avoid obstacles and reach their predetermined destination, and can quickly and effectively identify the obstacles on the way out. The mapping and positioning accuracy of the system's laser SLAM is 1 m ± 7 cm, and the laser SLAM speed of this system is 25~31 FPS, which can realize the short-distance obstacle avoidance and navigation function both in indoor and outdoor environments. The improved YOLOv5 helps to identify 86 types of objects. The recognition rates for pedestrian crosswalks and for vehicles are 84.6% and 71.8%, respectively; the overall recognition rate for 86 types of objects is 61.2%, and the obstacle recognition rate of the intelligent guide system is 25-26 FPS.
Collapse
Affiliation(s)
- Chunming Mai
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
| | - Huaze Chen
- College of Information Science and Technology, Hainan Normal University, Haikou 571158, China;
| | - Lina Zeng
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Zaijin Li
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Guojun Liu
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Zhongliang Qiao
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Yi Qu
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Lianhe Li
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| | - Lin Li
- College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China; (C.M.); (L.Z.); (Z.L.); (G.L.); (Z.Q.); (Y.Q.)
- Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China
- Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China;
| |
Collapse
|
|
2
|
Gharghan SK, Al-Kafaji RD, Mahdi SQ, Zubaidi SL, Ridha HM. Indoor Localization for the Blind Based on the Fusion of a Metaheuristic Algorithm with a Neural Network Using Energy-Efficient WSN. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2022. [DOI: 10.1007/s13369-022-07188-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
|
3
|
Improved Extreme Learning Machine Based UWB Positioning for Mobile Robots with Signal Interference. MACHINES 2022. [DOI: 10.3390/machines10030218] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
For the purpose of tackling ultra-wideband (UWB) indoor positioning with signal interference, a binary classifier for signal interference discrimination and positioning errors compensation model combining genetic algorithm (GA) and extreme learning machine (ELM) are put forward. Based on the distances between four anchors and the target which are calculated with time of flight (TOF) ranging technique, GA-ELM-based binary classifier for judging the existence of signal interference, and GA-ELM-based positioning errors compensation model are built up to compensate for the result of the preliminary evaluated positioning model. Finally, the datasets collected in the actual scenario are used for verification and analysis. The experimental results indicate that the root-mean-square error (RMSE) of positioning without signal interference is 14.5068 cm, which is reduced by 71.32% and 59.72% compared with those results free of compensation and optimization, respectively. Moreover, the RMSE of positioning with signal interference is 28.0861 cm, which is decreased by 64.38% and 70.16%, in comparison to their counterparts without compensation and optimization, respectively. Consequently, these calculated results of numerical examples lead to the conclusion that the proposed method displays its wide application, high precision and rapid convergence in improving the positioning accuracy for mobile robots.
Collapse
|