Road surface condition monitoring is one of the most crucial tasks for vehicle perception systems. The presence of water, snow, ice, or any other substance covering the road surface directly affects the rolling resistance and controllability of the vehicle, which is directly related to the safety of the traffic participants. Many sensors, such as RGB cameras, infrared sensors, and mmWave sensors, are used to monitor and inspect road surfaces. The research aims to provide a tool to segment an input image into correct classes. The DeepHyperX toolbox was used for the rapid prototyping of deep learning (DL) classification models for hyperspectral images. The effectiveness of the developed algorithm in several case studies is presented, and it is verified that the low number of iterations is enough to detect the water, snow, and ice on the road surface.
1. WHO. Global Status Report on Road Safety 2018.
https://www.who.int/publications/i/item/9789241565684 (accessed 2024-01-24).
2. Ziebinski, A., Cupek, R., Grzechca, D. and Chruszczyk, L. Review of advanced driver assistance systems (ADAS). AIP Conf. Proc., 2017, 1906(1), 120002.
https://doi.org/10.1063/1.5012394
3. Zang, S., Ding, M., Smith, D., Tyler, P., Rakotoarivelo, T. and Kaafar, M. A. The impact of adverse weather conditions on autonomous vehicles: how rain, snow, fog, and hail affect the performance of a self-driving car. IEEE Veh. Technol. Mag., 2019, 14(2), 103–111.
https://doi.org/10.1109/MVT.2019.2892497
4. Munawar, H. S. Image and video processing for defect detection in key infrastructure. In Machine Vision Inspection Systems: Image Processing, Concepts, Methodologies and Applications (Malarvel, M., Nayak, S. R., Panda, S. N., Pattnaik, P. K. and Muangnak, N., eds). Wiley Online Library, 2020, 159–177.
https://doi.org/10.1002/9781119682042.CH7
5. Roychowdhury, S., Zhao, M., Wallin, A., Ohlsson, N. and Jonasson, M. Machine learning models for road surface and friction estimation using front-camera images. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018. IEEE, 2018, 1–8.
https://doi.org/10.1109/IJCNN.2018.8489188
6. Chen, C., Zhu, H., Li, M. and You, S. A review of visual-inertial simultaneous localization and mapping from filtering-based and optimization-based perspectives. Robotics, 2018, 7(3), 45.
https://doi.org/10.3390/ROBOTICS7030045
7. Shinmoto, Y., Takagi, J., Egawa, K., Murata, Y. and Takeuchi, M. Road surface recognition sensor using an optical spatial filter. In Proceedings of Conference on Intelligent Transportation Systems, Boston, MA, USA, 12 November 1997. IEEE, 1997, 1000–1004.
https://doi.org/10.1109/ITSC.1997.660610
8. Liyanage, D. C., Hudjakov, R. and Tamre, M. Hyperspectral imaging methods improve RGB image semantic segmentation of unstructured terrains. In Proceedings of the 15th International Conference Mechatronic Systems and Materials (MSM), Bialystok, Poland, 1–3 July 2020. IEEE, 2020, 1–5.
https://doi.org/10.1109/MSM49833.2020.9201738
9. Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P. and Benediktsson, J. A. Deep learning for hyperspectral image classification: an overview. IEEE Trans. Geosci. Remote Sens., 2019, 57(9), 6690–6709.
https://doi.org/10.1109/TGRS.2019.2907932
10. Basterretxea, K., Martínez, V., Echanobe, J., Gutiérrez-Zaballa, J. and Del Campo, I. HSI-drive: a dataset for the research of hyperspectral image processing applied to autonomous driving systems. In Proceedings of the 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan, 11–17 July 2021. IEEE, 2021, 866–873.
https://doi.org/10.1109/IV48863.2021.9575298
11. Lu, J., Liu, H., Yao, Y., Tao, S., Tang, Z. and Lu, J. Hsi road: a hyper spectral image dataset for road segmentation. In Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK, 6–10 July 2020. IEEE, 2020, 1–6.
https://doi.org/10.1109/ICME46284.2020.9102890
12. Specim. Specim IQ Technical Specifications.
https://www.specim.fi/iq/tech-specs/ (accessed 2022-12-14).
13. Audebert, N., Saux, B. and Lefèvre, S. Deep learning for classification of hyperspectral data: a comparative review. IEEE Geosci. Remote Sens. Mag., 2019, 7(2), 159–173.
https://doi.org/10.1109/MGRS.2019.2912563
14. Fauvel, M., Chanussot, J., Benediktsson, J. A. and Sveinsson, J. R. Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles. IEEE Trans. Geosci. Remote Sens., 2008, 46(11P2), 3804–3814.
15. Li, J., Bioucas-Dias, J. M. and Plaza, A. Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Trans. Geosci. Remote Sens., 2010, 48(11), 4085–4098.
https://doi.org/10.1109/TGRS.2010.2060550
16. Rychlewski, D. Hyperspectral image classification of satellite images using compressed neural networks. Master’s thesis, 2020.
https://www.researchgate.net/publication/344817503_Hyperspectral_Image_Classification_of_Satellite_Images_
Using_Compressed_Neural_Networks (accessed 2022-12-05).
17. Labelbox.
https://labelbox.com/ (accessed 2022-12-05).
18. Ben Hamida, A., Benoit, A., Lambert, P. and Ben Amar, C. 3-D deep learning approach for remote sensing image classification. IEEE Trans. Geosci. Remote Sens., 2018, 56(8), 4420–4434.
https://doi.org/10.1109/TGRS.2018.2818945
