Skip to main content
SpringerLink
Log in

One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects

  • Original Paper
  • Published:
Journal of Civil Structural Health Monitoring Aims and scope Submit manuscript

Abstract

In this study, a supervised machine learning network model is proposed to detect and classify various types of cracks developed in asphalt pavements, including lane markers. Crack images captured from a digital camera are classified into nine categories following the pavement distress identification manual proposed by the Federal Highways Administration (FHWA). These categories are three different types of cracks, such as fatigue, longitudinal, and transverse cracks with three severity levels of the low, medium, and high for each crack type. To establish a training dataset for crack detection, 1000 images with the original size of 3704 × 10,000 pixels are divided into 20,000 smaller images of 1852 × 1000 pixels image size. The training images are labeled based on the nine categories and trained using an updated version of faster R-CNN called RetinaNet. The trained network model is validated using pavement surface images obtained from 2400 m of two road sections. It is observed from the validation study that the detection and classification accuracy of the trained network model is 84.9% considering both the crack type and severity level. When considering the crack type only, the detection accuracy of the network model is 89.1%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Oliveira H, Correia PL (2009) Automatic road crack segmentation using entropy and image dynamic thresholding. In: 2009 17th European signal processing conference. IEEE, Glasgow, Scotland, pp 622-626. ISBN:978-161-7388-76-7

  2. Zhao H, Qin G, Wang X (2010) Improvement of canny algorithm based on pavement edge detection. In: 2010 3rd International congress on image and signal processing. IEEE, Yantai, China 2:964–967. https://doi.org/10.1109/CISP.2010.5646923

  3. Tanaka N, Uematsu K (1998) A crack detection method in road surface images using morphology. In: Proceedings of International association for pattern recognition workshop on machine vision applications, MVA 1998. Chiba, Japan 98:154-157. ISBN: 4901122983

  4. Koch C, Brilakis I (2011) Pothole detection in asphalt pavement images. Adv Eng Inform 25(3):507–515. https://doi.org/10.1016/j.aei.2011.01.002

    Article  Google Scholar 

  5. Ying L, Salari E (2010) Beamlet transform-based technique for pavement crack detection and classification. Computer Aided Civil Infrastruct Eng 25(8):572–580. https://doi.org/10.1111/j.1467-8667.2010.00674.x

    Article  Google Scholar 

  6. Sheerin SN, Kavitha S, Raghuraman G (2018) Review and analysis of crack detection and classification techniques based on crack types. Int J Appl Eng Res 13(8):6056–6062

    Google Scholar 

  7. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: advances in neural information processing systems 25 (NIPS 2012). Nevada, USA 1097–1105. https://doi.org/10.1145/3065386

  8. Fan Z, Wu Y, Lu J, Li W (2018) Automatic pavement crack detection based on structured prediction with the convolutional neural network. In: computer vision and pattern recognition 2018. Utah, USA. arXiv preprint arXiv:1802.02208

  9. Li S, Zhao X (2018) Convolutional neural networks-based crack detection for real concrete surface. In conference: sensors and smart structures technologies for civil, mechanical, and aerospace systems 2018. Colorado, USA. https://doi.org/10.1117/12.2296536

  10. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: proceedings of the 2014 IEEE conference on computer vision and pattern recognition. Ohio, USA, 580–587. https://doi.org/10.1109/CVPR.2014.81

  11. Girshick R (2015) Fast R-CNN. In: Proceedings of the 2015 IEEE international conference on computer vision. Santiago, Chile 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  12. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: conference on neural information processing systems 2015. Montreal, Canada 91–99. https://doi.org/10.1109/TPAMI.2016.2577031

  13. Liu W et al (2016) SSD: single shot multibox detector. In: European conference on computer vision 2016. Amsterdam, The Netherlands 21–37. https://doi.org/10.1007/978-3-319-46448-0_2

  14. Dai J, Li K, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems, 2016. Barcelona, Spain 379–387. arXiv:1605.06409

  15. Lin T-Y , Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, 2017. Venice, Italy 2980–2988. https://doi.org/10.1109/ICCV.2017.324

  16. Mohan A, Poobal S (2018) Crack detection using image processing: a critical review and analysis. Alex Eng J 57(2):787–798. https://doi.org/10.1016/j.aej.2017.01.020

    Article  Google Scholar 

  17. Yao X, Yao M, Xu B (2008) Automated detection and identification of area-based distress in concrete pavements. In: 7th international conference on managing pavement assets, 2008. Alberta, Canada, pp 1–11

    Google Scholar 

  18. Radopoulou SC, Brilakis I (2015) Patch detection for pavement assessment. Autom Constr 53:95–104. https://doi.org/10.1016/j.autcon.2015.03.010

    Article  Google Scholar 

  19. Hadjidemetriou GM, Christodoulou SE, Vela PA (2016) Automated detection of pavement patches utilizing support vector machine classification. In: 2016 18th Mediterranean electrotechnical conference (MELECON). IEEE, Limassol, Cyprus 1–5. https://doi.org/10.1109/MELCON.2016.7495460

  20. Miller JS, Bellinger WY (2014) Distress identification manual for the long-term pavement performance program. United States, Federal Highway Administration, Office of Infrastructure

    Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. Nevada, USA 770–778. https://doi.org/10.1109/CVPR.2016.90

  22. Lin T-Y , Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. Hawaii, USA, 2117–2125. https://doi.org/10.1109/CVPR.2017.106

  23. ASTM D6433–11 (2011) Standard practice for roads and parking lots pavement condition index surveys

  24. Brown E, Kandhal PS, Roberts FL, Kim YR, Lee D-Y, Kennedy TW (2009) Hot mix asphalt materials, mixture design, and construction. NAPA research and education foundation, Lanham, Maryland

    Google Scholar 

  25. Gaiser H, de Vries M, Lacatusu V (2018) Fizyr/keras-retinanet 0.5.1. In: GitHub repository. https://doi.org/10.5281/zenodo.3250670

  26. He K, Gkioxari G, Dollár P, Girshick (2017) Mask R-CNN. In: proceedings of the IEEE international conference on computer vision, 2017. Venice, Italy 2961–2969. https://doi.org/10.1109/ICCV.2017.322

  27. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, 2015. Munich, Germany 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

Download references

Acknowledgements

 The authors would like to acknowledge the support given by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 20POQW-B152690-02), Sejong University, Seoul Metropolitan Government, and the Korea Institute of Civil Engineering and Building Technology (KICT). 

Author information

Authors and Affiliations

  1. IRIS Technology Company, 98 Gunja-dong, Gwangjin-gu, Seoul, 143-747, Republic of Korea

    Van Phuc Tran

  2. Department of Civil and Environmental Engineering, Sejong University, 98 Gunja-dong, Gwangjin-gu, Seoul, 143-747, Republic of Korea

    Thai Son Tran, Hyun Jong Lee, Ki Deok Kim & Thanh Tu Nguyen

  3. Pavement Research Center, Seoul Metropolitan Government, Seoul, South Korea

    Jongeun Baek

Authors
  1. Van Phuc Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thai Son Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hyun Jong Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ki Deok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jongeun Baek
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Thanh Tu Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van Phuc Tran.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tran, V.P., Tran, T.S., Lee, H.J. et al. One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects. J Civil Struct Health Monit 11, 205–222 (2021). https://doi.org/10.1007/s13349-020-00447-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13349-020-00447-8

Keywords

Search

Navigation