Pavement damage is the main factor affecting road performance. Pavement cracking, a common type of road damage, is a key challenge in road maintenance. In order to achieve an accurate crack classification, segmentation, and geometric parameter calculation, this paper proposes a method based on a deep convolutional neural network fusion model for pavement crack identification, which combines the advantages of the multitarget single-shot multibox detector (SSD) convolutional neural network model and the U-Net model. First, the crack classification and detection model is applied to classify the cracks and obtain the detection confidence. Next, the crack segmentation network is applied to accurately segment the pavement cracks. By improving the feature extraction structure and optimizing the hyperparameters of the model, pavement crack classification and segmentation accuracy were improved. Finally, the length and width (for linear cracks) and the area (for alligator cracks) are calculated according to the segmentation results. Test results show that the recognition accuracy of the pavement crack identification method for transverse, longitudinal, and alligator cracks is 86.8%, 87.6%, and 85.5%, respectively. It is demonstrated that the proposed method can provide the category information for pavement cracks as well as the accurate positioning and geometric parameter information, which can be used directly for evaluating the pavement condition.
2020 fusion version 18 cracked
Pavement distress is the main factor affecting road performance. Timely and accurate detection of pavement damages is a crucial step in pavement maintenance. Cracks are the initial manifestation of various types of pavement diseases. Pavement cracks will not only affect pavement appearance and driving comfort but also can easily expand to cause pavement structural damage and shorten the overall service performance and life of the pavement [1, 2]. Therefore, early crack detection and timely maintenance of the cracked pavement can reduce the economic cost of pavement repairing and ensure the safety of vehicles and drivers transiting on the pavement.
In 2016, Zhang et al. [18] proposed a crack detection method based on deep learning. They trained a deep CNN based on supervised learning, proving the feasibility of combining deep learning with pavement crack recognition. In 2017, Zhao et al. [19] proposed a pavement crack detection method based on a CNN using images of different scales and taken at different angles for training, achieving the detection of cracks of various shapes. However, owing to road surface interference and noise, the detection accuracy of this system peaked at 82.5%. In 2017, Markus et al. developed the open dataset GAPs for the training of deep neural network and evaluated the pavement damage detection technology for the first time, which is of great significance [20, 21]. In 2018, Nhat-Duc et al. [22] established an intelligent method for the automatic recognition of pavement crack morphology; this study constructs a machine learning model for pavement crack classification that included multiple support vector machines and an artificial swarm optimization algorithm. Using feature analysis, a set of features is extracted from the image projection integral, which can significantly improve the prediction performance. However, the algorithm is complex and programming it becomes very difficult. In 2020, Zhaoyun Sun et al. [23] proposed a method to detect pavement expansion cracks with the improved Faster R-CNN, which can achieve accurate expansion crack location detection through the optimization model. The aforementioned studies only detect and classify pavement cracks and their location but cannot quantify certain crack characteristics, such as crack width and area. On the other hand, there are also many studies on crack segmentation. In 2018, Zhang and Wang [24] proposed CrackNet, which is an efficient architecture based on CNN to predict the class of each image pixel, but its network structure is related to input image size, which prevents the generalization of the method. In the same year, Sen Wang et al. [25] proposed to use the full convolutional networks (FCNs) to detect cracks and built the Crack-FCN model taking into account the shortcomings of the FCN model in the crack segmentation experiment and obtained a complete crack image. However, the highest accuracy obtained by their method is only 67.95%; thus, segmentation performance needs to be improved. In 2019, Piao Weng et al. [26] proposed a pavement crack segmentation method based on the VGG-U-Net model. It solves the problem of fracture in the crack segmentation result in complex background, but its training time is slightly longer and its efficiency is low. In 2020, Zhun Fan et al. [27] proposed an encoder-decoder architecture based on hierarchical feature learning and dilated convolution (U-HDN) detects cracks in an end-to-end manner. The U-HDN method can extract and fuse different context sizes and different levels of feature mapping, so it has high performance. In the same year, Zhun Fan et al. [28] proposed an ensemble of convolutional neural network based on probability fusion for automatic detection and measurement of pavement cracks, and the predicted crack morphology is measured by skeleton extraction algorithm. In summary, these previous studies only use the segmentation method, which cannot achieve accurate crack classification and location determination.
Given the abovementioned problems in pavement crack identification, this paper proposes a method based on a deep convolutional neural network fusion model for pavement crack identification, which is applicable in many crack detection cases (including detector vehicle and smartphone). By training on a learning image data having a variety of sources and sizes, the method can effectively identify cracks, and recognition accuracy can be guaranteed. At the same time, a detected crack can be segmented, and the segmented binary image can be used to calculate the geometric parameters of the crack. Therefore, the proposed model is of great significance for intelligent pavement detection and it can also achieve detection and segmentation simultaneously, thereby significantly improving model efficiency.
In this paper, a crack identification method based on a deep CNN fusion model is proposed. First, the image dataset is established, and the image noise in the dataset is filtered out to increase the contrast between road cracks and background. Next, the processed images are provided as input into an improved single shot multibox detector (SSD) crack detection model and an improved U-Net crack segmentation model for training. Then, the binary image of a crack obtained by the segmentation model is used to calculate the geometric parameters of the crack. By integrating the advantages of the two models, this pavement crack identification method can effectively overcome the single-model limitations of inaccurate positioning and imperfect information. The overall process flowchart is shown in Figure 1. The details of each step are discussed in Section 2.1.
The characteristic feature of the SSD network model is its capability of performing multiscale feature map detections. The model adds some convolutional feature layers at the end of the feature extraction network, and the feature maps extracted from these convolutional layers have the feature of decreasing in size. Image prediction is carried out by means of fusion of the multiscale detection results. In Figure 5, the feature fusion of conv4_3 and , two convolution layers, is given as an illustrative example. conv4_3_norm_priorbox sets each point to generate four preselected boxes. The sample dataset used in this study contains three categories; thus, the value of the conv4_3_norm_mbox_conf channel is 12 (). Each preselected box returns four position transitions, so the value of the conv4_3_norm_mbox_loc channel is 16 (). generates six preselected boxes per point, in addition to the others. Finally, mbox_conf and mbox_loc are merged, mbox_conf behind reshapes, and then softmax classification is performed. As can be seen from the above example, each convolution feature layer will produce corresponding prediction results, and finally, the prediction results on different scales will be fused to obtain the best fracture prediction results.
By replacing the feature extraction structure of the original SSD network with the deep residual network, the network accuracy and recall rate in predicting pavement cracks were substantially improved. This analysis of the experimental results shows that the proposed method achieves good results in the classification and detection of cracks. From the prediction effect, however, a classification by the pavement crack detection method based on the single SSD crack location model is incomplete, is not conducive to subsequent crack geometry parameter computation steps, and will produce larger calculation errors. Thus, as the practical application value is still lacking at this point, this study adopted the fusion segmentation model approach to address this problem.
The crack segmentation model uses the cascade mode of multiple residual elements, which can effectively extract the morphological characteristics of the pavement crack and improve the learning effectiveness of the neural network on the crack characteristics. A single crack segmentation method based on U-Net can provide the crack pixel location information, but it cannot classify the crack [35]. Therefore, in this study, a fusion of two models was adopted to identify pavement crack images and to obtain the crack category, location information, and geometric parameters, thereby facilitating accurate quantification and evaluation of pavement cracks.
As shown in Figure 17, when only the SSD detection network is used, the number of cracks can be accurately obtained, but the crack width cannot be quantified. Use of only the U-Net segmentation model will lead to misjudgment of the number of cracks; e.g., a fractured crack may be misidentified as multiple cracks. The fusion of the detection and segmentation networks can avoid this phenomenon and ensure that the crack is identified as a single crack. Thus, the advantage of the fusion model is that it can accurately identify the number of cracks and ensure that cracks are quantified correctly. 2ff7e9595c
Commentaires