CN110853034A

CN110853034A - Crack detection method, crack detection device, electronic equipment and computer-readable storage medium

Info

Publication number: CN110853034A
Application number: CN202010039454.5A
Authority: CN
Inventors: 王皓冉; 李永龙; 陈永灿; 张华�; 刘昭伟
Original assignee: Sichuan Energy Internet Research Institute EIRI Tsinghua University
Current assignee: Sichuan Energy Internet Research Institute EIRI Tsinghua University
Priority date: 2020-01-15
Filing date: 2020-01-15
Publication date: 2020-02-28

Abstract

The invention provides a crack detection method, a crack detection device, electronic equipment and a computer-readable storage medium, and relates to the field of image processing. The method comprises the following steps: acquiring an image to be detected of a target building; inputting an image to be detected into a crack detection model to obtain a preliminary prediction image of a target building; the crack detection model is obtained according to a plurality of crack training images of the target building; fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building. On the basis of an image to be detected of the target building, a preliminary prediction image acquired through the crack detection model is fused so as to acquire a more accurate crack detection image, the crack condition of the target building can be acquired more accurately through the crack detection image, and compared with destructive detection, the target building cannot be damaged; compared with human eye detection, the crack detection efficiency is improved by using a machine for crack detection.

Description

Crack detection method, crack detection device, electronic equipment and computer-readable storage medium

Technical Field

The invention relates to the field of image processing, in particular to a crack detection method, a crack detection device, electronic equipment and a computer-readable storage medium.

Background

Concrete buildings such as high dams, bridges and concrete pavements are affected by factors such as earthquakes and landslides, and the surface of the concrete building is easy to form crack defects which seriously threaten the safety of the building. In order to evaluate the health condition of the buildings, crack detection is generally performed on the cracks, and the cracks are used as an important index for evaluating the safety of the buildings.

The traditional concrete crack detection mainly adopts methods of manual naked eye detection, destructive detection and traditional image processing technology. The manual naked eye detection has the defects of high labor cost, strong subjectivity, high false detection rate, long time consumption and the like; the destructive detection can cause secondary damage to the concrete structure in the detection process; the traditional image processing technology needs to manually design a specific target detection target, and has the defects of complex workload, low generalization capability and low efficiency; the traditional concrete crack detection cannot realize the crack segmentation detection of a target building quickly and accurately.

Disclosure of Invention

In view of the above, the present invention provides a crack detection method, a crack detection apparatus, an electronic device, and a computer-readable storage medium.

In order to achieve the purpose, the technical scheme adopted by the invention is as follows:

in a first aspect, the present invention provides a crack detection method, the method comprising: and acquiring an image to be detected of the target building. Inputting the image to be detected into a crack detection model to obtain a preliminary prediction image of the target building; wherein the crack detection model is obtained according to a plurality of crack training images of the target building. Fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building.

In an optional embodiment, the fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building includes: acquiring at least one effective receptive field of the preliminary prediction image; the effective receptive field is used to indicate an area in the target structure having a crack to be identified. Clustering the image to be detected and the effective receptive field to obtain the crack detection image; the crack detection image comprises background pixels of the crack to be confirmed and the target building.

In an optional embodiment, the clustering the image to be detected and the effective receptive field to obtain the crack detection image includes: acquiring a clustering sample set according to the image to be detected and the effective receptive fields; the clustering sample set comprises a plurality of clustering samples of the image to be detected and the crack to be confirmed; acquiring a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set; the crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected; and fusing the crack clustering result and the background clustering result to obtain the crack detection image.

In an optional embodiment, the obtaining, according to the clustering sample set, a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel includes: acquiring a fracture centroid vector and a background centroid vector from a plurality of the clustered samples; the fracture centroid vector is a clustering sample which is the smallest from a fracture average value in a plurality of clustering samples, and the fracture average value is the average value of the plurality of clustering samples belonging to a fracture class; the background centroid vector is a cluster sample which is the smallest from a background average value in the plurality of cluster samples, and the background average value is an average value of the plurality of cluster samples except the crack class. And iterating the fracture centroid vector to converge according to the plurality of clustering samples belonging to the fracture category to obtain the fracture clustering result. And iterating the background centroid vector to converge according to the clustering samples except the crack classes to obtain the background clustering result.

In an optional embodiment, the inputting the image to be detected to a crack detection model to obtain a preliminary prediction image of the target building includes: preprocessing the image to be detected to obtain a plurality of images to be processed; the pretreatment comprises any one or combination of the following: horizontal turning, vertical turning, mirror image, contrast adjustment and brightness adjustment. Inputting the multiple images to be processed into the crack detection model to obtain multiple characteristic graphs; the feature size of each of the feature maps is different. And fusing a plurality of feature maps to obtain the preliminary prediction image.

In a second aspect, the present invention provides a crack detection device, comprising: the device comprises an acquisition module and a processing module. The acquisition module is used for acquiring an image to be detected of the target building. The processing module is used for inputting the image to be detected into a crack detection model so as to obtain a preliminary prediction image of the target building; wherein the crack detection model is obtained according to a plurality of crack training images of the target building. The processing module is further used for fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building.

In an alternative embodiment, the processing module is further configured to obtain at least one effective receptive field of the preliminary prediction image; the effective receptive field is used to indicate an area in the target structure having a crack to be identified. The processing module is further used for clustering the image to be detected and the effective receptive field to obtain the crack detection image; the crack detection image comprises background pixels of the crack to be confirmed and the target building.

In an optional embodiment, the processing module is further configured to obtain a cluster sample set according to the image to be detected and the plurality of effective receptive fields; the clustering sample set comprises the images to be detected and a plurality of clustering samples of the cracks to be confirmed. The processing module is further used for acquiring a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set; the crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected. The processing module is further used for fusing the crack clustering result and the background clustering result to obtain the crack detection image.

In an optional embodiment, the processing module is further configured to obtain a fracture centroid vector and a background centroid vector from a plurality of the clustered samples; the fracture centroid vector is a clustering sample which is the smallest from a fracture average value in a plurality of clustering samples, and the fracture average value is the average value of the plurality of clustering samples belonging to a fracture class; the background centroid vector is a cluster sample which is the smallest from a background average value in the plurality of cluster samples, and the background average value is an average value of the plurality of cluster samples except the crack class. The processing module is further configured to iterate the fracture centroid vector to converge according to the plurality of clustered samples belonging to the fracture category to obtain the fracture clustering result. The processing module is further configured to iterate the background centroid vector to converge according to the plurality of clustered samples belonging to other than the fracture class to obtain the background clustering result.

In an optional embodiment, the processing module is further configured to pre-process the image to be detected to obtain a plurality of images to be processed; the pretreatment comprises any one or combination of the following: horizontal turning, vertical turning, mirror image, contrast adjustment and brightness adjustment. The processing module is further used for inputting the multiple images to be processed to the crack detection model to obtain multiple characteristic graphs; the feature size of each of the feature maps is different. The processing module is further configured to fuse a plurality of feature maps to obtain the preliminary prediction image.

In a third aspect, the present invention provides an electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor to implement the method of any one of the preceding embodiments.

In a fourth aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the preceding embodiments.

Compared with the prior art, the invention provides a crack detection method, a crack detection device, electronic equipment and a computer-readable storage medium, and relates to the field of image processing. The crack detection method comprises the following steps: acquiring an image to be detected of a target building; inputting the image to be detected into a crack detection model to obtain a preliminary prediction image of the target building; the crack detection model is obtained according to a plurality of crack training images of the target building; fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building. On the basis of an image to be detected of the target building, a preliminary prediction image acquired through the crack detection model is fused so as to acquire a more accurate crack detection image, the crack condition of the target building can be acquired more accurately through the crack detection image, and compared with destructive detection, the target building cannot be damaged; compared with human eye detection, the crack detection efficiency is improved by using a machine for crack detection.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1 is a schematic flow chart of a crack detection method according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of another crack detection method according to an embodiment of the present invention;

FIG. 3 is a schematic flow chart of another crack detection method according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of another crack detection method according to an embodiment of the present invention;

FIG. 5 is a schematic flow chart of another crack detection method according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a crack detection model according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a prediction box according to an embodiment of the present invention;

FIG. 8 is a schematic flow chart of another crack detection method according to an embodiment of the present invention;

FIG. 9 is a schematic diagram illustrating the effect of crack detection according to an embodiment of the present invention;

FIG. 10 is a block diagram of a crack detection device according to an embodiment of the present invention;

fig. 11 is a block diagram of an electronic device according to an embodiment of the present invention.

Icon: 50-crack detection device, 51-acquisition module, 52-processing module, 60-electronic device, 61-memory, 62-processor, 63-communication interface.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.

It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Concrete buildings such as high dams, bridges and concrete pavements are affected by factors such as earthquakes and landslides, and the surface of the concrete building is easy to form crack defects which seriously threaten the safety of the building. Crack detection is therefore an important indicator of the health of these buildings.

Periodic crack detection plays a crucial role in the maintenance and operation of existing concrete buildings: according to the geometrical form and the position distribution of the cracks, the potential cause of the internal damage of the cracks can be deduced, and reasonable guidance opinions can be provided for the health safety and risk assessment of the concrete structure.

In the three technical solutions in the prior art: the manual naked eye detection has the defects of high labor cost, strong subjectivity, high false detection rate, long time consumption and the like; the destructive detection can cause secondary damage to the concrete structure in the detection process; the traditional image processing technology needs to manually design a specific target detection target, is tedious in workload, low in generalization capability and low in efficiency, and cannot quickly and accurately realize crack segmentation detection at a pixel level.

Based on the above problems and the deficiencies suggested in the background art, an embodiment of the present invention provides a crack detection method, please refer to fig. 1, and fig. 1 is a schematic flow chart of the crack detection method provided in the embodiment of the present invention. The crack detection method comprises the following steps:

and S31, acquiring an image to be detected of the target building.

And S32, inputting the image to be detected into the crack detection model to obtain a preliminary prediction image of the target building.

The crack detection model is obtained according to a plurality of crack training images of the target building. For example, the crack detection model may be an end-to-end target detection model (SSD), and the SSD model may obtain the crack defect preliminary positioning information of the image to be detected by extracting the features of the image to be detected.

And S33, fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building.

The crack detection image is used for determining the crack condition of the target building. It should be understood that the crack detection image may or may not include a crack image of the target structure, and whether the crack is included in the crack detection image is related to the actual safety condition of the target structure.

On the basis of an image to be detected of the target building, a preliminary prediction image acquired through the crack detection model is fused so as to acquire a more accurate crack detection image, the crack condition of the target building can be acquired more accurately through the crack detection image, and compared with destructive detection, the target building cannot be damaged; compared with human eye detection, the crack detection efficiency is improved by using a machine for crack detection; compared with the traditional image processing technology, the crack detection method provided by the embodiment of the invention can enable the detection result of the target building to be more accurate.

In an optional implementation manner, the image to be detected and the preliminary prediction image may be channel superposition, may also be simple superposition between the images, and may also be other image fusion manners, and in order to quickly and effectively obtain the crack condition of the target building, on the basis of fig. 1, for example, the image to be detected and the preliminary prediction image are clustered and fused, please refer to fig. 2, and fig. 2 is a flow diagram of another crack detection method provided by an embodiment of the present invention. The above S33 may include:

and S331, acquiring at least one effective receptive field of the preliminary prediction image.

The effective field is used to indicate the area in the target structure having a crack to be identified. For example, when the crack detection model is an SSD model, to prevent important information from being lost too much in the downsampling process, more image information can be retained by increasing the number of channels of the convolution kernel; predefining prediction frames with various aspect ratios, so that the convolutional neural network of the SSD model has a larger receptive field area in the process of extracting the features of the image to be processed, thereby predicting small targets and target objects with various forms, and more accurately determining the cracks to be confirmed when the cracks on the target building are small; the network structure such as the SSD model uses a plurality of feature maps with different scales to detect cracks with different sizes, namely a low layer (a feature extraction layer with a small channel number) predicts small cracks, and a high layer (a feature extraction layer with a large channel number) predicts large cracks.

S332, clustering the image to be detected and the effective receptive field to obtain a crack detection image.

The crack detection image comprises a crack to be confirmed and background pixels of a target building; it will be appreciated that when clustering the image to be detected and the effective receptive field, two categories may be clustered: and integrating the clustering results corresponding to the background pixels and the target pixels (cracks to be confirmed) to obtain a crack detection image.

In an optional implementation manner, in the clustering process, supervised clustering may be used, or unsupervised clustering may also be used, and on the basis of fig. 2, the embodiment of the present invention takes unsupervised clustering as an example, please refer to fig. 3, and fig. 3 is a schematic flow chart of another crack detection method provided in the embodiment of the present invention. The above S332 may include:

s332a, obtaining a clustering sample set according to the image to be detected and the effective receptive fields.

The cluster sample set comprises an image to be detected and a plurality of cluster samples of cracks to be confirmed.

S332b, acquiring a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set and the image to be detected.

The crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected. It should be understood that when unsupervised clustering is implemented, the above-mentioned clustering sample set may be divided into different categories by using a K-Means clustering method, and assuming that the euclidean distance between data of the same category is closer, the more similar the features of the data closer, and the more different the features of the data farther away, that is, the similarity is inversely proportional to the euclidean distance between the data.

S332c, fusing the crack clustering result and the background clustering result to obtain a crack detection image.

For example, assuming that the target building is a concrete structure, when the target building is segmented aiming at common crack defects in the concrete defects, only the background and the cracks need to be clustered, and then the background and the cracks are combined, so that a crack detection image of the target building can be obtained, and further the crack condition of the target building is determined.

To facilitate understanding of the above clustering process, on the basis of fig. 3, the embodiment of the present invention takes unsupervised clustering as an example, please refer to fig. 4, and fig. 4 is a schematic flow chart of another crack detection method provided in the embodiment of the present invention. The above S322b may include:

and S332b1, acquiring a fracture centroid vector and a background centroid vector from the plurality of clustering samples.

The fracture centroid vector is a clustering sample which is the smallest distance from the fracture average value in the clustering samples, and the fracture average value is the average value of the clustering samples belonging to the fracture category. The background centroid vector is the clustering sample with the smallest distance to the background average value in the clustering samples, and the background average value is the average value of the clustering samples except for the crack category.

S332b2, iterating the crack mass center vector to converge according to the plurality of clustering samples belonging to the crack category, so as to obtain a crack clustering result.

And S332b3, iterating the background centroid vector to converge according to the plurality of clustering samples except for the crack category to obtain a background clustering result.

In order to facilitate understanding of the unsupervised clustering process, the embodiment of the present invention provides a possible implementation manner: input clustering sample set

Assuming output cluster partitioning

Wherein the number of the clustered categories is k, and the maximum iteration number is N; k =2 when only background pixels and cracks to be confirmed need to be clustered

The ith effective receptive field in the image to be detected and the preliminary prediction image is obtained.

The first step is as follows: randomly selecting 2 cluster samples from the cluster sample set D as initial fracture centroid vectors (C:)

) And a background centroid vector: (

）。

The second step is that: partitioning initialization clusters into

. Calculate each sample

Respective and fracture centroid vector: (

) And a background centroid vector: (

) Is a distance of

Will be

Marking as minimum distance

Corresponding category

(crack or background pixel to be confirmed) and will

Is divided into

In categories, and by executing formulasTo accomplish the partitioning and updating of the plurality of clustered samples. And iterating and updating the centroid vectors of all the class samples until convergence, wherein the expression of the iterative updating of the centroid is as follows:

wherein the content of the first and second substances,

represents the centroid vector, j =1, 2; x represents the number of the cluster samples,

representing the class (crack or background pixel to be confirmed) in the output cluster partition, and if the centroid vector no longer changes, proceeding to the third step.

The third step: output cluster partitioning

And obtaining each sample clustering result (crack clustering result and background clustering result) and obtaining a final crack detection image so as to realize crack detection on the target building.

In an alternative embodiment, for a single image to be detected, the image to be detected is directly input to the crack detection model, a better preliminary prediction image may not be obtained, and in order to obtain a better preliminary prediction, a possible implementation is provided on the basis of fig. 1, please refer to fig. 5, where fig. 5 is a schematic flow diagram of another crack detection method provided in the embodiment of the present invention. The above S32 may include:

s321, preprocessing the image to be detected to obtain a plurality of images to be processed.

The pretreatment comprises any one or combination of the following: horizontal flipping, vertical flipping, mirroring, contrast adjustment, brightness adjustment, and the like.

And S322, inputting the multiple to-be-processed images into the crack detection model, and acquiring multiple characteristic graphs.

It should be understood that the feature sizes of each feature map are different.

And S323, fusing the plurality of feature maps to obtain a preliminary prediction image.

In order to obtain the above feature map, the embodiment of the present invention takes the fracture detection model as the SSD deep convolutional network as an example, please refer to fig. 6, and fig. 6 is a schematic structural diagram of the fracture detection model provided in the embodiment of the present invention. The network model (crack detection model) can simultaneously improve the precision and speed of crack defect detection by acquiring multi-scale information and setting various aspect ratios.

The whole crack detection model comprises two parts: the first part is to extract features of the input image (to-be-processed image) using the convolution layer of the first 5 layers of Visual geometry (VGG), such as VGG 16; and the second part uses the conventional convolution operation to realize down-sampling and extract high-dimensional characteristic information.

In order to prevent the important information of the image to be detected from being lost too much in the down-sampling process, more information can be reserved by increasing the number of channels of the convolution kernel. By predefining prediction frames with various aspect ratios, the network has a larger receptive field area in the feature extraction process, so that small targets and target objects with various forms can be predicted. The network structure (crack detection model) shown in fig. 6 uses feature maps of six different scales to detect targets of different sizes, namely, a low-level predicted small target and a high-level predicted large target.

Where D41 is the input image (image size 300 × 300); d42 is a three-layer convolution layer of a visual geometry model (VGG), the convolution processing with a convolution kernel of 3 x 24 is carried out on the image to be detected, and the size of the output characteristic diagram is 38 x 38; d43~ D48 are the characteristic extraction layer of crack detection model: d43 is used to instruct the image to be detected to be processed by convolution with a convolution kernel of 3 × 3 × 36, and the size of the output feature map is 19 × 19; d44 is used to instruct the image to be detected to be processed by convolution with a convolution kernel of 3 × 3 × 36, and the size of the output feature map is 19 × 19; d45 is used to instruct the image to be detected to be processed by convolution with a convolution kernel of 3 × 3 × 36, and the size of the output feature map is 10 × 10; d46 is used to instruct the image to be detected to be processed by convolution with a convolution kernel of 3 × 3 × 36, and the size of the output feature map is 5 × 5; d47 is used to instruct the image to be detected to be processed by convolution with a convolution kernel of 3 × 3 × 24, and the size of the output feature map is 3 × 3; d48 is used to instruct the image to be detected to be subjected to convolution processing with a convolution kernel of 3 × 3 × 24, and the feature map size of the output is 1 × 1. And (3) putting the feature maps obtained from D42-D48 into a multi-scale feature map candidate area, wherein each pixel point on each feature map corresponds to a receptive field, so that the SSD is equivalent to classifying and regressing all receptive fields in an image to be detected, and further selecting an effective receptive field through non-maximum value inhibition.

To facilitate understanding of the prediction block, please refer to fig. 7, and fig. 7 is a schematic diagram of a prediction block according to an embodiment of the present invention. On the basis of the image to be detected, the preliminary prediction image can be obtained through a crack detection model and a prediction frame.

To facilitate understanding of any crack detection method, please refer to fig. 8, and fig. 8 is a schematic flow chart of another crack detection method according to an embodiment of the present invention. After an image to be detected is subjected to preprocessing and a crack detection model, a preliminary prediction image is obtained; and clustering the image to be detected and the preliminary prediction image to obtain a crack detection image so as to realize crack detection of the target building.

Referring to fig. 9, fig. 9 is a schematic diagram illustrating an effect of crack detection according to an embodiment of the invention. The cracks in the image to be detected are not clear, the cracks in the crack detection image are clustered, the contrast between the background and the cracks is obvious, and the cracks of the target building can be effectively detected.

To implement any of the crack detection methods described above, please refer to fig. 10, and fig. 10 is a block diagram illustrating a crack detection apparatus according to an embodiment of the present invention. The crack detection device 50 includes: an acquisition module 51 and a processing module 52.

The obtaining module 51 is configured to obtain an image to be detected of a target building.

The processing module 52 is configured to input the image to be detected into the crack detection model to obtain a preliminary prediction image of the target building. The crack detection model is obtained according to a plurality of crack training images of the target building. The processing module 52 is further configured to fuse the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building. The crack detection image is used for determining the crack condition of the target building.

It should be understood that the obtaining module 51 and the processing module 52 may cooperatively implement the above-described S31-S33 and possible sub-steps thereof.

In an alternative embodiment, the processing module 52 is further configured to obtain at least one valid field of the preliminary prediction image. The effective field is used to indicate the area in the target structure having a crack to be identified. The processing module 52 is further configured to cluster the image to be detected and the effective receptive field to obtain a crack detection image. The crack detection image includes background pixels of the crack to be confirmed and the target structure.

It should be understood that the processing module 52 may implement S331-S332 and possible sub-steps thereof described above.

In an optional embodiment, the processing module 52 is further configured to obtain a cluster sample set according to the image to be detected and a plurality of effective receptive fields. The clustering sample set comprises a plurality of clustering samples of the images to be detected and the cracks to be confirmed. The processing module 52 is further configured to obtain a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set. And the crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected. The processing module 52 is further configured to fuse the crack clustering result and the background clustering result to obtain a crack detection image.

It should be appreciated that the processing module 52 may implement the above-described S332 a-S332 c and possible sub-steps thereof.

In an alternative embodiment, the processing module 52 is further configured to obtain a fracture centroid vector and a background centroid vector from a plurality of clustered samples. The fracture centroid vector is a clustering sample which is the smallest from the fracture average value in the plurality of clustering samples, and the fracture average value is the average value of the plurality of clustering samples belonging to the fracture category. The background centroid vector is the clustering sample with the smallest distance to the background average value in the clustering samples, and the background average value is the average value of the clustering samples except for the crack category. The processing module 52 is further configured to iterate the fracture centroid vector to converge according to the plurality of clustered samples belonging to the fracture category to obtain a fracture clustering result. The processing module 52 is further configured to iterate the background centroid vector to converge according to the plurality of clustered samples belonging to other than the fracture class, so as to obtain a background clustering result.

It should be appreciated that the processing module 52 may implement the above-described S33b 1-S33 b3 and possible sub-steps thereof.

In an alternative embodiment, the processing module 52 is further configured to pre-process the image to be detected to obtain a plurality of images to be processed. The pretreatment comprises any one or combination of the following: horizontal turning, vertical turning, mirror image, contrast adjustment and brightness adjustment. The processing module 52 is further configured to input a plurality of images to be processed to the crack detection model, and obtain a plurality of feature maps. The feature size of each feature map is different. The processing module 52 is further configured to fuse the plurality of feature maps to obtain a preliminary prediction image.

It should be understood that the processing module 52 may implement S321-S323 and possible sub-steps thereof described above.

Fig. 11 shows a block diagram of an electronic device according to an embodiment of the present invention, where fig. 11 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device 60 comprises a memory 61, a processor 62 and a communication interface 63. The memory 61, processor 62 and communication interface 63 are electrically connected to each other, directly or indirectly, to enable transmission or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 61 may be used to store software programs and modules, such as program instructions/modules corresponding to the crack detection method provided in the embodiment of the present invention, and the processor 62 executes various functional applications and data processing by executing the software programs and modules stored in the memory 61. The communication interface 63 may be used for communicating signaling or data with other node devices. The electronic device 60 may have a plurality of communication interfaces 63 in the present invention.

The Memory 61 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.

The processor 62 may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc.

The electronic device 60 may implement any of the crack detection methods provided by the present invention. The electronic device 60 may be, but is not limited to, a cell phone, a tablet computer, a notebook computer, a server, or other electronic device with processing capabilities.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

In summary, the present invention provides a crack detection method, a crack detection apparatus, an electronic device, and a computer-readable storage medium, and relates to the field of image processing. The crack detection method comprises the following steps: acquiring an image to be detected of a target building; inputting an image to be detected into a crack detection model to obtain a preliminary prediction image of a target building; the crack detection model is obtained according to a plurality of crack training images of the target building; fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building. On the basis of an image to be detected of the target building, a preliminary prediction image acquired through the crack detection model is fused so as to acquire a more accurate crack detection image, the crack condition of the target building can be acquired more accurately through the crack detection image, and compared with destructive detection, the target building cannot be damaged; compared with human eye detection, the crack detection efficiency is improved by using a machine for crack detection.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A crack detection method, characterized in that the method comprises:

acquiring an image to be detected of a target building;

inputting the image to be detected into a crack detection model to obtain a preliminary prediction image of the target building; the crack detection model is obtained according to a plurality of crack training images of the target building;

fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building.

2. The method according to claim 1, wherein the fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building comprises:

acquiring at least one effective receptive field of the preliminary prediction image; the effective receptive field is used for indicating an area with cracks to be confirmed in the target building;

clustering the image to be detected and the effective receptive field to obtain the crack detection image; the crack detection image comprises background pixels of the crack to be confirmed and the target building.

3. The method according to claim 2, wherein the clustering the image to be detected and the effective receptive field to obtain the crack detection image comprises:

acquiring a clustering sample set according to the image to be detected and the effective receptive fields; the clustering sample set comprises a plurality of clustering samples of the image to be detected and the crack to be confirmed;

acquiring a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set; the crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected;

and fusing the crack clustering result and the background clustering result to obtain the crack detection image.

4. The method according to claim 3, wherein the obtaining of the crack clustering result of the crack to be confirmed and the background clustering result of the background pixel according to the clustering sample set comprises:

acquiring a fracture centroid vector and a background centroid vector from a plurality of the clustered samples; the fracture centroid vector is a clustering sample which is the smallest from a fracture average value in a plurality of clustering samples, and the fracture average value is the average value of the plurality of clustering samples belonging to a fracture class; the background centroid vector is a clustering sample which is the smallest from a background average value in the clustering samples, and the background average value is the average value of the clustering samples except the crack class;

iterating the fracture centroid vector to converge according to a plurality of the clustering samples belonging to the fracture category to obtain the fracture clustering result;

and iterating the background centroid vector to converge according to the clustering samples except the crack classes to obtain the background clustering result.

5. The method according to any of claims 1-4, wherein said inputting said image to be detected to a crack detection model for obtaining a preliminary prediction image of said target structure comprises:

preprocessing the image to be detected to obtain a plurality of images to be processed; the pretreatment comprises any one or combination of the following: horizontal turning, vertical turning, mirror image, contrast adjustment and brightness adjustment;

inputting the multiple images to be processed into the crack detection model to obtain multiple characteristic graphs; the feature size of each feature map is different;

and fusing a plurality of feature maps to obtain the preliminary prediction image.

6. A crack detection device, comprising: the device comprises an acquisition module and a processing module;

the acquisition module is used for acquiring an image to be detected of the target building;

the processing module is used for inputting the image to be detected into a crack detection model so as to obtain a preliminary prediction image of the target building; the crack detection model is obtained according to a plurality of crack training images of the target building;

the processing module is further used for fusing the image to be detected and the preliminary prediction image to obtain a crack detection image of the target building; the crack detection image is used for determining the crack condition of the target building.

7. The apparatus according to claim 6, wherein the processing module is further configured to obtain at least one valid field of the preliminary prediction image; the effective receptive field is used for indicating an area with cracks to be confirmed in the target building;

the processing module is further used for clustering the image to be detected and the effective receptive field to obtain the crack detection image; the crack detection image comprises background pixels of the crack to be confirmed and the target building.

8. The apparatus according to claim 7, wherein the processing module is further configured to obtain a cluster sample set according to the image to be detected and a plurality of effective receptive fields; the clustering sample set comprises a plurality of clustering samples of the image to be detected and the crack to be confirmed;

the processing module is further used for acquiring a crack clustering result of the crack to be confirmed and a background clustering result of the background pixel according to the clustering sample set; the crack clustering result represents the distribution condition of the cracks to be confirmed in the image to be detected, and the background clustering result represents the background pixel information of the target building in the image to be detected;

the processing module is further used for fusing the crack clustering result and the background clustering result to obtain the crack detection image.

9. An electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor to perform the method of any one of claims 1 to 5.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1-5.