CN115375683A - Image processing-based waterlogging point detection method, system, storage medium and equipment - Google Patents

Image processing-based waterlogging point detection method, system, storage medium and equipment Download PDF

Info

Publication number
CN115375683A
CN115375683A CN202211299943.XA CN202211299943A CN115375683A CN 115375683 A CN115375683 A CN 115375683A CN 202211299943 A CN202211299943 A CN 202211299943A CN 115375683 A CN115375683 A CN 115375683A
Authority
CN
China
Prior art keywords
waterlogging
image
reference object
feature map
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211299943.XA
Other languages
Chinese (zh)
Inventor
李芬
冯祥胜
唐传师
何瑶
魏明明
胡沁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi Atmospheric Exploration Technology Center
Original Assignee
Jiangxi Atmospheric Exploration Technology Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangxi Atmospheric Exploration Technology Center filed Critical Jiangxi Atmospheric Exploration Technology Center
Priority to CN202211299943.XA priority Critical patent/CN115375683A/en
Publication of CN115375683A publication Critical patent/CN115375683A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration using local operators
    • G06T5/30Erosion or dilatation, e.g. thinning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/155Segmentation; Edge detection involving morphological operators
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a waterlogging point detection method, a waterlogging point detection system, a storage medium and a device based on image processing, wherein the conventional mobile phone has high popularization rate and is provided with a shooting function, a waterlogging point photo shot by the mobile phone is obtained by taking the mobile phone as a waterlogging point monitoring device, and accumulated water in the waterlogging point photo is identified, so that the waterlogging point detection mode is more intelligent and efficient, monitoring of waterlogging points in urban roads does not need to be carried out by relying on an observation station, specifically, by obtaining a shot picture uploaded by a user, when the shot picture is judged to be the waterlogging point position in a preset area, the shot picture is input into an image processing model and is subjected to post-processing to obtain a post-processed image, and finally, whether the post-processed image belongs to the category of waterlogging is judged, and waterlogging early warning information is sent out.

Description

Image processing-based waterlogging point detection method, system, storage medium and equipment
Technical Field
The invention belongs to the technical field of waterlogging point detection, and particularly relates to a waterlogging point detection method, a waterlogging point detection system, a storage medium and equipment based on image processing.
Background
Urban waterlogging disasters are natural disasters of accumulated water formed in low-lying places of urban terrains or under the conditions of unsmooth drainage and the like due to excessive runoff caused by short-duration heavy rainfall, continuous rainfall or large rainfall in the process, and the influencing factors are mainly heavy rainfall and urban drainage capacity.
Generally, if high-intensity rainstorm occurs, accumulated water can be formed at different positions of a city in a short time, namely, waterlogging points are formed, in order to monitor all the waterlogging points in the city, an observation station is usually installed at the waterlogging points and used for monitoring whether accumulated water exists at the waterlogging points, when the accumulated water at the waterlogging points is serious, broadcasting is carried out to remind people to go around to go out, although the method can effectively monitor all the waterlogging points in real time, the observation station needs to consume a large amount of manpower and material resources, the maintenance cost of a subsequent observation station is high, in addition, when the observation station breaks down, the accumulated water monitoring at the waterlogging points stops, waterlogging data can not be provided for users, and inconvenience is brought to the people to a certain extent when the observation station breaks down.
Disclosure of Invention
Based on the above, the invention provides a waterlogging point detection method, a waterlogging point detection system, a storage medium and equipment based on image processing, and aims to solve the problems that in the prior art, an observation station is relied on to monitor waterlogging points in urban roads, the cost is high, and waterlogging data cannot be provided for users in time when the observation station breaks down.
A first aspect of an embodiment of the present invention provides a waterlogging point detection method based on image processing, where the method includes:
pre-acquiring each waterlogging point position in a preset area and a feature reference object of each waterlogging point position, pre-establishing a mapping model of each waterlogging point position and the corresponding feature reference object, and establishing a feature reference object library according to all the feature reference objects;
acquiring a shot picture uploaded by a user, identifying a target reference object in the shot picture, and judging whether the target reference object exists in the characteristic reference object library;
if yes, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging segmentation image;
post-processing the waterlogging point segmentation image to obtain a post-processing image, and judging whether the post-processing image belongs to the waterlogging category or not;
if so, sending out waterlogging early warning information, wherein the waterlogging early warning information at least comprises the waterlogging category and the target waterlogging point position.
Further, the step of inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging-point segmentation image includes:
acquiring a historical waterlogging image, and marking and dividing a first waterlogging area, a second waterlogging area and a non-waterlogging area in the historical waterlogging image to obtain training set data, wherein the waterlogging depth of the first waterlogging area is greater than a first threshold and less than or equal to a second threshold, and the waterlogging depth of the second waterlogging area is greater than or equal to the second threshold;
inputting the training set data into an image segmentation network for training, wherein the training process is to input the training set data into a backbone network to obtain a feature map;
and sequentially inputting the feature maps into a position network and a path network to obtain corresponding position feature maps and path feature maps, and fusing the position feature maps and the path feature maps to obtain a historical waterlogging point segmentation image so as to complete the establishment of the image processing model.
Further, the location network may be represented as:
Figure 549167DEST_PATH_IMAGE001
wherein S is ji A prediction probability map representing the (j, i) th grid point, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Transforming to obtain the characteristic diagram of the ith column with the size of NxC j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j The feature map of the ith column with the size of C multiplied by N is obtained through conversion, E represents a position feature map, alpha represents a position feature parameter, C represents the path number of the feature map, H represents the height of the feature map, W represents the length of the feature map, and N represents the total column number of pixels in the feature map.
Further, the path network may be represented as:
Figure 614075DEST_PATH_IMAGE002
wherein, X ji Weight value representing the (j, i) th path, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Transforming to obtain the characteristic diagram of the ith column with the size of NxC j Is represented by A j Transforming to obtain a feature map of line j with size of C × N, D i Is represented by A j The feature map of the ith column with the size of C multiplied by N is obtained through conversion, F represents a path feature map, beta represents a path feature parameter, C represents the path number of the feature map, H represents the height of the feature map, and W represents the length of the feature map.
Further, the step of performing post-processing on the waterlogging point segmentation image to obtain a post-processing image, and judging whether the post-processing image belongs to the waterlogging category includes:
determining a pixel point region threshold according to the historical waterlogging point segmentation image, and filtering an interference region in the waterlogging point segmentation image according to the pixel point region threshold to obtain a target image;
and performing mathematical morphology operation on the target image to obtain a post-processing image with clear boundary.
Further, the steps of acquiring a shot picture uploaded by a user, identifying a target reference object in the shot picture, and judging whether the target reference object exists in the feature reference object library include:
when the target reference object is judged to be in the feature reference object library, acquiring the shooting time of the shot picture and meteorological data within a preset time period of the shooting time, and judging whether rainwater weather exists or not according to the meteorological data;
and if so, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging segmentation image.
Further, when it is determined that the target reference object exists in the feature reference object library, the step of acquiring the shooting time of the shot picture and the meteorological data within a preset time period of the shooting time, and determining whether rainwater weather exists according to the meteorological data includes:
acquiring the shooting time and uploading time of uploading the shot picture by a user, and judging whether the interval time between the shooting time and the uploading time exceeds preset time or not;
if not, acquiring meteorological data within a preset time period of the shooting time, and judging whether rainwater weather exists or not according to the meteorological data.
A second aspect of an embodiment of the present invention provides a waterlogging point detection system based on image processing, including:
the acquisition module is used for acquiring the positions of the waterlogging points and the characteristic references of the waterlogging points in a preset area in advance, establishing a mapping model of the waterlogging points and the corresponding characteristic references in advance, and establishing a characteristic reference library according to all the characteristic references;
the reference object judging module is used for acquiring shot pictures uploaded by a user, identifying a target reference object in the shot pictures and judging whether the target reference object exists in the characteristic reference object library or not;
the image processing module is used for inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position when the target reference object is judged to be in the feature reference object library, inputting the shot picture into the image processing model and outputting a waterlogging point segmentation image;
the waterlogging judging module is used for carrying out post-processing on the waterlogging point segmentation image to obtain a post-processing image and judging whether the post-processing image belongs to the waterlogging category or not;
and the sending module is used for sending out waterlogging early warning information when the post-processing image is judged to belong to the waterlogging category, wherein the waterlogging early warning information at least comprises the waterlogging category and the target waterlogging point position.
A third aspect of embodiments of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the image-processing-based waterlogging point detection method provided in the first aspect.
A fourth aspect of the embodiments of the present invention provides a waterlogging point detection apparatus based on image processing, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the waterlogging point detection method based on image processing provided in the first aspect when executing the computer program.
The waterlogging point detection method, the waterlogging point detection system, the storage medium and the waterlogging point detection equipment based on image processing have the following beneficial effects that:
the existing mobile phones are high in popularization rate and provided with shooting functions, a waterlogging point photo shot by the mobile phone is obtained by taking the mobile phone in the hand of a user as waterlogging point monitoring equipment, accumulated water in the waterlogging point photo is identified, the waterlogging point detection mode is more intelligent and efficient, the waterlogging point in an urban road does not need to be monitored by an observation station any more, specifically, the shot picture uploaded by the user is obtained, when the shot picture is judged to be the waterlogging point position in a preset area, the shot picture is input into an image processing model and is subjected to post-processing to obtain a post-processing image, and finally whether the post-processing image belongs to the waterlogging category or not is judged, if yes, waterlogging early warning information is sent out.
Drawings
Fig. 1 is a flowchart of an implementation of a waterlogging point detection method based on image processing according to a first embodiment of the present invention;
fig. 2 is a flowchart of an implementation of a waterlogging point detection method based on image processing according to a second embodiment of the present invention;
fig. 3 is a block diagram illustrating a waterlogging point detection system based on image processing according to a third embodiment of the present invention;
fig. 4 is a block diagram of a waterlogging point detection apparatus based on image processing according to a fourth embodiment of the present invention.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Several embodiments of the invention are presented in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Example one
Referring to fig. 1, fig. 1 illustrates a waterlogging point detection method based on image processing according to a first embodiment of the present invention, where the method specifically includes steps S01 to S05.
Step S01, pre-obtaining the positions of the waterlogging points and the characteristic reference objects of the waterlogging points in a preset area, pre-establishing a mapping model of the waterlogging points and the corresponding characteristic reference objects, and establishing a characteristic reference object library according to all the characteristic reference objects.
Wherein, in a city, often can have a plurality of waterlogging points, when meetting the rainwater weather of persistence or the heavy rainfall weather of short time promptly, this position all can have ponding, in order to detect these waterlogging points, need acquire the position of these waterlogging points, promptly the waterlogging point position, and this position is usually for showing with the longitude and latitude, location that so can be more accurate.
It should be noted that, because the image processing-based waterlogging point detection is performed, a photo needs to be taken at a waterlogging point, in order to determine whether a taken waterlogging point picture is a picture corresponding to the waterlogging point, a feature reference for each waterlogging point needs to be obtained in advance, wherein the feature reference is an object capable of representing the waterlogging point, such as a guideboard, a landmark building, a garden and the like, and a mapping model between the waterlogging point and the feature reference is established, that is, when a specific feature reference is identified in the waterlogging point picture, a specific waterlogging point position corresponding to the taken picture can be known, and in addition, the feature references obtained in advance are collected to establish a feature reference library, and the feature reference library is used for comparing with the feature reference in the feature library after a target reference in the taken picture is identified.
And S02, acquiring a shot picture uploaded by a user, identifying a target reference object in the shot picture, judging whether the target reference object exists in the feature reference object library, and if so, executing S03.
Specifically, when the target reference object in the shot picture is identified and compared with the feature reference object in the feature reference object library, and the feature reference object consistent with the target reference object exists in the feature reference object library, the shot picture is indicated to be the waterlogging position in the preset area.
And S03, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging point segmentation image.
Specifically, firstly, an image processing model needs to be established, in order to establish the image processing model, a large number of historical waterlogging point images are obtained, the historical waterlogging point images are urban waterlogging images which are actually acquired, and after preprocessing, training set data, verification set data and test set data are obtained, wherein the preprocessing aims at eliminating irrelevant information in the images, recovering useful real information, enhancing the detectability of relevant information and simplifying the data to the maximum extent, so that the reliability of feature extraction, image segmentation, matching and identification is improved.
Furthermore, after preparing training set data, verification set data and test set data, an image segmentation network is constructed, and it should be noted that the image segmentation network includes a location network and a path network, wherein, since misclassification of objects often occurs in semantic segmentation, introducing a location attention mechanism can effectively enhance feature representation capability by establishing rich context links on local features and performing feature representation on context information from a larger perspective. The location awareness mechanism enhances the capability of the network model to identify details and edges of the object through selective fusion of local features, and specifically, the location network can be expressed as:
Figure 403040DEST_PATH_IMAGE003
wherein S is ji A prediction probability map representing the (j, i) th grid point, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of NxC through transformation, wherein C is j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of C multiplied by N through conversion, wherein E represents a position characteristic diagram, alpha represents a position characteristic parameter, C represents the path number of the characteristic diagram, H represents the height of the characteristic diagram, W represents the length of the characteristic diagram, N represents the total column number of pixels in the characteristic diagram, and concretely, firstly, the characteristic diagram is obtained through 1 multiplied by 1 convolution operationThe number of paths is reduced to obtain a feature map a of size C × H × W, and then a size transformation is performed on a to obtain a feature map B of size N × C (C = H × W) and a feature map C and a feature map D of size C × N. Matrix multiplication and softmax operation of the feature maps B and C are performed to obtain a predicted probability map S for each point. Then, matrix multiplication of the feature map D and the prediction probability map S is performed to obtain a dimension transform. And finally, carrying out summation fusion on corresponding features of the feature map A to obtain a position feature map E with dimensions of C multiplied by H multiplied by W.
In addition, the deeper the element path graph in the network model, the faster it will respond to a particular category. The path attention module explicitly models the dependency relationship between paths, can highlight the feature mapping of mutual dependency, improve the feature representation of specific semantics, and enhance the overall recognition of the same category network model, so the path network can be represented as:
Figure 36147DEST_PATH_IMAGE004
wherein, X ji Weight value representing the (j, i) th path, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of NxC through transformation, wherein C is j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j Specifically, firstly, the number of paths is reduced through convolution operation of 1 × 1 to obtain a feature map A with the dimension of C × H × W, and then dimension transformation is performed on the feature map A to obtain a feature map B and a feature map D with the dimension of C × N (C = H × W), and a feature map C with the dimension of N × C. Matrix multiplication and softmax conversion are carried out on the characteristic diagram B and the characteristic diagram C to obtain a weight value X of each path, and then matrix multiplication is carried out on the characteristic diagram D and the weight value X to obtain dimension transformation. Finally, the corresponding features of the feature map A are summed and fused to obtain the dimension of C multiplied by H multiplied by WThe path profile F of (a).
After the image processing model is established, inputting the acquired shot picture into the image processing model to output the segmentation image of the waterlogging point, wherein the acquired shot picture can be shot by a mobile phone or a camera fixedly arranged at the waterlogging point.
And S04, performing post-processing on the waterlogging point segmentation image to obtain a post-processing image, judging whether the post-processing image belongs to the waterlogging category, and if so, executing the step S05.
It should be noted that, an image is segmented according to historical waterlogging points, a pixel point region threshold is determined, an interference region in the waterlogging point segmented image is filtered according to the pixel point region threshold, a target image is obtained, specifically, a reasonable threshold obtained by using priori knowledge is used for processing the segmented image, and a region predicted to be waterlogging and having a too small area is filtered, specifically, a main network in the image segmentation network adopts a ResNet50 with a hole convolution, and a size scaling ratio from an input image to a detected feature map is 8. In the present embodiment, the threshold is determined to be 8 times, that is, the region of the output image which is predicted to be waterlogging and less than 500 pixels, will not be treated as a waterlogging region.
Furthermore, mathematical morphology operation is carried out on the segmented image after threshold filtering, mainly corrosion re-expansion operation is carried out on the area predicted as waterlogging, isolated dots, burrs and bridges predicted as waterlogging are removed, and meanwhile the original shape and size of the area predicted as waterlogging are kept.
And step S05, sending out early warning information of waterlogging, wherein the early warning information of waterlogging at least comprises the type of waterlogging and the position of the target waterlogging point.
In summary, in the image processing-based waterlogging point detection method in the embodiments of the present invention, the captured image uploaded by the user is obtained, and when the captured image is judged to be the waterlogging point position in the preset area, the captured image is input into the image processing model, and is subjected to post-processing to obtain a post-processed image, and finally, whether the post-processed image belongs to the waterlogging category is judged, and if so, the waterlogging early warning information is sent.
Example two
Referring to fig. 2, fig. 2 shows a waterlogging point detection method based on image processing according to a second embodiment of the present invention, where the difference between the second embodiment and the first embodiment is that two pairs of information about taken pictures are identified to ensure that the input image processing model is a useful picture, and the method specifically includes steps S20 to S25.
And S20, pre-acquiring the waterlogging positions in a preset area and the characteristic reference objects of the waterlogging positions, pre-establishing a mapping model of the waterlogging positions and the corresponding characteristic reference objects, and establishing a characteristic reference object library according to all the characteristic reference objects.
Step S21, acquiring a shot picture uploaded by a user, identifying a target reference object in the shot picture, judging whether the target reference object exists in the feature reference object library, and if so, executing step S22.
Step S22, acquiring the shooting time of the shot picture and meteorological data within a preset time period of the shooting time, judging whether rainwater weather exists according to the meteorological data, and if so, executing step S23.
It should be noted that, in order to avoid uploading a past captured historical waterlogging point picture at the waterlogging point position by a user to predict the waterlogging point water level, providing false information, acquiring the capturing time of the waterlogging point picture, acquiring meteorological data within a preset time period of the capturing time according to the uploading time of the captured picture uploaded by the user, when the interval time between the uploading time and the capturing time is too long, not inputting the captured image into the image processing model for detection, when the interval time between the uploading time and the capturing time is within the preset time, judging whether rainwater weather exists according to the meteorological data, and if not, stopping inputting the captured image into the image processing model.
And S23, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging point segmentation image.
And S24, performing post-processing on the waterlogging point segmentation image to obtain a post-processing image, judging whether the post-processing image belongs to the waterlogging category, and if so, executing the step S25.
And S25, sending out waterlogging early warning information, wherein the waterlogging early warning information at least comprises the waterlogging category and the target waterlogging point position.
In summary, in the image processing-based waterlogging point detection method in the embodiments of the present invention, by obtaining a shot picture uploaded by a user, when the shot picture is judged to be a waterlogging point position in a preset area, the shot picture is input into an image processing model, post-processing is performed to obtain a post-processed image, and finally, whether the post-processed image belongs to the waterlogging category is judged, and if so, waterlogging early warning information is sent.
EXAMPLE III
Referring to fig. 3, fig. 3 is a block diagram illustrating a waterlogging point detection system 300 based on image processing according to a third embodiment of the present invention. The image-processing-based waterlogging point detection system 300 in the present embodiment includes units for performing the steps in the above-described embodiments. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 3, the image processing-based waterlogging point detection system 300 includes: the system comprises an acquisition module 31, a reference object judgment module 32, an image processing module 33, an inland inundation judgment module 34 and a sending module 35, wherein:
the acquisition module 31 is configured to acquire each waterlogging point position in a preset area and a feature reference at each waterlogging point position in advance, establish a mapping model between each waterlogging point position and the corresponding feature reference in advance, and establish a feature reference library according to all the feature references;
a reference object judgment module 32, configured to obtain a shot picture uploaded by a user, identify a target reference object in the shot picture, and judge whether the target reference object exists in the feature reference object library;
the image processing module 33 is configured to, when it is determined that the target reference object exists in the feature reference object library, input the target reference object into the mapping model to obtain a corresponding target waterlogging point position, input the captured image into the image processing model, and output a waterlogging point segmentation image;
the waterlogging judging module 34 is configured to perform post-processing on the waterlogging point segmentation image to obtain a post-processing image, and judge whether the post-processing image belongs to the waterlogging category;
and the sending module 35 is configured to send out waterlogging early warning information when it is determined that the post-processing image belongs to the waterlogging category, where the waterlogging early warning information at least includes the waterlogging category and the target waterlogging point position.
Further, in some optional embodiments of the present invention, the image processing-based waterlogging point detection system 300 further includes:
the training set data acquisition module is used for acquiring a historical waterlogging image, and marking and dividing a first ponding area, a second ponding area and a non-ponding area in the historical waterlogging image to obtain training set data, wherein the ponding depth of the first ponding area is greater than a first threshold value and less than or equal to a second threshold value, and the ponding depth of the second ponding area is greater than or equal to the second threshold value;
the training module is used for inputting the training set data into an image segmentation network for training, wherein the training process is to input the training set data into a backbone network to obtain a characteristic diagram;
the image processing model establishing module is used for inputting the feature map into a position network and a path network in sequence to obtain a corresponding position feature map and a corresponding path feature map, and fusing the position feature map and the path feature map to obtain a historical waterlogging point segmentation image so as to complete establishment of the image processing model, wherein the position network can be expressed as:
Figure 543351DEST_PATH_IMAGE005
wherein S is ji A prediction probability map representing the (j, i) th lattice point, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of NxC through transformation, wherein C is j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of C multiplied by N through conversion, wherein E represents a position characteristic diagram, alpha represents a position characteristic parameter, C represents the path number of the characteristic diagram, H represents the height of the characteristic diagram, W represents the length of the characteristic diagram, and N represents the total column number of pixels in the characteristic diagram;
the path network may be represented as:
Figure 146371DEST_PATH_IMAGE006
wherein, X ji Weight value representing the (j, i) th path, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Transforming to obtain the characteristic diagram of the ith column with the size of NxC j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j The feature map of the ith column with the size of C multiplied by N is obtained through conversion, F represents a path feature map, beta represents a path feature parameter, C represents the path number of the feature map, H represents the height of the feature map, and W represents the length of the feature map.
Further, in some optional embodiments of the present invention, the waterlogging judging module 34 includes:
the filtering unit is used for determining a pixel point region threshold according to the historical waterlogging point segmentation image, and filtering an interference region in the waterlogging point segmentation image according to the pixel point region threshold to obtain a target image;
and the operation unit is used for performing mathematical morphology operation on the target image to obtain a post-processing image with clear boundary.
Further, in some optional embodiments of the present invention, the image processing-based waterlogging point detection system 300 further includes:
and the rainwater weather judging module is used for acquiring the shooting time of the shot picture and the meteorological data within the preset time period of the shooting time when judging that the target reference object exists in the characteristic reference object library, and judging whether rainwater weather exists according to the meteorological data, wherein when the judgment result of the rainwater weather judging module is yes, the step of inputting the target reference object into the mapping model to obtain the corresponding target waterlogging position, inputting the shot picture into the image processing model and outputting a waterlogging point segmentation image is executed.
Further, in some optional embodiments of the present invention, the rainwater weather determination module includes:
the first judgment unit is used for acquiring the shooting time and the uploading time of the shot picture uploaded by the user, and judging whether the interval time between the shooting time and the uploading time exceeds the preset time, wherein when the judgment result of the first judgment unit is negative, the weather data in the preset time period of the shooting time is acquired, and whether rainwater weather exists is judged according to the weather data.
Example four
In another aspect, the present invention further provides an image-processing-based waterlogging point detection apparatus, please refer to fig. 4, which shows an image-processing-based waterlogging point detection apparatus according to a fourth embodiment of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor, where when the processor 10 executes the computer program 30, an image-processing-based waterlogging point detection method is implemented.
The processor 10 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor or other data Processing chip in some embodiments, and is used to execute program codes stored in the memory 20 or process data, such as executing an access restriction program.
The memory 20 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 20 may in some embodiments be an internal storage unit of an image processing based waterlogging detection device, such as a hard disk of the image processing based waterlogging detection device. The memory 20 may also be an external storage device of the image processing-based waterlogging detection device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the image processing-based waterlogging detection device. Further, the memory 20 may also include both an internal storage unit and an external storage device of a waterlogging point detection apparatus based on image processing. The memory 20 can be used not only to store application software of a waterlogging point detection apparatus based on image processing and various types of data, but also to temporarily store data that has been output or is to be output.
It is noted that the configuration shown in fig. 4 does not constitute a limitation of an image processing based waterlogging detection apparatus, which in other embodiments may comprise fewer or more components than shown, or combine certain components, or a different arrangement of components.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the waterlogging point prediction method as described above.
Those of skill in the art will understand that the logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be viewed as implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent should be subject to the appended claims.

Claims (10)

1. A waterlogging point detection method based on image processing is characterized by comprising the following steps:
pre-acquiring each waterlogging point position in a preset area and a feature reference object of each waterlogging point position, pre-establishing a mapping model of each waterlogging point position and the corresponding feature reference object, and establishing a feature reference object library according to all the feature reference objects;
acquiring a shot picture uploaded by a user, identifying a target reference object in the shot picture, and judging whether the target reference object exists in the characteristic reference object library;
if yes, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging segmentation image;
performing post-processing on the waterlogging point segmentation image to obtain a post-processing image, and judging whether the post-processing image belongs to the waterlogging category or not;
if yes, sending out waterlogging early warning information, wherein the waterlogging early warning information at least comprises the waterlogging category and the target waterlogging point position.
2. The image processing-based waterlogging point detection method according to claim 1, wherein the step of inputting the target reference object into the mapping model to obtain a corresponding target waterlogging point position, inputting the shot picture into the image processing model, and outputting a waterlogging point segmentation image comprises:
acquiring a historical waterlogging image, and marking and dividing a first ponding area, a second ponding area and a non-ponding area in the historical waterlogging image to obtain training set data, wherein the ponding depth of the first ponding area is greater than a first threshold value and less than or equal to a second threshold value, and the ponding depth of the second ponding area is greater than or equal to the second threshold value;
inputting the training set data into an image segmentation network for training, wherein the training process is to input the training set data into a backbone network to obtain a feature map;
and sequentially inputting the feature map into a position network and a path network to obtain a corresponding position feature map and a corresponding path feature map, and fusing the position feature map and the path feature map to obtain a historical waterlogging point segmentation image so as to complete the establishment of the image processing model.
3. The image-processing-based waterlogging detection method of claim 2, wherein said location network can be expressed as:
Figure 951682DEST_PATH_IMAGE001
wherein S is ji A prediction probability map representing the (j, i) th grid point, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Obtaining a characteristic diagram of an ith column with the size of NxC through transformation, wherein C is j Is represented by A j Transforming to obtain a feature map of line j with size C × N, D i Is represented by A j The feature map of the ith column with the size of C multiplied by N is obtained through conversion, E represents a position feature map, alpha represents a position feature parameter, C represents the path number of the feature map, H represents the height of the feature map, W represents the length of the feature map, and N represents the total column number of pixels in the feature map.
4. The image processing-based waterlogging detection method of claim 2, wherein the path network can be expressed as:
Figure 272942DEST_PATH_IMAGE002
wherein, X ji Weight value representing the (j, i) th path, A j Feature map of j-th row with size C × H × W, B i Is represented by A j Transforming to obtain the characteristic diagram of the ith column with the size of NxC j Is represented by A j Transforming to obtain a feature map of line j with size of C × N, D i Is represented by A j Transforming to obtain sizeC × N ith row feature map, F path feature map, β path feature parameter, C number of paths in feature map, H height of feature map, and W length of feature map.
5. The image processing-based waterlogging point detection method according to claim 2, wherein the step of performing post-processing on the waterlogging point segmentation image to obtain a post-processed image, and judging whether the post-processed image belongs to the waterlogging category comprises:
determining a pixel point region threshold according to the historical waterlogging point segmentation image, and filtering an interference region in the waterlogging point segmentation image according to the pixel point region threshold to obtain a target image;
and performing mathematical morphology operation on the target image to obtain a post-processing image with clear boundary.
6. The image-processing-based waterlogging point detection method of claim 2, wherein the steps of obtaining a picture taken uploaded by a user, identifying a target reference object in the picture taken, and determining whether the target reference object exists in the feature reference object library, are followed by:
when the target reference object is judged to be in the feature reference object library, acquiring the shooting time of the shot picture and meteorological data within a preset time period of the shooting time, and judging whether rainwater weather exists or not according to the meteorological data;
and if so, inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position, inputting the shot picture into an image processing model, and outputting a waterlogging segmentation image.
7. The image-processing-based waterlogging point detection method of claim 6, wherein said step of obtaining a shooting time of said shot picture and meteorological data within a preset time period of said shooting time when said target reference object is judged to be present in said feature reference object library, and judging whether there is rainy weather according to said meteorological data comprises:
acquiring the shooting time and uploading time of uploading the shot picture by a user, and judging whether the interval time between the shooting time and the uploading time exceeds preset time or not;
if not, acquiring meteorological data within a preset time period of the shooting time, and judging whether rainwater weather exists or not according to the meteorological data.
8. A waterlogging point detection system based on image processing, the system comprising:
the acquisition module is used for acquiring the positions of the waterlogging points and the characteristic references of the waterlogging points in a preset area in advance, establishing a mapping model of the waterlogging points and the corresponding characteristic references in advance, and establishing a characteristic reference library according to all the characteristic references;
the reference object judging module is used for acquiring shot pictures uploaded by a user, identifying a target reference object in the shot pictures and judging whether the target reference object exists in the characteristic reference object library or not;
the image processing module is used for inputting the target reference object into the mapping model to obtain a corresponding target waterlogging position when the target reference object is judged to be in the feature reference object library, inputting the shot picture into the image processing model and outputting a waterlogging point segmentation image;
the waterlogging judging module is used for carrying out post-processing on the waterlogging point segmentation image to obtain a post-processing image and judging whether the post-processing image belongs to the waterlogging category or not;
and the sending module is used for sending out waterlogging early warning information when the post-processing image is judged to belong to the waterlogging category, wherein the waterlogging early warning information at least comprises the waterlogging category and the target waterlogging point position.
9. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out a method for image-processing-based waterlogging point detection as set forth in any one of claims 1-7.
10. An image processing based waterlogging point detection apparatus, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing an image processing based waterlogging point detection method according to any one of claims 1-7 when executing the program.
CN202211299943.XA 2022-10-24 2022-10-24 Image processing-based waterlogging point detection method, system, storage medium and equipment Pending CN115375683A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211299943.XA CN115375683A (en) 2022-10-24 2022-10-24 Image processing-based waterlogging point detection method, system, storage medium and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211299943.XA CN115375683A (en) 2022-10-24 2022-10-24 Image processing-based waterlogging point detection method, system, storage medium and equipment

Publications (1)

Publication Number Publication Date
CN115375683A true CN115375683A (en) 2022-11-22

Family

ID=84072881

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211299943.XA Pending CN115375683A (en) 2022-10-24 2022-10-24 Image processing-based waterlogging point detection method, system, storage medium and equipment

Country Status (1)

Country Link
CN (1) CN115375683A (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345815A (en) * 2013-06-08 2013-10-09 清华大学 Urban storm flood monitoring and traffic controlling and guiding system and method
CN104281646A (en) * 2014-08-28 2015-01-14 浙江大学 Urban water-logging detecting method on basis of microblog data
CN109801327A (en) * 2018-12-21 2019-05-24 杭州电子科技大学 A kind of urban waterlogging depth of accumulated water information extracting method based on video data
CN110501044A (en) * 2019-08-20 2019-11-26 石家庄学院 A kind of urban waterlogging monitoring system and its monitoring method
CN110893828A (en) * 2018-09-13 2020-03-20 沈阳美行科技有限公司 Method and device for early warning of surface water accumulation
CN111144254A (en) * 2019-12-18 2020-05-12 江河瑞通(北京)技术有限公司 Urban waterlogging degree identification method and device
CN111754564A (en) * 2019-03-28 2020-10-09 杭州海康威视***技术有限公司 Video display method, device, equipment and storage medium
CN114119776A (en) * 2022-01-26 2022-03-01 江西省大气探测技术中心 Urban waterlogging point management method, system, storage medium and equipment
CN114220079A (en) * 2021-12-22 2022-03-22 中国第一汽车股份有限公司 Method and device for identifying surface water, computer equipment and storage medium
CN114332487A (en) * 2021-12-31 2022-04-12 北京精英路通科技有限公司 Image-based accumulated water early warning method, device, equipment, storage medium and product
CN114663827A (en) * 2022-02-28 2022-06-24 北京慧图科技(集团)股份有限公司 Method and system for judging depth of waterlogging in scene
CN114821340A (en) * 2022-06-01 2022-07-29 河南大学 Land utilization classification method and system
CN114838784A (en) * 2022-04-12 2022-08-02 中国市政工程华北设计研究总院有限公司 Lower bridge water level monitoring and early warning system based on rainfall ponding attitude analysis

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345815A (en) * 2013-06-08 2013-10-09 清华大学 Urban storm flood monitoring and traffic controlling and guiding system and method
CN104281646A (en) * 2014-08-28 2015-01-14 浙江大学 Urban water-logging detecting method on basis of microblog data
CN110893828A (en) * 2018-09-13 2020-03-20 沈阳美行科技有限公司 Method and device for early warning of surface water accumulation
CN109801327A (en) * 2018-12-21 2019-05-24 杭州电子科技大学 A kind of urban waterlogging depth of accumulated water information extracting method based on video data
CN111754564A (en) * 2019-03-28 2020-10-09 杭州海康威视***技术有限公司 Video display method, device, equipment and storage medium
CN110501044A (en) * 2019-08-20 2019-11-26 石家庄学院 A kind of urban waterlogging monitoring system and its monitoring method
CN111144254A (en) * 2019-12-18 2020-05-12 江河瑞通(北京)技术有限公司 Urban waterlogging degree identification method and device
CN114220079A (en) * 2021-12-22 2022-03-22 中国第一汽车股份有限公司 Method and device for identifying surface water, computer equipment and storage medium
CN114332487A (en) * 2021-12-31 2022-04-12 北京精英路通科技有限公司 Image-based accumulated water early warning method, device, equipment, storage medium and product
CN114119776A (en) * 2022-01-26 2022-03-01 江西省大气探测技术中心 Urban waterlogging point management method, system, storage medium and equipment
CN114663827A (en) * 2022-02-28 2022-06-24 北京慧图科技(集团)股份有限公司 Method and system for judging depth of waterlogging in scene
CN114838784A (en) * 2022-04-12 2022-08-02 中国市政工程华北设计研究总院有限公司 Lower bridge water level monitoring and early warning system based on rainfall ponding attitude analysis
CN114821340A (en) * 2022-06-01 2022-07-29 河南大学 Land utilization classification method and system

Similar Documents

Publication Publication Date Title
CN109033950B (en) Vehicle illegal parking detection method based on multi-feature fusion cascade depth model
CN101807352B (en) Method for detecting parking stalls on basis of fuzzy pattern recognition
CN110533950A (en) Detection method, device, electronic equipment and the storage medium of parking stall behaviour in service
CN110826429A (en) Scenic spot video-based method and system for automatically monitoring travel emergency
CN112666553B (en) Road ponding identification method and equipment based on millimeter wave radar
CN114332513B (en) New energy automobile abnormal parking amplification data detection method for smart city
CN111723773B (en) Method and device for detecting carryover, electronic equipment and readable storage medium
CN113627299B (en) Wire floater intelligent recognition method and device based on deep learning
CN106339657A (en) Straw incineration monitoring method and device based on monitoring video
CN112084892B (en) Road abnormal event detection management device and method thereof
CN114299457A (en) Accumulated water depth detection method and device
CN110852236A (en) Target event determination method and device, storage medium and electronic device
CN113326783A (en) Edge early warning method for water conservancy industry
CN115359392A (en) Roadside construction behavior detection method based on improved YOLOv5 model
CN115588047A (en) Three-dimensional target detection method based on scene coding
CN111239131A (en) AI intelligent water environmental protection real-time monitoring platform
CN113963314A (en) Rainfall monitoring method and device, computer equipment and storage medium
CN115375683A (en) Image processing-based waterlogging point detection method, system, storage medium and equipment
CN116311039A (en) Urban precipitation inversion method, device, computer equipment and storage medium
CN116843946A (en) Tunnel rock mass main structural surface identification method and device based on image identification
Ashqer et al. Parking lot space detection based on image processing
CN113902739B (en) NUT wire clamp defect identification method, device and equipment and readable storage medium
CN107886049B (en) Visibility recognition early warning method based on camera probe
CN113793069A (en) Urban waterlogging intelligent identification method of deep residual error network
CN115294774A (en) Non-motor vehicle road illegal parking detection method and device based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination