CN117346846B - Automatic correction type water measuring weir flow photographic monitoring method and device - Google Patents
Automatic correction type water measuring weir flow photographic monitoring method and device Download PDFInfo
- Publication number
- CN117346846B CN117346846B CN202311219793.1A CN202311219793A CN117346846B CN 117346846 B CN117346846 B CN 117346846B CN 202311219793 A CN202311219793 A CN 202311219793A CN 117346846 B CN117346846 B CN 117346846B
- Authority
- CN
- China
- Prior art keywords
- weir
- image
- target
- water
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 268
- 238000012937 correction Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012544 monitoring process Methods 0.000 title claims abstract description 45
- 230000011218 segmentation Effects 0.000 claims abstract description 61
- 238000003709 image segmentation Methods 0.000 claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 230000007246 mechanism Effects 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000002372 labelling Methods 0.000 claims description 15
- 238000012806 monitoring device Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 6
- 238000003702 image correction Methods 0.000 claims description 4
- 230000001133 acceleration Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 abstract description 2
- 239000011159 matrix material Substances 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004568 cement Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 229910052744 lithium Inorganic materials 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000013049 sediment Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01F—MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
- G01F1/00—Measuring the volume flow or mass flow of fluid or fluent solid material wherein the fluid passes through a meter in a continuous flow
- G01F1/002—Measuring the volume flow or mass flow of fluid or fluent solid material wherein the fluid passes through a meter in a continuous flow wherein the flow is in an open channel
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C13/00—Surveying specially adapted to open water, e.g. sea, lake, river or canal
- G01C13/002—Measuring the movement of open water
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01F—MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
- G01F23/00—Indicating or measuring liquid level or level of fluent solid material, e.g. indicating in terms of volume or indicating by means of an alarm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Fluid Mechanics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Hydrology & Water Resources (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of hydrologic tests, in particular to an automatic-correction water weir flow photography monitoring method and device, comprising the steps of performing perspective correction on an original water gauge image, and performing weir groove inner wall segmentation processing by utilizing a SAM image segmentation model fused with a VIT model and a cross attention mechanism module to obtain a target weir groove inner wall segmentation result; obtaining a target weir groove water level according to a target weir groove inner wall segmentation result and a weir groove water level identification model; and obtaining the flow of the water measuring weir by using the water level flow relation model and the target weir groove water level. According to the invention, the SAM image segmentation model and the YOLO v5 target detection model are used for automatically correcting and identifying the weir groove water gauge image, so that the problems of high cost and limited stability of the conventional water gauge flow monitoring method are solved, and the method has good segmentation and identification capability on the water gauge or the inner wall of the weir groove under various environments or observation angles, improves the accuracy and efficiency of identification and monitoring, and has good universality.
Description
Technical Field
The invention relates to the technical field of hydrologic tests, in particular to a method and a device for automatically-corrected water weir flow photography monitoring.
Background
The water measuring weir is a common device for measuring water flow, and the working principle is that a water level gauge or various ultrasonic open channel flowmeters, radar fluviographs and the like are usually used for monitoring the water level of the water measuring weir at present, wherein the float fluviographs are stable and reliable, have better measuring precision, but usually need to build a water level logging and have higher cost; the ultrasonic flowmeter and the radar water level gauge are high in price and are easy to be interfered by water environments such as sediment, water temperature and aquatic organisms.
In recent years, a non-contact type measuring weir flow monitoring method based on image processing has been attracting attention, such as: the patent with publication number CN115638835A discloses a method and a device for automatically monitoring the flow of a water measuring weir based on an image recognition algorithm, wherein circular edge detection is adopted for a disc image of a float type water level device, and the opening operation and Hough straight line detection are carried out on boundary lines to obtain a connecting rod angle, so that the flow is calculated; the patent with publication number CN113237534A discloses a rotary disk type water weir water level monitoring system, a YOLO v3 convolutional neural network is built by a system monitoring control module, the number of isosceles triangles exposed out of the water surface from a water level ruler is detected, and the water level and the flow of a weir groove can be calculated according to the real height of the triangles, but the incomplete triangle exposed out of the water surface cannot be accurately identified by the method; hu Minquan et al introduce a novel water level in the automatic measuring technique of water head on a water weir based on machine vision, and propose an automatic measuring algorithm in cooperation with the water level, however, the identification accuracy of the non-contact water weir flow monitoring method or device based on image processing is often limited by environmental conditions (illumination, shadow, float grass, biological residues and the like), and the generalization performance is limited; meanwhile, error correction cannot be realized, once the marker is offset, the identification accuracy is obviously reduced, the stability is limited, and the monitoring cost is high.
Disclosure of Invention
The invention provides a method and a device for automatically-corrected water weir flow photography monitoring, which solve the technical problems that the existing water weir flow monitoring cost is high and the stability of an image water level identification method is limited.
In order to solve the technical problems, the invention provides a method and a device for monitoring the flow of a water weir in a photographic way, which can be automatically corrected.
In a first aspect, the present invention provides an automatically modifiable water weir flow photography monitoring method, the method comprising the steps of:
Acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image;
Labeling foreground points and background points in the perspective correction image to generate labeling point information;
Acquiring a pre-constructed SAM image segmentation model fused with a VIT model and a cross attention mechanism module, inputting the perspective correction image and the annotation point information into the SAM image segmentation model for weir-notch inner wall segmentation processing, and obtaining a target weir-notch inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information;
obtaining a target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model;
And calculating to obtain the flow of the water measuring weir by using the water level flow relation model and the target weir groove water level.
In a further embodiment, before the step of obtaining the target weir groove water level according to the target weir groove inner wall segmentation result and the pre-established weir groove water level identification model, the method further includes performing offset detection on the water gauge according to the target weir groove inner wall segmentation result, and when the water gauge is offset, correcting the mark point information by using a pre-trained YOLO v5 target detection model, where the step of correcting the mark point information by using the pre-trained YOLO v5 target detection model specifically includes:
identifying the perspective correction image by using a pre-trained YOLO v5 target detection model to obtain a water gauge offset position coordinate;
Calculating to obtain the water gauge offset according to the water gauge offset position coordinates and the water gauge standard position coordinates;
correcting the marking point information according to the water gauge offset, generating corrected marking point information, and re-dividing the original water gauge image according to the corrected marking point information through the SAM image segmentation model, and dynamically updating the target weir groove inner wall segmentation result.
In a further embodiment, the step of performing offset detection on the water gauge according to the target weir groove inner wall segmentation result includes:
And determining the boundary line between the water measuring weir groove and the water gauge according to the dividing result of the inner wall of the target weir groove, detecting the deviation of the water gauge by utilizing the average included angle between the boundary line between the water measuring weir groove and the water gauge and the plumb line direction, and judging that the water gauge deviates if the average included angle is not within the preset water gauge included angle range.
In a further embodiment, the step of performing perspective correction on the original water gauge image to obtain a perspective corrected image further includes:
And detecting whether the region with the pixel missing value exists in the perspective correction image, and if so, performing image interpolation by adopting a nearest neighbor interpolation method or a Lagrange interpolation method.
In a further embodiment, the step of inputting the perspective correction image and the annotation point information into the SAM image segmentation model to perform a weir groove inner wall segmentation process, and obtaining a target weir groove inner wall segmentation result includes:
Extracting features of the perspective correction image through a VIT model to obtain image features;
Traversing the image features, and decoding and calculating the image features and the mark point information through a cross attention mechanism to obtain decoded image features and mask features;
And carrying out convolution, up-sampling and multi-layer perceptron processing on the decoded image features by using the SAM image segmentation model to obtain final image features, wherein the mask features are subjected to dimension adjustment by a multi-layer perceptron to be consistent with the final image features, and the final image features are multiplied by the mask features to obtain a target weir groove inner wall segmentation result.
In a further embodiment, the step of obtaining the target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model comprises the following steps:
calculating the pixel area of the inner wall of the target weir groove by utilizing the dividing result of the inner wall of the target weir groove;
inputting the pixel area of the inner wall of the target weir groove into a pre-established weir groove water level identification model to obtain the target weir groove water level, wherein the weir groove water level identification model is as follows:
L＝h-k·S
wherein L represents a target weir trough water level; h. k represents regression coefficients; s represents the pixel area of the inner wall of the target weir groove.
In a further embodiment, the water level flow relationship model is:
Wherein Q represents the flow rate of the water measuring weir; c e represents the flow experience coefficient; θ represents the V-shaped weir apex angle; g represents gravitational acceleration; h e represents the height of the V-shaped weir apex angle; l represents the target weir trough water level.
In a second aspect, the present invention provides an automatically modifiable water-measuring weir flow photographic monitoring device, the device comprising:
The image correction module is used for acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image;
the image labeling module is used for labeling foreground points and background points in the perspective correction image and generating labeling point information;
The image segmentation module is used for acquiring a pre-constructed SAM image segmentation model fused with the VIT model and the cross attention mechanism module, inputting the perspective correction image and the mark point information into the SAM image segmentation model for weir groove inner wall segmentation processing, and obtaining a target weir groove inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information;
the water level identification module is used for obtaining a target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model;
and the flow monitoring module is used for calculating the flow of the water measuring weir by utilizing the water level and flow relation model and the target water level of the weir groove.
The invention provides a method and a device for monitoring flow of a water weir capable of being automatically corrected, wherein the method carries out perspective correction on an original water gauge image, and carries out weir groove inner wall segmentation processing by utilizing a SAM image segmentation model fused with a VIT model and a cross attention mechanism module to obtain a target weir groove inner wall segmentation result; obtaining a target weir groove water level according to a target weir groove inner wall segmentation result and a pre-established weir groove water level identification model; and calculating to obtain the flow of the water measuring weir by using the water level flow relation model and the target weir groove water level. Compared with the prior art, the method adopts the SAM model and the YOLO v5 target detection model to establish an automatically-corrected weir groove water level image recognition model, and simultaneously utilizes the relationship model of the water level and the flow of the water measuring weir to obtain a real-time flow value, so that the method has good recognition performance and adaptability to the weir groove water level and the flow under various environments or observation angles, improves the accuracy and stability of monitoring the water measuring weir flow, is beneficial to long-term continuous flow monitoring, and has the advantages of high monitoring efficiency, low cost, wide application range and the like.
Drawings
FIG. 1 is a schematic flow chart of an automatically modifiable water weir flow photography monitoring method provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of a water weir flow monitoring device according to an embodiment of the present invention;
FIG. 3 is a perspective corrected image schematic diagram provided by an embodiment of the present invention;
FIG. 4 is a schematic view of a perspective corrected image of a foreground point and a background point marked as provided by an embodiment of the present invention;
FIG. 5 is a diagram of a binary mask in the target weir groove inner wall segmentation result according to the embodiment of the present invention;
FIG. 6 is a diagram of a weir trough image with cue points and a mask output by a SAM image segmentation model according to an embodiment of the present invention;
FIG. 7 is a block diagram of an automatically modifiable water weir flow camera monitoring device according to embodiments of the present invention.
Detailed Description
The following examples are given for the purpose of illustration only and are not to be construed as limiting the invention, including the drawings for reference and description only, and are not to be construed as limiting the scope of the invention as many variations thereof are possible without departing from the spirit and scope of the invention.
Referring to fig. 1, an embodiment of the present invention provides an automatically modifiable water-measuring weir flow photographic monitoring method, applied to a water-measuring weir flow monitoring device, as shown in fig. 1, the method includes the following steps:
S1, acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image.
As shown in fig. 2, in this embodiment, a water gauge is installed on the channel or the inner wall of the water tank of the water measuring weir in the direction perpendicular to the water surface, and a column is fixed at a position of a proper distance from one side of the water measuring weir to the water gauge, and a delay monitoring camera and a solar power panel are erected on the column, wherein the power of the solar power panel is 100W, a 60AH lithium battery is equipped, and the solar power panel is used for supplying power to the monitoring camera; the camera is preferably selected to be a model of a pupil YT-CAM8008LA-4G camera, 800 ten thousand pixels are clear, fixed focus is 4mm, a 4G communication module is built in, the camera is about 3m away from the ground, a 60-degree dip scale is formed, and the size of a shot picture is 3 multiplied by 3840 multiplied by 2160; it should be noted that, in order to clearly see the scale of the water gauge from the monitoring picture and accurately read the water level, in this embodiment, the camera lens angle needs to be adjusted, so that the full length of the water gauge is located in the monitoring picture of the camera and is located in the center of the picture as much as possible, meanwhile, the embodiment sets the shooting interval of the delay camera to 5 minutes, the camera shoots the water gauge picture at regular time and uploads the picture to the cloud server synchronously, the picture processing module is transmitted to calculate the water level, and the monitoring personnel or the spot check personnel can view and download the picture of the water gauge from the cloud.
Since perspective distortion exists in a water gauge picture photographed in the dip mode, which means that in an image coordinate system, the real world length of the water gauge corresponding to the unit pixel is inconsistent, after the original water gauge image is photographed by the monitoring camera, the photographed original water gauge image needs to be subjected to perspective correction, and the photographed original water gauge image is corrected to the water gauge forward-looking direction, so that a perspective correction image is obtained, wherein the calculation formula of perspective correction is as follows:
Wherein, (x, y) represents the pixel point coordinates of the original water gauge image; (m, n) represents pixel coordinates of the perspective correction image; m represents a spatial perspective transformation matrix; a 0、a1、a2、a3 denotes a linear transformation parameter; b 0、b1 denotes a translation parameter; c 0、c1 denotes perspective transformation parameters.
In this embodiment, since the spatial perspective transformation matrix M has 8 unknowns, at least 4 sets of corresponding points before and after correction are needed to be solved, in this embodiment, 4 points are selected on the captured original water gauge image, the coordinates are (1817.0, 966.4), (1856.0, 966.5), (1806.0, 1457.2), and (1841.4, 1457.6), respectively, corresponding to intersections of scale lines of "9.5" and "0.5" of the water gauge and left and right edges of the water gauge, the coordinates of the four corrected points are (450, 300), (550, 300), (450, 1600), and (550, 1600), respectively, and straight lines formed by the left and right edges in the image coordinate system are vertical lines, and the perspective corrected image obtained after the spatial perspective correction is as shown in fig. 3, in this embodiment, the size of the perspective corrected image is 3×1000×2000.
After perspective correction, if a partial region has a missing pixel value, the pixel value of the nearest non-missing point is copied to the point or lagrangian interpolation is adopted to obtain an image after missing value repair, which specifically comprises the following steps: and detecting whether the region with the pixel missing value exists in the perspective correction image, and if so, performing image interpolation by adopting a nearest neighbor interpolation method or a Lagrange interpolation method.
S2, labeling foreground points and background points in the perspective correction image, and generating labeling point information.
In the embodiment, a prompting point is marked on a perspective correction image, background indication information is given to the prompting point, the background indication information refers to whether the prompting point is a foreground prompting point (input is 1) or a background prompting point (input is 0), as shown in fig. 4, the embodiment preferentially marks three prompting points on the perspective correction image, wherein two prompting points are positioned on the inner wall of a weir groove, coordinates are (171, 896) and (880, 1103), and the two prompting points are foreground points and are areas focused or interested in the embodiment; the third cue point is located on the water gauge, and coordinates are 475, 800, and the third cue point is a background point and is an irrelevant area in the embodiment.
S3, acquiring a pre-constructed SAM image segmentation model fused with a VIT model and a cross attention mechanism module, inputting the perspective correction image and the mark point information into the SAM image segmentation model for weir groove inner wall segmentation treatment, and obtaining a target weir groove inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross-attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information.
In this embodiment, the step of inputting the perspective correction image and the labeling point information into the SAM image segmentation model to perform segmentation processing, and obtaining a target weir groove inner wall segmentation result includes:
Extracting features of the perspective correction image through a VIT model to obtain image features;
Traversing the image features, and decoding and calculating the image features and the mark point information through a cross attention mechanism to obtain decoded image features and mask features;
And carrying out convolution, up-sampling and multi-layer perceptron processing on the decoded image features by using the SAM image segmentation model to obtain final image features, wherein the mask features are subjected to dimension adjustment by a multi-layer perceptron to be consistent with the final image features, and the mask features and the final image features are multiplied to obtain a target mask, so that a target weir groove inner wall segmentation result is obtained.
Specifically, considering that the segmentation of the water gauge pixel region is difficult to identify with naked eyes due to the interference of water gauge reflection on the segmentation result, and the water surface line between the inner wall of a weir groove where the water gauge leans against and the water body is noticeable to be white, the recognition of the water surface line by the model is facilitated, for this purpose, the inner wall is mask segmented by using a SAM image segmentation model, wherein the SAM vision model fuses a plurality of texts or vision classical models with a transducer module, the image segmentation is performed on the target region based on the prompt engineering, the image and the annotation point information (annotation point, annotation frame or text semantics) are encoded by using a VIT model (Vision Transformer, vision self-attention model) and a Bert model respectively, and the pixel point or region highly related to the annotation point information is calculated by adopting a cross attention mechanism.
The SAM image segmentation model traverses all image features obtained by the VIT model, calculates whether each image feature is related to the mark point information through a cross attention mechanism, wherein whether each image feature is related to the mark point information is reflected by a weight matrix A, the weight matrix A obtained through softmax calculation can reflect the relative importance degree of each feature, the bigger the value is, the more relevant is, the smaller the value is, the less relevant is, if the correlation is judged, the mask '1' is output corresponding to a pixel region, and the mask '0' is output without the correlation, and the cross attention mechanism is specifically as follows:
y＝Av
Wherein A represents a weight matrix; q represents an image feature; k represents mark point information; k T is the transpose of k; d h denotes a feature dimension; v represents a value vector; y represents the decoded image feature.
The SAM image segmentation model carries out convolution, up-sampling and multi-layer perceptron processing on the image features y output by the cross attention mechanism module to obtain final image features. The mask features output by the SAM model are subjected to dimension adjustment through the multi-layer perceptron, so that the mask features are consistent with the final image features, the object mask is obtained by multiplying the object mask and the final image features, the mask with the highest confidence is reserved through non-maximum suppression, and finally, a mask matrix of the inner wall of the weir groove above the water surface with the highest confidence, namely a segmentation result of the inner wall of the object weir groove, is output. In the mask matrix (binarization matrix) of the inner wall of the weir groove above the water surface output by the SAM image segmentation model, the pixel region with the value of 1 is used as a target segmentation region, the segmentation result of the inner wall of the target weir groove is shown in fig. 5, and the feature dimension can be adjusted by carrying out convolution, up-sampling and multi-layer perceptron processing on the image features output by the cross attention mechanism module, so that the calculation between tensors is facilitated.
Because the impact of water flow on the water gauge and fatigue and aging of the fixing screw over time are considered, the water gauge can deviate by a small extent, and at this time, the marking point may not play a role in prompting in the SAM model, so before the step of obtaining the target weir trough water level according to the target weir trough inner wall segmentation result and the pre-established weir trough water level recognition model, the method for monitoring the flow of the water gauge provided by the embodiment further comprises the step of detecting the deviation of the water gauge according to the target weir trough inner wall segmentation result, and when the water gauge deviates, correcting the marking point information by using the pre-trained YOLO v5 target detection model, specifically:
Determining the boundary line (the middle edge line of the dividing result) between the water gauge and the water gauge according to the dividing result of the inner wall of the target water gauge, calculating the average included angle between the boundary line between the water gauge and the plumb line direction, and performing offset detection on the water gauge according to the average included angle, wherein if the average included angle is not within the preset water gauge included angle range, the water gauge is judged to be offset, and the marking point information is required to be corrected; if the average included angle is within the preset included angle range of the water gauge, judging that the water gauge is not deviated;
When the water gauge is deviated, the perspective correction image is identified by utilizing a pre-trained YOLO v5 target detection model, so that a water gauge deviation position coordinate is obtained;
Calculating to obtain the water gauge offset according to the water gauge offset position coordinates and the water gauge standard position coordinates;
correcting the marking point information according to the water gauge offset, generating corrected marking point information, and re-dividing the original water gauge image according to the corrected marking point information through the SAM image segmentation model, and dynamically updating the target weir groove inner wall segmentation result.
Specifically, for ease of understanding, the present embodiment exemplifies offset detection: observing the boundary line between the water measuring weir groove and the water gauge, calculating the average included angle between the boundary line and the plumb direction, and if the average included angle is smaller than 3 degrees, considering that the water gauge is not deviated; if the average included angle exceeds 3 degrees, the water gauge is considered to deviate, and the marked point information needs to be corrected.
The YOLO v5 target detection model in this embodiment adopts a network structure similar to Darknet, and the network structure includes a plurality of convolution layers, a pooling layer and a full-connection layer, where the convolution layers are used to extract image features, the pooling layer is used to reduce the dimension of a feature map, the full-connection layer is used for final target detection, and the loss function adopted in the training process of the YOLO v5 target detection model includes a prediction frame regression loss function, a classification loss function and a confidence loss function, and the specific training process of the YOLO v5 target detection model is as follows:
The method comprises the steps of manually marking a data set of a perspective correction image, marking the position of a water gauge in the image by adopting a rectangular detection frame, forming a training set consisting of about 100 samples, wherein the training set comprises water gauge diagrams under different hydrologic seasons (high water, level water and dead water) and environmental conditions (rainy days, foggy days and night), and coordinate information corresponding to four vertexes of the rectangular detection frame, and training a YOLO v5 target detection model by using a training set to obtain a trained YOLO v5 target detection model for automatically identifying the position of the water gauge.
The trained YOLO v5 target detection model can better identify the position of the water gauge in the image through the rectangular detection frame, and the coordinate (x 1,y1) of the center point of the rectangular detection frame is calculated; wherein the offset of (x 1,y1) compared with the center coordinate (x 2,y2) of the rectangular frame of the water gauge before the offset isIn this embodiment, the coordinates (m, n) of the marking point in the marking point information are corrected according to Δx and Δy, so as to obtain corrected coordinates (m ', n') of the marking point, and corrected marking point information is obtained, where/>And the SAM image segmentation model re-executes the segmentation process according to the corrected marking point information, and dynamically corrects the mask of the inner wall of the target weir groove.
S4, obtaining the target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model.
In this embodiment, the step of obtaining the target weir groove water level according to the target weir groove inner wall segmentation result and the pre-established weir groove water level identification model includes:
calculating the pixel area of the inner wall of the target weir groove by utilizing the dividing result of the inner wall of the target weir groove;
and inputting the pixel area of the inner wall of the target weir groove into a pre-established weir groove water level identification model to obtain the target weir groove water level.
Specifically, as shown in fig. 6, since there is a small number of gaps in the mask edges (inner wall and water gauge, inner wall and water body) of the inner wall of the weir groove, and the water surface of the weir groove is not horizontal (the lower edge of the mask is not completely horizontal) due to the uneven bottom of the weir groove and the convex cement block on the inner wall surface, if the height of the pixels of the inner wall is directly calculated, a large error is generated for water level identification, and the pixel area of the inner wall of the weir groove above the water surface line under the image coordinate system is calculated by using the mask matrix, the error can be greatly reduced, therefore, the embodiment reads the water level through the scale mark of the water gauge, and establishes a linear relationship between the pixel area S of the inner wall of the weir groove and the real water level L, and generates a weir groove water level identification model, wherein the weir groove water level identification model is:
L＝h-k·S
Wherein L represents a target weir trough water level; h. k represents regression coefficients; s represents the pixel area of the inner wall of the target weir groove, wherein, because the mask matrix of the dividing result of the inner wall of the target weir groove is a binarization matrix and the pixel area of the inner wall of the target weir groove is the number of image pixels occupied by the inner wall, the embodiment can obtain the area of the inner wall of the target weir groove by summing the mask matrices of the dividing result of the inner wall of the target weir groove, such as: when the measuring range of the water gauge is 100cm and the area S of the mask of the inner wall of the weir groove under the image coordinate system is 361826 pixel points, the established weir groove water level identification model is as follows:
L＝68.50-8.71×10-5×S。
S5, calculating to obtain the flow of the water measuring weir by using a water level flow relation model and a target weir groove water level, wherein the water level flow relation model is as follows:
Wherein Q represents the flow rate of the water measuring weir; c e represents the flow experience coefficient; θ represents the V-shaped weir apex angle; g represents gravitational acceleration; h e represents the height of the V-shaped weir apex angle; l represents the target weir trough water level.
In this embodiment, the perspective correction image is subjected to prompt point information correction and mask calculation, the target weir trough water level is calculated through the weir trough water level identification model, and then the flow value corresponding to the calculation of the water measuring weir water level-flow calculation formula is substituted, the water measuring weir in this embodiment is a V-shaped weir, and the coefficient of the water measuring weir flow calculation formula and the use condition refer to national standard jjjg-711-1990.
The embodiment of the invention provides an automatic correction water measuring weir flow shooting monitoring method, which comprises the steps of performing perspective correction on an original water gauge image, performing segmentation processing on the perspective correction image by utilizing a SAM image segmentation model fused with a VIT model and a cross attention mechanism module, establishing a relation between mask pixels and weir groove water level, simultaneously utilizing a YOLO v5 model to identify offset distances of the water gauge due to environmental factors, automatically correcting segmentation masks, and having good segmentation identification capability on the inner wall of the water gauge or the weir groove under various environments or observation angles, thereby greatly improving the accuracy of monitoring the water level and the flow of the water measuring weir, realizing rapid, continuous, low-cost and automatic monitoring of the water level and the flow of the water measuring weir, and having strong application value.
It should be noted that, the sequence number of each process does not mean that the execution sequence of each process is determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.
In one embodiment, as shown in fig. 7, an embodiment of the present invention provides an automatically modifiable water weir flow photography monitoring device, the device comprising:
The image correction module 101 is used for acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image;
The image labeling module 102 is configured to label foreground points and background points in the perspective correction image, and generate labeling point information;
The image segmentation module 103 is configured to obtain a pre-constructed SAM image segmentation model fused with the VIT model and the cross attention mechanism module, input the perspective correction image and the mark point information into the SAM image segmentation model for segmentation processing, and obtain a target weir groove inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information;
The water level identification module 104 is configured to obtain a target water level of the weir groove according to the dividing result of the inner wall of the target weir groove and a pre-established water level identification model of the weir groove;
the flow monitoring module 105 is used for calculating the flow of the water weir by using the water level and flow relation model and the target water level of the weir groove.
The specific limitation of an automatically modifiable water-measuring weir flow photographic monitoring device can be referred to above for the limitation of an automatically modifiable water-measuring weir flow photographic monitoring method, and will not be described herein. Those of ordinary skill in the art will appreciate that the various modules and steps described in connection with the disclosed embodiments of the application may be implemented in hardware, software, or a combination of both. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application provides an automatic-correction water weir flow shooting monitoring device, which performs perspective correction on an original water gauge image through an image correction module; dividing the perspective correction image through an image dividing module; the automatic monitoring of the water level and the flow of the water measuring weir is realized through the water level identification module and the flow monitoring module. Compared with the prior art, the method and the device have the advantages that the offset distance of the water gauge due to environmental factors is identified by using the YOLO v5 model, the segmentation mask is automatically corrected, the correction process is fast, the real-time performance is good, the stability and the controllability are high, the automatic segmentation and correction of the water gauge image are effectively realized, and the method and the device have the advantages of being simple in implementation mode, efficient, low in cost and the like.
The foregoing examples represent only a few preferred embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the application. It should be noted that modifications and substitutions can be made by those skilled in the art without departing from the technical principles of the present application, and such modifications and substitutions should also be considered to be within the scope of the present application. Therefore, the protection scope of the patent of the application is subject to the protection scope of the claims.
Claims (7)
1. The automatic correction water weir flow photography monitoring method is characterized by comprising the following steps of:
Acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image;
Labeling foreground points and background points in the perspective correction image to generate labeling point information;
Acquiring a pre-constructed SAM image segmentation model fused with a VIT model and a cross attention mechanism module, inputting the perspective correction image and the annotation point information into the SAM image segmentation model for weir-notch inner wall segmentation processing, and obtaining a target weir-notch inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information;
obtaining a target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model;
Calculating to obtain the flow of the water measuring weir by utilizing the water level flow relation model and the target weir groove water level;
Inputting the perspective correction image and the annotation point information into the SAM image segmentation model for weir groove inner wall segmentation processing, and obtaining a target weir groove inner wall segmentation result comprises the following steps:
Extracting features of the perspective correction image through a VIT model to obtain image features;
Traversing the image features, and decoding and calculating the image features and the mark point information through a cross attention mechanism to obtain decoded image features and mask features;
And carrying out convolution, up-sampling and multi-layer perceptron processing on the decoded image features by using the SAM image segmentation model to obtain final image features, wherein the mask features are subjected to dimension adjustment by a multi-layer perceptron to be consistent with the final image features, and the final image features are multiplied by the mask features to obtain a target weir groove inner wall segmentation result.
2. The automatic correction water measuring weir flow photography monitoring method according to claim 1, wherein before the step of obtaining the target weir trough water level according to the target weir trough inner wall segmentation result and the pre-established weir trough water level identification model, the method further comprises the steps of detecting the deviation of the water gauge according to the target weir trough inner wall segmentation result, and correcting the marked point information by using a pre-trained YOLO v5 target detection model when the water gauge deviates, wherein the step of correcting the marked point information by using the pre-trained YOLO v5 target detection model specifically comprises:
identifying the perspective correction image by using a pre-trained YOLO v5 target detection model to obtain a water gauge offset position coordinate;
Calculating to obtain the water gauge offset according to the water gauge offset position coordinates and the water gauge standard position coordinates;
correcting the marking point information according to the water gauge offset, generating corrected marking point information, and re-dividing the original water gauge image according to the corrected marking point information through the SAM image segmentation model, and dynamically updating the target weir groove inner wall segmentation result.
3. The method for automatically modifiable water-measuring weir flow photographic monitoring as defined in claim 2, wherein said step of detecting the deviation of the water gauge based on the dividing result of the inner wall of the target weir trough comprises:
And determining the boundary line between the water measuring weir groove and the water gauge according to the dividing result of the inner wall of the target weir groove, detecting the deviation of the water gauge by utilizing the average included angle between the boundary line between the water measuring weir groove and the water gauge and the plumb line direction, and judging that the water gauge deviates if the average included angle is not within the preset water gauge included angle range.
4. The method for automatically modifiable water-measuring weir flow photographic monitoring as defined in claim 1, wherein the step of performing perspective correction on the original water gauge image to obtain a perspective corrected image further comprises:
And detecting whether the region with the pixel missing value exists in the perspective correction image, and if so, performing image interpolation by adopting a nearest neighbor interpolation method or a Lagrange interpolation method.
5. The method of automatically modifiable water-measuring weir flow photographic monitoring as claimed in claim 1, wherein said step of obtaining a target weir trough water level based on said target weir trough inner wall segmentation and a pre-established weir trough water level identification model comprises:
calculating the pixel area of the inner wall of the target weir groove by utilizing the dividing result of the inner wall of the target weir groove;
inputting the pixel area of the inner wall of the target weir groove into a pre-established weir groove water level identification model to obtain the target weir groove water level, wherein the weir groove water level identification model is as follows:
L＝h-k·S
wherein L represents a target weir trough water level; h. k represents regression coefficients; s represents the pixel area of the inner wall of the target weir groove.
6. The automatically modifiable water-measuring weir flow photographic monitoring method as claimed in claim 1, wherein said water-level flow relationship model is:
Wherein Q represents the flow rate of the water measuring weir; c e represents the flow experience coefficient; θ represents the V-shaped weir apex angle; g represents gravitational acceleration; h e represents the height of the V-shaped weir apex angle; l represents the target weir trough water level.
7. An automatically modifiable water-measuring weir flow photographic monitoring device, comprising:
The image correction module is used for acquiring an original water gauge image, and performing perspective correction on the original water gauge image to obtain a perspective correction image;
the image labeling module is used for labeling foreground points and background points in the perspective correction image and generating labeling point information;
The image segmentation module is used for acquiring a pre-constructed SAM image segmentation model fused with the VIT model and the cross attention mechanism module, inputting the perspective correction image and the mark point information into the SAM image segmentation model for weir groove inner wall segmentation processing, and obtaining a target weir groove inner wall segmentation result; the VIT model is used for extracting image features of the perspective correction image; the cross attention mechanism module is used for decoding and calculating the image characteristics and the annotation point information;
the water level identification module is used for obtaining a target weir groove water level according to the target weir groove inner wall segmentation result and a pre-established weir groove water level identification model;
the flow monitoring module is used for calculating the flow of the water measuring weir by utilizing the water level flow relation model and the target water level of the weir groove;
Inputting the perspective correction image and the annotation point information into the SAM image segmentation model for weir groove inner wall segmentation processing to obtain a target weir groove inner wall segmentation result, wherein the method specifically comprises the following steps of:
Extracting features of the perspective correction image through a VIT model to obtain image features;
Traversing the image features, and decoding and calculating the image features and the mark point information through a cross attention mechanism to obtain decoded image features and mask features;
And carrying out convolution, up-sampling and multi-layer perceptron processing on the decoded image features by using the SAM image segmentation model to obtain final image features, wherein the mask features are subjected to dimension adjustment by a multi-layer perceptron to be consistent with the final image features, and the final image features are multiplied by the mask features to obtain a target weir groove inner wall segmentation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311219793.1A CN117346846B (en) | 2023-09-20 | 2023-09-20 | Automatic correction type water measuring weir flow photographic monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311219793.1A CN117346846B (en) | 2023-09-20 | 2023-09-20 | Automatic correction type water measuring weir flow photographic monitoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117346846A CN117346846A (en) | 2024-01-05 |
| CN117346846B true CN117346846B (en) | 2024-06-25 |
Family
ID=89354938
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311219793.1A Active CN117346846B (en) | 2023-09-20 | 2023-09-20 | Automatic correction type water measuring weir flow photographic monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117346846B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109443480A (en) * | 2018-11-02 | 2019-03-08 | 南京邮电大学 | Gauge positioning and water level measurement method based on image procossing |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9541432B2 (en) * | 2013-05-17 | 2017-01-10 | The United States Of America As Represented By The Administrator Of The U.S. Environmental Protection Agency | Flow imaging and monitoring for synchronized management of wide area drainage |
| WO2023049247A1 (en) * | 2021-09-24 | 2023-03-30 | Hydro Radar, LLC | Flow and level monitor for fluid systems |
| CN114812701A (en) * | 2022-04-08 | 2022-07-29 | 新疆坎儿井灌溉技术有限责任公司 | Flow measuring method for gentle slope channel |
| CN115638835A (en) * | 2022-10-09 | 2023-01-24 | 中国电建集团华东勘测设计研究院有限公司 | Automatic flow monitoring method and device for measuring weir based on image recognition algorithm |
| CN115861616A (en) * | 2022-12-09 | 2023-03-28 | 复旦大学 | Semantic segmentation system for medical image sequence |
| CN219328482U (en) * | 2023-01-10 | 2023-07-11 | 湖北清江水电开发有限责任公司 | Manual observation device for dam measuring weir |
| CN116189180A (en) * | 2023-04-28 | 2023-05-30 | 青岛理工大学 | Urban streetscape advertisement image segmentation method |
-
2023
- 2023-09-20 CN CN202311219793.1A patent/CN117346846B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109443480A (en) * | 2018-11-02 | 2019-03-08 | 南京邮电大学 | Gauge positioning and water level measurement method based on image procossing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117346846A (en) | 2024-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112766274B (en) | Water gauge image water level automatic reading method and system based on Mask RCNN algorithm | |
| CN104484648B (en) | Robot variable visual angle obstacle detection method based on outline identification | |
| CN108764257A (en) | A kind of pointer instrument recognition methods of various visual angles | |
| KR102339465B1 (en) | Autonomous navigation ship system for removing sea waste based on deep learning-vision recognition | |
| CN108562250B (en) | Keyboard keycap flatness rapid measurement method and device based on structured light imaging | |
| CN111998910B (en) | Visual measurement method and system for water level of multi-stage water gauge | |
| CN113137920A (en) | Underwater measurement equipment and underwater measurement method | |
| CN111292371B (en) | Intelligent water level detection equipment, intelligent water level monitoring management device and detection method | |
| CN112785655A (en) | Method, device and equipment for automatically calibrating external parameters of all-round camera based on lane line detection and computer storage medium | |
| CN110889829A (en) | Monocular distance measurement method based on fisheye lens | |
| CN113134683A (en) | Laser marking method and device based on machine learning | |
| CN115077414B (en) | Device and method for measuring bottom contour of sea surface target by underwater vehicle | |
| CN110796681A (en) | Visual positioning system and method for cooperative work of ship | |
| CN112215125A (en) | Water level identification method based on YOLOv3 | |
| CN115205286B (en) | Method for identifying and positioning bolts of mechanical arm of tower-climbing robot, storage medium and terminal | |
| CN115578315A (en) | Bridge strain close-range photogrammetry method based on unmanned aerial vehicle image | |
| CN110851978B (en) | Camera position optimization method based on visibility | |
| CN116168028A (en) | High-speed rail original image processing method and system based on edge filtering under low visibility | |
| CN117346846B (en) | Automatic correction type water measuring weir flow photographic monitoring method and device | |
| CN110992291B (en) | Ranging method, system and storage medium based on three-eye vision | |
| CN116883483A (en) | Fish body measuring method based on laser camera system | |
| CN115830140A (en) | Offshore short-range photoelectric monitoring method, system, medium, equipment and terminal | |
| CN113822104B (en) | Artificial intelligence surface of water detecting system based on virtual scale of many candidates | |
| Bi et al. | Method for detecting surface defects of underwater buildings: Binocular vision based on sinusoidal grating fringe assistance | |
| CN116222693B (en) | Natural river channel water level automatic monitoring method based on water gauge reflection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |