CN116128883A - Photovoltaic panel quantity counting method and device, electronic equipment and storage medium - Google Patents
Photovoltaic panel quantity counting method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116128883A CN116128883A CN202310420646.4A CN202310420646A CN116128883A CN 116128883 A CN116128883 A CN 116128883A CN 202310420646 A CN202310420646 A CN 202310420646A CN 116128883 A CN116128883 A CN 116128883A
- Authority
- CN
- China
- Prior art keywords
- target object
- frame
- image
- target
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000012549 training Methods 0.000 claims abstract description 71
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 31
- 238000013135 deep learning Methods 0.000 claims abstract description 24
- 238000002372 labelling Methods 0.000 claims abstract description 13
- 238000004519 manufacturing process Methods 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 230000007704 transition Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 abstract description 10
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000013528 artificial neural network Methods 0.000 description 7
- 210000003128 head Anatomy 0.000 description 7
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a photovoltaic panel quantity counting method, which comprises the following steps: acquiring a video of an area to be counted; selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set; inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image; the number of target objects in each frame of image of the video is counted based on the target detector. The construction supervision unit only needs to provide a construction site video, the result can be automatically counted by the photovoltaic panel quantity counting method based on target tracking, the operation is simple and convenient, compared with a manual counting method, the efficiency is improved by more than 3 times, and the working intensity of staff is greatly reduced.
Description
Technical Field
The patent application relates to the technical field of artificial intelligence, in particular to a photovoltaic panel quantity counting method, a device, electronic equipment and a storage medium.
Background
At present, the construction progress statistics of the photovoltaic station mainly utilizes unmanned aerial vehicle shooting field video manual statistics, construction supervision personnel utilize unmanned aerial vehicle high altitude to beat photovoltaic construction field, identify photovoltaic panels through human eyes, and count out photovoltaic panel quantity.
When the number of the photovoltaic panels is counted manually, the human eyes need to watch videos carefully from beginning to end, the manual counting time length is about 3 times or more of the video time length, and when the human eyes do the work for a long time, the human eyes are easy to fatigue so as to make mistakes, so that the manual counting work efficiency is low and the accuracy is low.
Disclosure of Invention
The invention aims to solve one of the technical problems in the related art to at least a certain extent, and therefore, the first aim of the invention is to provide a photovoltaic panel quantity counting method, and only a construction site video is needed to be provided, and the photovoltaic panel quantity counting method based on target tracking can automatically count the result, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
A second object of the present invention is to provide a photovoltaic panel counting device.
A third object of the present invention is to propose an electronic device.
A fourth object of the present invention is to propose a storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for counting the number of photovoltaic panels, including the following steps:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, partial images are selected based on each frame of video image, a target object in the selected images is marked with an external rectangular frame, and a training data set is manufactured, and the method comprises the following steps:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
According to the method for counting the number of the photovoltaic panels, the training data set of the circumscribed rectangular frame is input into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image, and the method comprises the following steps:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
According to the method for counting the number of the photovoltaic panels, the counting of the number of the target objects of each frame of image of the video based on the target detector comprises the following steps:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, each frame of image of the video of the area to be counted is input into the target detector, and the apparent characteristics of the target object in each frame of image are obtained, and the method comprises the following steps:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
The expression is as follows: the invention aims to solve one of the technical problems in the related art to at least a certain extent, and therefore, the first aim of the invention is to provide a photovoltaic panel quantity counting method, and only a construction site video is needed to be provided, and the photovoltaic panel quantity counting method based on target tracking can automatically count the result, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
A second object of the present invention is to provide a photovoltaic panel counting device.
A third object of the present invention is to propose an electronic device.
A fourth object of the present invention is to propose a storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for counting the number of photovoltaic panels, including the following steps:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, partial images are selected based on each frame of video image, a target object in the selected images is marked with an external rectangular frame, and a training data set is manufactured, and the method comprises the following steps:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
According to the method for counting the number of the photovoltaic panels, the training data set of the circumscribed rectangular frame is input into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image, and the method comprises the following steps:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
According to the method for counting the number of the photovoltaic panels, the counting of the number of the target objects of each frame of image of the video based on the target detector comprises the following steps:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, each frame of image of the video of the area to be counted is input into the target detector, and the apparent characteristics of the target object in each frame of image are obtained, and the method comprises the following steps:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
wherein ,
is the coordinate information of the i-th target object,
is the x coordinate value of the upper left corner of the circumscribed rectangle of the target object,
is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,
is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object,
the y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object; category information collectionThe expression is as follows:
wherein ,is the category of the i-th target object, +. >The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object; />
Creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker, comprising:
video first frame image I based on object detector 1 Detecting based on the detected coordinate information set of the target objectInitializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>,/>,/>Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->,,/>Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
for the state information of the target object at time k-1, i.e.>, wherein ,/>) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->Is the abscissa speed, +.>Is the ordinate speed, +.>Is the area velocity; />Estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
each frame of image of the video of the area to be counted is operated according to the steps;
Matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frame
And prediction result
Matching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frame
Updating
While continuing to predict the t+1st frame target object coordinates
The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, unmatched target objects are utilized
Creating a new Kalman tracker and the number of the target object;
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
each frame of image of the video of the area to be counted is operated according to the steps;
matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frameAnd prediction resultMatching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frameUpdate->While continuing to predict the t+1st frame target object coordinates +.>The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>Creating a new Kalman tracker and the number of the target object;
Counting the number set of each target object qualified by matching, and calculating the number of the target objects, wherein the number set comprises the following steps:
sequentially counting target objects of each frame of imageNumbering sets of individual ID occurrences and />Number set of occurrence frequency of each ID of 1 +.>, wherein ,the expression of (2) is as follows:
,/>is numbered->The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
,/>is numbered->The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
and counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
According to the photovoltaic panel quantity counting method provided by the embodiment of the invention, for the unmatched target object coordinate informationInitializing and creating a new Kalman tracker, generating new numbers at the same time, and updating the number set of occurrence frequency +.> and />。
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, the number of the target objects is obtained by counting the ID category number of which the frequency of occurrence of the target objects exceeds 3 times based on the number set, and the number of the target objects is obtained by accumulation and statistics, and the method comprises the following steps:
Based on numbered setsStatistics of->ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered setsStatistics of->ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the number of target objects M, m=clsw+clsh is counted.
An embodiment of a second aspect of the present invention provides a photovoltaic panel count apparatus, including:
the acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
the training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
and the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
An embodiment of the third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements a method for counting the number of photovoltaic panels according to the embodiment of the first aspect of the present invention when the processor executes the program.
An embodiment of a fourth aspect of the present invention proposes a computer readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements a method for counting the number of photovoltaic panels as proposed by an embodiment of the first aspect of the present invention.
Compared with the prior art, the invention has the beneficial effects that: according to the photovoltaic panel number counting method, the target detector is obtained through training, the apparent characteristics of the target object in each frame of video image are obtained through the target detector, the detection result and the prediction result are matched, the ID category number is counted, and the number of the photovoltaic panels is counted. The construction supervision unit only needs to provide a construction site video, the result can be automatically counted by the photovoltaic panel quantity counting method based on target tracking, the operation is simple and convenient, compared with a manual counting method, the efficiency is improved by more than 3 times, and the working intensity of staff is greatly reduced.
Drawings
FIG. 1 is a flow chart of a method for counting the number of photovoltaic panels according to one embodiment of the present application;
FIG. 2 is a detailed flow chart of step S4 according to one embodiment of the present application;
FIG. 3 is a schematic view of a frame of an image captured in an embodiment of the present application;
FIG. 4 is a schematic view of a circumscribed rectangular frame of an image of a photovoltaic panel according to one embodiment of the present application;
FIG. 5 is a schematic diagram of target detector identification in one embodiment of the present application;
FIG. 6 is a schematic diagram of a Mosaic image enhancement technique in one embodiment of the present application;
FIG. 7 is a statistical flow chart of one embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
Other advantages and effects of the present application will be readily apparent to those skilled in the art from the present disclosure, by describing embodiments of the present application with specific examples. This application is also intended to cover any adaptations or uses of various embodiments and can be practiced in different but specific details of the subject matter within the scope of the description and from various points of view. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
Other advantages and effects of the present application will be readily apparent to those skilled in the art from the present disclosure, by describing embodiments of the present application with specific examples. This application is also intended to cover any adaptations or uses of various embodiments and can be practiced in different but specific details of the subject matter within the scope of the description and from various points of view. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
The following describes in detail a method, an apparatus, an electronic device, and a storage medium for counting the number of photovoltaic panels provided in the embodiments of the present application through specific embodiments and application scenarios thereof with reference to the accompanying drawings.
In the embodiment of the application, the number of the photovoltaic panels can be calculated by constructing a photovoltaic panel number counting device, executing a photovoltaic panel number counting method in the photovoltaic panel number counting device, obtaining a target detector through training, obtaining apparent characteristics of a target object in each frame of image of a video through the target detector, matching a detection result and a prediction result, counting the number of ID categories, and calculating the number of the counted photovoltaic panels. The result can be automatically counted by only providing the construction site video and the photovoltaic panel quantity counting method based on target tracking, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
The photovoltaic panel quantity counting device can comprise an acquisition module, a training set making module, a training module and a counting module.
The acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
The training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
and the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
The device can be applied to a terminal, and can be executed by hardware or software in the terminal.
The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet having a touch sensitive surface (e.g., a touch screen display and/or a touch pad). It should also be appreciated that in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).
In the following various embodiments, a terminal including a display and a touch sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.
The execution main body of the photovoltaic panel number counting method provided by the embodiment of the application may be an electronic device or a functional module or a functional entity capable of implementing the photovoltaic panel number counting method in the electronic device, and the electronic device mentioned in the embodiment of the application includes, but is not limited to, a mobile phone, a tablet computer, a camera, a wearable device and the like.
Fig. 1 is a flowchart of a method for counting the number of photovoltaic panels according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
s1, acquiring videos of areas to be counted;
it can be understood that the video of the area to be counted is obtained by shooting by adopting the camera on the unmanned aerial vehicle, specifically, according to the area position of the photovoltaic power station, the flight parameters set by the unmanned aerial vehicle are as follows: the flying height is between 200 and 300m, the flying speed is 8m/s, shooting is carried out at an angle vertical to the ground, flying is carried out in a parallel or vertical photovoltaic plate mode, the resolution of video is 1920 x 1080 or higher based on definition.
S2, selecting partial images based on each frame of video image, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
in the step, the selected target object of the partial image is marked with an external rectangular frame, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, rectangular frame information is stored, an image data set comprising the rectangular frame is obtained, and in the embodiment, the target object refers to a photovoltaic panel.
The external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, and the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information.
According to one embodiment of the invention, in a photovoltaic power plant, a rectangular frame contains photovoltaic panels and a spacing region between the photovoltaic panels, the rectangular frame of a complete photovoltaic panel is named "white_zc", the incomplete photovoltaic panel is named "half_zc", and rectangular frame information is stored in an xml file.
It should be noted that, a complete photovoltaic panel refers to two rows of panels and the edges are visible, and an incomplete photovoltaic panel refers to one row of panels and one row of frames and the edges are visible.
S3, inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
in the step, a training data set of an external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module performs feature fusion on the first feature map and outputs a second feature map, the Head module performs convolution on the second feature map, the offset of a target object prediction frame is output, and an NMS non-maximum suppression is adopted to generate a target detector of a target object.
According to one embodiment of the invention, during training, the image data of the training data set is input into a yolov5 deep learning algorithm, the algorithm reads the image data, the neural network extracts the image features, the neural network identifies according to the features, then the identification error is calculated according to the loss function, and when the error is smaller than a threshold value, model training is completed.
According to an embodiment of the present invention, taking a photovoltaic panel as a target image and taking a video image resolution of 1920×1080 as an example, the training data set preprocessing of the circumscribed rectangular frame includes: setting the image input size to 1088 x 1088, and expanding the photovoltaic panel data set by utilizing a Mosaic image enhancement technology.
It should be noted that, since yolov5 neural network downsamples the image by a factor of 32, the scaling size must be a factor of 32; because the photovoltaic panel target is larger, original pictures 1920 x 1080 scale to 1088 x 1088, key characteristic information of the photovoltaic panel cannot be lost, and meanwhile, video memory resources can be saved and training speed can be improved.
The technological process of the Mosaic: as shown in fig. 6, a gray (R channel value 114, g channel value 114, B channel value 114) base map with a size of 2176 x 2176 is first constructed, one point C (cut_x, cut_y) is piled up and selected as a splicing point in a rectangle defined by a point a (544) and a point B (1632), 4 images are randomly selected, and the regions are spliced into the base map according to the splicing points; the preprocessed data set is input to the Backbone module of the yolov5 neural network.
In accordance with one embodiment of the present invention, the yolov5 deep learning algorithm model is trained by first extracting photovoltaic panel features from a back plane module through a convolutional layer, a csp1_x structure and an SPPF structure, and outputting a first feature map of 136X 512, 68X 512, 34X 512 three scales, then, feature fusion is carried out on the three-scale first feature graphs through up-sampling and down-sampling operations in a Neck module, 128-136-68-34-512-136 second feature graphs are output, and finally, photovoltaic panel predicted frame offset parameters are output from the three second feature graphs through a 1*1 convolution kernel in a Head module, wherein ,/>、/>The offset of the width and the height of the photovoltaic panel prediction frame relative to the prior frame is respectively; />Is the confidence of the photovoltaic panel prediction frame; />Is a specific category of photovoltaic panel predictions, 0 is a complete photovoltaic panel, and 1 is an incomplete photovoltaic panel.
It should be noted that, the position of the photovoltaic panel prediction frame is calculated according to the following formula:
the grid upper left corner coordinates of the central point of the photovoltaic panel prediction frame; />Is the center coordinate of the photovoltaic panel prediction frame relative +.>Offset of (2); />Is the center point coordinate of the photovoltaic panel prediction frame; />、The width and the height of the photovoltaic panel prediction frame are respectively; / >Is a Sigmoid function, limiting the predicted offset to [0,1 ]]The predicted central point does not exceed the corresponding grid area; />、/>The width and height of the a priori frame, respectively.
According toValue versus all photovoltaic panel predicted frame position parametersScreening and sorting, will->A box greater than 0.8 as a preselected box;
then NMS non-maximum suppression is adopted, inSelecting the maximum +.>Value +.>As a benchmark, calculate it and other +>Removing the boxes having an IOU greater than 0.85; repeating the step, selecting a new reference from the remaining preselected frames, and removing the frames with IOU greater than 0.85 until no frames with IOU greater than 0.85 are available; finally, each datum represents a complete photovoltaic panel, so that a prediction frame of the photovoltaic panel is obtained; the same is true of the acquisition step of the incomplete photovoltaic panel prediction frame.
The IOU refers to the ratio of intersection and union of a photovoltaic panel prediction frame and a real frame, and the calculation formula is as follows:
Defining a loss function and training a target detector of the photovoltaic panel; the loss function is defined as the sum of the location loss, the confidence loss, and the category loss, and the calculation formula is as follows:
wherein, K, (-) -is>B is the output characteristic diagram, grids and the number of anchor boxes on each grid, wherein each anchor box is a rectangular frame in which a photovoltaic panel can exist; / >For the weight of the corresponding item +.>,/>,/>;The kth output characteristic diagram, the ith grid and the jth anchor box are represented, if yes, the output characteristic diagram is 1, otherwise, the output characteristic diagram is 0; />Respectively a prediction frame and a real frame; />The weight of the output characteristic diagram for balancing each scale is [4.0,1.0,0.4 ]]Output feature maps corresponding to 136 x 136, 68 x 68, 34 x 34 in turn;
loss of positioning,, wherein />For the distance of the center point of the predicted and real frames, < >>For the diagonal length of the smallest bounding rectangle of the prediction and real frames, V is the aspect ratio similarity of the prediction and real frames,/o>For the width of the real frame +.>For the height of the real frame +.>For predicting the width of the frame, +.>Is the height of the prediction frame; confidence loss->,/>Prediction of frame confidence for photovoltaic panels, +.>IOU values for photovoltaic panels prediction and real frames,>for the two classification cross entropy loss->A weight representing a positive sample;
category loss
For the photovoltaic panel prediction frame class,
as a true frame class of photovoltaic panels,
for the purpose of the two-class cross entropy loss,
the weights representing the photovoltaic panel categories.
And judging the error of the predicted photovoltaic panel information and the real photovoltaic panel information of the neural network by analyzing the change of the result value of the loss function along with the training times, wherein the closer the predicted information is to the expected information, the smaller the loss function is, and the higher the accuracy of the target detector of the photovoltaic panel is.
S4, counting the number of target objects of each frame of image of the video based on the target detector.
In this step, as shown in fig. 2, the following specific steps are included:
s41, inputting each frame of image of the video of the region to be counted into a target detector, and obtaining apparent characteristics of a target object in each frame of image;
in the step, the video of the region to be counted is input into a target detector, the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
wherein ,is the coordinate information of the i-th target object, < >>Is the x coordinate value of the left upper corner of the circumscribed rectangle of the target object,/->Is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,/->Is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object, < >>The y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object;
wherein ,is the category of the i-th target object, +.>The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object.
S42, creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
in this step, a first frame image I is displayed to the video based on the object detector 1 Detecting based on the detected coordinate information set of the target objectInitializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>,/>,/>Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->,,/>Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
for the state information of the target object at time k-1, i.e.>, wherein ,/>) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->Is the abscissa speed, +.>Is the ordinate speed, +.>Is the area velocity;
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
it should be noted that, each frame of image of the video of the region to be counted is operated according to the steps.
S43, matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
In the step, the IOU distance is used as a weight by using the KM algorithm, and the t frame detection result is obtainedAnd prediction result->Matching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frameUpdate->While continuing to predict the t+1st frame target object coordinates +.>The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>Creating a new Kalman tracker and the number of the target object;
for the non-matched target object coordinate informationInitializing and creating a new Kalman tracker, generating new numbers at the same time, and updating a number set of occurrence frequencies and />。
S44, counting the number sets of all the target objects qualified in matching, and calculating the number of the target objects.
In this step, the target object of each frame image is counted in sequenceNumber set of frequency of occurrence of each ID +.> and />Number set of occurrence frequency of each ID of 1 +.>, wherein ,/>The expression of (2) is as follows: />
,/>Is numbered->The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
,/>is numbered->The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
And counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
Specifically, based on a set of numbersStatistics of->ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered setsStatistics of->ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the total number of target objects M, m=clsw+clsh is counted.
In order that the above-described embodiments may be better illustrated, an example will now be described.
For example, taking a video to be detected as a flight shot video, taking a photovoltaic panel as an example, as shown in fig. 7, the flow of the photovoltaic panel number statistics method is as follows:
taking a certain photovoltaic power station as an example, determining the regional range of a photovoltaic panel in the photovoltaic power station, setting flight parameters of an unmanned aerial vehicle, setting the flight height of 200m and the flight speed of 8m/s, as shown in fig. 3, flying in the direction parallel or perpendicular to the photovoltaic panel, shooting perpendicular to the ground, and enabling the video resolution to be 1920 x 1080.
After the video is shot according to the parameters, selecting partial images and labeling external rectangular frames, wherein the rectangular frames need to comprise photovoltaic panels and interval areas between the photovoltaic panels, the name of the rectangular frames of the whole photovoltaic panels (two rows of panels and the edges of the rectangular frames are visible) is named as "white_zc", the name of the incomplete photovoltaic panels is named as "half_zc", and rectangular frame information is stored in an xml file as shown in fig. 4.
Training and learning are carried out on the training set by utilizing a yolov5 deep learning algorithm model, a target detector of the target object is generated, and the apparent characteristics of the target object are described through the generation of the target detector, as shown in fig. 5.
When the video to be detected is input, the target detector can detect the circumscribed rectangular coordinates of the photovoltaic panel in each frame of videoAnd category->;
is x coordinate value of the left upper corner of the circumscribed rectangle of the photovoltaic panel, < >>Is the y coordinate value of the left upper corner of the circumscribed rectangle of the photovoltaic panel, < >>Is x coordinate value of right lower corner of the rectangle circumscribed by the photovoltaic panel, < >>The y coordinate value of the right lower corner of the circumscribed rectangle of the photovoltaic panel; />
Is a category of i-th photovoltaic panel, < >>The value 0 or 1,0 represents a complete photovoltaic panel, and 1 represents an incomplete photovoltaic panel;
first frame photovoltaic panel image I with target detector 1 Detecting, using the detected coordinate information set of the photovoltaic panelInitializing and creating a Kalman tracker; kalman tracker generates a set of photovoltaic panel numbers,/>,/>Is the unique number of the ith photovoltaic panel, and predicts the second frame of photovoltaic panel image I 2 Photovoltaic panel coordinate information->,/>,Is the predicted coordinates of the ith photovoltaic panel;
Statistics I 1 In (a)Frequency of occurrence of each ID> and />1 frequency of occurrence of the respective IDs +.>,
,/>Is numbered->The frequency of occurrence of the complete photovoltaic panel is 1, and 1 is accumulated every 1 occurrence in 1 frame of image;
,/>is numbered->The frequency of occurrence of incomplete photovoltaic panels, the initial value is 1, and 1 is accumulated every 1 occurrence in 1 frame of image;
detecting the subsequent image by using the target detector, and detecting the coordinate information of the photovoltaic panel detected by the t-th frame imageAnd predictive information +.>;
Using KM algorithm, taking IOU distance as weight, and detecting the t frameAnd prediction resultMatching is carried out; based on preset unmanned aerial vehicle flight parameters and actual flight conditions, the IOU distance threshold is set to be 0.1, and when the IOU is less than 0.1, the unmanned aerial vehicle is not considered to be the same target;
photovoltaic panel coordinate information matched by using t-th frameUpdating a Kalman tracker with simultaneous relayContinuously predicting coordinates of photovoltaic panel in t+1th frame +.>;
For unmatched photovoltaic panelsA new kalman tracker should be initialized and created while generating new numbers, updating the set of numbers +.> and />;
According to the number setStatistics of->Obtaining the number Clsw of the complete photovoltaic panels according to ID category number more than 3; />
According to the number setStatistics of->Obtaining the number Clsh of incomplete photovoltaic panels according to ID category numbers more than 3;
Finally, photovoltaic panel number = clsw+clsh/2.
According to one embodiment of the invention, in the photovoltaic panel quantity counting device, the acquisition module shoots videos obtained from the area to be counted by adopting a camera on the unmanned aerial vehicle, specifically, according to the area position of the photovoltaic power station, the flight parameters set by the unmanned aerial vehicle are as follows: the flying height is between 200 and 300m, the flying speed is 8m/s, shooting is carried out at an angle vertical to the ground, flying is carried out in a parallel or vertical photovoltaic plate mode, the resolution of video is 1920 x 1080 or higher based on definition.
The training set making module is used for selecting partial images based on each frame of image of the video, and making a training data set by marking a target object in the selected images with an external rectangular frame.
According to one embodiment of the invention, in a photovoltaic power plant, a rectangular frame contains photovoltaic panels and a spacing region between the photovoltaic panels, the rectangular frame of a complete photovoltaic panel is named "white_zc", the incomplete photovoltaic panel is named "half_zc", and rectangular frame information is stored in an xml file.
It should be noted that, a complete photovoltaic panel refers to two rows of panels and the edges are visible, and an incomplete photovoltaic panel refers to one row of panels and one row of frames and the edges are visible.
And the training module inputs the training data set into a yolov5 deep learning algorithm model to perform training learning, generates a target detector of the target object, and describes the apparent characteristics of the target object by generating the target detector.
According to one embodiment of the invention, during training, the image data of the training set is input into a yolov5 deep learning algorithm, the algorithm reads the image data, the neural network extracts the image features, the neural network performs recognition according to the features, then calculates recognition errors according to a loss function, and completes model training when the errors are smaller than a threshold value.
And the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
The photovoltaic panel number counting device in the embodiment of the application can be an electronic device, and also can be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.
The photovoltaic panel number statistics device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.
The photovoltaic panel number statistics device provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to 2, and in order to avoid repetition, a detailed description is omitted here.
In some embodiments, as shown in fig. 8, the embodiment of the present application further provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and capable of running on the processor 701, where the program when executed by the processor 701 implements the respective processes of the above embodiment of the method for counting the number of photovoltaic panels, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.
The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device described above.
The invention also provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is set to realize the method for counting the number of the photovoltaic panels according to the embodiment of the invention when the computer program runs.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (10)
1. A method for counting the number of photovoltaic panels, comprising the steps of:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
2. The method according to claim 1, wherein selecting a partial image based on each frame of video image, labeling a target object in the selected image with an external rectangular frame, and creating a training data set, includes:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
3. The method according to claim 1, wherein inputting the training data set of the circumscribed rectangle frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image comprises:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
4. The method for counting the number of photovoltaic panels according to claim 1, wherein the counting the number of target objects in each frame of image of the video based on the target detector comprises:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
5. The method for counting the number of the photovoltaic panels according to claim 4, wherein inputting each frame of image of the video of the area to be counted into the target detector, and obtaining the apparent characteristics of the target object in each frame of image, comprises:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target objectAnd category information set->Coordinate information set->The expression is as follows:
wherein ,is the t-th frame image I t A set of coordinate information of the detected target object; />
wherein ,is the coordinate information of the i-th target object, < >>Is the x coordinate value of the left upper corner of the circumscribed rectangle of the target object,/->Is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,/->Is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object, < >>The y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object;
wherein ,is the category of the i-th target object, +.>The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker, comprising:
video first frame image I based on object detector 1 Detecting based on the detected coordinate information set of the target objectInitializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>,/>,/>Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->,,/>Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
for the state information of the target object at time k-1, i.e.>, wherein ,/>) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->Is the abscissa speed, +.>Is the ordinate speed, +.>Is the area velocity;
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
Each frame of image of the video of the area to be counted is operated according to the steps;
matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frameAnd prediction result->Matching, when IOU is more than or equal to 0.1, using target object coordinate information matched by t frame +.>Update->While continuing to predict the t+1st frame target object coordinates +.>The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>Creating a new Kalman tracker and the number of the target object;
counting the number set of each target object qualified by matching, and calculating the number of the target objects, wherein the number set comprises the following steps:
sequentially counting target objects of each frame of imageNumbering sets of individual ID occurrences and />Number set of occurrence frequency of each ID of 1 +.>, wherein ,the expression of (2) is as follows:
,is numbered->The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
,is numbered->The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
and counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
7. The method according to claim 5, wherein counting the number of ID categories of the target object occurring more than 3 times based on the number set, obtaining the number of target objects, and accumulating the statistics to obtain the number of target objects, comprises:
based on numbered setsStatistics of->ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered setsStatistics of->ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the total number of target objects M, m=clsw+clsh is counted.
8. A photovoltaic panel count apparatus, comprising:
the acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
the training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
And the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of counting the number of photovoltaic panels according to any one of claims 1-7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method of counting the number of photovoltaic panels according to any one of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310420646.4A CN116128883A (en) | 2023-04-19 | 2023-04-19 | Photovoltaic panel quantity counting method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310420646.4A CN116128883A (en) | 2023-04-19 | 2023-04-19 | Photovoltaic panel quantity counting method and device, electronic equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116128883A true CN116128883A (en) | 2023-05-16 |
Family
ID=86312201
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310420646.4A Pending CN116128883A (en) | 2023-04-19 | 2023-04-19 | Photovoltaic panel quantity counting method and device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116128883A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117132419A (en) * | 2023-10-26 | 2023-11-28 | 北京图知天下科技有限责任公司 | Numbering method of photovoltaic module |
CN117611929A (en) * | 2024-01-23 | 2024-02-27 | 湖北经济学院 | LED light source identification method, device, equipment and medium based on deep learning |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106127812A (en) * | 2016-06-28 | 2016-11-16 | 中山大学 | A kind of passenger flow statistical method of non-gate area, passenger station based on video monitoring |
US20190304102A1 (en) * | 2018-03-30 | 2019-10-03 | Qualcomm Incorporated | Memory efficient blob based object classification in video analytics |
WO2020207038A1 (en) * | 2019-04-12 | 2020-10-15 | 深圳壹账通智能科技有限公司 | People counting method, apparatus, and device based on facial recognition, and storage medium |
CN111860282A (en) * | 2020-07-15 | 2020-10-30 | 中国电子科技集团公司第三十八研究所 | Subway section passenger flow volume statistics and pedestrian retrograde motion detection method and system |
CN113469955A (en) * | 2021-06-15 | 2021-10-01 | 车嘉祺 | Photovoltaic module fault area image detection method and system |
KR20220051055A (en) * | 2020-10-16 | 2022-04-26 | 이지스로직 주식회사 | Solar panel inspection device using drone |
CN114677554A (en) * | 2022-02-25 | 2022-06-28 | 华东理工大学 | Statistical filtering infrared small target detection tracking method based on YOLOv5 and Deepsort |
WO2022134120A1 (en) * | 2020-12-26 | 2022-06-30 | 西安科锐盛创新科技有限公司 | Target motion prediction-based parking lot management and control method, apparatus, and electronic device |
-
2023
- 2023-04-19 CN CN202310420646.4A patent/CN116128883A/en active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106127812A (en) * | 2016-06-28 | 2016-11-16 | 中山大学 | A kind of passenger flow statistical method of non-gate area, passenger station based on video monitoring |
US20190304102A1 (en) * | 2018-03-30 | 2019-10-03 | Qualcomm Incorporated | Memory efficient blob based object classification in video analytics |
WO2020207038A1 (en) * | 2019-04-12 | 2020-10-15 | 深圳壹账通智能科技有限公司 | People counting method, apparatus, and device based on facial recognition, and storage medium |
CN111860282A (en) * | 2020-07-15 | 2020-10-30 | 中国电子科技集团公司第三十八研究所 | Subway section passenger flow volume statistics and pedestrian retrograde motion detection method and system |
KR20220051055A (en) * | 2020-10-16 | 2022-04-26 | 이지스로직 주식회사 | Solar panel inspection device using drone |
WO2022134120A1 (en) * | 2020-12-26 | 2022-06-30 | 西安科锐盛创新科技有限公司 | Target motion prediction-based parking lot management and control method, apparatus, and electronic device |
CN113469955A (en) * | 2021-06-15 | 2021-10-01 | 车嘉祺 | Photovoltaic module fault area image detection method and system |
CN114677554A (en) * | 2022-02-25 | 2022-06-28 | 华东理工大学 | Statistical filtering infrared small target detection tracking method based on YOLOv5 and Deepsort |
Non-Patent Citations (3)
Title |
---|
LBJ_WZ: "Deepsort源码详解及个人理解", 《HTTPS://BLOG.CSDN.NET/QQ_42779673/ARTICLE/DETAILS/116950649》, pages 1 - 16 * |
TAO HONG等: "A Real-Time Tracking Algorithm for Multi-Target UAV Based on Deep Learning", 《MDPI》, pages 1 - 18 * |
李广: "基于视频图像的车流量的检测研究", 《中国优秀硕士学位论文全文数据库(电子期刊)》, pages 17 - 32 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117132419A (en) * | 2023-10-26 | 2023-11-28 | 北京图知天下科技有限责任公司 | Numbering method of photovoltaic module |
CN117132419B (en) * | 2023-10-26 | 2024-01-23 | 北京图知天下科技有限责任公司 | Numbering method of photovoltaic module |
CN117611929A (en) * | 2024-01-23 | 2024-02-27 | 湖北经济学院 | LED light source identification method, device, equipment and medium based on deep learning |
CN117611929B (en) * | 2024-01-23 | 2024-04-23 | 湖北经济学院 | LED light source identification method, device, equipment and medium based on deep learning |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10210418B2 (en) | Object detection system and object detection method | |
CN109829398B (en) | Target detection method in video based on three-dimensional convolution network | |
US8462987B2 (en) | Detecting multiple moving objects in crowded environments with coherent motion regions | |
JP5181704B2 (en) | Data processing apparatus, posture estimation system, posture estimation method and program | |
CN116128883A (en) | Photovoltaic panel quantity counting method and device, electronic equipment and storage medium | |
CN111079739B (en) | Multi-scale attention feature detection method | |
CN110309842B (en) | Object detection method and device based on convolutional neural network | |
CN108182695B (en) | Target tracking model training method and device, electronic equipment and storage medium | |
CN116152863B (en) | Personnel information identification method and device, electronic equipment and storage medium | |
CN112541403B (en) | Indoor personnel falling detection method by utilizing infrared camera | |
CN114926747A (en) | Remote sensing image directional target detection method based on multi-feature aggregation and interaction | |
CN116824335A (en) | YOLOv5 improved algorithm-based fire disaster early warning method and system | |
US20170053172A1 (en) | Image processing apparatus, and image processing method | |
CN115690545B (en) | Method and device for training target tracking model and target tracking | |
CN113537026B (en) | Method, device, equipment and medium for detecting graphic elements in building plan | |
US20220351486A1 (en) | Object detection device, object detection method, and program | |
CN117132649A (en) | Ship video positioning method and device for artificial intelligent Beidou satellite navigation fusion | |
CN116758419A (en) | Multi-scale target detection method, device and equipment for remote sensing image | |
CN116433722A (en) | Target tracking method, electronic device, storage medium, and program product | |
CN112926681B (en) | Target detection method and device based on deep convolutional neural network | |
CN114067359B (en) | Pedestrian detection method integrating human body key points and visible part attention characteristics | |
CN115205806A (en) | Method and device for generating target detection model and automatic driving vehicle | |
CN115239700A (en) | Spine Cobb angle measurement method, device, equipment and storage medium | |
JP2022531029A (en) | Image recognition method, device and storage medium | |
CN117523428B (en) | Ground target detection method and device based on aircraft platform |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230516 |