WO2021139494A1 - Animal body online claim settlement method and apparatus based on monocular camera, and storage medium - Google Patents

Animal body online claim settlement method and apparatus based on monocular camera, and storage medium Download PDF

Info

Publication number
WO2021139494A1
WO2021139494A1 PCT/CN2020/136401 CN2020136401W WO2021139494A1 WO 2021139494 A1 WO2021139494 A1 WO 2021139494A1 CN 2020136401 W CN2020136401 W CN 2020136401W WO 2021139494 A1 WO2021139494 A1 WO 2021139494A1
Authority
WO
WIPO (PCT)
Prior art keywords
animal body
area
image
calibration board
animal
Prior art date
Application number
PCT/CN2020/136401
Other languages
French (fr)
Chinese (zh)
Inventor
梅栋
汤鑫
余勇健
齐宪标
肖嵘
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021139494A1 publication Critical patent/WO2021139494A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/08—Insurance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This application relates to artificial intelligence, and in particular to a method, device and storage medium for online animal claims verification based on a monocular camera.
  • the insurance of livestock is a very large and stable insurance type.
  • farmers will insure their livestock, such as pigs and cows, for each individual when they are young, or they will also be insured when transporting livestock. Livestock is insured.
  • the insurance company will make compensation for each animal.
  • Insurance compensation refers to the process by which the insurer verifies the insured subject information and decides whether to pay and the amount of compensation. This usually requires obtaining the facial features, body size, and weight characteristics of the animal body to assist.
  • the insurance applicant contacts the company, and the company dispatches a salesperson to the site to verify the compensation.
  • Salespersons generally use their own tools to measure animal body size, weight and other information, establish insurance files, and complete the work of compensation.
  • the agricultural insurance and livestock insurance is characterized by the fact that the single amount is relatively small compared to people and vehicles. Generally, the amount involved is only a few hundred yuan, but the frequency is very high. The cost of labor and transportation for a single input The cost of compensation for urban residents and cars is much higher. If the salesperson's business level is not high or reach an agreement with the policyholder, it is easy to cause insurance fraud.
  • the applicant found that as of now, there is no technology that can adapt to the measurement of animal bodies on the mobile terminal and can be conveniently applied to the online claims of animal bodies.
  • the inventor Realize that it is possible to use mobile phones for animal identification, body size, and weight identification. In view of this, it is necessary to develop a method that can obtain the body size and weight of the animal body based on the mobile terminal for online compensation.
  • the present application provides a method, device and storage medium for online claims of an animal body based on a monocular camera, the main purpose of which is to obtain the body size and weight information of the animal body by identifying the two-dimensional image of the animal body.
  • the present application provides an online animal compensation method based on a monocular camera, which includes the following steps:
  • the monocular camera Control the monocular camera to focus on the animal body and the calibration board.
  • the animal body is placed on the ground and the calibration board is placed under the abdomen of the animal body.
  • the shooting screen of the monocular camera shows the animal body area frame and the calibration board area frame. Place in the animal area frame;
  • Control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
  • the pre-segmented image of the preset size is segmented with the animal body as the center and sent to the trained weight recognition model, and the body length and weight information are output, and combined with the type information obtained by the facial image recognition of the animal body, the compensation result is determined.
  • This application also provides an online animal compensation device based on a monocular camera, including:
  • Focusing module used to control the monocular camera to focus on the animal body and the calibration board, where the animal body is placed on the ground side, the calibration board is placed under the animal body's abdomen, and the animal body area frame and the calibration board are displayed in the shooting screen of the monocular camera Area frame, the animal body is placed in the animal body area frame;
  • the shooting compliance judgment module judges whether the IOU of the smallest outsourcing rectangular area of the animal body in the current frame and the animal body area frame displayed in the shooting screen is greater than the preset intersection threshold, if it is greater, then continue to execute, otherwise it prompts to refocus;
  • the animal body segmentation module is used to control the monocular camera to capture the image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
  • the weight recognition module which uses the animal body as the center to segment a pre-segmented image of a preset size and sends it to the trained weight recognition model, outputs the body length and weight information, and combines the type information obtained by the animal body's facial image recognition to determine the compensation result.
  • This application also provides an electronic device, which includes:
  • At least one processor and,
  • a memory communicatively connected with the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the animal body based on the monocular camera as described above. Online compensation method.
  • the application also provides a computer-readable storage medium that stores a computer program that, when executed by a processor, realizes the above-mentioned method for online animal compensation based on a monocular camera.
  • the identification result of the unified model is used as the compensation index, thereby ensuring the fairness of the most fundamental criteria for compensation to all users.
  • FIG. 1 is a schematic diagram of the steps of an online animal compensation method based on a monocular camera according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of the calibration board of the embodiment of the present application being smaller than the animal body area frame;
  • FIG. 3 is a schematic diagram of the alignment of the calibration board and the animal body area frame of the embodiment of the present application.
  • FIG. 4 is a schematic diagram of the calibration board of the embodiment of the present application being larger than the animal body area frame;
  • Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present application.
  • Fig. 6 is a block diagram of an online animal compensation device based on a monocular camera according to an embodiment of the present application.
  • This embodiment proposes an online animal compensation method based on a monocular camera, which includes the following steps:
  • Step S1 control the monocular camera to focus on the animal body and the calibration board.
  • the animal body 50 is placed on the ground and the calibration board 20 is placed under the abdomen of the animal body 50, so that the side of the animal body and the calibration board 20 are In the horizontal direction, an animal body area frame 30 and a calibration board area frame 80 are displayed in the shooting picture 60 of the monocular camera.
  • the calibration board area frame 80 is located in the animal body area frame, and the animal body 50 is placed in the animal body area frame 30.
  • the calibration plate 20 and the calibration plate area 80 are completely overlapped during shooting, so that the size of the photographed animal body can have the same reference scale, so as to facilitate the weight recognition in the later stage. It can be seen from FIG. 4 and FIG. 5 that in the case of the same animal body area frame 30, due to the different shooting distances, the smallest outer rectangles of the obtained animal bodies are different. If the calibration board 20 is not used to lock the proportion of the captured animal body, it is possible that the distance between the monocular camera and the animal body is different, causing the same animal body to be photographed in different sizes, which will cause body size and weight recognition. error. By fixing the calibration plate 20 as a marker, the size of the photographed animal body is based on the calibration plate, which can prevent the same animal body from getting a different animal body size due to different shooting distances, which leads to deviations in weight recognition.
  • the size coordinates of the animal area frame and the calibration board area frame can be: the upper left, upper right, lower right, and lower left coordinates of the animal area frame are (0.2*Width,0.1*height), (0.8*Width,0.1*height), (0.2*Width,0.6*height), (0.2*Width,0.6*height), the coordinates of the upper left, upper right, lower right, and lower left of the calibration board area are (0.4*Width, 0.65*height), (0.6*Width, 0.65*height), (0.4*Width,0.8*height), (0.6*Width,0.8*height), where Width is the width parameter and height is the height parameter.
  • Step S2 Perform image quality judgment on the captured picture, including the calibration scale judgment and the target animal body area judgment.
  • the calibration scale determination means that the ratio scale I between the area of the calibration plate detected in the shooting image and the area of the calibration plate area is greater than a set ratio scale threshold, for example, the ratio scale threshold is 0.8. If it is less than the set ratio scale threshold, it will prompt to refocus.
  • I ratio scale is the meaning of S b1 and S b2 area of intersection coincides with S b1, S b2 ratio of coverage area of the physical meaning of the two degree of agreement, I calculated ratio scale as follows:
  • S b1 represents the area of the calibration plate obtained in the current frame
  • S b2 represents the area of the calibration board preset in the screen.
  • the judgment condition of the target animal body area is that the IOU (IOU is the abbreviation of the intersection ratio) of the animal body's smallest outer rectangular area 40 in the current frame and the animal body area frame 30 preset in the screen is greater than the intersection threshold.
  • IOU IOU is the abbreviation of the intersection ratio
  • select the intersection The threshold is 0.75. If it is less than, it will prompt to refocus.
  • the specific formula of IOU is:
  • Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame
  • Sh2 is the area of the animal body area frame preset in the screen
  • FindChessboardCorners can be called to determine the detection and determination of the calibration board.
  • FindChessboardCorners is a commonly used image processing library Opencv built-in method, for example, a 40x40cm calibration board is composed of 8x8 squares. Therefore, the detection corners (black squares) The intersection point is the corner point) The threshold for the number of points is 49.
  • the calibration board is detected, and the pixel area set occupied by the calibration board is calculated according to the corresponding area of 8x8.
  • the pre-determination of the shooting picture is also performed before the determination of the calibration scale and the determination of the target animal body area.
  • the pre-determination of the shooting picture includes the determination of the general quality attributes of the image to be shot, including the resolution of the image, the degree of blurring of the image, and
  • the detection of the animal body in the picture includes the following steps:
  • the Laplacian operator can be used to detect the picture blur degree.
  • Opencv a cross-platform computer vision library issued under the BSD license, BSD is Unix The derivative system of
  • BSD is Unix The derivative system of
  • the Laplacian operator is used to measure the second derivative of the picture, which can reflect the rapidly changing area of the density in the picture ( That is, the boundary), convolve each pixel of the picture with the Laplacian operator, and then calculate the output variance.
  • the boundary variance of a clear picture will be relatively large, while the boundary information contained in a blurred picture is less, and the variance is small .
  • the variance of consecutive 2s is less than the blur threshold, it is regarded as blur.
  • the smaller the variance the higher the blur degree of the picture.
  • Step S3 Control the monocular camera to shoot images containing the animal body and the calibration board, and identify the animal body area and the calibration board on the image.
  • image recognition is used, and the image judged by the image quality is input into the Cascade RCNN network model for animal Recognition of the body area, the output is the mask of the smallest outer rectangular area of the animal body, and the coordinates of the four corner points of the smallest outer rectangular area of the animal body are obtained (X1, Y1), (X2, Y2), (X3, Y3), ( X4, Y4), where X1, X2, X3, X4 are the abscissa of the upper left corner, the abscissa of the upper right corner, the abscissa of the lower left corner, and the abscissa of the lower right corner of the animal body's smallest enclosing rectangular area.
  • Y1, Y2, Y3, Y4 are The ordinate of the upper left corner, the ordinate of the upper right corner, the ordinate of the lower left corner, and the ordinate of the lower right corner of the animal body's smallest outer rectangular area.
  • the Cascade RCNN network model is a target detection model integrated in mmdetection (an open source target detection toolkit based on PyTorch).
  • Step S4 segmenting a rectangular area of preset size with the animal body as the center as a pre-segmented image for the calculation of the weight recognition model.
  • the specific steps for obtaining the pre-segmented image containing the animal body are as follows:
  • the area of the calibration board is exactly the same as the area of the animal body area frame 30 for shooting.
  • the animal body size benchmarks obtained should be the same, but in practice, due to the shooting angle, shooting method, The visual difference is that there must be an error in the degree of overlap between the calibration board and the animal body area frame 30. Therefore, the ratio scale I between the area of the detection calibration board and the area frame of the animal body is used to compensate the error caused by human shooting through the ratio scale. From this, calculate the size D width and D height of the pre-segmented image that each image needs to be segmented.
  • D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
  • the four corner coordinates are calculated as follows:
  • P hgxs is the coordinate of the lower left corner of the pre-segmented image
  • X hgxs is the abscissa of the lower left corner of the pre-segmented image
  • Y hgxs is the ordinate of the lower left corner of the pre-segmented image.
  • Phgyx is the coordinate of the lower right corner of the pre-segmented image
  • X hgyx is the abscissa of the lower right corner of the pre-segmented image
  • X hgyx is the ordinate of the lower right corner of the pre-segmented image.
  • P hgys is the coordinates of the upper right corner of the pre-segmented image
  • X hgys is the abscissa of the upper right corner of the pre-segmented image
  • Y hgys is the ordinate of the upper right corner of the pre-segmented image.
  • Phgzs is the coordinates of the upper left corner of the pre-segmented image
  • X hgzs is the abscissa of the upper left corner of the pre-segmented image
  • Y hgzs is the ordinate of the upper left corner of the pre-segmented image.
  • Step S5 Send the obtained pre-segmented image to the weight recognition model.
  • the weight recognition model is the RESNET-50 network, which is a deep learning model that has undergone multi-task training and verification, and can output animal bodies.
  • the body length and weight information of the animal body is combined with the animal body type information to output the compensation result.
  • different types of pigs and different body size and weight characteristics correspond to different compensation prices.
  • the compensation price can be determined according to the type information, body size and weight information of the animal.
  • This embodiment uses 2000 pre-segmented images. Among them, 1800 are used as training images and 200 are used as verification images.
  • All pre-segmented images are uniformly scaled to a reasonable size, and this embodiment is scaled to a size of 640x640.
  • the network outputs two branches, the body length recognition branch and the weight recognition branch.
  • the body length recognition branch is used as the auxiliary input information of the weight recognition branch.
  • the preset accuracy threshold is reached.
  • the model can be identified.
  • G Input the scaled image to the RESNET-50 network, and output the relative indicators L_0 and W_0 of body length and weight.
  • step S1 it also includes step S0, taking a photo of the face of the insured animal, and using a fine-grained classification algorithm to determine the category of the animal body.
  • the fine-grained classification algorithm can distinguish pigs of various breeds with very similar biological characteristics. Cow picture. For example, what kind of pigs belong to (different types of pigs correspond to different compensation prices).
  • the fine-grained classification algorithm may be MTCNN, B-CNN algorithm, and the B-CNN algorithm has a high classification accuracy rate on the CUB bird data set.
  • the body weight and body measurement data finally obtained may be untrue due to the forgery of the calibration board.
  • the insured should use the calibration board designated by the insurance company to take pictures of animals.
  • the insured can imitate the calibration board, reduce the size of the calibration board, and make the calibration board fill the reference area of the calibration board of the mobile phone by shortening the shooting distance, so that the calibration board of the specified size can be combined
  • the weight and body size calculated by the algorithm are distorted in reference to the calculation model, and a larger prediction error is obtained.
  • This embodiment adopts the following method to ensure that it is possible to identify whether there is a forged calibration board: call the AI interface provided by the mobile phone supplier through the mobile phone to obtain the geographic location, camera parameters, IMU parameters (inertial measurement unit), and shoot according to the policyholder Perform 3D reconstruction of the scene, obtain the actual size of the calibration board, obtain the shooting distance according to the displacement sensor of the mobile phone, and determine whether the calibration board is specified according to the acquired shooting height and the actual size of the calibration board Calibration board. Specifically, for the same calibration board, the image size of the monocular camera at a fixed shooting height should be the same. For the calibration board designated by the insurance company, the corresponding data has its shooting size according to different shooting heights. The size of the calibration board obtained from the 3D scene should be consistent with the data, otherwise it is considered that the calibration board is not a designated calibration board.
  • Obtaining a 3D scene includes the following steps:
  • Camera calibration Regard the internal parameters of the camera, due to the integration of many sensors inside the mobile phone, such as IMU, camera and even structured light module, various mobile phone companies or supporting companies also provide AI interface platforms. For example, Google's AI_core, Facebook's, Huawei's HUAWEI_AI, and Apple's AR_KIT can all call the AI interface of the mobile phone supplier through the app. So that each application can call the AI interface to obtain corresponding data as needed. According to the camera internal parameters combined with the pose estimation algorithm, the camera external parameters can be obtained.
  • the animal online compensation device 200 based on a monocular camera can be installed in an electronic device.
  • the online animal compensation device 200 based on a monocular camera may include a focusing module 201, a shooting compliance determination module 202, an animal body segmentation module 203, and a weight recognition module 204.
  • the module mentioned in the present invention refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, which are stored in the memory of the electronic device.
  • each module is as follows:
  • the focusing module 201 is used to control the monocular camera to focus on the animal body and the calibration board.
  • the animal body 50 is placed on the ground on the side, and the calibration board 20 is placed under the abdomen of the animal body 50, so that the side of the animal body is aligned with the calibration board.
  • the boards 20 are all in the horizontal direction.
  • the shooting screen 60 of the monocular camera displays the animal body area frame 30 and the calibration board area frame 80.
  • the calibration board area frame 80 is located in the animal body area frame, and the animal body 50 is placed in the animal body area. Box 30.
  • the calibration plate 20 and the calibration plate area 80 are completely overlapped during shooting, so that the size of the photographed animal body can have the same reference scale, so as to facilitate the weight recognition in the later stage. It can be seen from FIG. 4 and FIG. 5 that in the case of the same animal body area frame 30, due to the different shooting distances, the smallest outer rectangles of the obtained animal bodies are different. If the calibration board 20 is not used to lock the proportion of the captured animal body, it is possible that the distance between the monocular camera and the animal body is different, causing the same animal body to be photographed in different sizes, which will cause body size and weight recognition. error. By fixing the calibration plate 20 as a marker, the size of the photographed animal body is based on the calibration plate, which can prevent the same animal body from getting a different animal body size due to different shooting distances, which leads to deviations in weight recognition.
  • the shooting compliance determination module 202 is used to determine the image quality of the shooting picture, including the calibration scale determination and the target animal body area determination.
  • the calibration scale determination means that the ratio scale I between the area of the calibration plate detected in the shooting image and the area of the calibration plate area is greater than a set ratio scale threshold, for example, the ratio scale threshold is 0.8. If it is less than the set ratio scale threshold, it will prompt to refocus.
  • I ratio scale is the meaning of S b1 and S b2 area of intersection coincides with S b1, S b2 ratio of coverage area of the physical meaning of the two degree of agreement, I calculated ratio scale as follows:
  • S b1 represents the area of the calibration plate obtained in the current frame
  • S b2 represents the area of the calibration board preset in the screen.
  • the judgment condition of the target animal body area is that the IOU (IOU is the abbreviation of the intersection ratio) of the animal body's smallest outer rectangular area 40 in the current frame and the animal body area frame 30 preset in the screen is greater than the intersection threshold.
  • IOU IOU is the abbreviation of the intersection ratio
  • select the intersection The threshold is 0.75. If it is less than, it will prompt to refocus.
  • the specific formula of IOU is:
  • Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame
  • Sh2 is the area of the animal body area frame preset in the screen
  • Opencv's FindChessboardCorners function can be called to determine the detection and determination of the calibration board.
  • FindChessboardCorners is a commonly used image processing library built-in method of Opencv.
  • a 40x40cm calibration board is used, which is composed of 8x8 squares. Therefore, the detection corners are set. (The point where the black squares intersect is the corner point)
  • the threshold of the number is 49.
  • the pre-determination of the shooting picture is also performed before the determination of the calibration scale and the determination of the target animal body area.
  • the pre-determination of the shooting picture includes the determination of the general quality attributes of the image to be shot, including the resolution of the image, the degree of blurring of the image, and
  • the detection of the animal body in the picture includes the following steps:
  • the Laplacian operator can be used to detect the picture blur degree.
  • Opencv a cross-platform computer vision library issued under the BSD license, BSD is Unix The derivative system of
  • BSD is Unix The derivative system of
  • the Laplacian operator is used to measure the second derivative of the picture, which can reflect the rapidly changing area of the density in the picture ( That is, the boundary), convolve each pixel of the picture with the Laplacian operator, and then calculate the output variance.
  • the boundary variance of a clear picture will be relatively large, while the boundary information contained in a blurred picture is less, and the variance is small .
  • the variance of consecutive 2s is less than the blur threshold, it is regarded as blur.
  • the smaller the variance the higher the blur degree of the picture.
  • the animal body segmentation module 203 controls the monocular camera to capture images containing the animal body and the calibration board, and recognizes the animal body area and the calibration board on the image. Specifically, image recognition is used, and the image judged by the image quality is input to Cascade RCNN
  • the network model recognizes the animal body area, and the output is the mask of the smallest outer rectangular area of the animal body, and the coordinates (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), where X1, X2, X3, X4 are the abscissa of the upper left corner, the abscissa of the upper right corner, the abscissa of the lower left corner, and the abscissa of the lower right corner of the animal body's smallest enclosing rectangular area, Y1, Y2, Y3 and Y4 are the ordinate of the upper left corner, the ordinate of the upper right corner, the ordinate of the lower left corner, and
  • Step S4 segmenting a rectangular area of preset size with the animal body as the center as a pre-segmented image for the calculation of the weight recognition model.
  • the specific steps for obtaining the pre-segmented image containing the animal body are as follows:
  • the area of the calibration board is exactly the same as the area of the animal body area frame 30 for shooting.
  • the animal body size benchmarks obtained should be the same, but in practice, due to the shooting angle, shooting method, The visual difference is that there must be an error in the degree of overlap between the calibration board and the animal body area frame 30. Therefore, the ratio scale I between the area of the detection calibration board and the area frame of the animal body is used to compensate the error caused by human shooting through the ratio scale. From this, calculate the size D width and D height of the pre-segmented image that each image needs to be segmented.
  • D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
  • the four corner coordinates are calculated as follows:
  • P hgxs is the coordinate of the lower left corner of the pre-segmented image
  • X hgxs is the abscissa of the lower left corner of the pre-segmented image
  • Y hgxs is the ordinate of the lower left corner of the pre-segmented image.
  • Phgyx is the coordinate of the lower right corner of the pre-segmented image
  • X hgyx is the abscissa of the lower right corner of the pre-segmented image
  • X hgyx is the ordinate of the lower right corner of the pre-segmented image.
  • P hgys is the coordinates of the upper right corner of the pre-segmented image
  • X hgys is the abscissa of the upper right corner of the pre-segmented image
  • Y hgys is the ordinate of the upper right corner of the pre-segmented image.
  • Phgzs is the coordinates of the upper left corner of the pre-segmented image
  • X hgzs is the abscissa of the upper left corner of the pre-segmented image
  • Y hgzs is the ordinate of the upper left corner of the pre-segmented image.
  • the weight recognition module 204 is used to send the obtained pre-segmented images to the weight recognition model.
  • the weight recognition model is the RESNET-50 network, which is a deep learning model that has undergone multi-task training and verification. This can output the body length and weight information of the animal body. And combined with the type of animal information to output the results of the compensation. For example, different types of pigs and different body size and weight characteristics correspond to different compensation prices.
  • the compensation price can be determined according to the type information, body size and weight information of the animal.
  • This embodiment uses 2000 pre-segmented images. Among them, 1800 are used as training images and 200 are used as verification images.
  • the network outputs two branches, the body length recognition branch and the weight recognition branch.
  • the body length recognition branch is used as the auxiliary input information of the weight recognition branch.
  • the preset accuracy threshold is reached.
  • the model can be identified.
  • G Input the scaled image to the RESNET-50 network, and output the relative indicators L_0 and W_0 of body length and weight.
  • the electronic device 2 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions.
  • the electronic device 2 can be a mobile device that is easy to carry, such as a smart mobile device, a tablet computer, and a notebook computer.
  • the electronic device 2 at least includes a memory 21 and a processor 22 that are communicatively connected to each other through a line, and the monocular camera 70 is connected to the processor.
  • the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or a memory of the electronic device 2.
  • the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk, a smart media card (SMC), and a secure digital device equipped on the electronic device 2.
  • SMC smart media card
  • secure digital device equipped on the electronic device 2.
  • SD Secure Digital, SD
  • flash card Flash Card
  • the memory 21 may also include both the internal storage unit of the electronic device 2 and its external storage device.
  • the memory 21 is generally used to store an operating system and various application software installed in the electronic device 2, for example, the code of an animal body online compensation program based on a monocular camera.
  • the memory 21 can also be used to temporarily store various types of data that have been output or will be output.
  • the processor 22 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. It is used for running the program code or processing data stored in the memory 21, for example, running an animal body online compensation program based on a monocular camera.
  • CPU central processing unit
  • controller a controller
  • microcontroller a microprocessor
  • other data processing chips it is used for running the program code or processing data stored in the memory 21, for example, running an animal body online compensation program based on a monocular camera.
  • FIG. 6 only shows the electronic device 2 with the memory 21 and the processor 22, but it should be understood that it is not required to implement all the illustrated components, and more or fewer components may be implemented instead. .
  • the memory 21 containing a readable storage medium may include an operating system, an animal body online compensation program based on a monocular camera, and the like.
  • the processor 22 implements the steps from S1 to S5 as described above when the animal body online compensation program based on the monocular camera in the memory 21 is executed, which will not be repeated here.
  • the embodiment of the present application also proposes a computer-readable storage medium.
  • the computer-readable storage medium may be non-volatile or volatile.
  • the computer-readable storage medium may be a hard disk, a multimedia card, SD card, flash memory card, SMC, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, etc. or Any combination of several.
  • the computer-readable storage medium includes an animal online compensation program based on a monocular camera, etc., and the following operations are implemented when the monocular camera-based online animal compensation program is executed by the processor 22:
  • S1 control the monocular camera to focus on the animal body and the calibration board, where the animal body is placed on the ground side, and the calibration board is placed under the abdomen of the animal body.
  • the shooting screen of the monocular camera displays the animal body area frame and the calibration board area frame. The animal body is placed in the animal body area frame;
  • S3 control the monocular camera to take images containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
  • the specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned monocular camera-based animal online compensation method and the electronic device 2, and will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed are an animal body online claim settlement method and apparatus based on a monocular camera, and a storage medium. The method comprises: controlling a monocular camera to focus on an animal body and a calibration plate, wherein the animal body is laterally placed on the ground, the calibration plate is placed below the abdomen of the animal body, there is an animal body area frame and a calibration plate area frame in a photographed picture, and the animal body is placed in the animal body area frame; if an IOU, in the current frame, of the minimum wrapping rectangular area of the animal body and the animal body area frame is greater than a preset intersection-union threshold value, continuing to execute same, otherwise, prompting re-focusing; photographing an image which includes the animal body and the calibration plate, inputting the image into a Cascade network model to perform animal body area identification, and outputting a minimum wrapping rectangular area mask of the animal body; and segmenting a pre-segmented image by means of taking the animal body as the center, sending the pre-segmented image to a weight recognition model, outputting body length and weight information, and performing claim settlement by means of combining type information obtained by facial image recognition of the animal body. The method guarantees the fairness criterion and reduces the claim settlement cost by means of performing on-line claim settlement by using a model.

Description

基于单目摄像头的动物体在线核赔方法、装置及存储介质Monocular camera-based animal body online compensation method, device and storage medium
本申请要求于2020年08月27日提交中国专利局、申请号为202010879333.1，发明名称为“基于单目摄像头的动物体在线核赔方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on August 27, 2020, with the application number 202010879333.1, and the invention titled "Monocular Camera-based On-line Animal Compensation Method, Device and Storage Medium". The entire content is incorporated into this application by reference.
技术领域Technical field
本申请涉及人工智能，尤其涉及一种基于单目摄像头的动物体在线核赔方法、装置及存储介质。This application relates to artificial intelligence, and in particular to a method, device and storage medium for online animal claims verification based on a monocular camera.
背景技术Background technique
在农险行业中，对家畜的投保是一项很庞大且稳定的险种，一般农户会在家畜，比如猪、牛幼年时期即针对每一个体进行投保，或者，农户在运输家畜时也会对家畜进行投保。在家畜死亡时，保险公司会针对每一个动物体进行核赔，保险核赔是指保险人核实投保的标的信息，决定是否赔付以及赔付数额的过程。这通常需要获取动物体的面部特征、体尺、体重特征来辅助。In the agricultural insurance industry, the insurance of livestock is a very large and stable insurance type. Generally, farmers will insure their livestock, such as pigs and cows, for each individual when they are young, or they will also be insured when transporting livestock. Livestock is insured. When the livestock dies, the insurance company will make compensation for each animal. Insurance compensation refers to the process by which the insurer verifies the insured subject information and decides whether to pay and the amount of compensation. This usually requires obtaining the facial features, body size, and weight characteristics of the animal body to assist.
相对于人险和车险，动物的核赔具有很大的个体辨识区分困难度，在线通过业务人员肉眼估计的方法，对业务员的经验要求很高，培养成本大，同时也违背了对维护各被保险个体之间的公平的保险准则，不利于公司业务发展。Compared with human insurance and auto insurance, the verification of animal claims has great difficulty in individual identification. The online estimation method by business personnel requires high experience and high training costs. It also violates the maintenance requirements. Fair insurance standards among insured individuals are not conducive to the development of the company's business.
针对动物核赔，业界主流方案有如下几种：For animal claims, there are several mainstream solutions in the industry:
1、业务员现场传统方法人工核赔：1. On-site traditional methods of manual compensation for salesmen:
一般通过投保人联系公司，公司派遣业务员前往现场核赔。Generally, the insurance applicant contacts the company, and the company dispatches a salesperson to the site to verify the compensation.
业务员一般通过自带的工具来测量动物的体尺、体重等信息，建立保险档案，完成核赔的工作。Salespersons generally use their own tools to measure animal body size, weight and other information, establish insurance files, and complete the work of compensation.
(1)优点：业务员现场核赔是最传统的核赔方法，有着各项完整的流程，不容易出现未知的问题。而且随着业务员处理的经验增多，同一业务员处理的效果会越来越好，达到一个熟练的水平。(1) Advantages: On-site verification of claims by salespersons is the most traditional method of verification, with various complete procedures, and unknown problems are not easy to appear. And as the salesperson's handling experience increases, the same salesperson's handling effect will get better and better, reaching a level of proficiency.
(2)缺点：农险家畜的投保，特点是单体金额相对人、车较小，一般涉及的金额也只是几百元人民币，但频次很高，单次投入核赔的人力、交通成本比城市人、车的核赔，成本占比高很多。如果业务员业务水平不高或者和投保人达成协议，也容易造成骗保。(2) Disadvantages: The agricultural insurance and livestock insurance is characterized by the fact that the single amount is relatively small compared to people and vehicles. Generally, the amount involved is only a few hundred yuan, but the frequency is very high. The cost of labor and transportation for a single input The cost of compensation for urban residents and cars is much higher. If the salesperson's business level is not high or reach an agreement with the policyholder, it is easy to cause insurance fraud.
2、业务员现场采集数据远程辅助核赔：2. The salesman collects data on-site and remotely assists in compensation:
针对方案1对业务员水平的较高要求以及不可控的风险存在，在业界各公司，开始加入一些信息化的举措，比如业务员拍摄的图片、测量的体重同步上传公司数据库，公司采用申请或抽检的方式，由高级专业专员远程核赔。In response to the relatively high requirements for the level of salespersons and uncontrollable risks in Option 1, companies in the industry have begun to add some information-based measures, such as pictures taken by salespersons, and the measured weights are uploaded to the company database simultaneously. The company adopts application or The method of random inspection is remotely verified by a senior professional commissioner.
(1)优点：这样使得高经验水平的人员可以远程协助现场业务员，加上信息的线上化，可以提高核赔的下限水平。同时多人的核赔机制，使得核赔的标准，从多数业务员各异，做到主要取决于数量较少的高级专员，这样一定程度可以保障核赔的公平性。(1) Advantages: This allows highly experienced personnel to remotely assist on-site salespersons, and online information can increase the lower limit of compensation. At the same time, the multi-person compensation mechanism makes compensation standards different from those of most salespersons, and it mainly depends on a small number of high-level commissioners, so that the fairness of compensation can be guaranteed to a certain extent.
(2)缺点：还是需要投入相当多的人力来进行现场数据采集，通过少量丰富经验人员和现场多数业务员的协同，依然还是依赖人来评价核赔的标准。不利于业务的自动化。(2) Disadvantages: It still needs to invest a lot of manpower to collect on-site data. Through the collaboration of a small number of experienced personnel and most on-site salespersons, it is still dependent on people to evaluate the standard of compensation. It is not conducive to the automation of the business.
综上，申请人发现，截至目前还未有能够适应移动端应用动物体的测量，便捷的应用在动物体在线核赔上的技术，随着深度学习技术的发展与手机功能的成熟，发明人意识到，利用手机进行动物身份鉴别、体尺、体重识别成为了可能。有鉴于此，有必要开发一种能够基于移动端获得动物体的体尺、体重进行在线核赔的方法。In summary, the applicant found that as of now, there is no technology that can adapt to the measurement of animal bodies on the mobile terminal and can be conveniently applied to the online claims of animal bodies. With the development of deep learning technology and the maturity of mobile phone functions, the inventor Realize that it is possible to use mobile phones for animal identification, body size, and weight identification. In view of this, it is necessary to develop a method that can obtain the body size and weight of the animal body based on the mobile terminal for online compensation.
发明内容Summary of the invention
本申请提供一种基于单目摄像头的动物体在线核赔方法、装置及存储介质，其主要目的在于通过识别动物体的二维图像，获得动物体的体尺和体重信息。The present application provides a method, device and storage medium for online claims of an animal body based on a monocular camera, the main purpose of which is to obtain the body size and weight information of the animal body by identifying the two-dimensional image of the animal body.
为实现上述目的，本申请提供的一种基于单目摄像头的动物体在线核赔方法，包括以下步骤：In order to achieve the above-mentioned purpose, the present application provides an online animal compensation method based on a monocular camera, which includes the following steps:
控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Control the monocular camera to focus on the animal body and the calibration board. The animal body is placed on the ground and the calibration board is placed under the abdomen of the animal body. The shooting screen of the monocular camera shows the animal body area frame and the calibration board area frame. Place in the animal area frame;
判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；Determine whether the minimum outsourcing rectangle area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting screen are greater than the preset intersection threshold, if it is greater, continue to execute, otherwise prompt to refocus;
控制单目摄像头拍摄包含动物体和标定板的图像，将所述图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；Control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The pre-segmented image of the preset size is segmented with the animal body as the center and sent to the trained weight recognition model, and the body length and weight information are output, and combined with the type information obtained by the facial image recognition of the animal body, the compensation result is determined.
本申请还提供一种基于单目摄像头的动物体在线核赔装置，包括：This application also provides an online animal compensation device based on a monocular camera, including:
对焦模块，用于控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Focusing module, used to control the monocular camera to focus on the animal body and the calibration board, where the animal body is placed on the ground side, the calibration board is placed under the animal body's abdomen, and the animal body area frame and the calibration board are displayed in the shooting screen of the monocular camera Area frame, the animal body is placed in the animal body area frame;
拍摄合规判定模块，判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；The shooting compliance judgment module judges whether the IOU of the smallest outsourcing rectangular area of the animal body in the current frame and the animal body area frame displayed in the shooting screen is greater than the preset intersection threshold, if it is greater, then continue to execute, otherwise it prompts to refocus;
动物体分割模块，用于控制单目摄像头拍摄包含动物体和标定板的图像，将图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；The animal body segmentation module is used to control the monocular camera to capture the image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
体重识别模块，以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The weight recognition module, which uses the animal body as the center to segment a pre-segmented image of a preset size and sends it to the trained weight recognition model, outputs the body length and weight information, and combines the type information obtained by the animal body's facial image recognition to determine the compensation result.
本申请还提供一种电子设备，所述电子设备包括：This application also provides an electronic device, which includes:
至少一个处理器；以及，At least one processor; and,
与所述至少一个处理器通信连接的存储器；其中，A memory communicatively connected with the at least one processor; wherein,
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上所述的基于单目摄像头的动物体在线核赔方法。The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the animal body based on the monocular camera as described above. Online compensation method.
本申请还提供一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的基于单目摄像头的动物体在线核赔方法。The application also provides a computer-readable storage medium that stores a computer program that, when executed by a processor, realizes the above-mentioned method for online animal compensation based on a monocular camera.
而本方案针对远程动物核赔的问题，提出一套远程端对端的解决方案，具有以下有益的技术效果：In response to the problem of remote animal compensation, this program proposes a remote end-to-end solution, which has the following beneficial technical effects:
1、通过统一模型的识别结果作为核赔指标，从而保证了核赔最根本的准则对所有用户的公平性准则。1. The identification result of the unified model is used as the compensation index, thereby ensuring the fairness of the most fundamental criteria for compensation to all users.
2、将业务员现场核赔的人力节省采用现阶段人人配备的智能手机的单目摄像头作为采集设备，不需要为投保人额外配备采集设备，节省了公司经费，同时智能手机作为最新科技的集中平台，具有非常高的可信度，通过智能手机官方的AI接口获取的指标，亦可以作为防伪的输入。2. Use the monocular camera of the smart phone equipped by everyone at this stage as the collection device for the labor saving of the salesman’s on-site verification of claims. There is no need to equip the policyholder with additional collection equipment, which saves the company’s expenses. At the same time, the smart phone is the latest technology. The centralized platform has very high credibility. The indicators obtained through the official AI interface of the smartphone can also be used as the input for anti-counterfeiting.
3、利用深度学习的相关模型进行核赔，是一种可训练、可迭代的方法，并且每一次核赔输入的数据亦可以作为训练集的扩充，这样随着采集的数据不断扩增，能够不断的提高核赔测量的准确性。能节省高额的熟练业务员人力成本和指标的误赔开支(人工方法或者传统算法不具备可持续的迭代优化能力，投保人如果测得指标对其不利，比如体重测量较实际值偏轻则一定会申诉，如果对其有利，比如较实际值偏重，则会给公司造成额外的赔付金额)。3. It is a trainable and iterable method to use deep learning related models to verify claims, and the data input for each claim can also be used as an expansion of the training set, so that as the collected data continues to expand, it can be Continuously improve the accuracy of compensation measurement. It can save the high manpower cost of skilled salespersons and the miscompensation expenditure of indicators (manual methods or traditional algorithms do not have sustainable iterative optimization capabilities. If the policyholder measures indicators to its disadvantage, for example, the weight measurement is lighter than the actual value Will definitely appeal, if it is beneficial to it, such as heavier than the actual value, it will cause additional compensation to the company).
附图说明Description of the drawings
通过结合下面附图对其实施例进行描述，本申请的上述特征和技术优点将会变得更加清楚和容易理解。By describing its embodiments in conjunction with the following drawings, the above-mentioned features and technical advantages of the present application will become clearer and easier to understand.
图1是本申请实施例的基于单目摄像头的动物体在线核赔方法的步骤示意图；FIG. 1 is a schematic diagram of the steps of an online animal compensation method based on a monocular camera according to an embodiment of the present application;
图2是本申请实施例的标定板比动物体区域框小的示意图；FIG. 2 is a schematic diagram of the calibration board of the embodiment of the present application being smaller than the animal body area frame;
图3是本申请实施例的标定板与动物体区域框重合的示意图；FIG. 3 is a schematic diagram of the alignment of the calibration board and the animal body area frame of the embodiment of the present application;
图4是本申请实施例的标定板比动物体区域框大的示意图；FIG. 4 is a schematic diagram of the calibration board of the embodiment of the present application being larger than the animal body area frame;
图5是本申请实施例的电子设备的示意图；Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present application;
图6是本申请实施例的基于单目摄像头的动物体在线核赔装置的模块构成图。Fig. 6 is a block diagram of an online animal compensation device based on a monocular camera according to an embodiment of the present application.
具体实施方式Detailed ways
下面将参考附图来描述本申请所述的基于单目摄像头的动物体在线核赔方法、装置及存储介质的实施例。本领域的普通技术人员可以认识到，在不偏离本申请的精神和范围的情况下，可以用各种不同的方式或其组合对所描述的实施例进行修正。因此，附图和描述在本质上是说明性的，而不是用于限制权利要求的保护范围。此外，在本说明书中，附图未按比例画出，并且相同的附图标记表示相同的部分。Hereinafter, embodiments of the method, device and storage medium for online animal compensation based on a monocular camera described in the present application will be described with reference to the accompanying drawings. Those of ordinary skill in the art may realize that the described embodiments can be modified in various different ways or combinations thereof without departing from the spirit and scope of the present application. Therefore, the drawings and description are illustrative in nature, and are not used to limit the scope of protection of the claims. In addition, in this specification, the drawings are not drawn to scale, and the same reference numerals denote the same parts.
本实施例提出一种基于单目摄像头的动物体在线核赔方法，包括以下步骤：This embodiment proposes an online animal compensation method based on a monocular camera, which includes the following steps:
步骤S1，控制单目摄像头对焦动物体和标定板，如图2所示，动物体50侧放于地面,标定板20置于动物体50腹部下方，使得动物体的侧面与标定板20都在水平方向上，单目摄像头的拍摄画面60中显示有动物体区域框30和标定板区域框80，标定板区域框80位于动物体区域框内，动物体50置于动物体区域框30中。Step S1, control the monocular camera to focus on the animal body and the calibration board. As shown in Figure 2, the animal body 50 is placed on the ground and the calibration board 20 is placed under the abdomen of the animal body 50, so that the side of the animal body and the calibration board 20 are In the horizontal direction, an animal body area frame 30 and a calibration board area frame 80 are displayed in the shooting picture 60 of the monocular camera. The calibration board area frame 80 is located in the animal body area frame, and the animal body 50 is placed in the animal body area frame 30.
其中，优选地，是拍摄时使得标定板20与标定板区域80完全重合，可以使得拍摄的动物体的大小参照的尺度相同，以利于后期的体重识别。从图4、图5可以看出，在同样的动物体区域框30的情况下，由于拍摄距离不同，导致得到的动物体最小外包矩形不同。而不采用标定板20来锁定拍摄的动物体的比例的话，则可能由于单目摄像头距离动物体的距离不同，导致拍摄的相同的动物体呈现出大小不同的形式，会造成体尺、体重识别误差。通过固定标定板20作为标记物，使得所拍摄的动物体的大小都是以标定板为基准的，可以防止由于拍摄距离不同导致相同的动物体却得到不同的动物体大小导致体重识别出现偏差。Wherein, preferably, the calibration plate 20 and the calibration plate area 80 are completely overlapped during shooting, so that the size of the photographed animal body can have the same reference scale, so as to facilitate the weight recognition in the later stage. It can be seen from FIG. 4 and FIG. 5 that in the case of the same animal body area frame 30, due to the different shooting distances, the smallest outer rectangles of the obtained animal bodies are different. If the calibration board 20 is not used to lock the proportion of the captured animal body, it is possible that the distance between the monocular camera and the animal body is different, causing the same animal body to be photographed in different sizes, which will cause body size and weight recognition. error. By fixing the calibration plate 20 as a marker, the size of the photographed animal body is based on the calibration plate, which can prevent the same animal body from getting a different animal body size due to different shooting distances, which leads to deviations in weight recognition.
动物体区域框和标定板区域框的尺寸坐标可以为：动物体区域框坐标左上、右上、右下、左下分别为(0.2*Width,0.1*height)、(0.8*Width,0.1*height)、(0.2*Width,0.6*height)、(0.2*Width,0.6*height)，标定板区域框坐标左上、右上、右下、左下分别为(0.4*Width,0.65*height)、(0.6*Width,0.65*height)、(0.4*Width,0.8*height)、(0.6*Width,0.8*height)，其中，Width为宽度参数，height为高度参数。The size coordinates of the animal area frame and the calibration board area frame can be: the upper left, upper right, lower right, and lower left coordinates of the animal area frame are (0.2*Width,0.1*height), (0.8*Width,0.1*height), (0.2*Width,0.6*height), (0.2*Width,0.6*height), the coordinates of the upper left, upper right, lower right, and lower left of the calibration board area are (0.4*Width, 0.65*height), (0.6*Width, 0.65*height), (0.4*Width,0.8*height), (0.6*Width,0.8*height), where Width is the width parameter and height is the height parameter.
步骤S2，对拍摄画面进行图像质量判定，包括标定尺度判定、目标动物体区域判定。Step S2: Perform image quality judgment on the captured picture, including the calibration scale judgment and the target animal body area judgment.
其中，标定尺度判定是指拍摄画面中检测的标定板面积与标定板区域框面积之间的比例尺度I大于设定的比例尺度阈值，例如比例尺度阈值为0.8。如果小于设定的比例尺度阈值，则提示重新对焦。比例尺度I的含义是S b1与S b2的相交重合区域面积与S b1、S b2覆盖区域面积的比值，物理意义的是两者吻合程度，比例尺度I的计算公式如下： Wherein, the calibration scale determination means that the ratio scale I between the area of the calibration plate detected in the shooting image and the area of the calibration plate area is greater than a set ratio scale threshold, for example, the ratio scale threshold is 0.8. If it is less than the set ratio scale threshold, it will prompt to refocus. I ratio scale is the meaning of S b1 and S b2 area of intersection coincides with S b1, S b2 ratio of coverage area of the physical meaning of the two degree of agreement, I calculated ratio scale as follows:
Figure PCTCN2020136401-appb-000001
Figure PCTCN2020136401-appb-000001
其中，S b1代表为当前帧获得的标定板的面积； Among them, S b1 represents the area of the calibration plate obtained in the current frame;
S b2代表为画面中预设的标定板区域面积。 S b2 represents the area of the calibration board preset in the screen.
目标动物体区域的判断条件是当前帧的动物体最小外包矩形区域40和画面中预设的动物体区域框30的IOU(IOU是交并比的缩写)大于交并阈值，此处选取交并阈值为0.75，若小于，则提示重新对焦，IOU的具体公式为：The judgment condition of the target animal body area is that the IOU (IOU is the abbreviation of the intersection ratio) of the animal body's smallest outer rectangular area 40 in the current frame and the animal body area frame 30 preset in the screen is greater than the intersection threshold. Here, select the intersection The threshold is 0.75. If it is less than, it will prompt to refocus. The specific formula of IOU is:
Figure PCTCN2020136401-appb-000002
Figure PCTCN2020136401-appb-000002
其中，S h1是当前帧的动物体最小外包矩形区域； Among them, Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame;
S h2是画面中预设的动物体区域框的区域； Sh2 is the area of the animal body area frame preset in the screen;
IOU是S h1与S h2的相交重合区域面积与S h1、S h2覆盖区域面积的比值。 IOU intersect with S h2 S h1 coincides with the area of S h1, the ratio of the area of coverage S h2.
具体地，可以是调用FindChessboardCorners进行标定板的检测判定，(FindChessboardCorners是常用的图像处理库Opencv的内置方法，例如采用40x40cm的标定板，由8x8个方格组成。因而设定检测角点(黑色方块相交的点为角点)个数的阈值为49个，当满足要求时通过标定板检测，按照8x8的对应区域计算出标定板所占的像素区域集合。Specifically, FindChessboardCorners can be called to determine the detection and determination of the calibration board. (FindChessboardCorners is a commonly used image processing library Opencv built-in method, for example, a 40x40cm calibration board is composed of 8x8 squares. Therefore, the detection corners (black squares) The intersection point is the corner point) The threshold for the number of points is 49. When the requirements are met, the calibration board is detected, and the pixel area set occupied by the calibration board is calculated according to the corresponding area of 8x8.
优选地，在进行标定尺度判定、目标动物体区域判定之前还进行拍摄画面预判定，拍摄画面预判定包括对将要拍摄的图像的通用质量属性的判定，包括图像的分辨率、图像的模糊程度和检测画面中动物体情况，包括如下步骤：Preferably, the pre-determination of the shooting picture is also performed before the determination of the calibration scale and the determination of the target animal body area. The pre-determination of the shooting picture includes the determination of the general quality attributes of the image to be shot, including the resolution of the image, the degree of blurring of the image, and The detection of the animal body in the picture includes the following steps:
B.检测当前拍摄画面的分辨率，如果分辨率设定小于阈值则弹出提示分辨率小于阈值(如1280*960)，提醒用户修改。B. Detect the resolution of the current shooting picture. If the resolution setting is less than the threshold, a pop-up prompts that the resolution is less than the threshold (such as 1280*960) to remind the user to modify.
C.满足B中条件后，提醒用户对焦动物体，逐帧检测画面模糊程度，可以采用拉普拉斯算子检测画面模糊程度，Opencv(基于BSD许可发行的跨平台计算机视觉库，BSD是Unix的衍生***)中提供了对拉普拉斯算子的封装方法，可以直接调用，其中，拉普拉斯算子是用来衡量图片的二阶导数，能够体现图片中密度快速变化的区域(即边界)，将画面的各点像素与拉普拉斯算子进行卷积，然后计算输出的方差，通常清晰图片的边界方差会比较大，而模糊图片中包含的边界信息少，方差较小。当连续2s的方差都小于模糊阈值则视为模糊，方差越小，画面模糊程度越高。C. After meeting the conditions in B, remind the user to focus on the animal body and detect the blur degree of the picture frame by frame. The Laplacian operator can be used to detect the picture blur degree. Opencv (a cross-platform computer vision library issued under the BSD license, BSD is Unix The derivative system of) provides an encapsulation method for the Laplacian operator, which can be directly called. Among them, the Laplacian operator is used to measure the second derivative of the picture, which can reflect the rapidly changing area of the density in the picture ( That is, the boundary), convolve each pixel of the picture with the Laplacian operator, and then calculate the output variance. Generally, the boundary variance of a clear picture will be relatively large, while the boundary information contained in a blurred picture is less, and the variance is small . When the variance of consecutive 2s is less than the blur threshold, it is regarded as blur. The smaller the variance, the higher the blur degree of the picture.
D.检测画面中有无动物体。如果检测不存在，则弹出提示对应目标检测失败，提醒用户将单目摄像头对准合格的动物体。D. Check whether there is an animal in the screen. If the detection does not exist, a pop-up prompt that the corresponding target detection failed, reminding the user to aim the monocular camera at the qualified animal body.
步骤S3，控制单目摄像头拍摄包含动物体和标定板的图像，对图像进行动物体区域和标定板识别，具体说，是采用图像识别，将通过图像质量判定的图像输入Cascade RCNN网络模型进行动物体区域的识别，输出为动物体最小外包矩形区域掩模，获得了动物体最小外包矩形区域的四个角点的坐标(X1，Y1)、(X2，Y2)、(X3，Y3)、(X4，Y4)，其中，X1、X2、X3、X4是动物体最小外包矩形区域的左上角横坐标、右上角横坐标、左下角横坐标、右下角横坐标，Y1、Y2、Y3、Y4是动物体最小外包矩形区域的左上角纵坐标、 右上角纵坐标、左下角纵坐标、右下角纵坐标。其中，Cascade RCNN网络模型是集成在mmdetection(基于PyTorch的开源目标检测工具包)中的一个目标检测模型。Step S3: Control the monocular camera to shoot images containing the animal body and the calibration board, and identify the animal body area and the calibration board on the image. Specifically, image recognition is used, and the image judged by the image quality is input into the Cascade RCNN network model for animal Recognition of the body area, the output is the mask of the smallest outer rectangular area of the animal body, and the coordinates of the four corner points of the smallest outer rectangular area of the animal body are obtained (X1, Y1), (X2, Y2), (X3, Y3), ( X4, Y4), where X1, X2, X3, X4 are the abscissa of the upper left corner, the abscissa of the upper right corner, the abscissa of the lower left corner, and the abscissa of the lower right corner of the animal body's smallest enclosing rectangular area. Y1, Y2, Y3, Y4 are The ordinate of the upper left corner, the ordinate of the upper right corner, the ordinate of the lower left corner, and the ordinate of the lower right corner of the animal body's smallest outer rectangular area. Among them, the Cascade RCNN network model is a target detection model integrated in mmdetection (an open source target detection toolkit based on PyTorch).
步骤S4，以动物体为中心分割出一个预设尺寸的长方形区域作为预分割图像，以便体重识别模型计算，获取包含动物体的预分割图像的具体步骤为：Step S4, segmenting a rectangular area of preset size with the animal body as the center as a pre-segmented image for the calculation of the weight recognition model. The specific steps for obtaining the pre-segmented image containing the animal body are as follows:
S41，理论上说，都以标定板的面积恰好与动物体区域框30的区域重合进行拍摄，则得到的动物体大小的基准应该都是一样的，但实际中，由于拍摄角度、拍摄手法、视觉上的差异，标定板与动物体区域框30的重合度肯定是有误差的。所以以检测标定板的面积与动物体区域框的比例尺度I，将人为拍摄的误差通过比例尺度来补偿。由此计算出各图像需要分割出的预分割图像的尺寸D 宽度、D 高度， S41. In theory, the area of the calibration board is exactly the same as the area of the animal body area frame 30 for shooting. The animal body size benchmarks obtained should be the same, but in practice, due to the shooting angle, shooting method, The visual difference is that there must be an error in the degree of overlap between the calibration board and the animal body area frame 30. Therefore, the ratio scale I between the area of the detection calibration board and the area frame of the animal body is used to compensate the error caused by human shooting through the ratio scale. From this, calculate the size D width and D height of the pre-segmented image that each image needs to be segmented.
Figure PCTCN2020136401-appb-000003
Figure PCTCN2020136401-appb-000003
其中，D 预设宽度、D 预设高度是比例尺度I为1时的预分割图像的尺寸。 Among them, D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
S42，计算动物体最小外包矩形的中心坐标P h，其具体计算公式为： S42: Calculate the center coordinate P h of the smallest outer rectangle of the animal body, and the specific calculation formula is:
P h＝(X h，Y h) P h ＝(X h , Y h )
其中：among them:
X h＝0.25*(X1+X2+X3+X4) X h ＝0.25*(X1+X2+X3+X4)
Y h＝0.25*(Y1+Y2+Y3+Y4) Y h ＝0.25*(Y1+Y2+Y3+Y4)
S43，计算分割的预分割图像的四个顶点坐标：S43. Calculate the coordinates of the four vertices of the segmented pre-segmented image:
四个角点坐标的计算如下：The four corner coordinates are calculated as follows:
1)1)
P hg＝(X hgxs，Y hgxs) P hg = (X hgxs , Y hgxs )
X hgxs＝X h–0.5*D 宽度 X hgxs = X h -0.5*D width
Y hgxs＝Y h–0.5*D 高度 Y hgxs = Y h -0.5*D height
其中，P hgxs是预分割图像左下角坐标； Among them, P hgxs is the coordinate of the lower left corner of the pre-segmented image;
X hgxs是预分割图像左下角横坐标； X hgxs is the abscissa of the lower left corner of the pre-segmented image;
Y hgxs是预分割图像左下角纵坐标。 Y hgxs is the ordinate of the lower left corner of the pre-segmented image.
2)2)
P hgyx＝(X hgyx，Y hgyx) P hgyx = (X hgyx , Y hgyx )
X hgyx＝X h+0.5*D 宽度 X hgyx = X h +0.5*D width
Y hgyx＝Y h–0.5*D 高度 Y hgyx = Y h -0.5*D height
其中，P hgyx是预分割图像右下角坐标； Among them, Phgyx is the coordinate of the lower right corner of the pre-segmented image;
X hgyx是预分割图像右下角横坐标； X hgyx is the abscissa of the lower right corner of the pre-segmented image;
X hgyx是预分割图像右下角纵坐标。 X hgyx is the ordinate of the lower right corner of the pre-segmented image.
3)3)
P hgys＝(X hgys，Y hgys) P hgys = (X hgys , Y hgys )
X hgys＝X h+0.5*D 宽度 X hgys ＝X h +0.5*D width
Y hgys＝Y h+0.5*D 高度 Y hgys = Y h +0.5*D height
P hgys是预分割图像右上角坐标； P hgys is the coordinates of the upper right corner of the pre-segmented image;
X hgys是预分割图像右上角横坐标； X hgys is the abscissa of the upper right corner of the pre-segmented image;
Y hgys是预分割图像右上角纵坐标。 Y hgys is the ordinate of the upper right corner of the pre-segmented image.
4)4)
P hgzs＝(X hgzs，Y hgzs) P hgzs = (X hgzs , Y hgzs )
X hgzs＝X h–0.5*D 宽度 X hgzs = X h -0.5*D width
Y hgzs＝Y h+0.5*D 高度 Y hgzs = Y h +0.5*D height
其中，P hgzs是预分割图像左上角坐标； Among them, Phgzs is the coordinates of the upper left corner of the pre-segmented image;
X hgzs是预分割图像左上角横坐标； X hgzs is the abscissa of the upper left corner of the pre-segmented image;
Y hgzs是预分割图像左上角纵坐标。 Y hgzs is the ordinate of the upper left corner of the pre-segmented image.
S44，以S43中获得的预分割图像对应的四个角坐标来对图像裁剪，获取的裁剪图像即为预分割图像。S44, crop the image by using four corner coordinates corresponding to the pre-segmented image obtained in S43, and the obtained cropped image is the pre-segmented image.
步骤S5，将获取的预分割图像送入体重识别模型，体重识别模型是RESNET-50网络，RESNET-50网络是一种深度学习模型，其经过了多任务训练和验证，由此可以输出动物体的体长和体重信息，并结合动物体的种类信息输出核赔结果。例如不同种类的猪以及根据不同的体尺和体重特征对应有不同的赔付价格，在此根据动物体的种类信息、体尺和体重信息可以做出赔付价格的确定。Step S5: Send the obtained pre-segmented image to the weight recognition model. The weight recognition model is the RESNET-50 network, which is a deep learning model that has undergone multi-task training and verification, and can output animal bodies. The body length and weight information of the animal body is combined with the animal body type information to output the compensation result. For example, different types of pigs and different body size and weight characteristics correspond to different compensation prices. Here, the compensation price can be determined according to the type information, body size and weight information of the animal.
体重识别模型的训练、验证以及识别体重的具体步骤为：The specific steps of training, verifying and recognizing the weight of the weight recognition model are as follows:
S51，训练阶段：S51, training phase:
A.收集多张预分割图像，每张具有标注信息，标注信息有动物体的体长、体重，随机划分一部分作为训练用，另一部分作为验证用，本实施例是采用预分割图像2000张，其中，1800张作为训练图像，200张作为验证图像。A. Collect multiple pre-segmented images, each with labeling information, the labeling information has the body length and weight of the animal body, one part is randomly divided for training, and the other part is used for verification. This embodiment uses 2000 pre-segmented images. Among them, 1800 are used as training images and 200 are used as verification images.
B.进行归一化处理，具体地，是设定参考体长2M，每张图片标注的体长除以参考体长，获取归一化的标注参数，设定参考体重300KG，每张图片的标注的体重除以参考体重，获取归一化的标注参数。B. Perform normalization processing. Specifically, set the reference body length to 2M, divide the body length marked on each picture by the reference body length to obtain the normalized annotation parameters, set the reference weight to 300KG, and the weight of each picture Divide the labeled weight by the reference weight to obtain the normalized labeled parameters.
C.对每张预分割图像统一缩放至合理大小，本实施例缩放至640x640大小。C. All pre-segmented images are uniformly scaled to a reasonable size, and this embodiment is scaled to a size of 640x640.
D.训练过程中，网络输出两个分支，体长识别分支和体重识别分支，其中体长识别分支作为体重识别分支的辅助输入信息，经过训练和验证后，达到预设的准确率阈值后，该模型即可进行识别。D. During the training process, the network outputs two branches, the body length recognition branch and the weight recognition branch. The body length recognition branch is used as the auxiliary input information of the weight recognition branch. After training and verification, the preset accuracy threshold is reached. The model can be identified.
S52，使用阶段：S52, use stage:
E.将通过步骤S1至S4得到图像缩放至640x640。E. Scale the image obtained through steps S1 to S4 to 640x640.
G.将经过缩放的图像输入至RESNET-50网络，输出体长和体重的相对指标L_0和W_0。G. Input the scaled image to the RESNET-50 network, and output the relative indicators L_0 and W_0 of body length and weight.
H.读取***设定的参考体长、参考体重参数，以上的训练步骤中已将参考体长、参考体重参数设定为2M，300KG，所以将得到的相对指标L_0和W_0分别与对应的参考体长、参考体重相乘，从而得到识别的体长L、体重W。H. Read the reference body length and reference weight parameters set by the system. In the above training steps, the reference body length and reference weight parameters have been set to 2M, 300KG, so the obtained relative indicators L_0 and W_0 are respectively corresponding to the corresponding The reference body length and the reference weight are multiplied together to obtain the identified body length L and weight W.
进一步地，在步骤S1之前还包括步骤S0，拍摄投保动物的脸部照片，利用细粒度分类算法来确定动物体的类别，所述细粒度分类算法可以区分生物特征非常相似的各品种的猪、牛图片。例如属于什么品种的猪(不同种类的猪对应不同的赔付价格)。所述细粒度分类算法可以是MTCNN，B-CNN算法，B-CNN算法在CUB鸟类数据集上有着很高的分类准确率。Further, before step S1, it also includes step S0, taking a photo of the face of the insured animal, and using a fine-grained classification algorithm to determine the category of the animal body. The fine-grained classification algorithm can distinguish pigs of various breeds with very similar biological characteristics. Cow picture. For example, what kind of pigs belong to (different types of pigs correspond to different compensation prices). The fine-grained classification algorithm may be MTCNN, B-CNN algorithm, and the B-CNN algorithm has a high classification accuracy rate on the CUB bird data set.
在一个可选实施例中，对于核赔过程的真实性判断，可能会由于标定板的伪造使得最终获得的动物体的体重及体尺数据不真实。比如正常情况下应该是投保人使用保险公司指定的标定板来进行动物体拍摄。而投保人为了获取较大的动物体重测量值，可以仿造标定板，缩小标定板的尺寸，通过拉近拍摄距离使得其标定板充满手机的标定板参考区域，从而使得按照指定大小的标定板结合算法获得的体重、体尺计算模型的参考性失真、得到较大的预测误差。In an optional embodiment, for the authenticity judgment of the compensation process, the body weight and body measurement data finally obtained may be untrue due to the forgery of the calibration board. For example, under normal circumstances, the insured should use the calibration board designated by the insurance company to take pictures of animals. In order to obtain a larger animal weight measurement, the insured can imitate the calibration board, reduce the size of the calibration board, and make the calibration board fill the reference area of the calibration board of the mobile phone by shortening the shooting distance, so that the calibration board of the specified size can be combined The weight and body size calculated by the algorithm are distorted in reference to the calculation model, and a larger prediction error is obtained.
本实施例采用以下方式来确保可以识别是否有伪造标定板的情况：通过手机调用手机供应商提供的AI接口，获取包括地理位置、相机参数、IMU参数(惯性测量单元)，并根据投保人拍摄的多角度图片、相机参数和IMU参数，进行场景三维重建，获得标定板的实际尺寸，根据手机的位移传感器可以获得拍摄距离，根据获取的拍摄高度、标定板的实际尺寸判定标定板是否为指定的标定板。具体说，对于同一块标定板，单目摄像头在固定的拍摄高度，其拍摄的图像尺寸应是一样的，对于保险公司指定的标定板，对应的根据不同的拍摄高度具有其拍摄尺寸的数据，而根据三维场景获得的标定板尺寸应与该数据相符，否则则认为其标定板不是指定标定板。This embodiment adopts the following method to ensure that it is possible to identify whether there is a forged calibration board: call the AI interface provided by the mobile phone supplier through the mobile phone to obtain the geographic location, camera parameters, IMU parameters (inertial measurement unit), and shoot according to the policyholder Perform 3D reconstruction of the scene, obtain the actual size of the calibration board, obtain the shooting distance according to the displacement sensor of the mobile phone, and determine whether the calibration board is specified according to the acquired shooting height and the actual size of the calibration board Calibration board. Specifically, for the same calibration board, the image size of the monocular camera at a fixed shooting height should be the same. For the calibration board designated by the insurance company, the corresponding data has its shooting size according to different shooting heights. The size of the calibration board obtained from the 3D scene should be consistent with the data, otherwise it is considered that the calibration board is not a designated calibration board.
获得三维场景包括以下步骤：Obtaining a 3D scene includes the following steps:
(1)摄像头标定：其中，关于相机的内参，由于手机内部集成了很多的传感器，比如IMU、摄像头甚至结构光模组，各个手机公司或者提供支持的企业也针对的提供了AI的 接口平台，比如谷歌的AI_core、Facebook的、华为的HUAWEI_AI,苹果的AR_KIT都可以通过app调用手机供应商的AI接口。使得各应用可以按需调用该AI接口获取相应的数据。根据相机内参并结合位姿估计算法可以获得相机外参。(1) Camera calibration: Among them, regarding the internal parameters of the camera, due to the integration of many sensors inside the mobile phone, such as IMU, camera and even structured light module, various mobile phone companies or supporting companies also provide AI interface platforms. For example, Google's AI_core, Facebook's, Huawei's HUAWEI_AI, and Apple's AR_KIT can all call the AI interface of the mobile phone supplier through the app. So that each application can call the AI interface to obtain corresponding data as needed. According to the camera internal parameters combined with the pose estimation algorithm, the camera external parameters can be obtained.
(2)采用sift算子计算图片每个像素点的特征，对多张图片像素做匹配对应，通过像素点的特征，结合相机参数，得到稀疏点云信息，可以使用Bundler(能够利用无序的图片集合重建出3D的模型)和VisualSFM(一种三维重建软件)生成稀疏点云。(2) Use the sift operator to calculate the characteristics of each pixel in the picture, and match the pixels of multiple pictures. Through the characteristics of the pixels, combined with the camera parameters, the sparse point cloud information can be obtained. Bundler (can use the disordered The 3D model is reconstructed from the picture collection) and VisualSFM (a 3D reconstruction software) to generate sparse point clouds.
(3)使用PMVS对稀疏点云做范围扩展、范围滤波，得到稠密点云，并对稠密点云网格化，得到三维场景。(3) Use PMVS to expand and filter the sparse point cloud to obtain a dense point cloud, and mesh the dense point cloud to obtain a three-dimensional scene.
如图5所示，是本申请基于单目摄像头的动物体在线核赔装置的功能模块图。基于单目摄像头的动物体在线核赔装置200可以安装于电子设备中。根据实现的功能，所述基于单目摄像头的动物体在线核赔装置200可以包括对焦模块201、拍摄合规判定模块202、动物体分割模块203、体重识别模块204。本发所述模块是指一种能够被电子设备处理器所执行，并且能够完成固定功能的一系列计算机程序段，其存储在电子设备的存储器中。As shown in Fig. 5, it is a functional block diagram of the animal online compensation device based on a monocular camera in the present application. The animal online compensation device 200 based on a monocular camera can be installed in an electronic device. According to the realized functions, the online animal compensation device 200 based on a monocular camera may include a focusing module 201, a shooting compliance determination module 202, an animal body segmentation module 203, and a weight recognition module 204. The module mentioned in the present invention refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, which are stored in the memory of the electronic device.
在本实施例中，关于各模块的功能如下：In this embodiment, the functions of each module are as follows:
其中，对焦模块201用于控制单目摄像头对焦动物体和标定板，如图2所示，动物体50侧放于地面,标定板20置于动物体50腹部下方，使得动物体的侧面与标定板20都在水平方向上，单目摄像头的拍摄画面60中显示有动物体区域框30和标定板区域框80，标定板区域框80位于动物体区域框内，动物体50置于动物体区域框30中。Wherein, the focusing module 201 is used to control the monocular camera to focus on the animal body and the calibration board. As shown in FIG. 2, the animal body 50 is placed on the ground on the side, and the calibration board 20 is placed under the abdomen of the animal body 50, so that the side of the animal body is aligned with the calibration board. The boards 20 are all in the horizontal direction. The shooting screen 60 of the monocular camera displays the animal body area frame 30 and the calibration board area frame 80. The calibration board area frame 80 is located in the animal body area frame, and the animal body 50 is placed in the animal body area. Box 30.
其中，优选地，是拍摄时使得标定板20与标定板区域80完全重合，可以使得拍摄的动物体的大小参照的尺度相同，以利于后期的体重识别。从图4、图5可以看出，在同样的动物体区域框30的情况下，由于拍摄距离不同，导致得到的动物体最小外包矩形不同。而不采用标定板20来锁定拍摄的动物体的比例的话，则可能由于单目摄像头距离动物体的距离不同，导致拍摄的相同的动物体呈现出大小不同的形式，会造成体尺、体重识别误差。通过固定标定板20作为标记物，使得所拍摄的动物体的大小都是以标定板为基准的，可以防止由于拍摄距离不同导致相同的动物体却得到不同的动物体大小导致体重识别出现偏差。Wherein, preferably, the calibration plate 20 and the calibration plate area 80 are completely overlapped during shooting, so that the size of the photographed animal body can have the same reference scale, so as to facilitate the weight recognition in the later stage. It can be seen from FIG. 4 and FIG. 5 that in the case of the same animal body area frame 30, due to the different shooting distances, the smallest outer rectangles of the obtained animal bodies are different. If the calibration board 20 is not used to lock the proportion of the captured animal body, it is possible that the distance between the monocular camera and the animal body is different, causing the same animal body to be photographed in different sizes, which will cause body size and weight recognition. error. By fixing the calibration plate 20 as a marker, the size of the photographed animal body is based on the calibration plate, which can prevent the same animal body from getting a different animal body size due to different shooting distances, which leads to deviations in weight recognition.
拍摄合规判定模块202用于对拍摄画面进行图像质量判定，包括标定尺度判定、目标动物体区域判定。The shooting compliance determination module 202 is used to determine the image quality of the shooting picture, including the calibration scale determination and the target animal body area determination.
其中，标定尺度判定是指拍摄画面中检测的标定板面积与标定板区域框面积之间的比例尺度I大于设定的比例尺度阈值，例如比例尺度阈值为0.8。如果小于设定的比例尺度阈值，则提示重新对焦。比例尺度I的含义是S b1与S b2的相交重合区域面积与S b1、S b2覆盖区域面积的比值，物理意义的是两者吻合程度，比例尺度I的计算公式如下： Wherein, the calibration scale determination means that the ratio scale I between the area of the calibration plate detected in the shooting image and the area of the calibration plate area is greater than a set ratio scale threshold, for example, the ratio scale threshold is 0.8. If it is less than the set ratio scale threshold, it will prompt to refocus. I ratio scale is the meaning of S b1 and S b2 area of intersection coincides with S b1, S b2 ratio of coverage area of the physical meaning of the two degree of agreement, I calculated ratio scale as follows:
Figure PCTCN2020136401-appb-000004
Figure PCTCN2020136401-appb-000004
其中，S b1代表为当前帧获得的标定板的面积； Among them, S b1 represents the area of the calibration plate obtained in the current frame;
S b2代表为画面中预设的标定板区域面积。 S b2 represents the area of the calibration board preset in the screen.
目标动物体区域的判断条件是当前帧的动物体最小外包矩形区域40和画面中预设的动物体区域框30的IOU(IOU是交并比的缩写)大于交并阈值，此处选取交并阈值为0.75，若小于，则提示重新对焦，IOU的具体公式为：The judgment condition of the target animal body area is that the IOU (IOU is the abbreviation of the intersection ratio) of the animal body's smallest outer rectangular area 40 in the current frame and the animal body area frame 30 preset in the screen is greater than the intersection threshold. Here, select the intersection The threshold is 0.75. If it is less than, it will prompt to refocus. The specific formula of IOU is:
Figure PCTCN2020136401-appb-000005
Figure PCTCN2020136401-appb-000005
其中，S h1是当前帧的动物体最小外包矩形区域； Among them, Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame;
S h2是画面中预设的动物体区域框的区域； Sh2 is the area of the animal body area frame preset in the screen;
IOU是S h1与S h2的相交重合区域面积与S h1、S h2覆盖区域面积的比值。 IOU intersect with S h2 S h1 coincides with the area of S h1, the ratio of the area of coverage S h2.
具体地，可以是调用Opencv的FindChessboardCorners函数进行标定板的检测判定，(FindChessboardCorners是常用的图像处理库Opencv的内置方法，例如采用40x40cm的标定板，由8x8个方格组成。因而设定检测角点(黑色方块相交的点为角点)个数的阈值为49个，当满足要求时通过标定板检测，按照8x8的对应区域计算出标定板所占的像素区域集合。Specifically, Opencv's FindChessboardCorners function can be called to determine the detection and determination of the calibration board. (FindChessboardCorners is a commonly used image processing library built-in method of Opencv. For example, a 40x40cm calibration board is used, which is composed of 8x8 squares. Therefore, the detection corners are set. (The point where the black squares intersect is the corner point) The threshold of the number is 49. When the requirements are met, the calibration board is tested and the pixel area set occupied by the calibration board is calculated according to the corresponding area of 8x8.
优选地，在进行标定尺度判定、目标动物体区域判定之前还进行拍摄画面预判定，拍摄画面预判定包括对将要拍摄的图像的通用质量属性的判定，包括图像的分辨率、图像的模糊程度和检测画面中动物体情况，包括如下步骤：Preferably, the pre-determination of the shooting picture is also performed before the determination of the calibration scale and the determination of the target animal body area. The pre-determination of the shooting picture includes the determination of the general quality attributes of the image to be shot, including the resolution of the image, the degree of blurring of the image, and The detection of the animal body in the picture includes the following steps:
B.检测当前拍摄画面的分辨率，如果分辨率设定小于阈值则弹出提示分辨率小于阈值(如1280*960)，提醒用户修改。B. Detect the resolution of the current shooting picture. If the resolution setting is less than the threshold, a pop-up prompts that the resolution is less than the threshold (such as 1280*960) to remind the user to modify.
C.满足B中条件后，提醒用户对焦动物体，逐帧检测画面模糊程度，可以采用拉普拉斯算子检测画面模糊程度，Opencv(基于BSD许可发行的跨平台计算机视觉库，BSD是Unix的衍生***)中提供了对拉普拉斯算子的封装方法，可以直接调用，其中，拉普拉斯算子是用来衡量图片的二阶导数，能够体现图片中密度快速变化的区域(即边界)，将画面的各点像素与拉普拉斯算子进行卷积，然后计算输出的方差，通常清晰图片的边界方差会比较大，而模糊图片中包含的边界信息少，方差较小。当连续2s的方差都小于模糊阈值则视为模糊，方差越小，画面模糊程度越高。C. After meeting the conditions in B, remind the user to focus on the animal body and detect the blur degree of the picture frame by frame. The Laplacian operator can be used to detect the picture blur degree. Opencv (a cross-platform computer vision library issued under the BSD license, BSD is Unix The derivative system of) provides an encapsulation method for the Laplacian operator, which can be directly called. Among them, the Laplacian operator is used to measure the second derivative of the picture, which can reflect the rapidly changing area of the density in the picture ( That is, the boundary), convolve each pixel of the picture with the Laplacian operator, and then calculate the output variance. Generally, the boundary variance of a clear picture will be relatively large, while the boundary information contained in a blurred picture is less, and the variance is small . When the variance of consecutive 2s is less than the blur threshold, it is regarded as blur. The smaller the variance, the higher the blur degree of the picture.
D.检测画面中有无动物体。如果检测不存在，则弹出提示对应目标检测失败，提醒用户将单目摄像头对准合格的动物体。D. Check whether there is an animal in the screen. If the detection does not exist, a pop-up prompt that the corresponding target detection failed, reminding the user to aim the monocular camera at the qualified animal body.
其中，动物体分割模块203控制单目摄像头拍摄包含动物体和标定板的图像，对图像进行动物体区域和标定板识别，具体说，是采用图像识别，将通过图像质量判定的图像输入Cascade RCNN网络模型进行动物体区域的识别，输出为动物体最小外包矩形区域掩模，获得了动物体最小外包矩形区域的四个角点的坐标(X1，Y1)、(X2，Y2)、(X3，Y3)、(X4，Y4)，其中，X1、X2、X3、X4是动物体最小外包矩形区域的左上角横坐标、右上角横坐标、左下角横坐标、右下角横坐标，Y1、Y2、Y3、Y4是动物体最小外包矩形区域的左 上角纵坐标、右上角纵坐标、左下角纵坐标、右下角纵坐标。其中，Cascade RCNN网络模型是集成在mmdetection(基于PyTorch的开源目标检测工具包)中的一个目标检测模型。Among them, the animal body segmentation module 203 controls the monocular camera to capture images containing the animal body and the calibration board, and recognizes the animal body area and the calibration board on the image. Specifically, image recognition is used, and the image judged by the image quality is input to Cascade RCNN The network model recognizes the animal body area, and the output is the mask of the smallest outer rectangular area of the animal body, and the coordinates (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), where X1, X2, X3, X4 are the abscissa of the upper left corner, the abscissa of the upper right corner, the abscissa of the lower left corner, and the abscissa of the lower right corner of the animal body's smallest enclosing rectangular area, Y1, Y2, Y3 and Y4 are the ordinate of the upper left corner, the ordinate of the upper right corner, the ordinate of the lower left corner, and the ordinate of the lower right corner of the animal body's smallest outer rectangular area. Among them, the Cascade RCNN network model is a target detection model integrated in mmdetection (an open source target detection toolkit based on PyTorch).
步骤S4，以动物体为中心分割出一个预设尺寸的长方形区域作为预分割图像，以便体重识别模型计算，获取包含动物体的预分割图像的具体步骤为：Step S4, segmenting a rectangular area of preset size with the animal body as the center as a pre-segmented image for the calculation of the weight recognition model. The specific steps for obtaining the pre-segmented image containing the animal body are as follows:
S41，理论上说，都以标定板的面积恰好与动物体区域框30的区域重合进行拍摄，则得到的动物体大小的基准应该都是一样的，但实际中，由于拍摄角度、拍摄手法、视觉上的差异，标定板与动物体区域框30的重合度肯定是有误差的。所以以检测标定板的面积与动物体区域框的比例尺度I，将人为拍摄的误差通过比例尺度来补偿。由此计算出各图像需要分割出的预分割图像的尺寸D 宽度、D 高度， S41. In theory, the area of the calibration board is exactly the same as the area of the animal body area frame 30 for shooting. The animal body size benchmarks obtained should be the same, but in practice, due to the shooting angle, shooting method, The visual difference is that there must be an error in the degree of overlap between the calibration board and the animal body area frame 30. Therefore, the ratio scale I between the area of the detection calibration board and the area frame of the animal body is used to compensate the error caused by human shooting through the ratio scale. From this, calculate the size D width and D height of the pre-segmented image that each image needs to be segmented.
Figure PCTCN2020136401-appb-000006
Figure PCTCN2020136401-appb-000006
其中，D 预设宽度、D 预设高度是比例尺度I为1时的预分割图像的尺寸。 Among them, D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
S42，计算动物体最小外包矩形的中心坐标P h，其具体计算公式为： S42: Calculate the center coordinate P h of the smallest outer rectangle of the animal body, and the specific calculation formula is:
P h＝(X h，Y h) P h ＝(X h , Y h )
其中：among them:
X h＝0.25*(X1+X2+X3+X4) X h ＝0.25*(X1+X2+X3+X4)
Y h＝0.25*(Y1+Y2+Y3+Y4) Y h ＝0.25*(Y1+Y2+Y3+Y4)
S43，计算分割的预分割图像的四个顶点坐标：S43. Calculate the coordinates of the four vertices of the segmented pre-segmented image:
四个角点坐标的计算如下：The four corner coordinates are calculated as follows:
1)1)
P hg＝(X hgxs，Y hgxs) P hg = (X hgxs , Y hgxs )
X hgxs＝X h–0.5*D 宽度 X hgxs = X h -0.5*D width
Y hgxs＝Y h–0.5*D 高度 Y hgxs = Y h -0.5*D height
其中，P hgxs是预分割图像左下角坐标； Among them, P hgxs is the coordinate of the lower left corner of the pre-segmented image;
X hgxs是预分割图像左下角横坐标； X hgxs is the abscissa of the lower left corner of the pre-segmented image;
Y hgxs是预分割图像左下角纵坐标。 Y hgxs is the ordinate of the lower left corner of the pre-segmented image.
2)2)
P hgyx＝(X hgyx，Y hgyx) P hgyx = (X hgyx , Y hgyx )
X hgyx＝X h+0.5*D 宽度 X hgyx = X h +0.5*D width
Y hgyx＝Y h–0.5*D 高度 Y hgyx = Y h -0.5*D height
其中，P hgyx是预分割图像右下角坐标； Among them, Phgyx is the coordinate of the lower right corner of the pre-segmented image;
X hgyx是预分割图像右下角横坐标； X hgyx is the abscissa of the lower right corner of the pre-segmented image;
X hgyx是预分割图像右下角纵坐标。 X hgyx is the ordinate of the lower right corner of the pre-segmented image.
3)3)
P hgys＝(X hgys，Y hgys) P hgys = (X hgys , Y hgys )
X hgys＝X h+0.5*D 宽度 X hgys ＝X h +0.5*D width
Y hgys＝Y h+0.5*D 高度 Y hgys = Y h +0.5*D height
P hgys是预分割图像右上角坐标； P hgys is the coordinates of the upper right corner of the pre-segmented image;
X hgys是预分割图像右上角横坐标； X hgys is the abscissa of the upper right corner of the pre-segmented image;
Y hgys是预分割图像右上角纵坐标。 Y hgys is the ordinate of the upper right corner of the pre-segmented image.
4)4)
P hgzs＝(X hgzs，Y hgzs) P hgzs = (X hgzs , Y hgzs )
X hgzs＝X h–0.5*D 宽度 X hgzs = X h -0.5*D width
Y hgzs＝Y h+0.5*D 高度 Y hgzs = Y h +0.5*D height
其中，P hgzs是预分割图像左上角坐标； Among them, Phgzs is the coordinates of the upper left corner of the pre-segmented image;
X hgzs是预分割图像左上角横坐标； X hgzs is the abscissa of the upper left corner of the pre-segmented image;
Y hgzs是预分割图像左上角纵坐标。 Y hgzs is the ordinate of the upper left corner of the pre-segmented image.
S44，以S43中获得的预分割图像对应的四个角坐标来对图像裁剪，获取的裁剪图像即为预分割图像。S44, crop the image using the four corner coordinates corresponding to the pre-segmented image obtained in S43, and the obtained cropped image is the pre-segmented image.
其中，体重识别模块204用于将获取的预分割图像送入体重识别模型，体重识别模型是RESNET-50网络，RESNET-50网络是一种深度学习模型，其经过了多任务训练和验证，由此可以输出动物体的体长和体重信息。并结合动物体的种类信息输出核赔结果。例如不同种类的猪以及根据不同的体尺和体重特征对应有不同的赔付价格，在此根据动物体的种类信息、体尺和体重信息可以做出赔付价格的确定。Among them, the weight recognition module 204 is used to send the obtained pre-segmented images to the weight recognition model. The weight recognition model is the RESNET-50 network, which is a deep learning model that has undergone multi-task training and verification. This can output the body length and weight information of the animal body. And combined with the type of animal information to output the results of the compensation. For example, different types of pigs and different body size and weight characteristics correspond to different compensation prices. Here, the compensation price can be determined according to the type information, body size and weight information of the animal.
体重识别模型的训练、验证以及识别体重的具体步骤为：The specific steps of training, verifying and recognizing the weight of the weight recognition model are:
S51，训练阶段：S51, training phase:
A.收集多张预分割图像，每张具有标注信息，标注信息有动物体的体长、体重，随机划分一部分作为训练用，另一部分作为验证用，本实施例是采用预分割图像2000张，其中，1800张作为训练图像，200张作为验证图像。A. Collect multiple pre-segmented images, each with labeling information, the labeling information has the body length and weight of the animal body, one part is randomly divided for training, and the other part is used for verification. This embodiment uses 2000 pre-segmented images. Among them, 1800 are used as training images and 200 are used as verification images.
B.进行归一化处理，具体地，是设定参考体长2M，每张图片标注的体长除以参考体长，获取归一化的标注参数，设定参考体重300KG，每张图片的标注的体重除以参考体重，获取归一化的标注参数。B. Perform normalization processing. Specifically, set the reference body length to 2M, divide the body length marked on each picture by the reference body length to obtain the normalized annotation parameters, set the reference weight to 300KG, and the weight of each picture Divide the labeled weight by the reference weight to obtain the normalized labeled parameters.
C.对每张预分割图像统一缩放至合理大小，本实施例缩放至640x640大小。C. Uniformly scale each pre-segmented image to a reasonable size, and this embodiment scales to a size of 640x640.
D.训练过程中，网络输出两个分支，体长识别分支和体重识别分支，其中体长识别分支作为体重识别分支的辅助输入信息，经过训练和验证后，达到预设的准确率阈值后，该模型即可进行识别。D. During the training process, the network outputs two branches, the body length recognition branch and the weight recognition branch. The body length recognition branch is used as the auxiliary input information of the weight recognition branch. After training and verification, the preset accuracy threshold is reached. The model can be identified.
S52，使用阶段：S52, use stage:
E.将通过步骤S1至S4得到图像缩放至640x640。E. Scale the image obtained through steps S1 to S4 to 640x640.
G.将经过缩放的图像输入至RESNET-50网络，输出体长和体重的相对指标L_0和W_0。G. Input the scaled image to the RESNET-50 network, and output the relative indicators L_0 and W_0 of body length and weight.
H.读取***设定的参考体长、参考体重参数，以上的训练步骤中已将参考体长、参考体重参数设定为2M，300KG，所以将得到的相对指标L_0和W_0分别与对应的参考体长、参考体重相乘，从而得到识别的体长L、体重W。H. Read the reference body length and reference weight parameters set by the system. In the above training steps, the reference body length and reference weight parameters have been set to 2M, 300KG, so the obtained relative indicators L_0 and W_0 are respectively corresponding to the corresponding The reference body length and the reference weight are multiplied together to obtain the identified body length L and weight W.
本申请还提供一种电子设备，参阅图6所示，是本申请电子设备的实施例的硬件架构示意图。本实施例中，所述电子设备2是一种能够按照事先设定或者存储的指令，自动进行数值计算和/或信息处理的设备。例如，可以是智能移动设备、平板电脑、笔记本电脑等易于携带的移动设备。如图6所示，所述电子设备2至少包括通过线路相互通信连接的存储器21、处理器22，所述单目摄像头70与处理器连接。其中：所述存储器21可以是所述电子设备2的内部存储单元，例如该电子设备2的硬盘或内存。在另一些实施例中，所述存储器21也可以是所述电子设备2的外部存储设备，例如该电子设备2上配备的插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)等。This application also provides an electronic device. As shown in FIG. 6, it is a schematic diagram of the hardware architecture of an embodiment of the electronic device of this application. In this embodiment, the electronic device 2 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. For example, it can be a mobile device that is easy to carry, such as a smart mobile device, a tablet computer, and a notebook computer. As shown in FIG. 6, the electronic device 2 at least includes a memory 21 and a processor 22 that are communicatively connected to each other through a line, and the monocular camera 70 is connected to the processor. Wherein: the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or a memory of the electronic device 2. In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk, a smart media card (SMC), and a secure digital device equipped on the electronic device 2. (Secure Digital, SD) card, flash card (Flash Card), etc.
当然，所述存储器21还可以既包括所述电子设备2的内部存储单元也包括其外部存储设备。本实施例中，所述存储器21通常用于存储安装于所述电子设备2的操作***和各类应用软件，例如基于单目摄像头的动物体在线核赔程序的代码等。此外，所述存储器21还可以用于暂时地存储已经输出或者将要输出的各类数据。Of course, the memory 21 may also include both the internal storage unit of the electronic device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store an operating system and various application software installed in the electronic device 2, for example, the code of an animal body online compensation program based on a monocular camera. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.
所述处理器22在一些实施例中可以是中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。用于运行所述存储器21中存储的程序代码或者处理数据，例如运行基于单目摄像头的动物体在线核赔程序。In some embodiments, the processor 22 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. It is used for running the program code or processing data stored in the memory 21, for example, running an animal body online compensation program based on a monocular camera.
需要指出的是，图6仅示出了具有存储器21、处理器22的电子设备2，但是应理解的是，并不要求实施所有示出的组件，可以替代的实施更多或者更少的组件。It should be pointed out that FIG. 6 only shows the electronic device 2 with the memory 21 and the processor 22, but it should be understood that it is not required to implement all the illustrated components, and more or fewer components may be implemented instead. .
包含可读存储介质的存储器21中可以包括操作***、基于单目摄像头的动物体在线核赔程序等。处理器22执行存储器21中的基于单目摄像头的动物体在线核赔程序时实现如上所述的S1至S5的步骤，在此不再赘述。The memory 21 containing a readable storage medium may include an operating system, an animal body online compensation program based on a monocular camera, and the like. The processor 22 implements the steps from S1 to S5 as described above when the animal body online compensation program based on the monocular camera in the memory 21 is executed, which will not be repeated here.
此外，本申请实施例还提出一种计算机可读存储介质，所述计算机可读存储介质可以是非易失性，也可以是易失性，所述计算机可读存储介质可以是硬盘、多媒体卡、SD卡、 闪存卡、SMC、只读存储器(ROM)、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器等等中的任意一种或者几种的任意组合。所述计算机可读存储介质中包括基于单目摄像头的动物体在线核赔程序等，所述基于单目摄像头的动物体在线核赔程序被处理器22执行时实现如下操作：In addition, the embodiment of the present application also proposes a computer-readable storage medium. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium may be a hard disk, a multimedia card, SD card, flash memory card, SMC, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, etc. or Any combination of several. The computer-readable storage medium includes an animal online compensation program based on a monocular camera, etc., and the following operations are implemented when the monocular camera-based online animal compensation program is executed by the processor 22:
S1，控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；S1, control the monocular camera to focus on the animal body and the calibration board, where the animal body is placed on the ground side, and the calibration board is placed under the abdomen of the animal body. The shooting screen of the monocular camera displays the animal body area frame and the calibration board area frame. The animal body is placed in the animal body area frame;
S2，对拍摄画面进行图像质量判定，包括标定尺度判定、目标动物体区域判定；S2, judge the image quality of the shooting picture, including the judgment of the calibration scale and the judgment of the target animal body area;
S3，控制单目摄像头拍摄包含动物体和标定板的图像，将图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；S3, control the monocular camera to take images containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
S4，以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。S4, segmenting a pre-segmented image of a preset size with the animal body as the center and sending it to the trained weight recognition model, outputting body length and weight information, and combining the category information obtained by the facial image recognition of the animal body to determine the compensation result.
本申请之计算机可读存储介质的具体实施方式与上述基于单目摄像头的动物体在线核赔方法以及电子设备2的具体实施方式大致相同，在此不再赘述。The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned monocular camera-based animal online compensation method and the electronic device 2, and will not be repeated here.
以上所述仅为本申请的优选实施例，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。The foregoing descriptions are only preferred embodiments of the application, and are not used to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims (20)

  1. 一种基于单目摄像头的动物体在线核赔方法，其中，包括以下步骤：An online compensation method for animal bodies based on a monocular camera, which includes the following steps:
    控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Control the monocular camera to focus on the animal body and the calibration board. The animal body is placed on the ground and the calibration board is placed under the abdomen of the animal body. The shooting screen of the monocular camera shows the animal body area frame and the calibration board area frame. Place in the animal area frame;
    判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；Determine whether the minimum outsourcing rectangle area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting screen are greater than the preset intersection threshold, if it is greater, continue to execute, otherwise prompt to refocus;
    控制单目摄像头拍摄包含动物体和标定板的图像，将所述图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；Control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
    以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The pre-segmented image of the preset size is segmented with the animal body as the center and sent to the trained weight recognition model, and the body length and weight information are output, and combined with the type information obtained by the facial image recognition of the animal body, the compensation result is determined.
  2. 根据权利要求1所述的基于单目摄像头的动物体在线核赔方法，其中，在判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值之前，还包括：The animal body online compensation method based on a monocular camera according to claim 1, wherein in determining whether the minimum outsourcing rectangular area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting picture are greater than the preset intersection Before the threshold, it also includes:
    先进行拍摄画面预判定，包括对画面的分辨率、画面的模糊程度和动物体区域框中的动物体是否齐全的判定，First, make a pre-judgment of the shooting picture, including the judgment of the resolution of the picture, the degree of blurring of the picture and whether the animal body is complete in the animal body area frame.
    其中，画面模糊程度的判定是采用拉普拉斯算子进行检测，将画面的各点像素与拉普拉斯算子进行卷积计算输出方差，当连续2s的方差都小于模糊阈值则视为模糊。Among them, the degree of blurring of the picture is determined by using the Laplacian operator to detect, and each pixel of the picture is convolved with the Laplacian operator to calculate the output variance. When the variance of the continuous 2s is less than the blur threshold, it is regarded as blurry.
  3. 根据权利要求1所述的基于单目摄像头的动物体在线核赔方法，其中，在所述控制单目摄像头拍摄包含动物体和标定板的图像前，还包括：The method for online compensation of an animal body based on a monocular camera according to claim 1, wherein before the control monocular camera captures an image containing the animal body and the calibration board, the method further comprises:
    采用Opencv的FindChessboardCorners函数寻找标定板的角点，获得所述标定板的面积计算比例尺度I，并在所述比例尺度I小于预设的比例尺度阈值时提示重新对焦，所述比例尺度I的公式为：Use Opencv's FindChessboardCorners function to find the corner points of the calibration board, obtain the area of the calibration board, calculate the scale scale I, and prompt to refocus when the scale scale I is less than the preset scale scale threshold. The formula of the scale scale I for:
    Figure PCTCN2020136401-appb-100001
    Figure PCTCN2020136401-appb-100001
    其中，S b1代表为当前帧获得的标定板的面积； Among them, S b1 represents the area of the calibration plate obtained in the current frame;
    S b2代表为画面中预设的标定板区域面积。 S b2 represents the area of the calibration board preset in the screen.
  4. 根据权利要求1所述的基于单目摄像头的动物体在线核赔方法，其中，The online animal compensation method based on a monocular camera according to claim 1, wherein:
    动物体最小外包矩形区域和拍摄画面中预设的动物体区域框的IOU的计算公式为:The calculation formula for the minimum outsourcing rectangular area of the animal body and the IOU of the animal body area frame preset in the shooting screen is:
    Figure PCTCN2020136401-appb-100002
    Figure PCTCN2020136401-appb-100002
    其中，S h1是当前帧的动物体最小外包矩形区域； Among them, Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame;
    S h2是画面中预设的动物体区域框的区域； Sh2 is the area of the animal body area frame preset in the screen;
    IOU是S h1与S h2的相交重合区域面积与S h1、S h2覆盖区域面积的比值。 IOU intersect with S h2 S h1 coincides with the area of S h1, the ratio of the area of coverage S h2.
  5. 根据权利要求3所述的基于单目摄像头的动物体在线核赔方法，其中，所述以动物体为中心分割出预分割图像包括：The method for online compensation of animal body based on monocular camera according to claim 3, wherein said segmenting the pre-segmented image with the animal body as the center comprises:
    根据比例尺度I获得预分割图像的尺寸D 宽度、D 高度，其中，
    Figure PCTCN2020136401-appb-100003
    Figure PCTCN2020136401-appb-100004
     Obtain the size D width and D height of the pre-segmented image according to the scale I, where,
    Figure PCTCN2020136401-appb-100003
    Figure PCTCN2020136401-appb-100004
    其中，D 预设宽度、D 预设高度是比例尺度I为1时的预分割图像的尺寸。 Among them, D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
  6. 根据权利要求1所述的基于单目摄像头的动物体在线核赔方法，其中，The online animal compensation method based on a monocular camera according to claim 1, wherein:
    所述体重识别模型为RESNET-50网络，所述预分割图像送入经过训练的体重识别模型，输出体长和体重信息包括：The weight recognition model is the RESNET-50 network, the pre-segmented image is sent to the trained weight recognition model, and the output body length and weight information includes:
    收集多张标注有动物体的体长、体重信息的预分割图像，设定参考体长和参考体重，并将每张动物体图像标注的体长除以参考体长获取归一化的标注体长，每张动物体图像标注的体重除以参考体重获取归一化的标注体重，Collect multiple pre-segmented images marked with the body length and weight information of the animal body, set the reference body length and reference weight, and divide the body length marked on each animal body image by the reference body length to obtain the normalized annotation body Long, the weight of each animal body image is divided by the reference weight to obtain the normalized weight,
    将所述预分割图像中一部分作为训练图像，一部分作为验证图像；Use a part of the pre-segmented image as a training image and a part as a verification image;
    将训练图像输入RESNET-50网络进行训练，RESNET-50网络输出体长识别分支和体重识别分支，将验证图像输入RESNET-50网络，直至输出达到预设的准确率阈值；Input the training image into the RESNET-50 network for training, the RESNET-50 network outputs the body length recognition branch and the weight recognition branch, and input the verification image into the RESNET-50 network until the output reaches the preset accuracy threshold;
    将预分割图像输入至经过验证的RESNET-50网络，输出体长和体重的相对指标L_0和W_0，并分别与对应的参考体长、体重参数相乘，从而得到识别的体长L、体重W。Input the pre-segmented image to the verified RESNET-50 network, output the relative indexes L_0 and W_0 of body length and weight, and multiply them with the corresponding reference body length and weight parameters respectively to obtain the identified body length L and weight W .
  7. 根据权利要求1所述的基于单目摄像头的动物体在线核赔方法，其中，The online animal compensation method based on a monocular camera according to claim 1, wherein:
    在所述控制单目摄像头对焦动物体和标定板之前，还包括：Before controlling the monocular camera to focus on the animal body and the calibration board, the method further includes:
    先调用单目摄像头供应商提供的AI接口，获取包括地理位置、相机参数、IMU参数，并根据投保人拍摄的多角度图片、相机参数和IMU参数，进行场景三维重建，获得标定板的实际尺寸，根据手机的位移传感器获得拍摄距离，进而根据获取的拍摄距离、标定板的实际尺寸判定标定板是否为指定标定板。First call the AI interface provided by the monocular camera supplier to obtain the geographic location, camera parameters, IMU parameters, and perform 3D reconstruction of the scene according to the multi-angle pictures taken by the policyholder, camera parameters, and IMU parameters to obtain the actual size of the calibration board , Obtain the shooting distance according to the displacement sensor of the mobile phone, and then determine whether the calibration board is the designated calibration board according to the acquired shooting distance and the actual size of the calibration board.
  8. 一种基于单目摄像头的动物体在线核赔装置，其中，包括：An online animal compensation device based on a monocular camera, which includes:
    对焦模块，用于控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Focusing module, used to control the monocular camera to focus on the animal body and the calibration board, where the animal body is placed on the ground side, the calibration board is placed under the animal body's abdomen, and the animal body area frame and the calibration board are displayed in the shooting screen of the monocular camera Area frame, the animal body is placed in the animal body area frame;
    拍摄合规判定模块，判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；The shooting compliance judgment module judges whether the IOU of the smallest outsourcing rectangular area of the animal body in the current frame and the animal body area frame displayed in the shooting screen is greater than the preset intersection threshold, if it is greater, then continue to execute, otherwise it prompts to refocus;
    动物体分割模块，用于控制单目摄像头拍摄包含动物体和标定板的图像，将所述图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；The animal body segmentation module is used to control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
    体重识别模块，以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The weight recognition module, which uses the animal body as the center to segment a pre-segmented image of a preset size and sends it to the trained weight recognition model, outputs the body length and weight information, and combines the type information obtained by the animal body's facial image recognition to determine the compensation result.
  9. 一种电子设备，其中，所述电子设备包括：An electronic device, wherein the electronic device includes:
    至少一个处理器；以及，At least one processor; and,
    与所述至少一个处理器通信连接的存储器；其中，A memory communicatively connected with the at least one processor; wherein,
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如下步骤：The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the following steps:
    控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Control the monocular camera to focus on the animal body and the calibration board. The animal body is placed on the ground and the calibration board is placed under the abdomen of the animal body. The shooting screen of the monocular camera shows the animal body area frame and the calibration board area frame. Place in the animal area frame;
    判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；Determine whether the minimum outsourcing rectangle area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting screen are greater than the preset intersection threshold, if it is greater, continue to execute, otherwise prompt to refocus;
    控制单目摄像头拍摄包含动物体和标定板的图像，将所述图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；Control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
    以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The pre-segmented image of the preset size is segmented with the animal body as the center and sent to the trained weight recognition model, and the body length and weight information are output, and combined with the type information obtained by the facial image recognition of the animal body, the compensation result is determined.
  10. 根据权利要求9所述的电子设备，其中，在判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值之前，还包括：9. The electronic device according to claim 9, wherein before determining whether the minimum outsourcing rectangular area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting picture are greater than a preset intersection threshold, the method further comprises:
    先进行拍摄画面预判定，包括对画面的分辨率、画面模糊程度和动物体区域框中的动物体是否齐全的判定，First, make a pre-judgment of the shooting picture, including the judgment of the resolution of the picture, the degree of blurring of the picture, and whether the animal body is complete in the animal body area frame.
    其中，画面模糊程度的判断是采用拉普拉斯算子进行检测，将画面的各点像素与拉普拉斯算子进行卷积计算输出方差，当连续2s的所述方差都小于模糊阈值则视为画面模糊。Among them, the degree of blurring of the picture is determined by using the Laplacian operator to detect, and each pixel of the picture and the Laplacian operator are convolved to calculate the output variance. When the variance of the continuous 2s is less than the blur threshold, The picture is regarded as blurry.
  11. 根据权利要求9所述的电子设备，其中，在所述控制单目摄像头拍摄包含动物体和标定板的图像前，还包括：9. The electronic device according to claim 9, wherein before the control monocular camera captures an image containing an animal body and a calibration plate, the method further comprises:
    采用Opencv的FindChessboardCorners函数寻找所述标定板的角点，获得所述标定板的面积计算比例尺度I，并在所述比例尺度I小于预设的比例尺度阈值时提示重新对焦，所述比例尺度I的公式为：Use Opencv's FindChessboardCorners function to find the corner points of the calibration board, obtain the area of the calibration board, calculate the scale scale I, and prompt to refocus when the scale scale I is less than a preset scale scale threshold. The scale scale I The formula is:
    Figure PCTCN2020136401-appb-100005
    Figure PCTCN2020136401-appb-100005
    其中，S b1代表为当前帧获得的标定板的面积； Among them, S b1 represents the area of the calibration plate obtained in the current frame;
    S b2代表为画面中预设的标定板区域面积。 S b2 represents the area of the calibration board preset in the screen.
  12. 根据权利要求9所述的电子设备，其中，所述动物体最小外包矩形区域和拍摄画面中预设的动物体区域框的IOU的计算公式为:The electronic device according to claim 9, wherein the calculation formula of the IOU of the smallest outer rectangular area of the animal body and the preset animal body area frame in the shooting picture is:
    Figure PCTCN2020136401-appb-100006
    Figure PCTCN2020136401-appb-100006
    其中，S h1是当前帧的动物体最小外包矩形区域； Among them, Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame;
    S h2是画面中预设的动物体区域框的区域； Sh2 is the area of the animal body area frame preset in the screen;
    IOU是S h1与S h2的相交重合区域面积与S h1、S h2覆盖区域面积的比值。 IOU intersect with S h2 S h1 coincides with the area of S h1, the ratio of the area of coverage S h2.
  13. 根据权利要求11所述的电子设备，其中，所述以动物体为中心分割出预分割图像包括：The electronic device according to claim 11, wherein said segmenting the pre-segmented image centered on the animal body comprises:
    根据比例尺度I获得所述预分割图像的尺寸D 宽度、D 高度，其中，
    Figure PCTCN2020136401-appb-100007
    Figure PCTCN2020136401-appb-100008
     Obtain the size D width and D height of the pre-segmented image according to the scale I, where:
    Figure PCTCN2020136401-appb-100007
    Figure PCTCN2020136401-appb-100008
    其中，D 预设宽度、D 预设高度是比例尺度I为1时的预分割图像的尺寸。 Among them, D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
  14. 根据权利要求9所述的电子设备，其中，所述体重识别模型为RESNET-50网络，所述预分割图像送入经过训练的体重识别模型，输出体长和体重信息包括：The electronic device according to claim 9, wherein the weight recognition model is a RESNET-50 network, the pre-segmented image is sent to the trained weight recognition model, and the output body length and weight information includes:
    收集多张标注有动物体的体长、体重信息的预分割图像，设定参考体长和参考体重，并将每张动物体图像标注的体长除以参考体长获取归一化的标注体长，每张动物体图像标注的体重除以参考体重获取归一化的标注体重，Collect multiple pre-segmented images marked with the body length and weight information of the animal body, set the reference body length and reference weight, and divide the body length marked on each animal body image by the reference body length to obtain the normalized annotation body Long, the weight of each animal body image is divided by the reference weight to obtain the normalized weight,
    将所述预分割图像中一部分作为训练图像，一部分作为验证图像；Use a part of the pre-segmented image as a training image and a part as a verification image;
    将训练图像输入RESNET-50网络进行训练，RESNET-50网络输出体长识别分支和体重识别分支，将验证图像输入RESNET-50网络，直至输出达到预设的准确率阈值；Input the training image into the RESNET-50 network for training, the RESNET-50 network outputs the body length recognition branch and the weight recognition branch, and input the verification image into the RESNET-50 network until the output reaches the preset accuracy threshold;
    将预分割图像输入至经过验证的RESNET-50网络，输出体长和体重的相对指标L_0和W_0，并分别与对应的参考体长、体重参数相乘，从而得到识别的体长L、体重W。Input the pre-segmented image to the verified RESNET-50 network, output the relative indexes L_0 and W_0 of body length and weight, and multiply them with the corresponding reference body length and weight parameters respectively to obtain the identified body length L and weight W .
  15. 根据权利要求9所述的电子设备，其中，在所述控制单目摄像头对焦动物体和标定板之前，还包括：The electronic device according to claim 9, wherein before said controlling the monocular camera to focus on the animal body and the calibration plate, it further comprises:
    先调用单目摄像头供应商提供的AI接口，获取包括地理位置、相机参数、IMU参数，并根据投保人拍摄的多角度图片、相机参数和IMU参数，进行场景三维重建，获得标定板的实际尺寸，根据手机的位移传感器获得拍摄距离，进而根据获取的拍摄距离、标定板的实际尺寸判定标定板是否为指定标定板。First call the AI interface provided by the monocular camera supplier to obtain the geographic location, camera parameters, IMU parameters, and perform 3D reconstruction of the scene according to the multi-angle pictures taken by the policyholder, camera parameters, and IMU parameters to obtain the actual size of the calibration board , Obtain the shooting distance according to the displacement sensor of the mobile phone, and then determine whether the calibration board is the designated calibration board according to the acquired shooting distance and the actual size of the calibration board.
  16. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现如下步骤：A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the following steps:
    控制单目摄像头对焦动物体和标定板，其中，动物体侧放于地面,标定板置于动物体腹部下方，单目摄像头的拍摄画面中显示有动物体区域框和标定板区域框，动物体置于动物体区域框中；Control the monocular camera to focus on the animal body and the calibration board. The animal body is placed on the ground and the calibration board is placed under the abdomen of the animal body. The shooting screen of the monocular camera shows the animal body area frame and the calibration board area frame. Place in the animal area frame;
    判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值，若大于，则继续执行，否则提示重新对焦；Determine whether the minimum outsourcing rectangle area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting screen are greater than the preset intersection threshold, if it is greater, continue to execute, otherwise prompt to refocus;
    控制单目摄像头拍摄包含动物体和标定板的图像，将所述图像输入Cascade RCNN网络模型进行动物体区域的识别，输出动物体最小外包矩形区域掩模；Control the monocular camera to take an image containing the animal body and the calibration board, input the image into the Cascade RCNN network model to identify the animal body area, and output the minimum outsourcing rectangular area mask of the animal body;
    以动物体为中心分割出预设尺寸的预分割图像送入经过训练的体重识别模型，输出体长和体重信息，并结合动物体的面部图像识别获得的种类信息，确定核赔结果。The pre-segmented image of the preset size is segmented with the animal body as the center and sent to the trained weight recognition model, and the body length and weight information are output, and combined with the type information obtained by the facial image recognition of the animal body, the compensation result is determined.
  17. 根据权利要求16所述的计算机可读存储介质，其中，在判断当前帧的动物体最小外包矩形区域和拍摄画面中显示的动物体区域框的IOU是否大于预设的交并阈值之前，还包括：The computer-readable storage medium according to claim 16, wherein, before determining whether the minimum outsourcing rectangular area of the animal body in the current frame and the IOU of the animal body area frame displayed in the shooting picture are greater than a preset intersection threshold, the method further comprises :
    先进行拍摄画面预判定，包括对画面的分辨率、画面模糊程度和动物体区域框中的动物体是否齐全的判定，First, make a pre-judgment of the shooting picture, including the judgment of the resolution of the picture, the degree of blurring of the picture, and whether the animal body is complete in the animal body area frame.
    其中，画面模糊程度的判断是采用拉普拉斯算子进行检测，将画面的各点像素与拉普拉斯算子进行卷积计算输出方差，当连续2s的所述方差都小于模糊阈值则视为画面模糊。Among them, the degree of blurring of the picture is determined by using the Laplacian operator to detect, and each pixel of the picture and the Laplacian operator are convolved to calculate the output variance. When the variance of the continuous 2s is less than the blur threshold, The picture is regarded as blurry.
  18. 根据权利要求16所述的计算机可读存储介质，其中，在所述控制单目摄像头拍摄包含动物体和标定板的图像前，还包括：The computer-readable storage medium according to claim 16, wherein before the control monocular camera captures an image containing an animal body and a calibration plate, the method further comprises:
    采用Opencv的FindChessboardCorners函数寻找所述标定板的角点，获得所述标定板的面积计算比例尺度I，并在所述比例尺度I小于预设的比例尺度阈值时提示重新对焦，所述比例尺度I的公式为：Use Opencv's FindChessboardCorners function to find the corner points of the calibration board, obtain the area of the calibration board, calculate the scale scale I, and prompt to refocus when the scale scale I is less than a preset scale scale threshold. The scale scale I The formula is:
    Figure PCTCN2020136401-appb-100009
    Figure PCTCN2020136401-appb-100009
    其中，S b1代表为当前帧获得的标定板的面积； Among them, S b1 represents the area of the calibration plate obtained in the current frame;
    S b2代表为画面中预设的标定板区域面积。 S b2 represents the area of the calibration board preset in the screen.
  19. 根据权利要求16所述的计算机可读存储介质，其中，所述动物体最小外包矩形区域和拍摄画面中预设的动物体区域框的IOU的计算公式为:The computer-readable storage medium according to claim 16, wherein the calculation formula of the IOU of the smallest outer rectangular area of the animal body and the preset animal body area frame in the shooting picture is:
    Figure PCTCN2020136401-appb-100010
    Figure PCTCN2020136401-appb-100010
    其中，S h1是当前帧的动物体最小外包矩形区域； Among them, Sh1 is the smallest outsourcing rectangular area of the animal body in the current frame;
    S h2是画面中预设的动物体区域框的区域； Sh2 is the area of the animal body area frame preset in the screen;
    IOU是S h1与S h2的相交重合区域面积与S h1、S h2覆盖区域面积的比值。 IOU intersect with S h2 S h1 coincides with the area of S h1, the ratio of the area of coverage S h2.
  20. 根据权利要求16所述的计算机可读存储介质，其中，所述以动物体为中心分割出预分割图像包括：The computer-readable storage medium according to claim 16, wherein the segmenting the pre-segmented image centered on the animal body comprises:
    根据比例尺度I获得所述预分割图像的尺寸D 宽度、D 高度，其中，
    Figure PCTCN2020136401-appb-100011
    Figure PCTCN2020136401-appb-100012
     Obtain the size D width and D height of the pre-segmented image according to the scale I, where:
    Figure PCTCN2020136401-appb-100011
    Figure PCTCN2020136401-appb-100012
    其中，D 预设宽度、D 预设高度是比例尺度I为1时的预分割图像的尺寸。 Among them, D preset width and D preset height are the size of the pre-segmented image when the scale I is 1.
PCT/CN2020/136401 2020-08-27 2020-12-15 Animal body online claim settlement method and apparatus based on monocular camera, and storage medium WO2021139494A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010879333.1A CN111985477B (en) 2020-08-27 2020-08-27 Animal on-line core claim method, device and storage medium based on monocular camera
CN202010879333.1 2020-08-27

Publications (1)

Publication Number Publication Date
WO2021139494A1 true WO2021139494A1 (en) 2021-07-15

Family

ID=73439979

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/136401 WO2021139494A1 (en) 2020-08-27 2020-12-15 Animal body online claim settlement method and apparatus based on monocular camera, and storage medium

Country Status (2)

Country Link
CN (1) CN111985477B (en)
WO (1) WO2021139494A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114931112A (en) * 2022-04-08 2022-08-23 南京农业大学 Sow body ruler detection system based on intelligent inspection robot

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111985477B (en) * 2020-08-27 2024-06-28 平安科技（深圳）有限公司 Animal on-line core claim method, device and storage medium based on monocular camera
CN112749760A (en) * 2021-01-22 2021-05-04 淮阴师范学院 Waterfowl image recognition feature fusion model system and method based on deep convolutional network
CN112836904A (en) * 2021-04-07 2021-05-25 复旦大学附属中山医院 Body quality index prediction method based on face characteristic points
CN114399785B (en) * 2021-10-29 2023-02-21 平安科技（深圳）有限公司 Human height identification method and device, computer equipment and storage medium
CN116229518B (en) * 2023-03-17 2024-01-16 百鸟数据科技(北京)有限责任公司 Bird species observation method and system based on machine learning
CN116416260B (en) * 2023-05-19 2024-01-26 四川智迅车联科技有限公司 Weighing precision optimization method and system based on image processing

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108775863A (en) * 2018-03-27 2018-11-09 广州大学 Spectral Confocal displacement sensor positioning device and localization method
US20190050926A1 (en) * 2017-02-16 2019-02-14 Fusion Foundry, Llc Livestock trading platform
CN109714535A (en) * 2019-01-15 2019-05-03 南京信息工程大学 A kind of auto-focusing machine vision metrology device and method based on color difference
CN110580654A (en) * 2019-09-02 2019-12-17 江苏华鑫数据科技有限公司 Digital product underwriting and claims checking system
CN111985477A (en) * 2020-08-27 2020-11-24 平安科技（深圳）有限公司 Monocular camera-based animal body online claims checking method and device and storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106361345A (en) * 2016-11-29 2017-02-01 公安部第三研究所 System and method for measuring height of human body in video image based on camera calibration
CN108921026A (en) * 2018-06-01 2018-11-30 平安科技（深圳）有限公司 Recognition methods, device, computer equipment and the storage medium of animal identification
CN108921057B (en) * 2018-06-19 2021-06-01 厦门大学 Convolutional neural network-based prawn form measuring method, medium, terminal equipment and device
CN108871520A (en) * 2018-07-06 2018-11-23 平安科技（深圳）有限公司 Livestock body weight measurement and device
CN109165645B (en) * 2018-08-01 2023-04-07 腾讯科技（深圳）有限公司 Image processing method and device and related equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190050926A1 (en) * 2017-02-16 2019-02-14 Fusion Foundry, Llc Livestock trading platform
CN108775863A (en) * 2018-03-27 2018-11-09 广州大学 Spectral Confocal displacement sensor positioning device and localization method
CN109714535A (en) * 2019-01-15 2019-05-03 南京信息工程大学 A kind of auto-focusing machine vision metrology device and method based on color difference
CN110580654A (en) * 2019-09-02 2019-12-17 江苏华鑫数据科技有限公司 Digital product underwriting and claims checking system
CN111985477A (en) * 2020-08-27 2020-11-24 平安科技（深圳）有限公司 Monocular camera-based animal body online claims checking method and device and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114931112A (en) * 2022-04-08 2022-08-23 南京农业大学 Sow body ruler detection system based on intelligent inspection robot
CN114931112B (en) * 2022-04-08 2024-01-26 南京农业大学 Sow body ruler detection system based on intelligent inspection robot

Also Published As

Publication number Publication date
CN111985477A (en) 2020-11-24
CN111985477B (en) 2024-06-28

Similar Documents

Publication Publication Date Title
WO2021139494A1 (en) Animal body online claim settlement method and apparatus based on monocular camera, and storage medium
TWI709091B (en) Image processing method and device
KR102290040B1 (en) Imaging a body
WO2018107910A1 (en) Method and device for fusing panoramic video images
CN112633144A (en) Face occlusion detection method, system, device and storage medium
CN107679475B (en) Store monitoring and evaluating method and device and storage medium
CN109815843A (en) Object detection method and Related product
CN108961269A (en) Pig weight measuring method and system based on image
CN113689578B (en) Human body data set generation method and device
CN112017231B (en) Monocular camera-based human body weight identification method, monocular camera-based human body weight identification device and storage medium
CN108337551A (en) A kind of screen recording method, storage medium and terminal device
CN110287862B (en) Anti-candid detection method based on deep learning
CN109376631A (en) A kind of winding detection method and device neural network based
CN109951635A (en) It takes pictures processing method, device, mobile terminal and storage medium
CN111220235B (en) Water level monitoring method and device
CN111860652A (en) Method, device, equipment and medium for measuring animal body weight based on image detection
CN116343228A (en) Intelligent reading method and system for water meter
CN110766077A (en) Method, device and equipment for screening sketch in evidence chain image
CN113642639A (en) Living body detection method, living body detection device, living body detection apparatus, and storage medium
CN114511820A (en) Goods shelf commodity detection method and device, computer equipment and storage medium
CN112651333A (en) Silence living body detection method and device, terminal equipment and storage medium
CN111932462A (en) Training method and device for image degradation model, electronic equipment and storage medium
JP7396076B2 (en) Number recognition device, method and electronic equipment
CN117253022A (en) Object identification method, device and inspection equipment
CN112329845A (en) Method and device for replacing paper money, terminal equipment and computer readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20911791

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15/06/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20911791

Country of ref document: EP

Kind code of ref document: A1