WO2019200837A1

WO2019200837A1 - Method and system for measuring volume of parcel, and storage medium and mobile terminal

Info

Publication number: WO2019200837A1
Application number: PCT/CN2018/106796
Authority: WO
Inventors: 张帆
Original assignee: 南京阿凡达机器人科技有限公司
Priority date: 2018-04-17
Filing date: 2018-09-20
Publication date: 2019-10-24
Also published as: CN108627092A

Abstract

A method and system for measuring the volume of a parcel, and a storage medium and a mobile terminal. The method for measuring the volume of a parcel comprises: collecting a target image by means of a camera (1003) of a mobile terminal, wherein the target image comprises a plane marker, an upper surface of a parcel to be measured provided with the plane marker, and two side surfaces, adjacent to the upper surface, of the parcel to be measured (S100); carrying out image processing on the target image, so as to recognize the plane marker in the target image (S200); carrying out image edge detection on the target image, so as to identify, in the target image, angle points of the parcel to be measured (S300); and calculating the global coordinates of the angle points according to the plane marker, and a pre-acquired internal reference matrix and angle points of the camera, so as to obtain, according to the global coordinates of the angle points, the volume of the parcel to be measured (S400). By using the camera (1003) built in a mobile terminal to accurately measure, in real time, the volume of a parcel to be measured, the usage scenario is enriched, additional hardware facilities do not need to be added, and costs are saved.

Description

一种包裹体积的测量方法、***、储存介质及移动终端Method, system, storage medium and mobile terminal for measuring package volume

本申请要求2018年04月17日提交的申请号为：201810344830.4、发明名称为“一种包裹体积的测量方法、***、储存介质及移动终端”的中国专利申请的优先权，其全部内容合并在此。The present application claims the priority of the Chinese patent application filed on Apr. 17, 2018, the application number of which is: 201,810,344, 342, the invention is entitled "a method for measuring the volume of a package, a system, a storage medium and a mobile terminal", the entire contents of which are incorporated in this.

技术领域Technical field

本发明涉及计算机视觉领域，尤指一种包裹体积的测量方法、***、储存介质及移动终端。The invention relates to the field of computer vision, in particular to a method, a system, a storage medium and a mobile terminal for measuring a package volume.

背景技术Background technique

在快递行业发展迅猛的今天，有大量的快递待测包裹需要邮递。快递待测包裹的计费方式包括按重量或者体积的方式，快递员会取二者的最大值进行收费。再者，待测包裹的体积尺寸等会跟随该待测包裹的单号信息录入***，方便后续的装车及库存安排。所以，快速计算快递待测包裹的体积成为一种必要的技术。In today's rapid development of the express delivery industry, there are a large number of express parcels to be tested. The billing method of the parcel to be tested includes the method of weight or volume, and the courier will charge the maximum value of the two. Furthermore, the volume size of the package to be tested and the like will follow the single number information entry system of the package to be tested, facilitating subsequent loading and inventory arrangements. Therefore, it is a necessary technique to quickly calculate the volume of the package to be tested.

申请号为CN20151088222.6的发明专利公开了一种基于深度相机的体积测量方法及其***。该专利通过深度相机获取包含待测物体的深度图，然后根据深度信息将该物体从深度图像中分离出来。根据预先标定的相机参数，将待测物体的二维深度图像坐标转换为相机坐标系下的三维坐标，然后根据该待测对象的三维坐标计算该物体的尺寸。The invention patent of the application number CN20151088222.6 discloses a volume measurement method based on a depth camera and a system thereof. The patent acquires a depth map containing an object to be measured by a depth camera, and then separates the object from the depth image according to the depth information. The two-dimensional depth image coordinates of the object to be tested are converted into three-dimensional coordinates in the camera coordinate system according to the pre-calibrated camera parameters, and then the size of the object is calculated according to the three-dimensional coordinates of the object to be tested.

该方法有以下缺点：a.需要额外安装深度相机，深度相机价格比较昂贵，快递终端设备(或者手机)的成本比较低，如果全部配置上深度相机代价比较大。b.深度相机需要与计算模块(手机等移动终端设备)进行适配连接，实现技术比较麻烦。c.目前深度相机大多采用结构光或者TOF的方案，结构光易受强光干扰，不能在室外工作。而TOF深度相机的精度本来就比较低，在室外强光的干扰下误差更大。This method has the following disadvantages: a. An additional depth camera is required, the depth camera is relatively expensive, and the cost of the courier terminal device (or mobile phone) is relatively low, and if the depth camera is fully configured, the cost is relatively large. b. The depth camera needs to be adapted and connected with the computing module (mobile terminal device such as mobile phone), and the implementation technology is troublesome. c. At present, most of the depth cameras adopt the structure light or TOF scheme. The structure light is easily interfered by strong light and cannot work outdoors. The accuracy of the TOF depth camera is inherently low, and the error is greater under the interference of outdoor strong light.

发明内容Summary of the invention

本发明的目的是提供一种包裹体积的测量方法、***、储存介质及移动终端，实现使用手机等移动终端自带的摄像头实时、精确测量待测包裹体积，丰富使用场景，不需要新增额外的硬件设施，节省成本。The object of the present invention is to provide a method, a system, a storage medium and a mobile terminal for measuring a package volume, which realizes real-time and accurate measurement of the volume of the package to be tested by using a camera provided by a mobile terminal such as a mobile phone, and enriches the use scenario without adding additional Hardware facilities to save costs.

本发明提供的技术方案如下：The technical solution provided by the present invention is as follows:

本发明提供一种包裹体积的测量方法，应用于测量设有平面标识物的待测包裹的体积，包括步骤：S100通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面、与所述上表面相邻的所述待测包裹的两个侧表面；S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。The invention provides a method for measuring a package volume, which is used for measuring a volume of a package to be tested provided with a planar identifier, comprising the steps of: S100 collecting a target image by a camera of the mobile terminal; the target image comprising the planar identifier, An upper surface of the package to be tested of the planar identifier, and two side surfaces of the package to be tested adjacent to the upper surface; S200 performs image processing on the target image, thereby the target image Identifying the planar identifier; S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image; S400 according to the planar identifier, a pre-acquired camera The inner parameter matrix and the corner point, the world coordinates of the corner point are calculated, so that the volume of the package to be tested is obtained according to the world coordinates of the corner point.

优选的，所述步骤S100通过移动终端的摄像头采集目标图像之前包括步骤：S010对所述摄像头进行标定，获取所述摄像头的内参矩阵。Preferably, before the step S100 collects the target image by the camera of the mobile terminal, the method includes the following steps: S010: calibrating the camera to obtain an internal reference matrix of the camera.

优选的，所述步骤S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物具体包括步骤：S210对所述目标图像进行二值分割，并从分割后的二值化图像中提取外形轮廓；S220根据预存的所述平面标识物所对应的外形轮廓的共性特征，从提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；S230获取所述备选轮廓的正视图；S240当所述备选轮廓的正视图与预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像。Preferably, the step S200 performs image processing on the target image, so that the identifying the planar identifier in the target image specifically includes the following steps: S210 performs binary segmentation on the target image, and from the segmented Extracting a contour from the binarized image; S220: selecting, according to the common feature of the pre-stored contour identifier, the contour contour having the common feature as the candidate contour; A front view of the candidate contour; S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier.

优选的，所述平面标识物为正方形，所述步骤S220中预存的所述平面标识物所对应的预设轮廓特征为四边形；所述步骤S230获取所述备选轮廓的正视图具体为：S231读取所述备选轮廓的四个顶点的像素坐标；S232定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；S233将上述读取的四个顶点的像素坐标，以及所述定义的四个顶点的像素坐标分别代入平面射影变换公式(1)，求取射影变换矩阵M：Preferably, the plane identifier is a square, and the preset contour feature corresponding to the plane identifier pre-stored in the step S220 is a quadrangle; the front view of the candidate contour obtained in the step S230 is specifically: S231 Reading the pixel coordinates of the four vertices of the candidate contour; S232 defining the pixel coordinates of the four vertices after the candidate contour is projectively transformed into a front view as four pre-stored front views of the planar identifier Pixel coordinates of the vertex; S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projection transformation formula (1), respectively, and obtaining the projective transformation matrix M:

其中，X为备选轮廓顶点的齐次像素坐标，其非齐次像素坐标为

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

x1、x2、x3为X中的元素，x1'、x2'、x3'为X'中的元素；x、y分别为非对应的非齐次像素横坐标、纵坐标；u、v分别为射影变换后的非齐次像素横坐标、纵坐标； Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

S234根据所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour.

优选的，所述步骤S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点具体包括步骤：S310对所述目标图像进行边缘检测处理，得到边缘二值图像；S320从所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；S330计算所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。Preferably, the step S300 performs image edge detection on the target image, so that identifying a corner point of the package to be tested in the target image specifically includes the following steps: S310 performing edge detection processing on the target image to obtain An edge binary image; S320 searches for an edge line corresponding to the edge of the package to be tested from the edge binary image; S330 calculates a position of an intersection of any two edge lines of the edge line to obtain the package to be tested corner.

优选的，S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积具体包括步骤：Preferably, S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, thereby obtaining the volume of the package to be tested according to the world coordinates of the corner point. Specifically include the steps:

S410根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量；S410: calculating, according to the plane identifier and the pre-acquired internal reference matrix of the camera, an outer parameter matrix of a current field of view of the camera; the outer parameter matrix includes a rotation matrix and a translation vector;

S420根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标；S420: acquiring, according to the pixel coordinates of the corner point in the pixel coordinate system, the world coordinates of the corner point in combination with the rotation matrix and the translation vector;

S430根据所述角点的世界坐标计算所述待测包裹的长宽高；S430 calculates a length, width, and height of the package to be tested according to the world coordinates of the corner point;

S440将所述长宽高代入体积公式计算获取所述待测包裹的体积。S440 calculates the length and width of the volume into a volume formula to obtain the volume of the package to be tested.

优选的，所述步骤S410根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的旋转矩阵和平移向量具体包括步骤：Preferably, the step S410 calculates the rotation matrix and the translation vector of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera acquired in advance, and specifically includes the following steps:

S411以所述平面标识物的中点为原点建立世界坐标系；S411 establishes a world coordinate system with the midpoint of the planar identifier as an origin;

S412读取所述平面标识物的各个顶点在像素坐标系中的顶点像素坐标，以及各个顶点在世界坐标系中的顶点世界坐标；S412: reading vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

S413将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)得到所述摄像头当前视场的旋转矩阵和平移向量：S413, substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotation matrix and translation vector:

其中，(u，v，1)表示所述平面标识物中所述目标点在摄像机的像素坐标系中像素坐标的齐次坐标，(Xw，Yw，Zw，1)表示所述目标点在所述世界坐标系中世界坐标的齐次坐标，(Xc，Yc，Zc)表示所述目标点在摄像机的相机坐标系中的相机坐标；fx表示摄像机透镜的物理焦距F与成像仪在X轴方向每个单元尺寸的乘积，fy表示摄像机透镜的物理焦距F与成像仪在y轴方向每个单元尺寸的乘积，cx表示成像仪中心与光轴在X轴方向上的偏移，cy表示由于成像仪成像仪中心与光轴在Y轴方向上的偏移；R表示世界坐标系到相机坐标系的旋转矩阵，T表示所述世界坐标系到所述相机坐标系的平移向量。Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.

优选的，所述步骤S420根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标具体包括步骤：Preferably, the step S420 is performed according to the pixel coordinates of the corner point in the pixel coordinate system, and the world coordinates of the corner point are obtained by combining the rotation matrix and the translation vector.

S421读取所述角点在摄像机的像素坐标系中的角点的像素坐标；S421 reads the pixel coordinates of the corner point of the corner point in the pixel coordinate system of the camera;

S422将所述角点的像素坐标、所述预先获取的摄像头的内参矩阵以及所述摄像头当前视场的旋转矩阵和平移向量代入所述公式(4)中得到所述角点的世界坐标；其中，(u，v，1)表示所述角点的像素坐标的齐次坐标，(Xw，Yw，Zw)是所述角点的世界坐标，(Xw，Yw，Zw，1)表示所述角点的世界坐标的齐次坐标；S422, substituting the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4) to obtain the world coordinates of the corner point; (u, v, 1) represents the homogeneous coordinate of the pixel coordinates of the corner point, (Xw, Yw, Zw) is the world coordinate of the corner point, and (Xw, Yw, Zw, 1) represents the angle Homogeneous coordinates of the world coordinates of the point;

所述步骤S430根据所述角点的世界坐标计算所述待测包裹的长宽高具体包括步骤：The step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the following steps:

S431读取任意四个角点对应的世界坐标；所述四个角点中任意一个角点为第一角点，且剩余三个角点分别与所述第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴；S431 reads the world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively connected with the first corner point to generate a straight line Parallel to the XYZ axis of the world coordinate system, respectively;

S432将所述第一角点对应的世界坐标，以及第二角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的长度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的X轴；S432: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculating a length of the package to be tested; the second corner point and the first The line generated by the corner point connection is parallel to the X axis of the world coordinate system;

S433将所述第一角点对应的世界坐标，以及第三角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的宽度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的Y轴；S433: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculating a width of the package to be tested; the second corner point and the first corner The line generated by the point connection is parallel to the Y axis of the world coordinate system;

S434将第四角点对应的世界坐标以及角点的像素坐标，结合所述预先获取的摄像头的内参矩阵、所述摄像头当前视场的旋转矩阵和平移向量，代入所述公式(4)计算获得所述待测包裹的高度；所述第四角点与所述第一角点连接生成的直线平行于所述世界坐标系的Z轴。S434, the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point are combined with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector, and are obtained by substituting the formula (4). a height of the package to be tested; a line generated by the connection of the fourth corner point and the first corner point is parallel to a Z axis of the world coordinate system.

第二方面，本发明还提供一种包裹体积的测量***，应用于测量设有平面标识物的待测包裹的体积，所述包裹体积的测量***包括：图像获取模块，用于通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面和与上表面相邻的所述待测包裹的两个侧表面；In a second aspect, the present invention also provides a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar identifier, the measurement system of the package volume comprising: an image acquisition module, configured to pass through the mobile terminal The camera captures a target image; the target image includes the planar identifier, an upper surface of the package to be tested provided with the planar identifier, and two side surfaces of the package to be tested adjacent to the upper surface;

图像识别模块，用于对所述图像获取模块获取的所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；An image recognition module, configured to perform image processing on the target image acquired by the image acquisition module, thereby identifying the planar identifier in the target image;

图像处理模块，用于对所述图像获取模块获取的所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；An image processing module, configured to perform image edge detection on the target image acquired by the image acquiring module, so as to identify a corner point of the package to be tested in the target image;

体积测量模块，用于根据所述图像识别模块识别出的所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。a volume measuring module, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module, an internal parameter matrix of a pre-acquired camera, and the corner point, thereby, according to the corner point The world coordinates get the volume of the package to be tested.

优选的，所述的包裹体积的测量***，还包括：标定模块，用于在所述图像获取模块通过所述摄像头采集目标图像之前，对所述摄像头进行标定，获取所述摄像头的内参矩阵。Preferably, the measurement system of the package volume further includes: a calibration module, configured to calibrate the camera before acquiring the target image by the image acquisition module to obtain an internal parameter matrix of the camera.

优选的，所述图像识别模块包括：二值分割子模块，用于对所述提取模块获取的所述目标图像进行二值分割；外形提取子模块，用于从分割后的二值化图像中提取外形轮廓；存储子模块，用于存储所述平面标识物所对应的外形轮廓的共性特征、所述平面标识物模板；判断处理子模块，用于根据所述存储子模块中存储的所述平面标识物所对应的外形轮廓的共性特征，从所述外形提取子模块提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；正视图获取子模块，用于获取所述备选轮廓的正视图；识别子模块，用于当所述备选轮廓的正视图与所述存储子模块预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像。Preferably, the image recognition module includes: a binary segmentation sub-module for performing binary segmentation on the target image acquired by the extraction module; and an outline extraction sub-module for use in the segmented binarized image And extracting a profile; a storage submodule, configured to store a common feature of the outline corresponding to the planar identifier, the planar identifier template; and a determination processing submodule, configured to be configured according to the storage in the storage submodule a common feature of the contour corresponding to the planar identifier, and selecting, from the contour extracted by the shape extraction sub-module, an outline having the common feature as an alternative contour; and a front view obtaining sub-module for acquiring the preparation Selecting a front view of the contour; the identifying submodule, configured to identify, when the front view of the candidate contour is consistent with the planar identifier template prestored by the storage submodule, the image corresponding to the candidate contour is An image of a flat marker.

优选的，所述平面标识物为正方形，所述储存子模块储存的所述平面标识物所对应的外形轮廓的共性特征为四边形；所述正视图获取子模块包括：读取单元，用于读取所述备选轮廓的四个顶点的像素坐标；定义单元，用于定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；计算单元，用于将所述读取单元读取的四个顶点的像素坐标，以及所述定义单元定义的四个顶点的像素坐标分别代入下列平面射影变换公式(1)，求取射影变换矩阵M：Preferably, the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle; the front view obtaining sub-module includes: a reading unit for reading Taking pixel coordinates of four vertices of the candidate contour; defining a unit for defining pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as a pre-existing front view of the planar identifier a pixel coordinate of the four vertices; a calculation unit for substituting the pixel coordinates of the four vertices read by the reading unit and the pixel coordinates of the four vertices defined by the defining unit into the following planar projective transformation formula ( 1), find the projective transformation matrix M:

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

变换单元，用于根据所述计算单元求取得到的所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。a transforming unit, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit, to obtain an area included in the candidate contour Corresponding front view.

优选的，所述图像处理模块包括：边缘检测子模块，用于对所述提取模块获取的所述目标图像进行边缘检测处理，得到边缘二值图像；直线查找子模块，用于从所述边缘检测子模块得到的所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；角点获取子模块，用于计算所述直线查找子模块查找到的所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。Preferably, the image processing module includes: an edge detection sub-module, configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image; and a line search sub-module for using the edge Searching for the edge line corresponding to the edge of the package to be tested in the edge binary image obtained by the detection sub-module; the corner point acquisition sub-module is configured to calculate any two edges of the edge line found by the line search sub-module The position of the intersection formed by the straight line obtains the corner point of the package to be tested.

优选的，所述体积测量模块包括：外参矩阵获取子模块，用于根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量；Preferably, the volume measurement module includes: an outer parameter matrix acquisition submodule, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera acquired in advance; The outer parameter matrix includes a rotation matrix and a translation vector;

角点坐标获取子模块，用于根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标；a corner coordinate acquisition submodule, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in a pixel coordinate system, in combination with the rotation matrix and the translation vector;

长宽高获取子模块，用于根据所述角点的世界坐标计算所述待测包裹的长宽高；a length, width and height acquisition submodule, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

包裹体积获取子模块，用于将所述长宽高代入体积公式计算获取所述待测包裹的体积。a parcel volume acquisition sub-module, configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.

优选的，所述外参矩阵获取子模块包括：Preferably, the outer parameter matrix obtaining submodule comprises:

坐标系确定单元，用于以所述平面标识物的中点为原点建立世界坐标系；a coordinate system determining unit configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

顶点坐标读取单元，用于读取所述平面标识物的各个顶点在像素坐标系中的顶点像素坐标，以及各个顶点在世界坐标系中的顶点世界坐标；a vertex coordinate reading unit, configured to read vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

运算单元，用于将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)得到所述摄像头当前视场的旋转矩阵和平移向量：An arithmetic unit, configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and obtain the The rotation matrix and translation vector of the camera's current field of view:

优选的，所述角点坐标获取子模块包括：Preferably, the corner coordinate acquisition submodule comprises:

角点像素坐标获取单元，用于读取所述角点在摄像机的像素坐标系中的像素坐标；a corner pixel coordinate acquiring unit, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

角点世界坐标获取单元，用于将所述角点的像素坐标、所述预先获取的摄像头的内参矩阵以及所述摄像头当前视场的旋转矩阵和平移向量代入所述公式(4)中得到所述角点的世界坐标；其中，(u，v，1)表示所述角点的像素坐标的齐次坐标，(Xw，Yw，Zw)是所述角点的世界坐标，(Xw，Yw，Zw，1)表示所述角点的世界坐标的齐次坐标；a corner point world coordinate acquiring unit, configured to substitute pixel coordinates of the corner point, an internal parameter matrix of the pre-acquired camera, and a rotation matrix and a translation vector of a current field of view of the camera into the formula (4) The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw, Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

所述长宽高获取子模块包括：The long and wide height acquisition submodule includes:

角点世界坐标读取单元，用于读取任意四个角点对应的世界坐标；所述四个角点中任意一个角点为第一角点，且剩余三个角点分别与所述第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴；a corner point world coordinate reading unit, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points respectively correspond to the first A straight line generated by a pair of corner points is parallel to the XYZ axis of the world coordinate system, respectively;

长度运算单元，用于将所述第一角点对应的世界坐标，以及第二角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的长度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的X轴；a length operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the X axis of the world coordinate system;

宽度运算单元，用于将所述第一角点对应的世界坐标，以及第三角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的宽度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的Y轴；a width operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point and a line generated by the first corner connection is parallel to a Y axis of the world coordinate system;

高度运算单元，用于将第四角点对应的世界坐标以及角点的像素坐标，结合所述预先获取的摄像头的内参矩阵、所述摄像头当前视场的旋转矩阵和平移向量，代入所述公式(4)计算获得所述待测包裹的高度；所述第四角点与所述第一角点连接生成的直线平行于所述世界坐标系的Z轴。a height operation unit, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point, in combination with the pre-acquired internal parameter matrix of the camera, the rotation matrix of the current field of view of the camera, and the translation vector, and substitute the formula into the formula (4) Calculating a height of the package to be tested; a line generated by connecting the fourth corner point to the first corner point is parallel to a Z axis of the world coordinate system.

第三方面，本发明提供一种存储介质，所述存储介质存储有多条指令，所述多条指令被一个或者多个处理器执行，以实现本发明包裹体积的测量方法的步骤。In a third aspect, the present invention provides a storage medium storing a plurality of instructions that are executed by one or more processors to implement the steps of the method of measuring a package volume of the present invention.

第四方面，本发明提供一种移动终端，包括：处理器，实现各指令；存储介质，存储多条指令；其中：所述处理器执行所述存储介质存储的指令，以实现本发明包裹体积的测量方法的步骤。According to a fourth aspect, the present disclosure provides a mobile terminal, including: a processor that implements each instruction; a storage medium that stores a plurality of instructions; wherein: the processor executes an instruction stored by the storage medium to implement a package volume of the present invention The steps of the measurement method.

通过本发明提供的一种包裹体积的测量方法、***、储存介质及移动终端，能够带来以下至少一种有益效果：The method, system, storage medium and mobile terminal for measuring package volume provided by the invention can bring at least one of the following beneficial effects:

1)本发明通过在每个待测包裹的上表面设置一个平面标识物，然后摄像头采集包括平面标识物表面和任意一个或者多个与设有平面标识物表面相邻的待测包裹表面的目标视频，将目标视频提供世界坐标系、像素坐标系和相机坐标系之间的旋转平移计算得到待测包裹的长宽高，从而计算得到待测包裹的体积，由于使用移动终端自带的摄像头进行采集目标视频，能够实时、精确测量待测包裹体积。1) The present invention provides a planar marker by placing a planar marker on the upper surface of each package to be tested, and then the camera collects a surface including the planar marker surface and any one or more of the surface of the package to be tested adjacent to the surface on which the planar marker is disposed. Video, the target video is provided with a rotation translation between the world coordinate system, the pixel coordinate system and the camera coordinate system to obtain the length, width and height of the package to be tested, thereby calculating the volume of the package to be tested, which is performed by using the camera attached to the mobile terminal. The target video is collected, and the volume of the package to be tested can be measured in real time and accurately.

2)本发明使用移动终端自带的摄像头进行采集目标视频，不需要深度相机进行采集，并且不需要深度相机采集待测包裹时必须位于待测包裹的正上方，因此不需要复杂的布置，而且该摄像头不受强光干扰、可以在室外工作，丰富了使用场景，不需要新增额外的硬件设施，节省成本。2) The present invention uses the camera provided by the mobile terminal to acquire the target video, does not require a depth camera for acquisition, and does not require the depth camera to collect the package to be tested, and must be located directly above the package to be tested, so that no complicated arrangement is required, and The camera is protected from strong light and can work outdoors, enriching the use of the scene, eliminating the need for additional hardware and cost savings.

附图说明DRAWINGS

下面将以明确易懂的方式，结合附图说明优选实施方式，对一种包裹体积的测量方法、***、储存介质及移动终端的上述特性、技术特征、优点及其实现方式予以进一步说明。The above-described characteristics, technical features, advantages and implementation manners of a package volume measuring method, system, storage medium and mobile terminal will be further described below in a clear and understandable manner with reference to the accompanying drawings.

图1是本发明一种包裹体积的测量方法的一个实施例的流程图；1 is a flow chart of an embodiment of a method for measuring a package volume of the present invention;

图2是本发明一种包裹体积的测量方法的另一个实施例的流程图；2 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

图3是本发明一种包裹体积的测量方法的另一个实施例的流程图；3 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

图4是本发明一种包裹体积的测量方法的另一个实施例的流程图；4 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

图5是本发明一种包裹体积的测量方法和***的待测包裹的角点示意图；5 is a schematic view showing a corner point of a package to be tested according to a method and system for measuring a package volume according to the present invention;

图6是本发明一种包裹体积的测量方法和***的以平面标识物的中点为原点的世界坐标系的示意图；6 is a schematic diagram of a world coordinate system with a midpoint of a planar marker as an origin of a method and system for measuring a package volume according to the present invention;

图7是本发明一种包裹体积的测量方法的另一个实施例的流程图；Figure 7 is a flow chart showing another embodiment of a method for measuring a package volume of the present invention;

图8是本发明一种包裹体积的测量方法的另一个实施例的流程图；Figure 8 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

图9是本发明一种包裹体积的测量方法的另一个实施例中的待测包裹的体积测量步骤的流程图；9 is a flow chart showing a volume measuring step of a package to be tested in another embodiment of a method for measuring a package volume according to the present invention;

图10a是本发明一种包裹体积的测量方法的另一个实施例中的张正友平面标定方法的标定模板示意图；10a is a schematic diagram of a calibration template of a Zhang Zhengyou plane calibration method in another embodiment of a method for measuring a package volume according to the present invention;

图10b是本发明一种包裹体积的测量方法的另一个实施例中的张正友平面标定方法中检测特征点示意图；FIG. 10b is a schematic diagram showing the detected feature points in the Zhang Zhengyou plane calibration method in another embodiment of the method for measuring the package volume of the present invention; FIG.

图11是本发明一种包裹体积的测量方法的另一个实施例中的摄像头测量姿态的示意图；11 is a schematic diagram of a camera measuring attitude in another embodiment of a method for measuring a package volume according to the present invention;

图12是本发明一种包裹体积的测量方法的另一实施例中的旋转后的四边形轮廓正视图；Figure 12 is a front elevational view of the rotated quadrilateral profile in another embodiment of the method for measuring the package volume of the present invention;

图13是本发明一种包裹体积的测量***的一个实施例的结构示意图；Figure 13 is a schematic structural view of an embodiment of a package volume measuring system of the present invention;

图14是本发明一种包裹体积的测量***的另一个实施例的结构示意图；Figure 14 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

图15是本发明一种包裹体积的测量***的另一个实施例的结构示意图；15 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

图16是本发明一种包裹体积的测量***的另一个实施例的结构示意图；16 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

图17是本发明一种包裹体积的测量***的另一个实施例的结构示意图；17 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

图18是本发明一种包裹体积的测量***的另一个实施例的结构示意图；18 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

图19是本发明一种包裹体积的测量的移动终端的一个实施例的结构示意图。Figure 19 is a block diagram showing an embodiment of a mobile terminal for measuring the volume of the package of the present invention.

具体实施方式detailed description

为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对照附图说明本发明的具体实施方式。显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图，并获得其他的实施方式。In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings according to the drawings without obtaining creative labor, and obtain Other embodiments.

为使图面简洁，各图中只示意性地表示出了与本发明相关的部分，它们并不代表其作为产品的实际结构。另外，以使图面简洁便于理解，在有些图中具有相同结构或功能的部件，仅示意性地绘示了其中的一个，或仅标出了其中的一个。在本文中，“一个”不仅表示“仅此一个”，也可以表示“多于一个”的情形。In order to simplify the drawings, only the parts related to the present invention are schematically shown in the drawings, and they do not represent the actual structure of the product. In addition, in order to make the drawings simple and easy to understand, components having the same structure or function in some of the figures are only schematically illustrated, or only one of them is marked. In the present context, "a" means not only "only one" but also "more than one".

本方案适用于形状为长方体、正方体或者正四棱柱等矩形形状的待测包裹的体积测量，对于其他复杂形状、无规则形状的包裹无法测量。而由于目前快递物流行业的包裹普遍为矩形形状的纸箱、木架等等，因此普遍适用于快递物流行业的矩形形状的包裹的体积尺寸的测量。This scheme is applicable to the volume measurement of a package to be tested having a rectangular shape such as a rectangular parallelepiped, a cube, or a regular quadrangular prism. For other complicated shapes and irregular shapes, the package cannot be measured. Since the parcels in the express logistics industry are generally rectangular-shaped cartons, wooden frames, etc., they are generally suitable for measuring the volume size of rectangular shaped parcels in the express logistics industry.

本发明的第一个实施例，如图1所示，一种包裹体积的测量方法，应用于测量设有平面标识物的待测包裹的体积，包括：A first embodiment of the present invention, as shown in FIG. 1, is a method for measuring a package volume, which is used to measure the volume of a package to be tested provided with a planar marker, including:

S100通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面、与所述上表面相邻的所述待测包裹的两个侧表面；S100 collects a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the plane identifier, and the package to be tested adjacent to the upper surface Two side surfaces;

S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；S200 performing image processing on the target image, thereby identifying the planar identifier in the target image;

S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image;

S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinates of the corner point.

具体的，本实施例中，平面标识物是指预先设计好的具有特定几何形状和编码信息的图案，通常打印在一张纸上面。通过移动终端自带的前置摄像头或者后置摄像头均可以进行采集目标视频，使用移动终端进行采集目标视频时，需要将移动终端上的摄像头镜头朝向设于平面标识物的表面的斜上方，这样便于将待测包裹上设有平面标识物的上表面、与设有平面标识物的待测包裹的上表面相邻的任意两个侧表面进行采集。此处上表面并不一定是待测包裹放置时远离于地面的上表面，而是定义为摄像头在采集目标图像时在待测包裹设有平面标识物的任意一个摄像头可采集到的表面，即若是将平面标识物设于形状为长方体的待测包裹的远离底面的上表面，那么就能从与上表面相邻的四个侧表面中获取任意两个或者多个侧表面；若是将平面标识物设于矩形的待测包裹的任意一个侧表面，那么就能从与侧表面相邻的一个上表面和剩余三个侧表面中获取任意两个或者多个侧表面。此处只例举待测包裹的形状为长方体的示例，其他矩形形状的待测包裹均在本发明的保护范围内。Specifically, in the embodiment, the planar identifier refers to a pre-designed pattern having a specific geometric shape and encoding information, which is usually printed on a piece of paper. The target video can be collected by the front camera or the rear camera that is provided by the mobile terminal. When the mobile terminal is used to collect the target video, the camera lens on the mobile terminal needs to be oriented obliquely above the surface of the planar identifier. It is convenient to collect the upper surface of the package to be tested with the planar identifier and any two side surfaces adjacent to the upper surface of the package to be tested provided with the planar identifier. Here, the upper surface is not necessarily the upper surface away from the ground when the package to be tested is placed, but is defined as the surface that can be collected by any camera with a planar marker in the package to be tested when the target image is captured, ie If the planar identifier is disposed on the upper surface of the package to be tested having a rectangular parallelepiped shape away from the bottom surface, any two or more side surfaces can be obtained from the four side surfaces adjacent to the upper surface; The object is disposed on any one of the side surfaces of the rectangular package to be tested, and then any two or more side surfaces can be obtained from one upper surface and the remaining three side surfaces adjacent to the side surface. Here, only an example in which the shape of the package to be tested is a rectangular parallelepiped is exemplified, and other rectangular shaped packages to be tested are all within the scope of the present invention.

本发明为了全方位、完整的采集到目标图像，可以通过移动终端自带的摄像头进行采集视频，将采集到的视频进行图像处理获取目标图像，将目标图像进行处理后得到平面标识物的图像和待测包裹的角点，从而根据平面标识物和角点以及预先获取的摄像机的内参矩阵进行计算得到待测包裹的角点的世界坐标，根据角点的世界坐标计算得到待测包裹的体积。本发明由于使用移动终端自带的摄像头进行采集目标视频，不需要深度相机进行采集，并且不需要深度相机采集待测包裹时必须位于待测包裹的正上方，因此不需要复杂的布置，丰富了使用场景，不需要新增额外的硬件设施，节省成本。此外，本发明不需要将安装有深度相机与移动终端适配连接，测量用户的待测包裹的体积时，不仅仅要携带深度相机和移动终端，还需要在使用时对深度相机与移动终端进行实时的适配连接，本发明采用移动终端自带的摄像头和处理器，造价便宜，成本低的同时，简化待测包裹体积测量的步骤，测量效率高，并且由于待测包裹的体积由移动终端自带的处理器自动完成，操作简单，而且检测结果直观、可靠，测量精度高。因此，本发明能够全自动化的测量出包裹的体积，准确高效，同时也不用安装其他硬件设备，简单方便，用户体验度高。In order to acquire the target image in an all-round and complete manner, the invention can collect video through the camera provided by the mobile terminal, perform image processing on the captured video to obtain a target image, and process the target image to obtain an image of the planar marker. The corner point of the package to be tested is calculated according to the plane identifier and the corner point and the internal reference matrix of the pre-acquired camera to obtain the world coordinates of the corner point of the package to be tested, and the volume of the package to be tested is calculated according to the world coordinates of the corner point. The invention uses the camera provided by the mobile terminal to collect the target video, does not need the depth camera to collect, and does not need the depth camera to collect the package to be tested, and must be located directly above the package to be tested, so no complicated arrangement is needed, which enriches Use the scenario, no need to add additional hardware facilities, and save costs. In addition, the present invention does not need to be equipped with a depth camera and a mobile terminal, and when measuring the volume of the user's package to be tested, not only the depth camera and the mobile terminal but also the depth camera and the mobile terminal are used. The real-time adaptive connection, the invention adopts the camera and the processor provided by the mobile terminal, and has the advantages of low cost and low cost, simplifying the step of measuring the volume of the package to be tested, high measurement efficiency, and the volume of the package to be tested by the mobile terminal The built-in processor is automatically completed, the operation is simple, and the detection result is intuitive, reliable, and the measurement accuracy is high. Therefore, the invention can fully measure the volume of the package, is accurate and efficient, and does not need to install other hardware devices, is simple and convenient, and has high user experience.

本发明的第二个实施例，在S100通过移动终端的摄像头采集目标图像之前包括步骤：In a second embodiment of the present invention, before the S100 collects the target image through the camera of the mobile terminal, the method includes the following steps:

S010对所述摄像头进行标定，获取所述摄像头的内参矩阵。S010 calibrates the camera to obtain an internal reference matrix of the camera.

具体的，本实施例是上述第一实施例的优化实施例，本实施例与上述第一实施例相比，主要改进在于，新增了步骤S010来对摄像头进行标定，即本实施例中在采用上述方案进行测量待测包裹的体积之前，还需要对用来采集目标视频的摄像头进行标定，具体的，通过对所述摄像头进行标定，获取所述摄像头的内参矩阵，即确定所述摄像头成像的几何模型。摄像机标定是整个***的前期准备阶段，当选用一款摄像头进行视觉测量必须对其进行标定，从而获得摄像机的畸变系数及焦距等参数组成的内参矩阵。在后续的执行阶段，可以使用该摄像机的内参矩阵进行相应的视觉测量及计算。标定操作只需要执行一次。Specifically, the embodiment is an optimized embodiment of the foregoing first embodiment. Compared with the first embodiment, the main improvement is that the step S010 is added to calibrate the camera, that is, in this embodiment. Before measuring the volume of the package to be tested by using the above solution, the camera for collecting the target video needs to be calibrated. Specifically, by calibrating the camera, the internal reference matrix of the camera is obtained, that is, the camera is determined to be imaged. Geometric model. Camera calibration is the pre-preparation stage of the whole system. When a camera is selected for visual measurement, it must be calibrated to obtain the internal reference matrix composed of parameters such as distortion coefficient and focal length of the camera. In the subsequent implementation phase, the camera's internal reference matrix can be used for corresponding visual measurements and calculations. The calibration operation only needs to be performed once.

本发明采用张正友的棋盘标定法对摄像头进行标定，张正友标定法通过多张摄像头相对于棋盘不同角度时采集的图像对摄像头的内参系数进行计算，最终得到摄像头的内参矩阵。The invention uses Zhang Zhengyou's checkerboard calibration method to calibrate the camera. The Zhang Zhengyou calibration method calculates the internal reference coefficient of the camera by using images acquired by different cameras relative to different angles of the chessboard, and finally obtains the internal reference matrix of the camera.

该平面标定法的过程如下：The process of this plane calibration method is as follows:

①打印一张平面标定的模板并贴在一个平面上；1 print a plane-calibrated template and affix it to a flat surface;

②从不同角度拍摄若干张模板图像；2 taking several template images from different angles;

③检测出图像中的特征点；3 detecting feature points in the image;

④根据特征点在世界坐标系中坐标位置及其在图像中对应的像素坐标，联立方程组求出摄像机的内参矩阵，摄像机的内参矩阵是一个3x3的矩阵K，具体的形式如下：4 According to the coordinate position of the feature point in the world coordinate system and its corresponding pixel coordinates in the image, the internal parameter matrix of the camera is obtained by the simultaneous equations. The internal parameter matrix of the camera is a 3x3 matrix K. The specific form is as follows:

其中，fx表示摄像机透镜的物理焦距F与成像仪在X轴方向每个单元尺寸的乘积，fy表示摄像机透镜的物理焦距F与成像仪在y轴方向每个单元尺寸的乘积，cx表示成像仪中心与光轴在X轴方向上的偏移，cy表示由于成像仪成像仪中心与光轴在Y轴方向上的偏移。Where fx represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the X-axis direction, fy represents the product of the physical focal length F of the camera lens and the size of each element of the imager in the y-axis direction, cx represents the imager The offset of the center from the optical axis in the X-axis direction, cy represents the offset in the Y-axis direction due to the center of the imager imager and the optical axis.

本发明的第三个实施例，是上述第一、第二实施例的优化实施例，如图2所示，一种包裹体积的测量方法，应用于测量设有平面标识物的待测包裹的体积，包括：A third embodiment of the present invention is an optimized embodiment of the first and second embodiments, as shown in FIG. 2, a method for measuring a package volume, which is used for measuring a package to be tested provided with a planar marker. Volume, including:

所述步骤S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物具体包括；The step S200 performs image processing on the target image, so that the planar identifier is specifically included in the target image;

S210对所述目标图像进行二值分割，并从分割后的二值化图像中提取外形轮廓；S210 performs binary segmentation on the target image, and extracts a contour from the segmented binarized image;

S220根据预存的所述平面标识物所对应的外形轮廓的共性特征，从提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；S220: selecting, according to a common feature of the pre-stored contour identifiers of the plane identifiers, an outline contour having the common features as an alternative contour from the extracted contour contours;

S230获取所述备选轮廓的正视图；S230 acquires a front view of the candidate contour;

S240当所述备选轮廓的正视图与预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像；S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier;

具体的，本实施例是上述第一或第二实施例的优化实施例，本实施例与上述第一或第二实施例相比，主要改进在于，新增了步骤S210-S240来对如何对目标图像进行处理从而识别出屏幕平面标识物进行了进一步细化说明。本实施例对所述目标图像进行二值分割，二值分割即将图像上的像素点的灰度值设置为0或255，也就是将整个图像呈现出明显的黑白视觉效果，并将图像中具有特殊涵义的不同区域分开来，这些区域互不相交，每个区域都满足特定区域的一致性。也可以利用自适应性阈值对摄像头获取的图像进行二值分割。对所述目标图像进行二值分割处理后，需要识别出平面标识物，由于摄像头可能从不同角度(但是均能拍摄到平面标识物和包括平面标识物的表面和其他表面)拍摄得到了不同的目标视频，因此，采集的图像视频帧上的平面标识物并不一定是正视图的样子。那么需要先提取出从各个角度拍摄出的平面标识物，并对该平面标识物进行边缘检测，获取该平面标识物的外形轮廓的共性特征，再将这个共性特征预先进行存储，便于在采集视频帧后对该平面标识物进行识别。具体的，在对视频帧图像进行二值分割后，提取分割后的各图像的外形轮廓，根据平面标识物外形轮廓的的共性特征，查找出备选的外形轮廓，对轮廓进行多边形拟合，舍弃那些非凸多边形的，以及那些不是四边形的轮廓。此外，还可以使用一些额外的限制条件来剔除那些不可能是平面标识物图像的四边形轮廓，如四边形一边明显小于其余的边(形状比较瘦长)，轮廓周长或面积过小等，剩下的符合条件的轮廓即为平面标识物可能对应的的备选轮廓。比如，如果我们选择的屏幕平面标识物是正方形，那么经过边缘检测后可获得其外形轮廓的共性特征为四边形，那么我们便据此，从提取出的外形轮廓中，排除那些非凸多边形的，以及不是四边形的轮廓等。获得备选轮廓后，下一步就是获取各个备选轮廓的正视图，也就是说将平面标识物可能对应的备选轮廓图像区域变为正视图，从而进一步判断该备选轮廓区域是否为平面标识物对应的图像。具体的，通过将每个备选轮廓的正视图与预先存储的该平面标识物模板进行比对，当出现有比对一致的情况下便可判断出这个备选轮廓即为平面标识物的外形轮廓，这个备选轮廓对应的图像就是平面标识物的图像，至此，平面标识物被识别出来了。平面标识物的形状可以是矩形、菱形或者正方形等等形状。Specifically, this embodiment is an optimized embodiment of the foregoing first or second embodiment. Compared with the foregoing first or second embodiment, the main improvement is that steps S210-S240 are added to how to The target image is processed to identify the screen plane identifier for further refinement. In this embodiment, the target image is binary-divided, and the binary value is that the gray value of the pixel on the image is set to 0 or 255, that is, the entire image is presented with an obvious black-and-white visual effect, and the image has Different regions of special meaning are separated, and these regions do not intersect each other, and each region satisfies the consistency of a particular region. The image acquired by the camera can also be binary-divided using an adaptive threshold. After performing the binary segmentation processing on the target image, it is necessary to identify the planar identifier, since the camera may be photographed differently from different angles (but both the planar marker and the surface including the planar marker and other surfaces can be captured). The target video, therefore, the planar identifier on the captured image video frame does not have to be the front view. Then, it is necessary to extract the planar marker photographed from various angles, and perform edge detection on the planar marker to obtain the common features of the contour of the planar marker, and then store the common feature in advance to facilitate the video capture. The planar identifier is identified after the frame. Specifically, after the video frame image is binary-divided, the contours of the divided images are extracted, and according to the common features of the contours of the planar identifiers, an alternative contour is found, and the contour is polygon-fitted. Discard those non-convex polygons and those that are not quadrilateral. In addition, some additional constraints can be used to eliminate quadrilateral contours that are not likely to be planar marker images, such as the quadrilateral side is significantly smaller than the remaining edges (the shape is relatively thin), the contour perimeter or area is too small, etc., and the rest An eligible contour is an alternative contour that the planar marker may correspond to. For example, if the screen plane identifier we selected is a square, then after the edge detection, the common feature of the outline can be quadrilateral, then we can exclude those non-convex polygons from the extracted outline. And not the outline of the quadrilateral, etc. After the candidate contour is obtained, the next step is to obtain a front view of each candidate contour, that is, to change the candidate contour image area corresponding to the plane identifier into a front view, thereby further determining whether the candidate contour region is a plane identifier. The image corresponding to the object. Specifically, by comparing the front view of each candidate contour with the pre-stored planar identifier template, when the alignment is consistent, it can be determined that the candidate contour is the shape of the planar identifier. The contour, the image corresponding to this alternative contour is the image of the planar marker, and thus the planar marker is identified. The shape of the planar marker may be a rectangle, a diamond or a square or the like.

较佳的，在上述实施例的基础上，设置在待测包裹任意一个表面上的平面标识物为正方形，根据预存的所述平面标识物所对应的外形轮廓的共性特征中预存的所述平面标识物所对应的外形轮廓的共性特征是所述平面标识物所对应的外形轮廓为四边形。设置的平面标识物不同，那么对应的外形轮廓的共性特征也会不同。我们在选取平面标识物时尽量选取易于辨识的，特征点比较明显的平面标识物为佳。比如正方形，具有四个顶点，由于正方形的四个边长一致，加上四个顶点比较有利于后续的定位运算，使得运算起来也更为简单。Preferably, on the basis of the above embodiment, the planar identifier disposed on any surface of the package to be tested is a square, and the plane pre-stored in the common feature of the contour contour corresponding to the pre-stored planar identifier A common feature of the contour corresponding to the identifier is that the contour corresponding to the planar identifier is a quadrangle. If the set flat markers are different, the common features of the corresponding outlines will be different. When selecting a planar marker, we should try to select a flat marker that is easy to identify and has obvious feature points. For example, a square has four vertices. Since the four sides of the square are the same, adding four vertices is more conducive to subsequent positioning operations, making the operation simpler.

本发明的第四个实施例，如图3所示，是上述第三实施例的优化实施例，一种包裹体积的测量方法，应用于测量设有平面标识物的待测包裹的体积，包括：A fourth embodiment of the present invention, as shown in FIG. 3, is an optimized embodiment of the third embodiment, a method for measuring a package volume, which is used to measure the volume of a package to be tested provided with a planar marker, including :

所述步骤S230获取所述备选轮廓的正视图具体为：The step S230 obtains a front view of the candidate contour, which is specifically:

S231读取所述备选轮廓的四个顶点的像素坐标；S231 reading pixel coordinates of four vertices of the candidate contour;

S232定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；S232, the pixel coordinates of the four vertices after the projective transformation into the front view are defined as the pixel coordinates of the pre-stored four vertices of the front view of the planar identifier;

S233将上述读取的四个顶点的像素坐标，以及所述定义的四个顶点的像素坐标分别代入平面射影变换公式(1)，求取射影变换矩阵M：S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projective transformation formula (1), respectively, and obtaining the projective transformation matrix M:

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

S234根据所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。S240当所述备选轮廓的正视图与预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像；S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour. S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier;

具体的，本实施例是上述第三实施例的优化实施例，本实施例与上述第三实施例相比，主要改进在于，新增了步骤S231-S234来对如何获取得到备选轮廓的正视图进行了进一步细化说明。本实施例延续上述实施例中的平面标识物为正方形的例子继续举例说明，根据正方形平面标识物的正方形特性，比如，我们可以定义备选轮廓经过射影变换转换为正视图后的四个顶点的像素坐标，比如正方形的平面标识物的正视图的四个顶点像素坐标分别为(0，0)，(0，100)，(100，100)，(100，0)，那么可以将这四个顶点坐标作为四边形备选轮廓的四个顶点的像素坐标，此外，四边形备选轮廓的原图像的像素坐标可从图像中读取出来，通过四组顶点对应关系，可联立方程组求解出摄影变换矩阵M，利用该射影变换矩阵M便可以将四边形备选轮廓区域的所有像素都执行射影变换，即可得到该四边形备选轮廓的区域对应的正视图，该正视图图像大小为100×100，即可得到该四边形备选轮廓的区域对应的正视图，然后将该正视图与预存的该平面标识物模板进行对比，看是否一致，一致的话则说明这个四边形轮廓就是该平面标识物的轮廓，那么平面标识物便被识别出来了。这里选用正视图来对比是因为摄像头从各个角度拍摄出的平面标识物的图像是不同的，如果摄像头处在平面标识物的正上方，则平面标识物的图像也是一个矩形，如果摄像处于平面标识物的斜上方，那么矩形平面标识物的图像是一个不规则四边形，获取正视图进行对比的目的就是，不管摄像头从那个角度看，平面标识物的正视图都是一种从正上方往下看的图像，如此，便于平面标识物的识别。Specifically, this embodiment is an optimized embodiment of the foregoing third embodiment. Compared with the foregoing third embodiment, the main improvement is that steps S231-S234 are added to face how to obtain an alternative contour. The figure is further refined. This embodiment continues the example in which the planar identifier in the above embodiment is square. Continuing to illustrate, according to the square characteristic of the square planar identifier, for example, we can define the four contours after the candidate contour is converted into the front view by the projective transformation. The pixel coordinates, such as the four vertex pixel coordinates of the front view of the square planar identifier are (0, 0), (0, 100), (100, 100), (100, 0), then these four The vertex coordinates are used as the pixel coordinates of the four vertices of the quadrilateral candidate contour. In addition, the pixel coordinates of the original image of the quadrilateral candidate contour can be read from the image. Through the correspondence of the four sets of vertices, the simultaneous equations can be used to solve the photography. Transforming matrix M, by which all pixels of the quadrilateral candidate contour region can be subjected to projective transformation, and a front view corresponding to the region of the quadrilateral candidate contour can be obtained. The size of the front view image is 100×100. , to obtain a front view corresponding to the area of the quadrilateral alternative contour, and then compare the front view with the pre-stored planar identifier template To see if consistent, consistent, then it indicates that this is the outline of the plane quadrilateral outline marker, then the marker will be identified plane out. Here, the front view is selected for comparison because the image of the planar marker photographed by the camera from various angles is different. If the camera is directly above the planar marker, the image of the planar marker is also a rectangle, if the camera is in the plane logo The object is diagonally above, then the image of the rectangular flat marker is an irregular quadrilateral. The purpose of obtaining the front view for comparison is that the front view of the planar marker is viewed from the top directly regardless of the angle from which the camera looks. The image, in this way, facilitates the identification of flat markers.

本发明的第五个实施例，如图4所示，是上述第一实施例、第二实施例、第三实施例或者第四实施例的优化实施例，包括：The fifth embodiment of the present invention, as shown in FIG. 4, is an optimized embodiment of the first embodiment, the second embodiment, the third embodiment, or the fourth embodiment, and includes:

S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积；S400 calculates a world coordinate of the corner point according to the plane identifier, an internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain a volume of the package to be tested according to the world coordinate of the corner point;

所述S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点具体包括；The S300 performs image edge detection on the target image, so that the corner point of the package to be tested is specifically included in the target image;

S310对所述目标图像进行边缘检测处理，得到边缘二值图像；S310 performing edge detection processing on the target image to obtain an edge binary image;

S320从所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；S320: searching for the edge line corresponding to the edge of the package to be tested from the edge binary image;

S330计算所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。S330 calculates a position of an intersection of any two edge lines in the edge line to obtain a corner point of the package to be tested.

具体的，本实施例是上述第一实施例至第四实施例中任意一个实施例的优化实施例，本实施例与上述第一实施例至第四实施例中任意一个实施例相比，主要改进在于，通过步骤S310-S330来对如何检测识别出待测包裹的角点进行了进一步细化说明。本实施例中，对目标图像进行边缘检测处理，得到边缘二值图像，可以使用Canny边缘检测算法，即用高斯滤波器平滑目标图象检测目标图像中明显的边缘，用一阶偏导的有限差分来计算梯度的幅值和方向，对梯度幅值进行非极大值抑制，用双阈值算法检测和连接边缘，由于待测包裹的边缘与背景对比明显，所以通过边缘检测算法，可以突出这些明显的物体边缘，该过程会得到包含图像中待测包裹边缘的二值图像。Specifically, this embodiment is an optimized embodiment of any one of the foregoing first to fourth embodiments, and the present embodiment is mainly compared with any one of the foregoing first to fourth embodiments. The improvement is that the corner points of how to detect the package to be tested are further refined by steps S310-S330. In this embodiment, edge detection processing is performed on the target image to obtain an edge binary image, and a Canny edge detection algorithm can be used, that is, a Gaussian filter is used to smooth the target image to detect a significant edge in the target image, and the first-order partial guide is limited. The difference is used to calculate the amplitude and direction of the gradient, the non-maximum suppression of the gradient amplitude is used, and the edge is detected and connected by the double threshold algorithm. Since the edge of the package to be tested is obviously contrasted with the background, the edge detection algorithm can highlight these The apparent edge of the object, the process will get a binary image containing the edge of the package to be tested in the image.

对目标图像进行边缘检测处理得到包含图像中待测包裹边缘的边缘二值图像后，可以用Hough变换检测直线的方法从边缘二值图像中寻找所有存在的直线，Hough变换检测边缘二值图像中存在的直线的方法为，直线在极坐标系下的方程为下列公式(5)；After the edge detection processing of the target image is performed to obtain the edge binary image containing the edge of the package to be tested in the image, the Hough transform can be used to detect the straight line to find all the existing lines from the edge binary image, and the Hough transform detects the edge binary image. The method of the existence of the straight line is that the equation of the straight line in the polar coordinate system is the following formula (5);

ρ＝xcosθ+ysinθ (5)ρ=xcosθ+ysinθ (5)

对于某一个(xi，yi)，可以计算得到一条正弦曲线ρ＝xicosθ+yisinθ，而对于二值化边缘图像中的点，可以得到不同的正弦曲线，根据不同的θ，可以计算得到一系列的ρ，根据得到的(ρ，θ)建立起一个二维的累加器单元阵列，其中单元的大小是固定的，若对应的(ρ，θ)值落到某个单元内，则这个单元的累加器计数加一，最终累加器值超过一个预先定义的阈值T的对应的单元认为代表了一条直线。Hough变换检测直线的方法的已多有论述，此处不再深入讨论。For a certain (xi, yi), a sine curve ρ=xicosθ+yisinθ can be calculated, and for the points in the binarized edge image, different sinusoids can be obtained. According to different θ, a series of calculations can be obtained. ρ, according to the obtained (ρ, θ), establish a two-dimensional array of accumulator cells, wherein the size of the cell is fixed, and if the corresponding (ρ, θ) value falls within a certain unit, the accumulation of the cell The counter is incremented by one, and the corresponding unit whose final accumulator value exceeds a predefined threshold T is considered to represent a straight line. The Hough transform method for detecting straight lines has been discussed extensively and will not be discussed in depth here.

Hough变换检测直线也只是一种优选的方式，亦可选用其他的检测直线的方法，得到包含图像中待测包裹边缘的边缘二值图像，例如霍夫直线检测算法在边缘二值图像中寻找直线。霍夫直线检测算法基于明显的边缘寻找直线，二值图像中细小或者杂乱的边缘会自动被过滤掉。The Hough transform detection line is also a preferred method. Other methods of detecting the line can be used to obtain an edge binary image containing the edge of the package to be tested in the image. For example, the Hough line detection algorithm finds a line in the edge binary image. . The Hough line detection algorithm finds lines based on obvious edges, and small or cluttered edges in binary images are automatically filtered out.

霍夫直线检测算法检测直线具体步骤如下：The specific steps of the Hough line detection algorithm for detecting straight lines are as follows:

1、随机抽取图像中的一个特征点，即边缘点，如果该点已经被标定为是某一条直线上的点，则继续在剩下的边缘点中随机抽取一个边缘点，直到所有边缘点都抽取完了为止；1. Randomly extract a feature point in the image, that is, an edge point. If the point has been calibrated to be a point on a certain line, continue to randomly extract an edge point in the remaining edge points until all edge points are After the extraction is completed;

2、对该点进行霍夫变换，并进行累加和计算；2. Perform a Hough transform on the point and perform an accumulation and calculation;

3、选取在霍夫空间内值最大的点，如果该点大于阈值的，则进行步骤4，否则回到步骤1；3, select the point with the largest value in the Hough space, if the point is greater than the threshold, proceed to step 4, otherwise return to step 1;

4、根据霍夫变换得到的最大值，从该点出发，沿着直线的方向位移，从而找到直线的两个端点；4. According to the maximum value obtained by the Hough transform, from this point, the displacement is along the direction of the straight line, thereby finding the two end points of the straight line;

5、计算直线的长度，如果大于某个阈值，则被认为是好的直线输出，回到步骤1。5. Calculate the length of the line. If it is greater than a certain threshold, it is considered to be a good straight line output. Go back to step 1.

由于待测包裹的边缘比较明显，也比较长，所以能够检测出待测包裹边缘对应的边缘直线，如图5和图6所示，计算边缘直线中任意两条边缘直线组成的交点的位置，得到待测包裹的角点A1，A2，A3，A4，还可以包括待测包裹的其他角点A5，A6，A7，但是基于待测包裹的体积测量效率考虑，只计算得到A1，A2，A3，A4即可。当然，如果为了验证待测包裹的体积计算是否正确，可以使用A3，A5，A6，A7再次进行计算，比较与A1，A2，A3，A4计算得到的待测包裹的体积之间的差值是否小于预设差值范围，若是可以选取任意一个体积计算值作为待测包裹的体积测量值，否则，需要重新对待测包裹进行视频采集计算。本发明通过边缘图像获取、直线检测、确定候选多边形以及对候选多边形验证，免除了繁琐的神经网络训练过程和滑动操作，直接对目标图像中的待测包裹的边缘直线进行检测，更为简单直观，检测效率得到提高。Since the edge of the package to be tested is relatively obvious and long, the edge line corresponding to the edge of the package to be tested can be detected. As shown in FIG. 5 and FIG. 6, the position of the intersection of any two edge lines in the edge line is calculated. The corner points A1, A2, A3, and A4 of the package to be tested may be obtained, and may also include other corner points A5, A6, and A7 of the package to be tested, but only A1, A2, and A3 are calculated based on the volume measurement efficiency of the package to be tested. , A4 can be. Of course, if the calculation of the volume of the package to be tested is correct, you can use A3, A5, A6, A7 to calculate again, and compare the difference between the volume of the package to be tested calculated by A1, A2, A3, and A4. If it is less than the preset difference range, if any volume calculation value can be selected as the volume measurement value of the package to be tested, otherwise, the measurement package needs to be re-tested for video acquisition calculation. The invention adopts edge image acquisition, line detection, determination of candidate polygons and verification of candidate polygons, eliminating the cumbersome neural network training process and sliding operation, and directly detecting the edge line of the package to be tested in the target image, which is simpler and more intuitive. The detection efficiency is improved.

本发明的第六个实施例，如图7和图8所示，是上述第一实施例、第二实施例、第三实施例或者第四实施例的优化实施例，包括A sixth embodiment of the present invention, as shown in FIG. 7 and FIG. 8, is an optimized embodiment of the first embodiment, the second embodiment, the third embodiment, or the fourth embodiment, including

所述S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积具体包括：The S400 calculates the world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the camera acquired in advance, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinate of the corner point. include:

具体的，所述步骤S410根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量具体包括步骤，如图8所示：Specifically, the step S410 is to calculate an outer parameter matrix of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera obtained in advance; the outer parameter matrix includes a rotation matrix and a translation vector, and specifically includes steps As shown in Figure 8:

其中，(u，v，1)表示所述平面标识物中所述目标点在摄像机的像素坐标系中像素坐标的齐次坐标，(Xw，Yw，Zw，1)表示所述目标点在所述世界坐标系中世界坐标的齐次坐标，(Xc，Yc，Zc)表示所述目标点在摄像机的相机坐标系中的相机坐标；fx表示摄像机透镜的物理焦距F与成像仪在X轴方向每个单元尺寸的乘积，fy表示摄像机透镜的物理焦距F与成像仪在y轴方向每个单元尺寸的乘积，cx表示成像仪中心与光轴在X轴方向上的偏移，cy表示由于成像仪成像仪中心与光轴在Y轴方向上的偏移；R表示世界坐标系到相机坐标系的旋转矩阵，T表示所述世界坐标系到所述相机坐标系的平移向量；Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. An offset of the center of the imager from the optical axis in the Y-axis direction; R represents a rotation matrix of the world coordinate system to the camera coordinate system, and T represents a translation vector of the world coordinate system to the camera coordinate system;

例如，取平面标识物S的中心O1为世界坐标系的原点，该世界坐标系也是平面标识物S的平面标识物坐标系，则在平面标识物S所在的平面上的点的Z轴坐标都为0。设平面标识物S的边长为P，则其四个顶点平面标识物S坐标系O1中的坐标分别为(-P/2，-P/2，0)，(P/2，-P/2，0)，(P/2，P/2，0)，(-P/2，P/2，0)。For example, taking the center O1 of the plane identifier S as the origin of the world coordinate system, and the world coordinate system is also the plane identifier coordinate system of the plane identifier S, then the Z-axis coordinates of the point on the plane where the plane identifier S is located Is 0. If the side length of the plane identifier S is P, the coordinates of the four vertex plane identifiers S coordinate system O1 are (-P/2, -P/2, 0), (P/2, -P/, respectively). 2,0), (P/2, P/2, 0), (-P/2, P/2, 0).

根据***模型，设空间中一点P在相机坐标系下的坐标为Pc＝(Xc，Yc，Zc)，则其与该点在摄像机成像平面上像素坐标(u，v)存在公式(2)所列的关系，设P在世界坐标系下坐标为Pw＝(Xw，Yw，Zw)，则P点在世界坐标系下坐标Pw与相机坐标系下坐标Pc＝(Xc，Yc，Zc)存在公式(3)所列的转换关系，其中，3x3的矩阵R＝(r1，r2，r3)，矩阵R为由世界坐标系到相机坐标系的旋转矩阵，T为由世界坐标系到相机坐标系的平移向量，将公式(2)与公式(3)结合在一起，即可得到点P在世界坐标系中的坐标Pw与对应的图像平面上的像素坐标(u，v)的转换方程(4)，由于选定平面标识物S的中心点为原点建立世界坐标系，则对于平面标识物S的四个顶点中的任一顶点(Xw，Yw，0)，都存在公式(4)对应的等式。由于四个顶点在世界坐标系下的Z轴坐标Zw＝0，且矩阵R＝(r1，r2，r3)，所以公式(4)经简化可得如下等式(6)：According to the pinhole camera model, if the coordinates of a point P in the camera coordinate system are Pc=(Xc, Yc, Zc), then there is a formula (2, pixel coordinates) (u, v) on the imaging plane of the camera. ) The relationship listed, let P be the world coordinate system Pw = (Xw, Yw, Zw), then P point in the world coordinate system coordinates Pw and camera coordinate system coordinates Pc = (Xc, Yc, Zc) There is a conversion relationship listed in equation (3), where 3x3 matrix R = (r1, r2, r3), matrix R is a rotation matrix from the world coordinate system to the camera coordinate system, and T is from the world coordinate system to the camera coordinates The translation vector of the system, combining the formula (2) with the formula (3), can obtain the conversion equation of the coordinate Pw of the point P in the world coordinate system and the pixel coordinates (u, v) on the corresponding image plane ( 4), since the center point of the selected plane identifier S is the origin to establish a world coordinate system, there is a formula (4) corresponding to any of the four vertices (Xw, Yw, 0) of the plane identifier S. Equation. Since the four vertices have a Z-axis coordinate Zw=0 in the world coordinate system, and the matrix R=(r1, r2, r3), the equation (4) is simplified to obtain the following equation (6):

以上通过平面标识物S的一个顶点像素坐标及其对应的平面标识物坐标系中坐标可写出一个方程。通过平面标识物S的四个顶点对应可写出四个方程组，然后通过方程组的直接线性变换(DLT)算法，从而求出r1、r2、r3与T。其中，旋转矩阵R是正交矩阵，其列向量r1、r2、r3是相互正交的单位向量，当求出r1、r2时，r3可由r1、r2的向量积得到，从而顶点像素坐标和顶点世界坐标均已知的情况下，将预先获取的摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)就能计算得到摄像头当前视场的旋转矩阵和平移向量。The above equation can be written by a vertex pixel coordinate of the plane identifier S and its corresponding coordinate in the plane identifier coordinate system. Four equations can be written by the four vertices of the plane identifier S, and then through the direct linear transformation (DLT) algorithm of the equations, r1, r2, r3 and T are obtained. Wherein, the rotation matrix R is an orthogonal matrix, and the column vectors r1, r2, and r3 are mutually orthogonal unit vectors. When r1 and r2 are obtained, r3 can be obtained from the vector product of r1 and r2, thereby vertex pixel coordinates and vertices. When the world coordinates are known, the internal parameter matrix of the camera obtained in advance is substituted into the following formula (2) and formula (3), and the rotation matrix and translation of the current field of view of the camera can be calculated by combining formula (4). vector.

具体的，所述步骤S420根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标具体包括步骤，如图8所示：Specifically, the step S420, according to the pixel coordinates of the corner point in the pixel coordinate system, acquiring the world coordinates of the corner point in combination with the rotation matrix and the translation vector, specifically includes the steps, as shown in FIG. 8 :

如图5和图6所示，已知角点A1在像素坐标系中的像素坐标(u1，v1)，则将其代入公式(4)中，由于摄像机的内参矩阵K已知，此外，通过S411-S413已经求得旋转矩阵R及平移向量T，则该矩阵方程组的未知量为X _w、Y _w、Z _c，通过解矩阵方程组即可得到角点A1在世界坐标系下的世界坐标(X _w1，Y _w1，0)，同理也可以根据角点A2，A3，A4在像素坐标系中的对应的像素坐标(ui，vi)，i∈N(i≧2)分别求取出待测包裹的各个角点的对应在世界坐标系上的世界坐标(X _wi，Y _wi，0)，i∈N(i≧2)。 As shown in FIG. 5 and FIG. 6, the pixel coordinates (u1, v1) of the corner point A1 in the pixel coordinate system are known, and then substituted into the formula (4), since the internal reference matrix K of the camera is known, in addition, S411-S413 has obtained the rotation matrix R and the translation vector T, then the unknowns of the matrix equations are X _w , Y _w , Z _c , and the world of the corner point A1 in the world coordinate system can be obtained by solving the matrix equations. The coordinates (X _w1 , Y _w1 , 0) can be similarly extracted according to the corresponding pixel coordinates (ui, vi) and i∈N(i≧2) of the corner points A2, A3, and A4 in the pixel coordinate system. The respective corner points of the package to be tested correspond to world coordinates (X _wi , Y _wi , 0), i ∈ N (i ≧ 2) on the world coordinate system.

具体的所述步骤S430根据所述角点的世界坐标计算所述待测包裹的长宽高具体包括步骤，如图8所示：Specifically, the step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the steps, as shown in FIG. 8 :

具体的，本实施例中首先读取任意四个角点对应的世界坐标，将读取的四个角点中任意一个角点作为第一角点，剩余的三个角点分别与第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴，第二角点与第一角点连接生成的直线平行于世界坐标系的X轴，第二角点与第一角点连接生成的直线平行于世界坐标系的Y轴，第四角点与第一角点连接生成的直线平行于世界坐标系的Z轴。由于已经通过步骤S421-S422计算得到各个角点的世界坐标，如图5所示，假设第一角点为A1，第二角点为A3，第三角点为A2，第四角点为A4，那么第一角点A1的世界坐标为(X _w1，Y _w1，0)，第二角点A3的世界坐标为(X _w3，Y _w3，0)，第三角点A2的世界坐标为(X _w2，Y _w2，0)，第四角点A4的世界坐标为(X _w4，Y _w4，0)， Specifically, in this embodiment, the world coordinates corresponding to any four corner points are first read, and any one of the four corner points read is used as the first corner point, and the remaining three corner points are respectively associated with the first corner. The straight lines generated by the two-two connection are respectively parallel to the XYZ axis of the world coordinate system, and the line generated by the connection of the second corner point and the first corner point is parallel to the X-axis of the world coordinate system, and the second corner point and the first corner point The line generated by the connection is parallel to the Y-axis of the world coordinate system, and the line formed by the connection of the fourth corner point and the first corner point is parallel to the Z-axis of the world coordinate system. Since the world coordinates of the respective corner points have been calculated through steps S421-S422, as shown in FIG. 5, it is assumed that the first corner point is A1, the second corner point is A3, the third corner point is A2, and the fourth corner point is A4. Then the world coordinates of the first corner point A1 are (X _w1 , Y _w1 , 0), the world coordinates of the second corner point A3 are (X _w3 , Y _w3 , 0), and the world coordinates of the third corner point A2 are (X _w2 , Y _w2 , 0), the world coordinate of the fourth corner A4 is (X _w4 , Y _w4 , 0),

通过如下三维空间中的两点距离公式就可求得待测包裹的长度L＝|A ₁A ₃|，以及待测包裹的宽度W＝|A ₁A ₂|。 The length of the package to be tested, L=|A ₁ A ₃ |, and the width of the package to be tested, W=|A ₁ A ₂ |, can be obtained by the two-point distance formula in the three-dimensional space as follows.

由于直线A ₁A ₄与世界坐标系Z轴方向平行，所以角点A4的Z轴坐标即为高度H，角点A4的X轴坐标和Y轴坐标与角点A1相同，即X _w4＝X _w1，Y _w4＝Y _w1。已知角点A4在图像中的像素坐标(u4，v4)，则将其代入公式(4)中，由于摄像机的内参矩阵已知，通过步骤S411-S413步骤已经求得旋转矩阵R及平移向量T，则该矩阵方程组的未知量为Z _w、Z _c。显然，只需将X _w4或者Y _w4代入一个值即可通过解矩阵方程组得到未知量。优选的，通过轮流两次分别代入Xw4及Yw4计算Zw，然后取二者的平均值作为高度H，提升待测包裹高度测量的精确度和可靠性。通过上面计算出来的长宽高，由于包裹是长方体或者立方体，因此可直接计算出包裹的体积V＝L×W×H。同样，也可以根据角点A5，A6，A7在像素坐标系中的对应的像素坐标(ui，vi)，i∈N(i≧2)分别计算得到角点A5，A6，A7在世界坐标系中的世界坐标，此处计算得到角点A5，A6，A7对应的世界坐标系后，根据角点A3，A5，A6，A7计算得到待测包裹的体积，将根据角点A3，A5，A6，A7计算得到的长宽高用于验证计算得到的待测包裹的体积是否可靠，再次不在一一赘述。 Since the straight line A ₁ A ₄ is parallel to the Z-axis direction of the world coordinate system, the Z-axis coordinate of the corner point A4 is the height H, and the X-axis coordinate and the Y-axis coordinate of the corner point A4 are the same as the corner point A1, that is, X _w4 =X _W1 , Y _w4 =Y _w1 . Knowing the pixel coordinates (u4, v4) of the corner point A4 in the image, it is substituted into the formula (4). Since the internal parameter matrix of the camera is known, the rotation matrix R and the translation vector have been obtained through the steps S411-S413. T, then the unknowns of the matrix equations are Z _w , Z _c . Obviously, simply substituting X _w4 or Y _w4 into a value yields an unknown by solving the matrix equations. Preferably, Zw is calculated by substituting Xw4 and Yw4 twice, and then the average value of the two is taken as the height H, thereby improving the accuracy and reliability of the measurement of the height of the package to be tested. Through the above calculation of the length, width and height, since the package is a rectangular parallelepiped or a cube, the volume of the package V = L × W × H can be directly calculated. Similarly, corner points A5, A6, and A7 can be calculated in the world coordinate system according to the corresponding pixel coordinates (ui, vi) and i∈N(i≧2) of the corner points A5, A6, and A7 in the pixel coordinate system. In the world coordinates, after calculating the world coordinate system corresponding to the corner points A5, A6, A7, the volume of the package to be tested is calculated according to the corner points A3, A5, A6, A7, according to the corner points A3, A5, A6 The length, width and height calculated by A7 are used to verify whether the calculated volume of the package to be tested is reliable, and will not be repeated again.

此外，由于角点A1和角点A4连接生成的直线与世界坐标系Z轴方向平行，所以角点A1与角点A4距离也能计算得到待测包裹的高度，即：In addition, since the line generated by the connection of the corner point A1 and the corner point A4 is parallel to the Z-axis direction of the world coordinate system, the distance between the corner point A1 and the corner point A4 can also calculate the height of the package to be tested, that is:

下面，举个实例进行说明，待测包裹的体积测量步骤如图9所示：Below, an example is given to illustrate the volume measurement step of the package to be tested as shown in FIG. 9:

S1、摄像机标定：S1, camera calibration:

摄像机标定是整个***的前期准备阶段，当选用一款摄像头进行视觉测量必须对其进行标定，从而获得摄像机的畸变系数及焦距等参数组成的内参矩阵。在后续的执行阶段，可以使用该摄像机的内参矩阵进行相应的视觉测量及计算，标定操作只需要执行一次。Camera calibration is the pre-preparation stage of the whole system. When a camera is selected for visual measurement, it must be calibrated to obtain the internal reference matrix composed of parameters such as distortion coefficient and focal length of the camera. In the subsequent execution phase, the camera's internal parameter matrix can be used for corresponding visual measurement and calculation, and the calibration operation only needs to be performed once.

①打印一张平面标定的模板并贴在一个平面上，示意图如图10a所示；1 print a plane-calibrated template and affix it to a plane, as shown in Figure 10a;

③检测出图像中的特征点；示意图如图10b所示；3 detecting feature points in the image; a schematic diagram is shown in Figure 10b;

S2、获取视频帧：S2, obtain a video frame:

将平面标识物S放置在快递测量包裹尺寸的待测包裹的上表面上，以平面标识物S的中心O1作为世界坐标系原点。由于平面标识物的厚度比较薄可以忽略不计，则包裹的上表面与世界坐标系O1的X-Y平面共面，待测包裹为长方体或者立方体，则包裹的高度H方向与世界坐标系O1的Z轴方向平行。The plane marker S is placed on the upper surface of the package to be tested of the express measurement package size, and the center O1 of the plane marker S is taken as the origin of the world coordinate system. Since the thickness of the planar marker is negligible, the upper surface of the package is coplanar with the XY plane of the world coordinate system O1, and the package to be tested is a rectangular parallelepiped or a cube, and the height H direction of the package and the Z axis of the world coordinate system O1. The directions are parallel.

如图11所示，快递员使用自身携带的手持终端(或手机)的摄像头处于待测包裹的斜上方，保证能够看到待测包裹的上表面(包含平面标识物)及任意两个与上表面相邻的两个侧表面总共三个表面。As shown in Figure 11, the courier uses the camera of the handheld terminal (or mobile phone) carried by himself to be diagonally above the package to be tested, ensuring that the upper surface of the package to be tested (including the planar marker) and any two and The two side surfaces adjacent to the surface have a total of three surfaces.

S3、四边形检测：S3, quadrilateral detection:

平面标识物识别：该步骤用于在图像中识别特定的平面标识物，并确定平面标识物的四个角点，具体过程如下：利用自适应性阈值对摄像头获取的图像进行二值分割，然后从二值图像中提取外形类似平面标识物的四边形轮廓，通过四边形轮廓四个顶点，将该图像经过射影变换后转换为正方形的正视图。通过分析正视图图像中的编码，最终确定该四边形图像对应的平面标识物。Plane marker identification: This step is used to identify a specific plane identifier in the image and determine the four corner points of the plane identifier. The specific process is as follows: the image acquired by the camera is binary-divided by the adaptive threshold, and then A quadrilateral contour resembling a planar marker is extracted from the binary image, and the image is transformed into a square front view by projective transformation through four vertices of the quadrilateral contour. By analyzing the encoding in the front view image, the planar identifier corresponding to the quadrilateral image is finally determined.

①四边形轮廓检测1 quadrilateral contour detection

该过程检测出图像中平面标识物可能对应的四边形轮廓。利用自适应性阈值法对图像进行二值分割，然后从以上获取的二值图像中提取外形轮廓。对轮廓进行多边形拟合，舍弃那些非凸多边形的，以及那些不是四边形的轮廓。使用了一些额外的限制条件来剔除那些不可能是平面标识物图像的四边形轮廓，如四边形一边明显小于其余的边(形状比较瘦长)，轮廓周长或面积过小等。剩下的符合条件的轮廓即为平面标识物可能对应的的备选轮廓。The process detects a quadrilateral contour that the planar marker in the image may correspond to. The image is binarized by the adaptive threshold method, and then the outline is extracted from the binary image obtained above. Polygon fitting the contours, discarding those non-convex polygons, and those that are not quadrilateral. Some additional constraints are used to eliminate quadrilateral contours that are not likely to be planar marker images, such as the quadrilateral side being significantly smaller than the remaining edges (the shape is relatively elongated), the contour perimeter or area being too small. The remaining eligible contours are the alternative contours that the planar marker may correspond to.

②获取四边形轮廓区域的正视图2Get a front view of the quadrilateral outline area

该过程用于将平面标识物可能对应的四边形图像区域变为正方形的正视图，从而进一步判断该四边形区域是否为平面标识物对应的图像。定义四边形轮廓经过摄影变换转换为正视图后的四个顶点像素坐标分别为(0，0)，(0，100)，(100，100)，(100，0)。在原图像中四边形轮廓的四个顶点像素坐标已知，对于每一组点对应满足射影变换矩阵M：The process is used to change the quadrilateral image area corresponding to the planar identifier into a square front view, thereby further determining whether the quadrilateral area is an image corresponding to the planar identifier. The four vertex pixel coordinates after the quadrilateral contour is converted into a front view by photographic transformation are (0, 0), (0, 100), (100, 100), (100, 0). The coordinates of the four vertex pixels of the quadrilateral contour are known in the original image, and the projective transformation matrix M is satisfied for each set of points:

X为射影变换后顶点的齐次坐标，其非齐次像素坐标为

X is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

通过四组顶点对应关系，可联立方程组求解出射影变换矩阵M，对原图像的所有像素都执行射影变换M，即可得到该四边形轮廓区域对应的正视图，该正视图为正方形的平面标识物图像，即正视图图像大小为100×100。Through the four sets of vertex correspondences, the projective transformation matrix M can be solved by the simultaneous equations, and the projective transformation M can be performed on all the pixels of the original image to obtain a front view corresponding to the quadrilateral contour region, and the front view is a square plane. The marker image, that is, the front view image size is 100×100.

因为摄像头从各个角度拍摄出的平面标识物的图像是不同的，如果摄像头处在标识物的正上方，则标识物的图像也是一个矩形；如果摄像头斜着看，那么矩形平面标识物的图像是一个不规则四边形。本步骤中获取正视图的目的就是：不管摄像头从哪个角度看，得到的正视图都是一种从正上方往下看的图像。便于后续的平面标识物的识别，即图像的归一化。Because the image of the planar marker photographed by the camera from various angles is different, if the camera is directly above the marker, the image of the marker is also a rectangle; if the camera is obliquely viewed, the image of the rectangular planar marker is An irregular quadrilateral. The purpose of obtaining a front view in this step is that no matter which angle the camera looks from, the resulting front view is an image viewed from directly above. It facilitates the identification of subsequent planar markers, that is, the normalization of images.

S4、平面标识物识别：S4, flat marker identification:

如图12所示，将获取的平面标识物的正视图图像依次旋转90°，180°，270°从而最终得到四幅图像。将以上获取的四幅图像分别与平面标识物的图像模板进行匹配。当与任一副图像匹配成功时，该四边形轮廓即对应的是该平面标识物。As shown in FIG. 12, the front view image of the acquired planar marker is sequentially rotated by 90°, 180°, and 270° to finally obtain four images. The four images obtained above are respectively matched with the image templates of the planar markers. When the matching with any of the sub-images is successful, the quadrilateral contour corresponds to the planar identifier.

S5：计算外参矩阵(旋转矩阵R和平移向量T)S5: Calculate the outer parameter matrix (rotation matrix R and translation vector T)

计算以平面标识物S的中点为原点的世界坐标系O1相对于相机坐标系O2的旋转矩阵R1及平移向量T1：Calculating the rotation matrix R1 and the translation vector T1 of the world coordinate system O1 with the midpoint of the planar identifier S relative to the camera coordinate system O2:

取平面标识物S的中心O1为世界坐标系的原点，则在平面标识物S所在的平面上的点Z轴坐标都为0。设平面标识物S的边长为80mm，则其四个顶点平面标识物S坐标系O1中的坐标分别为(-40，-40，0)，(40，-40，0)，(40，40，0)，(-40，40，0)。Taking the center O1 of the plane identifier S as the origin of the world coordinate system, the point Z-axis coordinate on the plane where the plane identifier S is located is 0. If the side length of the plane identifier S is 80 mm, the coordinates of the four vertex plane identifiers S coordinate system O1 are (-40, -40, 0), (40, -40, 0), (40, 40,0), (-40,40,0).

根据***模型，设空间中一点P在摄像机坐标系下的坐标为Pc＝(Xc，Yc，Zc)，则其与该点在摄像机成像平面上像素坐标(u，v)存在以下关系：According to the pinhole camera model, if the coordinates of a point P in the camera coordinate system are Pc=(Xc, Yc, Zc), the following relationship exists with the pixel coordinates (u, v) of the point on the imaging plane of the camera:

设P在世界坐标系下坐标为Pw＝(Xw，Yw，Zw)，则P点在世界坐标系下坐标Pw与摄像机坐标系下坐标Pc＝(Xc，Yc，Zc)存在以下转换关系：Let P behave in the world coordinate system as Pw=(Xw, Yw, Zw), then the P point has the following conversion relationship between the coordinate Pw in the world coordinate system and the coordinate Pc=(Xc, Yc, Zc) in the camera coordinate system:

将公式(2)与公式(3)结合在一起，即可得到点P在世界坐标系中的坐标Pw与对应的图像平面上的像素坐标(u，v)的关系方程：Combining formula (2) with formula (3), we can get the relationship between the coordinates Pw of point P in the world coordinate system and the pixel coordinates (u, v) on the corresponding image plane:

将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)得到所述摄像头当前视场的旋转矩阵和平移向量。Substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotate matrix and translation vector.

选定平面标识物S的坐标系O1为世界坐标系Ow，则对于平面标识物S的四个顶点中的任一顶点(Xw，Yw，0)，都存在公式(4)的等式。由于四个顶点在世界坐标系下的Z轴坐标Zw＝0，所以公式(4)经简化可得如下等式(6)：When the coordinate system O1 of the selected plane identifier S is the world coordinate system Ow, the equation of the formula (4) exists for any of the four vertices (Xw, Yw, 0) of the plane identifier S. Since the Z-axis coordinates Zw=0 of the four vertices in the world coordinate system, the formula (4) is simplified to obtain the following equation (6):

以上通过平面标识物S的一个顶点在像素坐标系中顶点像素坐标，及其该顶点在世界坐标系中的顶点世界坐标可写出一个方程，那么平面标识物S的四个顶点对应可写出四个方程组，通过方程组的直接线性变换(DLT)算法，从而求出r1、r2、r3与T。其中，旋转矩阵R＝[r1 r2 r3]是正交矩阵，其列向量r1、r2、r3是相互正交的单位向量，当求出r1、r2时，r3可由r1、r2的向量积得到。The above can be written by the vertex pixel coordinates of a vertex of the planar identifier S in the pixel coordinate system and the vertex world coordinates of the vertex in the world coordinate system, and the four vertices of the planar identifier S can be written out. Four equations, through the direct linear transformation (DLT) algorithm of the equations, find r1, r2, r3 and T. The rotation matrix R=[r1 r2 r3] is an orthogonal matrix, and the column vectors r1, r2, and r3 are mutually orthogonal unit vectors. When r1 and r2 are obtained, r3 can be obtained from the vector product of r1 and r2.

S6、待测包裹的角点检测：S6, corner detection of the package to be tested:

在目标图像中找到待测包裹的各个角点，读取各个角点对应的像素坐标，后续使用每个角点对应的像素坐标进行长、宽、高测量。Find the corner points of the package to be tested in the target image, read the pixel coordinates corresponding to each corner point, and then use the pixel coordinates corresponding to each corner point to measure the length, width and height.

①在图像上使用Canny边缘检测算法，检测图像中明显的边缘。由于待测包裹的边缘与背景对比明显，所以通过边缘检测算法，可以突出这些明显的物体边缘。该过程会得到包含图像中包裹边缘的二值图像。1 Use the Canny edge detection algorithm on the image to detect significant edges in the image. Since the edge of the package to be tested is clearly contrasted with the background, these distinct object edges can be highlighted by the edge detection algorithm. This process will result in a binary image containing the edges of the wrap in the image.

②使用霍夫直线检测算法在边缘二值图像中寻找边缘直线。霍夫直线检测算法基于明显的及长的边缘寻找边缘直线，二值图像中细小或者杂乱的边缘会自动被过滤掉。由于待测包裹的边缘比较明显，也比较长，所以能够检测出待测包裹的边缘对应的边缘直线。2 Use the Hough line detection algorithm to find the edge line in the edge binary image. The Hough line detection algorithm looks for edge lines based on distinct and long edges, and small or cluttered edges in the binary image are automatically filtered out. Since the edge of the package to be tested is relatively obvious and relatively long, the edge line corresponding to the edge of the package to be tested can be detected.

③如图5和图6所示，计算待测包裹边缘对应的各边缘直线的交点的位置，得到角点A1，A2，A3，A4，该角点即为包裹对应的角点。3 As shown in FIG. 5 and FIG. 6, the position of the intersection point of each edge line corresponding to the edge of the package to be tested is calculated, and the corner points A1, A2, A3, and A4 are obtained, and the corner point is the corner point corresponding to the package.

S7、计算长度和宽度：如图5和图6所示，待测包裹的长L对应的是待测包裹的上表面上的角点A1，A3对应的距离，待测包裹的宽W对应的是其待测包裹的上表面的角点A1，A2对应的距离。S7. Calculating the length and width: As shown in FIG. 5 and FIG. 6, the length L of the package to be tested corresponds to the corner point A1 on the upper surface of the package to be tested, and the distance corresponding to A3, and the width W of the package to be tested corresponds to It is the distance corresponding to the corner points A1 and A2 of the upper surface of the package to be tested.

已知角点A1在图像中的像素坐标(u1，v1)，则将其代入公式(5)中。由于摄像机的内参矩阵K已知，以上步骤S411-S413已经求得旋转矩阵R及平移向量T，则该矩阵方程组的未知量为X _w、Y _w、Z _c，通过解矩阵方程组即可得到角点A1在世界坐标系下的坐标(X _w1，Y _w1，0)。 Knowing the pixel coordinates (u1, v1) of the corner point A1 in the image, it is substituted into the formula (5). Since the internal reference of K matrix camera is known, the above steps S411-S413 have been calculated rotation matrix R and translation vector T, the unknowns of the matrix equation _{_{_{X w, Y w, Z c}}} , can be set by solving the matrix equation The coordinates (X _w1 , Y _w1 , 0) of the corner point A1 in the world coordinate system are obtained.

同理，可以求得角点A2及A3在世界坐标系下的坐标(X _w2，Y _w2，0)，(X _w3，Y _w3，0)。通过如下三维空间中两点距离公式即可求得长L＝|A ₁A ₃|及宽W＝|A ₁A ₂|。 Similarly, the coordinates (X _w2 , Y _w2 , 0), (X _w3 , Y _w3 , 0) of the corner points A2 and A3 in the world coordinate system can be obtained. The length L=|A ₁ A ₃ | and the width W=|A ₁ A ₂ | can be obtained by the two-point distance formula in the three-dimensional space as follows.

S8、计算高度：如图5和图6所示，已知高度H＝|A ₁A ₄|与世界坐标系Z轴方向平行，所以角点A4的Z轴坐标即为高度H，角点A4的X轴坐标和Y轴坐标与角点A1相同，即X _w4＝X _w1，Y _w4＝Y _w1。 S8. Calculating the height: As shown in FIG. 5 and FIG. 6, the known height H=|A ₁ A ₄ | is parallel to the Z-axis direction of the world coordinate system, so the Z-axis coordinate of the corner point A4 is the height H, and the corner point A4 The X-axis coordinate and the Y-axis coordinate are the same as the corner point A1, that is, X _w4 = X _w1 , Y _w4 = Y _w1 .

已知角点A4在图像中的像素坐标(u4，v4)，则将其代入公式(4)中，由于摄像机的内参矩阵已知，以上步骤已经求得旋转矩阵R及平移向量T，则该矩阵方程组的未知量为Z _w、Z _c。显然，只需将X _w4或者Y _w4代入一个值即可通过解矩阵方程组得到未知量。优选的，通过轮流两次分别代入Xw4及Yw4计算Zw，然后取二者的平均值作为高度H，提升待测包裹高度测量的精确度和可靠性。 Knowing the pixel coordinates (u4, v4) of the corner point A4 in the image, it is substituted into the formula (4). Since the internal parameter matrix of the camera is known, the rotation matrix R and the translation vector T have been obtained in the above steps. The unknowns of the matrix equations are Z _w , Z _c . Obviously, simply substituting X _w4 or Y _w4 into a value yields an unknown by solving the matrix equations. Preferably, Zw is calculated by substituting Xw4 and Yw4 twice, and then the average value of the two is taken as the height H, thereby improving the accuracy and reliability of the measurement of the height of the package to be tested.

S9、计算体积：通过上面计算出来的长宽高，由于包裹是长方体或者立方体，因此可直接计算出包裹的体积V＝L×W×H。S9. Calculating the volume: The length, width and height calculated by the above, since the package is a rectangular parallelepiped or a cube, the volume of the package V=L×W×H can be directly calculated.

S10、判断是否退出；若否，返回步骤S2。S10. Determine whether to exit; if not, return to step S2.

本发明利用该摄像头在复杂的场景下实时准确的计算出待测包裹的长、宽、高尺寸，在室外强光下也能够正常工作，在保证精度的前提下计算出包裹的尺寸信息。由于本发明利用了手持终端设备(或者手机)上自带的摄像头，不需要额外安装深度摄像头等等更加昂贵的设备，节省了成本。此外快递员手持终端设备(或者手机)上本来就集成有后置摄像头，利用移动终端自带的摄像头在在室外强光下也能够正常工作，从而能够在保证精度的前提下实时的计算出待测包裹的长、宽、高尺寸，从而方便快捷的计算出待测包裹的尺寸信息。由于本发明利用了手持终端设备(或者手机)上摄像头，不需要额外增加昂贵的设备，节省了成本。此外，用户也可以使用储存用于测量包裹体积测量方法的指令的移动终端进行验证快递员估算的待测包裹的体积，从而了解自身包裹尺寸信息，避免快递员将待测包裹的体积多报而产生的收费，减少不必要的邮寄费用。The invention utilizes the camera to accurately calculate the length, width and height of the package to be tested in real time in a complicated scene, and can work normally under outdoor strong light, and calculates the size information of the package under the premise of ensuring accuracy. Since the present invention utilizes a camera attached to the handheld terminal device (or mobile phone), it does not require an additional expensive device such as a depth camera, and the cost is saved. In addition, the courier handheld terminal device (or mobile phone) is integrated with a rear camera, and the camera that is provided by the mobile terminal can work normally under the outdoor strong light, so that the real-time calculation can be performed under the premise of ensuring accuracy. The length, width and height of the package are measured, so that the size information of the package to be tested is conveniently and quickly calculated. Since the present invention utilizes a camera on a handheld terminal device (or a mobile phone), there is no need to add extra expensive equipment, which saves cost. In addition, the user can also use the mobile terminal storing the instruction for measuring the package volume measurement method to verify the volume of the package to be tested estimated by the courier, thereby knowing the size information of the package and avoiding the courier overreporting the volume of the package to be tested. Charges generated, reducing unnecessary mailing costs.

本发明的第七个实施例，如图13所示，一种包裹体积的测量***，应用于测量设有平面标识物的待测包裹的体积，包括：According to a seventh embodiment of the present invention, as shown in FIG. 13, a package volume measuring system is applied to measure the volume of a package to be tested provided with a planar marker, including:

图像获取模块1000，用于通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面和与上表面相邻的所述待测包裹的两个侧表面；The image acquisition module 1000 is configured to collect a target image by using a camera of the mobile terminal, where the target image includes the planar identifier, an upper surface of the package to be tested provided with the planar identifier, and the adjacent to the upper surface The two side surfaces of the package to be tested;

图像识别模块2000，用于对所述图像获取模块1000获取的所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；The image recognition module 2000 is configured to perform image processing on the target image acquired by the image acquisition module 1000, thereby identifying the planar identifier in the target image;

图像处理模块3000，用于对所述图像获取模块1000获取的所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；The image processing module 3000 is configured to perform image edge detection on the target image acquired by the image acquiring module 1000, so as to identify a corner point of the package to be tested in the target image;

体积测量模块4000，用于根据所述图像识别模块2000识别出的所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。a volume measurement module 4000, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module 2000, an internal parameter matrix of the camera acquired in advance, and the corner point, thereby The world coordinates of the point get the volume of the package to be tested.

具体的，本实施例是基于同一发明构思的***实施例，由于包裹体积的测量***解决问题的原理与包裹体积的测量方法相似，因此***的实施可以参见包裹体积的测量方法的实施，重复之处不再赘述。Specifically, the embodiment is a system embodiment based on the same inventive concept. Since the principle of solving the problem of the package volume measurement system is similar to the measurement method of the package volume, the implementation of the system can be referred to the implementation of the measurement method of the package volume, and the repetition is performed. I won't go into details here.

本发明的第八个实施例，是上述第七实施例的优化实施例，一种包裹体积的测量***，应用于测量设有平面标识物的待测包裹的体积，优选的，包裹体积的测量***还包括：An eighth embodiment of the present invention is an optimized embodiment of the seventh embodiment, a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar marker, preferably, a measurement of a package volume The system also includes:

标定模块5000，用于在所述图像获取模块通过所述摄像头采集目标图像之前，对所述摄像头进行标定，获取所述摄像头的内参矩阵。The calibration module 5000 is configured to calibrate the camera before the image acquisition module acquires the target image through the camera, and acquire an internal reference matrix of the camera.

本发明的第九个实施例，如图14所示，是上述第七或第八实施例的优化实施例，一种包裹体积的测量***，应用于测量设有平面标识物的待测包裹的体积，主要改进在于，所述图像识别模块2000包括：A ninth embodiment of the present invention, as shown in FIG. 14, is an optimized embodiment of the seventh or eighth embodiment, a measurement system for a package volume, which is used for measuring a package to be tested provided with a planar marker. Volume, the main improvement is that the image recognition module 2000 includes:

二值分割子模块2100，用于对所述提取模块获取的所述目标图像进行二值分割；a binary segmentation sub-module 2100, configured to perform binary segmentation on the target image acquired by the extraction module;

外形提取子模块2200，用于从分割后的二值化图像中提取外形轮廓；a shape extraction sub-module 2200, configured to extract a contour from the divided binarized image;

存储子模块2300，用于存储所述平面标识物所对应的外形轮廓的共性特征、所述平面标识物模板；a storage sub-module 2300, configured to store a common feature of the outline of the plane identifier, and the plane identifier template;

判断处理子模块2400，用于根据所述存储子模块2300中存储的所述平面标识物所对应的外形轮廓的共性特征，从所述外形提取子模块2200提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；a determination processing sub-module 2400, configured to select, according to the common features of the contours corresponding to the planar identifiers stored in the storage sub-module 2300, from the contours extracted by the shape extraction sub-module 2200 The contour of the feature is used as an alternative contour;

正视图获取子模块2500，用于获取所述备选轮廓的正视图；a front view obtaining sub-module 2500, configured to obtain a front view of the candidate contour;

识别子模块2600，用于当所述备选轮廓的正视图与所述存储子模块2300预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像。The identification sub-module 2600 is configured to: when the front view of the candidate profile is consistent with the planar identifier template pre-stored by the storage sub-module 2300, identify an image corresponding to the candidate profile as the planar identifier image.

本发明的第十个实施例，如图15所示，是上述第九实施例的优化实施例，一种包裹体积的测量***，应用于测量设有平面标识物的待测包裹的体积，主要改进在于，所述平面标识物为正方形，所述储存子模块储存的所述平面标识物所对应的外形轮廓的共性特征为四边形；A tenth embodiment of the present invention, as shown in FIG. 15, is an optimized embodiment of the ninth embodiment, a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar marker, mainly The improvement is that the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle;

所述正视图获取子模块2500包括：The front view obtaining submodule 2500 includes:

读取单元2510，用于读取所述备选轮廓的四个顶点的像素坐标；a reading unit 2510, configured to read pixel coordinates of four vertices of the candidate contour;

定义单元2520，用于定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；a defining unit 2520, configured to define pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as pixel coordinates of four pre-existing front views of the planar identifier;

计算单元2530，用于将所述读取单元2510读取的四个顶点的像素坐标，以及所述定义单元2520定义的四个顶点的像素坐标分别代入下列平面射影变换公式(1)，求取射影变换矩阵M：The calculating unit 2530 is configured to substitute the pixel coordinates of the four vertices read by the reading unit 2510 and the pixel coordinates of the four vertices defined by the defining unit 2520 into the following planar projective transformation formula (1), and obtain the same Projective transformation matrix M:

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

变换单元2540，用于根据所述计算单元2530求取得到的所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。a transforming unit 2540, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit 2530, to obtain the candidate contour inclusion The area corresponds to the front view.

本发明的第十一个实施例，如图16所示，是上述第七、第八或第十实施例的优化实施例，一种包裹体积的测量***，主要改进在于，所述图像处理模块3000包括：An eleventh embodiment of the present invention, as shown in FIG. 16, is an optimized embodiment of the seventh, eighth or tenth embodiment, a measurement system for a package volume, the main improvement being that the image processing module 3000 includes:

边缘检测子模块3100，用于对所述提取模块获取的所述目标图像进行边缘检测处理，得到边缘二值图像；The edge detection sub-module 3100 is configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image;

直线查找子模块3200，用于从所述边缘检测子模块3100得到的所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；a line search sub-module 3200, configured to search for an edge line corresponding to the edge of the package to be tested from the edge binary image obtained by the edge detection sub-module 3100;

角点获取子模块3300，用于计算所述直线查找子模块3200查找到的所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。The corner point acquisition sub-module 3300 is configured to calculate a position of an intersection of any two edge lines of the edge line found by the line search sub-module 3200, to obtain a corner point of the package to be tested.

本发明的第十二个实施例，如图17和图18所示，一种包裹体积的测量***，是上述第七实施例至第十一实施例中任意一个实施例的优化实施例，主要改进在于，所述体积测量模块4000包括：According to a twelfth embodiment of the present invention, as shown in FIG. 17 and FIG. 18, a measurement system for a package volume is an optimized embodiment of any one of the seventh embodiment to the eleventh embodiment, mainly The improvement is that the volume measuring module 4000 comprises:

外参矩阵获取子模块4100，用于根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量；An outer parameter matrix obtaining sub-module 4100, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera acquired in advance; the outer parameter matrix includes a rotation matrix and a translation vector ;

角点坐标获取子模块4200，用于根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标；a corner coordinate acquisition sub-module 4200, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in the pixel coordinate system, in combination with the rotation matrix and the translation vector;

长宽高获取子模块4300，用于根据所述角点的世界坐标计算所述待测包裹的长宽高；a length and width acquisition sub-module 4300, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

包裹体积获取子模块4400，用于将所述长宽高代入体积公式计算获取所述待测包裹的体积。The package volume acquisition sub-module 4400 is configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.

如图18所示，所述外参矩阵获取子模块4100包括：As shown in FIG. 18, the outer parameter matrix obtaining submodule 4100 includes:

坐标系确定单元4110，用于以所述平面标识物的中点为原点建立世界坐标系；a coordinate system determining unit 4110, configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

顶点坐标读取单元4120，用于读取所述平面标识物的各个顶点在像素坐标系中的顶点像素坐标，以及各个顶点在世界坐标系中的顶点世界坐标；a vertex coordinate reading unit 4120, configured to read vertex pixel coordinates of each vertex of the plane identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

运算单元4130，用于将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式 (4)得到所述摄像头当前视场的旋转矩阵和平移向量：The operation unit 4130 is configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combine the formula (4) to obtain the The rotation matrix and translation vector of the current field of view of the camera:

如图18示，所述角点坐标获取子模块4200包括：As shown in FIG. 18, the corner coordinate acquisition sub-module 4200 includes:

角点像素坐标获取单元4210，用于读取所述角点在摄像机的像素坐标系中的像素坐标；a corner pixel coordinate acquiring unit 4210, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

角点世界坐标获取单元4220，用于将所述角点的像素坐标、所述预先获取的摄像头的内参矩阵以及所述摄像头当前视场的旋转矩阵和平移向量代入所述公式(4)中得到所述角点的世界坐标；其中，(u，v，1)表示所述角点的像素坐标的齐次坐标，(Xw，Yw，Zw)是所述角点的世界坐标，(Xw，Yw，Zw，1)表示所述角点的世界坐标的齐次坐标；a corner point world coordinate obtaining unit 4220, configured to substitute the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4). The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw) , Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

如图18所示，所述长宽高获取子模块4300包括：As shown in FIG. 18, the length, width, and height acquisition submodule 4300 includes:

角点世界坐标读取单元4310，用于读取任意四个角点对应的世界坐标；所述四个角点中任意一个角点为第一角点，且剩余三个角点分别与所述第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴；a corner point world coordinate reading unit 4310, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively associated with the The straight lines generated by the two corners of the first corner point are respectively parallel to the XYZ axis of the world coordinate system;

长度运算单元4320，用于将所述第一角点对应的世界坐标，以及第二角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的长度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的X轴；a length operation unit 4320, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second angle a line formed by connecting the point to the first corner point is parallel to an X axis of the world coordinate system;

宽度运算单元4330，用于将所述第一角点对应的世界坐标，以及第三角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的宽度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的Y轴；a width operation unit 4330, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the Y axis of the world coordinate system;

高度运算单元4340，用于将第四角点对应的世界坐标以及角点的像素坐标，结合所述预先获取的摄像头的内参矩阵、所述摄像头当前视场的旋转矩阵和平移向量，代入所述公式(4)计算获得所述待测包裹的高度；所述第四角点与所述第一角点连接生成的直线平行于所述世界坐标系的Z轴。a height operation unit 4340, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector. Formula (4) calculates the height of the package to be tested; the line generated by the connection of the fourth corner point with the first corner point is parallel to the Z axis of the world coordinate system.

本发明第十三个实施例，一种存储介质，所述存储介质存储有多条指令，所述多条指令被一个或者多个处理器执行，以实现下述步骤：A thirteenth embodiment of the present invention is a storage medium storing a plurality of instructions, the plurality of instructions being executed by one or more processors to implement the following steps:

S100通过移动终端的摄像头采集目标视频，并从所述目标视频中获取目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的目标待测包裹表面和与所述目标待测包裹表面相邻的所述待测包裹的若干个表面；S100 collects a target video through a camera of the mobile terminal, and acquires a target image from the target video; the target image includes the plane identifier, a target to-be-measured package surface provided with the planar identifier, and the target a plurality of surfaces of the package to be tested adjacent to the surface of the package to be tested;

S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；S200 performs image processing on the target image to identify the planar identifier in the target image;

S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹在所述目标图像上的交点；S300 performing image edge detection on the target image, so that an intersection of the package to be tested on the target image is identified in the target image;

S400根据所述平面标识物、预先获取的内参矩阵和所述交点，计算所述待测包裹上的角点世界坐标，从而根据所述角点世界坐标得到所述待测包裹的体积。S400 calculates a corner point world coordinate on the package to be tested according to the plane identifier, the pre-acquired internal reference matrix, and the intersection point, so as to obtain the volume of the package to be tested according to the corner point world coordinate.

较佳的，在上述基础上，本发明存储介质的另一实施例，所述存储介质存储有多条指令，所述多条指令被一个或者多个处理器执行，以实现任一本发明所述的包裹体积的测量方法实施例的步骤。Preferably, on the basis of the above, another embodiment of the storage medium of the present invention, the storage medium stores a plurality of instructions, and the plurality of instructions are executed by one or more processors to implement any of the present inventions. The steps of the embodiment of the method of measuring the package volume.

本发明所述的包裹体积的测量方法的步骤可参见前面的方法实施例部分，为减少重复，此处不再赘叙。The steps of the method for measuring the package volume according to the present invention can be referred to the previous method embodiment part, and the repetition is not repeated here.

本发明第十四个实施例，一种移动终端，包括：处理器，实现各指令；存储介质，存储多条指令；其中：所述处理器执行所述存储介质存储的指令，以实现包裹体积的测量方法的步骤。According to a fourteenth embodiment of the present invention, a mobile terminal includes: a processor that implements each instruction; a storage medium that stores a plurality of instructions; wherein: the processor executes an instruction stored by the storage medium to implement a package volume The steps of the measurement method.

如图19所示，移动终端包括：存储器1001、一个或多个(图中仅示出一个)处理器1002、摄像头1003这些组件通过一条或多条通讯总线信号线相互通讯。As shown in FIG. 19, the mobile terminal includes: a memory 1001, one or more (only one shown) processor 1002, and a camera 1003. These components communicate with each other through one or more communication bus signal lines.

可以理解，图16所示的结构仅仅为示意，并不是对移动终端的结构造成限定，移动终端还可以包括比图16所示更多或者更少的组件，或者具有与图16所示不同的装置。图16所示的各组件可以采用硬件、软件或其组合实现。It can be understood that the structure shown in FIG. 16 is merely illustrative and does not limit the structure of the mobile terminal. The mobile terminal may further include more or less components than those shown in FIG. 16, or have different functions from those shown in FIG. Device. The components shown in FIG. 16 can be implemented in hardware, software, or a combination thereof.

存储器可用于存储软件程序以及模块，如本发明实施例中的包裹体积的测量方法及***实施例对应的程序指令/模块，处理器通过允许存储在存储器内的软件程序/模块，从而执行各种功能应用以及数据处理，即实现上述的包裹体积的测量方法/***。The memory can be used to store software programs and modules, such as the method for measuring the package volume and the program instructions/modules corresponding to the system embodiment in the embodiment of the present invention, and the processor executes various software programs/modules by allowing the software programs/modules stored in the memory. Functional application and data processing, that is, a method/system for measuring the above-described package volume.

存储器可包括高速随机存储器，还可包括非易失性存储器，比如一个或多个磁性存储装置、闪存或者其他非易失性固态存储器。所述的存储介质可为磁碟、光盘、只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)等。The memory can include high speed random access memory and can also include non-volatile memory such as one or more magnetic storage devices, flash memory or other non-volatile solid state memory. The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM), a random access memory (RAM), or the like.

处理器运行存储器内的各种软件、指令一致性移动终端的各种功能以及进行数据处理。摄像头用于拍摄视频，相当于是移动终端的眼睛，比如采用CCD摄像头等。The processor runs various software within the memory, commands various functions of the mobile terminal, and performs data processing. The camera is used to capture video, which is equivalent to the eyes of a mobile terminal, such as a CCD camera.

本发明实施例中，包裹体积的测量方法及***、存储介质与移动终端属于同一构思，在移动终端上，通过处理器执行存储介质中存储的指令，可以运行相应的包裹体积的测量方法实施例中提供的方法，其具体实现过程详见前面的包裹体积的测量方法实施例，此处不再赘述。In the embodiment of the present invention, the method and system for measuring the package volume, the storage medium and the mobile terminal belong to the same concept. On the mobile terminal, the method for measuring the corresponding package volume can be executed by executing the instruction stored in the storage medium by the processor. For the specific implementation process of the method provided in the above, please refer to the previous embodiment of the measurement method of the package volume, and details are not described herein again.

需要说明的是，本领域普通测试人员可以理解实现本发明实施例包裹体积的测量方法的全部或部分流程，是可以通过计算机程序来控制相关的硬件来完成，计算机程序可存储于一计算机可读取存储介质中，如存储在移动终端的存储器中，并被该移动终端内的至少一个处理器执行，在执行过程中可包括如包裹体积的测量方法实施例的流程。It should be noted that a general tester in the art can understand all or part of the process of implementing the measurement method of the package volume of the embodiment of the present invention, which can be completed by a computer program to control related hardware, and the computer program can be stored in a computer readable state. The storage medium is stored, for example, in a memory of the mobile terminal and executed by at least one processor in the mobile terminal, and may include a flow of a measurement method embodiment such as a package volume during execution.

应当说明的是，上述实施例均可根据需要自由组合。以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Claims

一种包裹体积的测量方法，其特征在于，应用于测量设有平面标识物的待测包裹的体积，包括步骤：A method for measuring a package volume, which is characterized by being used for measuring a volume of a package to be tested provided with a planar marker, comprising the steps of:

S100通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面、与所述上表面相邻的所述待测包裹的两个侧表面；S100 collects a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the plane identifier, and the package to be tested adjacent to the upper surface Two side surfaces;

S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；S200 performing image processing on the target image, thereby identifying the planar identifier in the target image;

S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image;

S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinates of the corner point.
根据权利要求1所述的包裹体积的测量方法，其特征在于，所述步骤S100通过移动终端的摄像头采集目标图像之前包括步骤：The method for measuring a package volume according to claim 1, wherein the step S100 comprises the steps of: before the target image is captured by the camera of the mobile terminal:

S010对所述摄像头进行标定，获取所述摄像头的内参矩阵。S010 calibrates the camera to obtain an internal reference matrix of the camera.
根据权利要求1所述的包裹体积的测量方法，其特征在于，所述步骤S200对所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物具体包括步骤：The method for measuring a package volume according to claim 1, wherein the step S200 performs image processing on the target image, so that the identifying the planar identifier in the target image comprises the following steps:

S210对所述目标图像进行二值分割，并从分割后的二值化图像中提取外形轮廓；S210 performs binary segmentation on the target image, and extracts a contour from the segmented binarized image;

S220根据预存的所述平面标识物所对应的外形轮廓的共性特征，从提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；S220: selecting, according to a common feature of the pre-stored contour identifiers of the plane identifiers, an outline contour having the common features as an alternative contour from the extracted contour contours;

S230获取所述备选轮廓的正视图；S230 acquires a front view of the candidate contour;

S240当所述备选轮廓的正视图与预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像。S240: When the front view of the candidate contour is consistent with the pre-stored planar identifier template, the image corresponding to the candidate contour is identified as an image of the planar identifier.
根据权利要求3所述的包裹体积的测量方法，其特征在于，所述平面标识物为正方形，所述步骤S220中预存的所述平面标识物所对应的预设轮廓特征为四边形；The method for measuring a package volume according to claim 3, wherein the planar identifier is a square, and the preset contour feature corresponding to the planar identifier prestored in step S220 is a quadrangle;

所述步骤S230获取所述备选轮廓的正视图具体为：The step S230 obtains a front view of the candidate contour, which is specifically:

S231读取所述备选轮廓的四个顶点的像素坐标；S231 reading pixel coordinates of four vertices of the candidate contour;

S232定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；S232, the pixel coordinates of the four vertices after the projective transformation into the front view are defined as the pixel coordinates of the pre-stored four vertices of the front view of the planar identifier;

S233将上述读取的四个顶点的像素坐标，以及所述定义的四个顶点的像素坐标分别代入平面射影变换公式(1)，求取射影变换矩阵M：S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projective transformation formula (1), respectively, and obtaining the projective transformation matrix M:

其中，X为备选轮廓顶点的齐次像素坐标，其非齐次像素坐标为

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

x1、x2、x3为X中的元素，x1'、x2'、x3'为X'中的元素；x、y分别为非对应的非齐次像素横坐标、纵坐标；u、v分别为射影变换后的非齐次像素横坐标、纵坐标； Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

S234根据所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour.
根据权利要求1-4任一项所述的包裹体积的测量方法，其特征在于，所述步骤S300对所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点具体包括步骤：The method for measuring a package volume according to any one of claims 1 to 4, wherein the step S300 performs image edge detection on the target image, thereby identifying the package to be tested in the target image. The corner points specifically include the steps:

S310对所述目标图像进行边缘检测处理，得到边缘二值图像；S310 performing edge detection processing on the target image to obtain an edge binary image;

S320从所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；S320: searching for the edge line corresponding to the edge of the package to be tested from the edge binary image;

S330计算所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。S330 calculates a position of an intersection of any two edge lines in the edge line to obtain a corner point of the package to be tested.
根据权利要求1-4任一项所述的包裹体积的测量方法，其特征在于，所述步骤S400根据所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积具体包括步骤：The method for measuring a package volume according to any one of claims 1 to 4, wherein the step S400 calculates the angle according to the plane identifier, the internal parameter matrix of the camera acquired in advance, and the corner point. The world coordinates of the point, so that the volume of the package to be tested is obtained according to the world coordinates of the corner point, specifically including the steps:

S410根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量；S410: calculating, according to the plane identifier and the pre-acquired internal reference matrix of the camera, an outer parameter matrix of a current field of view of the camera; the outer parameter matrix includes a rotation matrix and a translation vector;

S420根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标；S420: acquiring, according to the pixel coordinates of the corner point in the pixel coordinate system, the world coordinates of the corner point in combination with the rotation matrix and the translation vector;

S430根据所述角点的世界坐标计算所述待测包裹的长宽高；S430 calculates a length, width, and height of the package to be tested according to the world coordinates of the corner point;

S440将所述长宽高代入体积公式计算获取所述待测包裹的体积。S440 calculates the length and width of the volume into a volume formula to obtain the volume of the package to be tested.
根据权利要求6所述的包裹体积的测量方法，其特征在于，所述步骤S410根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的旋转矩阵和平移向量具体包括步骤：The method for measuring a package volume according to claim 6, wherein the step S410 calculates a rotation matrix and a translation of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera acquired in advance. The vector specifically includes the steps:

S411以所述平面标识物的中点为原点建立世界坐标系；S411 establishes a world coordinate system with the midpoint of the planar identifier as an origin;

S412读取所述平面标识物的各个顶点在像素坐标系中的顶点像素坐标，以及各个顶点在世界坐标系中的顶点世界坐标；S412: reading vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

S413将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)得到所述摄像头当前视场的旋转矩阵和平移向量：S413, substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotation matrix and translation vector:

其中，(u，v，1)表示所述平面标识物中所述目标点在摄像机的像素坐标系中像素坐标的齐次坐标，(Xw，Yw，Zw，1)表示所述目标点在所述世界坐标系中世界坐标的齐次坐标，(Xc，Yc，Zc)表示所述目标点在摄像机的相机坐标系中的相机坐标；fx表示摄像机透镜的物理焦距F与成像仪在X轴方向每个单元尺寸的乘积，fy表示摄像机透镜的物理焦距F与成像仪在y轴方向每个单元尺寸的乘积，cx表示成像仪中心与光轴在X轴方向上的偏移，cy表示由于成像仪成像仪中心与光轴在Y轴方向上的偏移；R表示世界坐标系到相机坐标系的旋转矩阵，T表示所述世界坐标系到所述相机坐标系的平移向量。Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.
根据权利要求7所述的包裹体积的测量方法，其特征在于，所述步骤S420根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标具体包括步骤：The method for measuring a package volume according to claim 7, wherein the step S420 acquires the world of the corner points according to pixel coordinates of the corner point in a pixel coordinate system in combination with the rotation matrix and the translation vector. The coordinates specifically include the steps:

S421读取所述角点在摄像机的像素坐标系中的角点的像素坐标；S421 reads the pixel coordinates of the corner point of the corner point in the pixel coordinate system of the camera;

S422将所述角点的像素坐标、所述预先获取的摄像头的内参矩阵以及所述摄像头当前视场的旋转矩阵和平移向量代入所述公式(4)中得到所述角点的世界坐标；其中，(u，v，1)表示所述角点的像素坐标的齐次坐标，(Xw，Yw，Zw)是所述角点的世界坐标，(Xw，Yw，Zw，1)表示所述角点的世界坐标的齐次坐标；S422, substituting the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4) to obtain the world coordinates of the corner point; (u, v, 1) represents the homogeneous coordinate of the pixel coordinates of the corner point, (Xw, Yw, Zw) is the world coordinate of the corner point, and (Xw, Yw, Zw, 1) represents the angle Homogeneous coordinates of the world coordinates of the point;

所述步骤S430根据所述角点的世界坐标计算所述待测包裹的长宽高具体包括步骤：The step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the following steps:

S431读取任意四个角点对应的世界坐标；所述四个角点中任意一个角点为第一角点，且剩余三个角点分别与所述第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴；S431 reads the world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively connected with the first corner point to generate a straight line Parallel to the XYZ axis of the world coordinate system, respectively;

S432将所述第一角点对应的世界坐标，以及第二角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的长度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的X轴；S432: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculating a length of the package to be tested; the second corner point and the first The line generated by the corner point connection is parallel to the X axis of the world coordinate system;

S433将所述第一角点对应的世界坐标，以及第三角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的宽度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的Y轴；S433: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculating a width of the package to be tested; the second corner point and the first corner The line generated by the point connection is parallel to the Y axis of the world coordinate system;

S434将第四角点对应的世界坐标以及角点的像素坐标，结合所述预先获取的摄像头的内参矩阵、所述摄像头当前视场的旋转矩阵和平移向量，代入所述公式(4)计算获得所述待测包裹的高度；所述第四角点与所述第一角点连接生成的直线平行于所述世界坐标系的Z轴。S434, the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point are combined with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector, and are obtained by substituting the formula (4). a height of the package to be tested; a line generated by the connection of the fourth corner point and the first corner point is parallel to a Z axis of the world coordinate system.
一种包裹体积的测量***，其特征在于，应用于测量设有平面标识物的待测包裹的体积，所述包裹体积的测量***包括：A package volume measuring system is characterized in that it is applied to measuring a volume of a package to be tested provided with a planar marker, the measurement system of the package volume comprising:

图像获取模块，用于通过移动终端的摄像头采集目标图像；所述目标图像包括所述平面标识物、设有所述平面标识物的待测包裹的上表面和与上表面相邻的所述待测包裹的两个侧表面；An image acquisition module, configured to acquire a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the planar identifier, and the to-before adjacent to the upper surface Measuring the two side surfaces of the package;

图像识别模块，用于对所述图像获取模块获取的所述目标图像进行图像处理，从而在所述目标图像中识别出所述平面标识物；An image recognition module, configured to perform image processing on the target image acquired by the image acquisition module, thereby identifying the planar identifier in the target image;

图像处理模块，用于对所述图像获取模块获取的所述目标图像进行图像边缘检测，从而在所述目标图像中识别出所述待测包裹的角点；An image processing module, configured to perform image edge detection on the target image acquired by the image acquiring module, so as to identify a corner point of the package to be tested in the target image;

体积测量模块，用于根据所述图像识别模块识别出的所述平面标识物、预先获取的摄像头的内参矩阵和所述角点，计算所述角点的世界坐标，从而根据所述角点的世界坐标得到所述待测包裹的体积。a volume measuring module, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module, an internal parameter matrix of a pre-acquired camera, and the corner point, thereby, according to the corner point The world coordinates get the volume of the package to be tested.
根据权利要求9所述的包裹体积的测量***，其特征在于，还包括：The package volume measuring system according to claim 9, further comprising:

标定模块，用于在所述图像获取模块通过所述摄像头采集目标图像之前，对所述摄像头进行标定，获取所述摄像头的内参矩阵。And a calibration module, configured to: after the image acquisition module acquires the target image by using the camera, perform calibration on the camera to obtain an internal parameter matrix of the camera.
根据权利要求9所述的包裹体积的测量***，其特征在于，所述图像识别模块包括：The package volume measuring system according to claim 9, wherein the image recognition module comprises:

二值分割子模块，用于对所述提取模块获取的所述目标图像进行二值分割；a binary segmentation submodule, configured to perform binary segmentation on the target image acquired by the extraction module;

外形提取子模块，用于从分割后的二值化图像中提取外形轮廓；a shape extraction sub-module for extracting a contour from the segmented binarized image;

存储子模块，用于存储所述平面标识物所对应的外形轮廓的共性特征、所述平面标识物模板；a storage submodule, configured to store a common feature of the outline corresponding to the planar identifier, and the planar identifier template;

判断处理子模块，用于根据所述存储子模块中存储的所述平面标识物所对应的外形轮廓的共性特征，从所述外形提取子模块提取的外形轮廓中选取具有所述共性特征的外形轮廓作为备选轮廓；a judging processing module, configured to select, according to a common feature of the contour contour corresponding to the plane identifier stored in the storage submodule, an outer shape extracted from the outer shape extracting submodule The contour is used as an alternative contour;

正视图获取子模块，用于获取所述备选轮廓的正视图；a front view obtaining submodule for obtaining a front view of the candidate contour;

识别子模块，用于当所述备选轮廓的正视图与所述存储子模块预存的所述平面标识物模板一致时，识别所述备选轮廓对应的图像为所述平面标识物的图像。And an identifying sub-module, configured to identify, when the front view of the candidate contour is consistent with the planar identifier template pre-stored by the storage sub-module, an image corresponding to the candidate contour as an image of the planar identifier.
根据权利要求11所述的包裹体积的测量***，其特征在于，所述平面标识物为正方形，所述储存子模块储存的所述平面标识物所对应的外形轮廓的共性特征为四边形；The measurement system of the package volume according to claim 11, wherein the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle;

所述正视图获取子模块包括：The front view obtaining submodule includes:

读取单元，用于读取所述备选轮廓的四个顶点的像素坐标；a reading unit, configured to read pixel coordinates of four vertices of the candidate contour;

定义单元，用于定义所述备选轮廓经过射影变换为正视图后的四个顶点的像素坐标为预存的所述平面标识物的正视图的四个顶点的像素坐标；a defining unit, configured to define pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as pixel coordinates of four pre-existing front views of the planar identifier;

计算单元，用于将所述读取单元读取的四个顶点的像素坐标，以及所述定义单元定义的四个顶点的像素坐标分别代入下列平面射影变换公式(1)，求取射影变换矩阵M：a calculating unit, configured to substitute the pixel coordinates of the four vertices read by the reading unit and the pixel coordinates of the four vertices defined by the defining unit into the following planar projective transformation formula (1), and obtain a projective transformation matrix M:

其中，X为备选轮廓顶点的齐次像素坐标，其非齐次像素坐标为

X′为射影变换后顶点的齐次坐标，其非齐次像素坐标为

x1、x2、x3为X中的元素，x1'、x2'、x3'为X'中的元素；x、y分别为非对应的非齐次像素横坐标、纵坐标；u、v分别为射影变换后的非齐次像素横坐标、纵坐标； Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

变换单元，用于根据所述计算单元求取得到的所述射影变换矩阵M，对所述备选轮廓对应的二值化图像的所有像素都执行射影变换，获得所述备选轮廓包含的区域对应的正视图。a transforming unit, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit, to obtain an area included in the candidate contour Corresponding front view.
根据权利要求9-12任一项所述的包裹体积的测量***，其特征在于，所述图像处理模块包括：The package volume measuring system according to any one of claims 9 to 12, wherein the image processing module comprises:

边缘检测子模块，用于对所述提取模块获取的所述目标图像进行边缘检测处理，得到边缘二值图像；An edge detection sub-module, configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image;

直线查找子模块，用于从所述边缘检测子模块得到的所述边缘二值图像中查找对应于待测包裹边缘的边缘直线；a line search sub-module, configured to search for an edge line corresponding to the edge of the package to be tested from the edge binary image obtained by the edge detection sub-module;

角点获取子模块，用于计算所述直线查找子模块查找到的所述边缘直线中任意两条边缘直线组成的交点的位置，得到所述待测包裹的角点。And a corner point obtaining sub-module, configured to calculate a position of an intersection of any two edge lines of the edge line found by the line search sub-module, to obtain a corner point of the package to be tested.
根据权利要求9所述的包裹体积的测量***，其特征在于，所述体积测量模块包括：The package volume measuring system according to claim 9, wherein the volume measuring module comprises:

外参矩阵获取子模块，用于根据所述平面标识物及预先获取的所述摄像头的内参矩阵，计算所述摄像头当前视场的外参矩阵；所述外参矩阵包括旋转矩阵和平移向量；An outer parameter matrix obtaining submodule, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera obtained in advance; the outer parameter matrix includes a rotation matrix and a translation vector;

角点坐标获取子模块，用于根据所述角点在像素坐标系中的像素坐标，结合所述旋转矩阵和平移向量获取所述角点的世界坐标；a corner coordinate acquisition submodule, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in a pixel coordinate system, in combination with the rotation matrix and the translation vector;

长宽高获取子模块，用于根据所述角点的世界坐标计算所述待测包裹的长宽高；a length, width and height acquisition submodule, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

包裹体积获取子模块，用于将所述长宽高代入体积公式计算获取所述待测包裹的体积。a parcel volume acquisition sub-module, configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.
根据权利要求14所述的包裹体积的测量***，其特征在于，所述外参矩阵获取子模块包括：The package volume measuring system according to claim 14, wherein the outer parameter matrix acquisition submodule comprises:

坐标系确定单元，用于以所述平面标识物的中点为原点建立世界坐标系；a coordinate system determining unit configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

顶点坐标读取单元，用于读取所述平面标识物的各个顶点在像素坐标系中的顶点像素坐标，以及各个顶点在世界坐标系中的顶点世界坐标；a vertex coordinate reading unit, configured to read vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

运算单元，用于将所述顶点像素坐标、顶点世界坐标以及所述预先获取的所述摄像头的内参矩阵分别代入下述公式(2)和公式(3)，再结合公式(4)得到所述摄像头当前视场的旋转矩阵和平移向量：An arithmetic unit, configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and obtain the The rotation matrix and translation vector of the camera's current field of view:

其中，(u，v，1)表示所述平面标识物中所述目标点在摄像机的像素坐标系中像素坐标的齐次坐标，(Xw，Yw，Zw，1)表示所述目标点在所述世界坐标系中世界坐标的齐次坐标，(Xc，Yc，Zc)表示所述目标点在摄像机的相机坐标系中的相机坐标；fx表示摄像机透镜的物理焦距F与成像仪在X轴方向每个单元尺寸的乘积，fy表示摄像机透镜的物理焦距F与成像仪在y轴方向每个单元尺寸的乘积，cx表示成像仪中心与光轴在X轴方向上的偏移，cy表示由于成像仪成像仪中心与光轴在Y轴方向上的偏移；R表示世界坐标系到相机坐标系的旋转矩阵，T表示所述世界坐标系到所述相机坐标系的平移向量。Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.
根据权利要求15所述的包裹体积的测量***，其特征在于，所述角点坐标获取子模块包括：The package volume measuring system according to claim 15, wherein the corner coordinate acquisition submodule comprises:

角点像素坐标获取单元，用于读取所述角点在摄像机的像素坐标系中的像素坐标；a corner pixel coordinate acquiring unit, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

角点世界坐标获取单元，用于将所述角点的像素坐标、所述预先获取的摄像头的内参矩阵以及所述摄像头当前视场的旋转矩阵和平移向量代入所述公式(4)中得到所述角点的世界坐标；其中，(u，v，1)表示所述角点的像素坐标的齐次坐标，(Xw，Yw，Zw)是所述角点的世界坐标，(Xw，Yw，Zw，1)表示所述角点的世界坐标的齐次坐标；a corner point world coordinate acquiring unit, configured to substitute pixel coordinates of the corner point, an internal parameter matrix of the pre-acquired camera, and a rotation matrix and a translation vector of a current field of view of the camera into the formula (4) The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw, Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

所述长宽高获取子模块包括：The long and wide height acquisition submodule includes:

角点世界坐标读取单元，用于读取任意四个角点对应的世界坐标；所述四个角点中任意一个角点为第一角点，且剩余三个角点分别与所述第一角点两两连接生成的直线分别平行于所述世界坐标系的XYZ轴；a corner point world coordinate reading unit, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points respectively correspond to the first A straight line generated by a pair of corner points is parallel to the XYZ axis of the world coordinate system, respectively;

长度运算单元，用于将所述第一角点对应的世界坐标，以及第二角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的长度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的X轴；a length operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the X axis of the world coordinate system;

宽度运算单元，用于将所述第一角点对应的世界坐标，以及第三角点对应的世界坐标代入坐标两点距离公式，计算获得所述待测包裹的宽度；所述第二角点与所述第一角点连接生成的直线平行于所述世界坐标系的Y轴；a width operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point and a line generated by the first corner connection is parallel to a Y axis of the world coordinate system;

高度运算单元，用于将第四角点对应的世界坐标以及角点的像素坐标，结合所述预先获取的摄像头的内参矩阵、所述摄像头当前视场的旋转矩阵和平移向量，代入所述公式(4)计算获得所述待测包裹的高度；所述第四角点与所述第一角点连接生成的直线平行于所述世界坐标系的Z轴。a height operation unit, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point, in combination with the pre-acquired internal parameter matrix of the camera, the rotation matrix of the current field of view of the camera, and the translation vector, and substitute the formula into the formula (4) Calculating a height of the package to be tested; a line generated by connecting the fourth corner point to the first corner point is parallel to a Z axis of the world coordinate system.
一种存储介质，其特征在于，所述存储介质存储有多条指令，所述多条指令被一个或者多个处理器执行，以实现权利要求1-8中任一项所述的包裹体积的测量方法的步骤。A storage medium, characterized in that the storage medium stores a plurality of instructions, the plurality of instructions being executed by one or more processors to implement the package volume of any one of claims 1-8 The steps of the measurement method.
一种移动终端，其特征在于，包括：A mobile terminal, comprising:

处理器，实现各指令；a processor that implements each instruction;

存储介质，存储多条指令；a storage medium that stores a plurality of instructions;

其中：所述处理器执行所述存储介质存储的指令，以实现权利要求1-8任一项所述的包裹体积的测量方法的步骤。Wherein the processor executes the instructions stored in the storage medium to implement the steps of the method for measuring a package volume according to any one of claims 1-8.