Background
The core idea of the intelligent sales counter is to infinitely reduce the distance between goods and consumers, and pay attention to convenience and convenience. This convenient retail business caters to the status of the young generation group, is widely popular domestically, and has a considerable international market. According to the technical direction, the intelligent sales counter is divided into two types: RFID type and computer vision type.
The traditional RFID type intelligent sales counter mainly utilizes RFID labels, each commodity is attached with the RFID label to identify the commodity, the type of intelligent sales counter has the advantages that the RFID technology is mature, products are easy to produce and popularize on a large scale, and most of the existing intelligent sales counter in the market adopts the RFID technology. The defects of the RFID technology are that the label is easy to shield and damage, and the goods damage rate is high; on the other hand, each commodity needs to be pasted with an RFID label, so that the cost of the label and the labor cost for pasting are high, and meanwhile, the technical threshold is low and the core competitiveness is lacked.
Computer vision type intelligent sales counter is a research hotspot in the field at present, the main technologies are static identification and dynamic identification, but no scheme which is especially mature and suitable for large-scale production exists. The static recognition determines the specific information of the commodity by recognizing the characteristics, the outline, the color and other obvious physical signs of the commodity, the direct static recognition faces a great challenge, and the outlines of a plurality of commodities in an intelligent sales counter are crossed and the commodity individuals are difficult to divide; the features are numerous and complicated and redundant, resulting in many mismatches; the rich colors also make it impossible to identify the goods by differentiating the colors. Only when the commodities are divided firstly, the static identification can be completed quickly and accurately. Traditional manual partitioning of goods results in extremely low efficiency and compatibility; affine transformation can convert the intelligent sales counter into a rectangle through stretching in the x-axis direction and the y-axis direction of the image, and the division of the goods passage is completed. However, the method is low in precision and inaccurate in segmentation, and meanwhile, the commodity feature deformation is caused by stretching, so that the identification is seriously influenced. Although the dynamic identification theoretically does not need to divide the virtual goods way, the complete division of the virtual goods way can remarkably reduce the time of dynamic identification by combining simple row and column judgment, and can assist in judging the accuracy of identification.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a method for dividing virtual channels of intelligent sales counter, compared with the existing dividing method, the method can obviously reduce the complexity of operation, save time and cost, obviously improve the dividing accuracy, and also has no influence on various physical signs on the surface of the commodity, so that the intelligent sales counter based on computer vision obtains good identification environment.
The invention is realized by the following technical scheme:
a parameter-configurable virtual goods path dividing method for intelligent sales counter is characterized by comprising the following steps:
s1, selecting an image or a video frame and importing the image or the video frame into a virtual goods path division algorithm;
s2, selecting the number parameters of rows and columns for dividing the virtual goods channels according to the physical condition of the intelligent sales counter;
s3, reading and displaying an image or a video frame, and acquiring the pixel size of the image or the video frame;
s4, when the image or the video frame drifts in S3, turning to S5, and when the image or the video frame does not drift, skipping S5;
s5, aiming at the drift of the image or video frame in the S4, selecting the accurate range of the image or video frame contained in the virtual cargo way by using the pixel size configuration parameter obtained in the S3;
s6, aiming at the angle of the camera for obtaining the image or the video frame, firstly turning to S5, and then configuring the line and column sight distance reduction coefficients.
Preferably, in S2, the intelligent sales counter is divided by using a geometric perspective phenomenon, so that the virtual lane is divided into postures in which the heights and widths decrease gradually layer by layer.
Preferably, in S2, the virtual lane divides the intelligent sales counter symmetrically by two sides of an image or video frame obtained by shooting in a manner that the camera is located in the middle or close to the middle; and dividing the virtual cargo channel by taking the position of the camera or the position close to the camera as an axis according to the condition of the container for the image or the video frame obtained by the camera which is not in the middle or is close to the middle.
Preferably, in the partition of the intelligent sales counter by the virtual goods way, the reduction speed of the weight reduced by the power is exponential, linear association exists between the virtual goods way and the goods way in the same row, when the exponential number is more than or equal to 5, the linear association is destroyed by the influence of the perspective phenomenon on the goods way in the row, and the linear association is difficult to correct; when the exponential number is not more than 5, the influence is not so large, and the display of the intelligent sales counter in the optional image or video frame is taken in a mode that the camera is positioned in the middle or close to the middle.
Preferably, for the division requiring extreme accuracy, two cameras are respectively responsible for the virtual division of half of the cargo ways, and tiny column adjusting parameters are added to adjust the cargo way weight ratio deviating from the central axis position.
Preferably, the coordinate parameters coordinate _ b and coordinate _ e of the image or video frame obtained by shooting in the camera approaching intermediate mode are configured according to the distance amount of the camera offset from the center.
Preferably, in S6, the configuration parameters, namely the row visual distance reduction coefficient constant _ row and the column visual distance reduction coefficient constant _ column, adapt to the change of the inclination angle of the intelligent sales counter photographed by the camera, and the expression form is the change of the included angle between the lens of the camera and the surface of the intelligent sales counter, and the expression form of the change of the included angle between the lens of the camera and the vertical line by the camera placed on the top of the intelligent sales counter on the horizontal ground.
Preferably, the same model, including but not limited to the intelligent sales counter produced in the same batch, applies the same coordinate parameters of coordinate _ b and coordinate _ e and row and column visual distance reduction coefficients constant _ row and constant _ column in the case of slight changes in the camera position and inclination.
The invention has the beneficial effects that:
1. the method can configure parameters according to the visual angles of different angles, and compared with a traditional dividing method, the accuracy is obviously improved.
2. The method has certain fault tolerance rate for the deviation and the view angle change of the image, and generally, the fault tolerance rate is higher when the goods way division weight is larger, namely, the deviation and the view angle change in a wider range can be allowed to float. The method disclosed by the invention can be used for carrying out virtual lane division on intelligent sales counter of different models in a large scale, and can also be used for automatically dividing virtual lanes by utilizing large-scale data training and a machine learning method.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
The method for dividing the virtual goods channel with configurable parameters for the intelligent sales counter as shown in FIG. 1 comprises the following steps:
step 1, selecting an image or video frame, and requiring samples from intelligent sales counter of the same type from the same angle in order to ensure the accuracy of a division result, wherein one frame is used as a division basis, and the rest frames are used as verification.
And 2, configuring row and column quantity parameters row and column of the virtual goods channel according to the actual situation of the intelligent sales counter.
And 3, reading and displaying the image, and selecting an actual effective area in the image as a roi area.
And 4, returning to the vertex coordinates of the roi selected in the step 3, and calculating the height and width of the roi.
And step 5, obtaining a calculation formula of standard height h _ std by intervening row-column visual distance reduction coefficients constant _ row and constant _ column according to row and column obtained in the step 2 and height and width obtained in the step 4, wherein the calculation formula is as follows
The standard width w _ std is
And 6, initializing a row-column visual distance reduction coefficient constant _ row and a constant _ column according to an empirical rule.
And 7, obtaining the widths and heights of all virtual cargo way rows and columns according to the h _ std and w _ st obtained in the step 5 and the rules of exponential reduction of the row-column visual distance reduction coefficients constant _ row and constant _ column respectively.
And 8, applying the result of the step 7 to the selected image or video frame, and adjusting the numerical values of the constant _ row and the constant _ column according to the display result to adapt to the intelligent sales counter in the angle of the model.
And 9, recording the numerical values of the constant _ row and the constant _ column of the current result, and storing the numerical values as relevant empirical rules.
Compared with the existing dividing method, the method has the advantages that the complexity of operation can be obviously reduced, the time cost is saved, the dividing accuracy is obviously improved, various physical signs on the surface of the commodity are not affected, and the intelligent sales counter based on computer vision obtains a good recognition environment.
Example 2
The method for dividing the virtual goods way implemented by the embodiment specifically includes:
step 1, selecting an image or a video frame, and introducing the image or the video frame into a virtual goods way division algorithm;
step 2, selecting the number parameters of rows and columns for dividing the virtual goods channels according to the physical condition of the intelligent sales counter;
it should be noted here that since the visual sense is rapidly reduced and there is a requirement for the number of rows and columns of the container itself, the method disclosed in the present invention is to further provide convenience and possibility for the identification of the goods in each channel, so that even if the virtual lane is divided by using the method, it is still necessary to consider whether there is a recognizable color and characteristic in the lane, and it should be noted that the consideration does not affect the correctness and applicability of the method.
Step 3, reading and displaying the image or video frame, reading the pixel size of the image or video frame, turning to step 4 when the image or video frame has drift, and skipping step 4 when the image or video frame has no drift;
step 4, aiming at the drift of the image or the video frame, configuring parameters to select the accurate range of the image or the video frame contained in the virtual cargo way on the basis of the pixel size obtained in the step 3;
it should be noted here that the actual selected range may exceed the scale of the image or video frame itself, due to the dead space between the aisle closest to the camera and the camera.
And 5, aiming at the angle of the camera for obtaining the image or the video frame, firstly turning to the step 4, and then configuring the line and column sight distance reduction coefficients, wherein all the parameters can be invariably applied to the intelligent sales counter of the same model.
The intelligent sales counter to be divided utilizes the geometric perspective phenomenon to enable the division of the virtual goods road to be in a posture that the height and the width are decreased gradually layer by layer, and the proportion of the goods layer with the farther spatial distance is smaller.
The principle is that the perspective phenomenon is the reason that the goods way in the image can not be divided in a conventional way, the weight of the goods way in the image is reduced due to the increase of the space distance from the shooting point, and the corresponding goods way should occupy smaller weight when being divided. Although the weights of different goods channels are different during division, the inherent rule of the gravity center of each goods channel is not completely changed due to the perspective phenomenon, and in an image, the goods channels in each row and each column are still linearly related, which means that the specific weight value of each goods channel can be obtained in a test mode, and the weight value can also be automatically obtained through a machine learning scoring method.
In the selected image or video frame, the intelligent sales counter is displayed and shot in a mode that the camera is positioned in the middle or close to the middle, and the virtual goods channel divides the intelligent sales counter in such a way that the two sides of the obtained image or video frame are symmetrically divided; for the image or video frame obtained by the camera which is not in the middle or is close to the middle position, the virtual cargo channel can be divided by taking the position of the camera or the position close to the camera as the axis according to the condition of the container.
The principle is that the reduction speed of the weight reduced according to the power is exponential, the method disclosed by the invention mainly utilizes the linear correlation of the goods lanes in the same row, when the exponential is more than or equal to 5, the linear correlation can be damaged by the influence of the perspective phenomenon on the goods lanes in the row, and the linear correlation is difficult to correct; when the exponential progression is not greater than 5, the influence is not great, and can be ignored due to the primary and secondary factors and the difficulty of parameter configuration, which is why the display of the intelligent sales counter in the selected image or video frame is required to be shot with the camera in the middle or near middle. For the division requiring extreme accuracy, two cameras can be adopted to be respectively responsible for the virtual division of half of the cargo ways, and tiny column adjusting parameters are also added to adjust the cargo way weight ratio deviating from the position of the central axis.
The obtained image or video frame is shot in a mode that the camera approaches the middle, and coordinate parameters coordinate _ b and coordinate _ e can be configured according to the distance of the offset center of the camera, so that the asymmetric errors on two sides caused by image drift are compensated, and the purpose of accurately modifying the virtual goods way to divide the target area is achieved.
The configurable parameter line-of-sight distance reduction coefficient constant _ row and the column-of-sight distance reduction coefficient constant _ column adapt to the change of the inclination angle of the intelligent sales counter shot by the camera, the expression form is the change of the included angle between the camera lens and the surface of the intelligent sales counter, and specifically, the change of the included angle between the camera lens and the vertical line is expressed by the camera lens on the top of the intelligent sales counter placed on the horizontal ground. This change also affects the distance of the sales counter from the bottom of the image in the image, and requires the co-ordinate parameters coordinate _ b and coordinate _ e to be combined.
The same type, including but not limited to the intelligent sales counter produced in the same batch, is suitable for the same coordinate parameters of coordinate _ b and coordinate _ e and row and column visual distance reduction coefficients constant _ row and constant _ column under the condition that the change of the camera position and the inclination angle is slight.
The invention discloses a virtual goods way dividing method, which is developed by combining the natural law of perspective phenomenon and the mathematical principle of geometric perspective, and the perspective effect diagram of the method is shown in figure 2.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.