CN112990115B - Shooting method and system for freezer display identification - Google Patents
Shooting method and system for freezer display identification Download PDFInfo
- Publication number
- CN112990115B CN112990115B CN202110427523.4A CN202110427523A CN112990115B CN 112990115 B CN112990115 B CN 112990115B CN 202110427523 A CN202110427523 A CN 202110427523A CN 112990115 B CN112990115 B CN 112990115B
- Authority
- CN
- China
- Prior art keywords
- image data
- centroid
- image
- point coordinate
- centroid point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/247—Aligning, centring, orientation detection or correction of the image by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
Abstract
The invention provides a shooting method and a shooting system for freezer display identification, wherein the method comprises the following steps: receiving a cabinet door opening signal, and starting a shooting device to shoot to obtain first image data; processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data; performing deformity correction on the second image data to obtain third image data without distortion; and processing the third image data through an image recognition technology to obtain a commodity display result, thereby completing the refrigerator display recognition. The shooting method and the shooting system for the refrigerator display identification enable a brand dealer to quickly and accurately obtain the refrigerator commodity display information under the condition of no salesman.
Description
Technical Field
The invention relates to the technical field of information transmission, in particular to a shooting method and a shooting system for freezer display identification.
Background
In the fast-moving field, the placement of freezers is an important means of sales, and the display of the merchandise placed in the freezer plays an important role in the ultimate sale. If the display of the goods put in the refrigerator is not up to standard, the goods are in short supply, and even other brands of goods are put in, the economic loss of the brands putting in the refrigerator is great. The brander therefore needs the service personnel to periodically visit the various store terminals and then check and confirm those released freezers. The method not only consumes a large amount of manpower and material resources, but also has low checking frequency and cannot know the current state of the freezer in real time.
For the above situations, some improved technical solutions exist at present, such as installing a camera at the top center of each layer of the refrigerator, and then uploading pictures to a service for analysis, but this method can only look at the top of the goods, and cannot distinguish the goods with extremely high similarity. Or the camera is arranged at the corner of the top of the refrigerator, and the mode can only check the commodity and cannot shoot complete commodity display; and install the camera in the refrigerator-freezer outside, though can shoot complete commodity display, probably because the refrigerator door has water smoke or shelters from, finally also can not obtain complete clear commodity display.
Disclosure of Invention
In order to solve the technical problems, the invention provides a shooting method and a shooting system for refrigerator display identification, so that brand merchants can quickly and accurately obtain refrigerator commodity display information without a salesman.
The invention provides a shooting method for freezer display identification, which comprises the following steps:
receiving a cabinet door opening signal, and starting a shooting device to shoot to obtain first image data;
processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data;
performing deformity correction on the second image data to obtain third image data without distortion;
and processing the third image data through an image recognition technology to obtain a commodity display result, thereby completing the refrigerator display recognition.
Further, the processing the first image data according to an algorithm for automatically analyzing the best shooting position to obtain second image data includes:
performing primary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing;
searching the object outline images in the fourth image data, and calculating the centroid point of each object outline image to obtain the coordinate set of the centroid point;
establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid point of each article outline image and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data;
sorting coordinates of centroid points in the first centroid point coordinate list and coordinates of centroid points in the second centroid point coordinate list respectively in an ascending sorting mode according to the distance relationship between the centroid point of each article outline image and a central line of the fourth image data to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data is, the more forward the coordinate sorting position of the center of mass point is;
and judging whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, if so, judging whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset rule, and if so, taking the fourth image data as second image data.
Further, the performing the preliminary processing on the first image data to obtain fourth image data includes:
preprocessing the first image data to obtain fifth image data; wherein the pre-processing comprises: zooming and rotating the image;
carrying out gray level processing on the fifth image data, and converting the fifth image data into a binary image to obtain sixth image data;
filtering the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the horizontal direction contour of the sixth image data and removing the region with the region area smaller than the threshold value in the sixth image data.
Further, the performing the deformity correction on the second image data to obtain a third image data without distortion includes:
acquiring internal parameters and a distortion matrix of the shooting device;
and mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain third image data without distortion.
Further, the processing the third image data by the image recognition technology to obtain the product display result includes:
performing target detection on the third image data to obtain commodity category information and commodity position information in the third image data;
obtaining freezer commodity display information according to the commodity category information and the commodity position information;
calculating a display index according to the ice chest commodity display information; wherein the display metrics include: saturation and purity.
A second aspect of the present invention provides a camera system for ice chest display identification, comprising:
the shooting module is used for receiving a cabinet door opening signal and starting the shooting device to shoot to obtain first image data;
the optimal shooting position algorithm processing module is used for processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data;
the deformity correction module is used for carrying out deformity correction on the second image data to obtain third image data without distortion;
and the pattern recognition module is used for processing the third image data through an image recognition technology to obtain a commodity display result and finish the refrigerator display recognition.
Further, the optimal shooting position algorithm processing module includes:
the image preliminary processing submodule is used for carrying out preliminary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing;
the centroid point calculation submodule is used for searching the article outline images in the fourth image data and calculating the centroid point of each article outline image to obtain a coordinate set of the centroid point;
the coordinate list establishing submodule of the centroid points is used for establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid points of the outline images of the objects and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data;
the coordinate list optimization submodule of the centroid points is used for sorting the coordinates of the centroid points in the first centroid point coordinate list and the coordinates of the centroid points in the second centroid point coordinate list respectively in an ascending sorting mode according to the distance relationship between the centroid points of each article contour image and the central line of the fourth image data to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data is, the more forward the coordinate sorting position of the center of mass point is;
and the judging submodule is used for judging whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, if so, judging whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset rule, and if so, taking the fourth image data as second image data.
Further, the image preliminary processing sub-module includes:
the image preprocessing submodule is used for preprocessing the first image data to obtain fifth image data; wherein the pre-processing comprises: zooming and rotating the image;
the image gray processing submodule is used for carrying out gray processing on the fifth image data and converting the fifth image data into a binary image to obtain sixth image data;
the image filtering processing submodule is used for carrying out filtering processing on the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the horizontal direction contour of the sixth image data and removing the region with the region area smaller than the threshold value in the sixth image data.
Further, the deformity correction module is further configured to:
acquiring internal parameters and a distortion matrix of the shooting device;
and mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain third image data without distortion.
Further, the pattern recognition module includes:
the target identification submodule is used for carrying out target detection on the third image data to obtain commodity category information and commodity position information in the third image data;
the freezer commodity display information acquisition submodule is used for acquiring freezer commodity display information according to the commodity category information and the commodity position information;
the display index calculation submodule is used for calculating a display index according to the commodity display information of the freezer; wherein the display metrics include: saturation and purity.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
1. the invention utilizes an algorithm for automatically analyzing the best shooting position to judge the position of the refrigerator door in real time, so that the camera can shoot all the commodities in the refrigerator at the best position.
2. The front of the commodity can be clearly shot, and the commodity display identification is facilitated.
3. The power supply module in the intelligent camera device can use a large-capacity mobile power supply, so that the rapid installation and use can be realized under the condition that the main body of the refrigerator is not changed aiming at the placed refrigerator.
4. The system replaces the salesman to collect the photos of the freezer, greatly improves the collection frequency and efficiency, and saves the cost
5. The automatic analysis algorithm can be suitable for various types of freezers, and has wide application range.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a block diagram of an apparatus for ice chest display identification according to one embodiment of the present invention;
FIG. 2 is a block diagram of an apparatus for ice chest display identification according to another embodiment of the present invention;
FIG. 3 is a block diagram of an apparatus for ice chest display identification according to another embodiment of the present invention;
FIG. 4 is a block diagram of an apparatus for ice chest display identification according to yet another embodiment of the present invention;
FIG. 5 is a block diagram of the mounting of an apparatus for ice chest display identification according to one embodiment of the present invention;
FIG. 6 is a block diagram of the mounting of an apparatus for ice chest display identification according to another embodiment of the present invention;
FIG. 7 is a flow chart of a photographing method for ice chest display identification according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a photographing method for ice chest display identification according to an embodiment of the present invention before image deformity correction;
FIG. 9 is a schematic diagram illustrating a camera method for ice chest display identification with image deformity correction according to an embodiment of the present invention;
FIG. 10 is a flow chart of a photographing method for ice chest display identification according to an embodiment of the present invention;
FIG. 11 is a flow chart of a photographing method for ice chest display identification according to another embodiment of the present invention;
FIG. 12 is a flow chart of a photographing method for ice chest display identification according to another embodiment of the present invention;
FIG. 13 is a flow chart of a photographing method for ice chest display identification according to another embodiment of the present invention;
FIG. 14 is a flow chart of a photographing method for ice chest display identification according to yet another embodiment of the present invention;
FIG. 15 is an apparatus diagram of a camera system for ice chest display identification according to one embodiment of the present invention;
FIG. 16 is an apparatus diagram of a camera system for ice chest display identification according to another embodiment of the present invention;
FIG. 17 is an apparatus diagram of a camera system for ice chest display identification according to another embodiment of the present invention;
FIG. 18 is an apparatus diagram of a camera system for ice chest display identification according to yet another embodiment of the present invention;
fig. 19 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be understood that the step numbers used herein are for convenience of description only and are not intended as limitations on the order in which the steps are performed.
It is to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
The terms "comprises" and "comprising" indicate the presence of the described features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The term "and/or" refers to and includes any and all possible combinations of one or more of the associated listed items.
A first aspect.
Referring to fig. 1-3, an embodiment of the present invention provides an apparatus for ice chest display identification, comprising:
the shooting module is used for collecting image data signals of commodities displayed in the refrigerator and sending the image data signals to the main control module;
a communication module for signal transmission, wherein the signal comprises: an image data signal;
the main control module is used for receiving the image data signal sent by the shooting module;
the switch sensor is used for detecting a frequency signal of opening the refrigerator door and sending the frequency signal to the main control module;
the temperature detection module is used for detecting a temperature signal in the refrigerator and sending the temperature signal to the main control module;
the shooting module, the switch sensor and the temperature detection module are connected with the main control module through the communication module;
the power supply module is used for supplying electric quantity to the ice chest display identification device; the power supply module is connected with the main control module.
In a specific embodiment, the shooting module comprises a wide-angle camera, and the communication module comprises a 4G network and/or a Wi-Fi network.
Referring to fig. 4, in an embodiment, the present invention provides an apparatus for refrigerator exhibition recognition, including a camera with a wide-angle lens, a wireless network module, a temperature sensor, a main control module, and a power supply module.
(1) The wide-angle camera can set parameters such as frame rate and resolution.
(2) The wireless network module may use a 4G or Wi-Fi network and record the GPS location.
(3) The temperature sensor may measure a temperature inside the refrigerator.
(4) The switch sensor can be used for opening the refrigerator door for times.
(5) The main control module can control all the modules and acquire corresponding data.
(6) The power supply module can supply power to all the modules, including but not limited to a mobile power supply or a household alternating current power supply.
Referring to fig. 5, the present invention provides a mounting method of an apparatus for ice chest display identification, comprising:
(1) the intelligent camera device is installed at the central position of the inner side of the refrigerator door handle.
Referring to fig. 6-7, after the device is installed, the device automatically takes a picture to collect a photo, and the specific flow is as follows:
(1) when the customer opens the refrigerator door, the main control module starts the camera after the switch sensor sends a signal.
(2) The main control module processes each frame of picture, obtains the best picture according to the algorithm of automatically analyzing the best shooting position and closes the camera.
The algorithm process for automatically analyzing the optimal shooting position comprises the following steps:
1) zoom and rotate the picture to the vertical direction first.
2) Converted into a gray scale image and converted into a binary image by a larger threshold value.
3) And filtering by using a vertical filter and a normal filter respectively to remove regions in the horizontal direction and with smaller area.
4) Finding the constants, picking the constants with larger area and calculating the centroid.
5) And respectively loading the centroid points into the left list and the right list according to whether the x of the centroids of the constraints is on the left side or the right side of the center line of the picture.
6) The left and right 2 lists are ordered with the centroid x coordinate being closer to the centerline and further forward.
7) If the 2 lists are not empty, and the x coordinate of the first mass center of the 2 lists is in a set range, the proper position is judged to be reached, otherwise, the proper position is not reached.
(3) And carrying out deformity correction on the optimal picture, acquiring a distortion-free picture, and storing the corrected picture.
As shown in fig. 8-9, the image deformity correction process includes:
1) and obtaining the internal reference and distortion matrix of the camera by using the picture containing the checkerboard square photographed by the camera.
2) And remapping the optimal picture obtained by the automatic analysis optimal shooting position algorithm according to the internal reference and the distortion matrix of the camera, wherein the mapped picture is the corrected picture.
(4) And the wireless network module sends the stored picture, the current equipment ID, the GPS position, the temperature information and the like to the background server.
(5) And the background server calls a related algorithm to identify the picture to obtain a display result of the commodity. And analyzing the display identification result according to the commodity display requirement of the brand party and feeding back the display identification result to the brand party.
Wherein, the identification process is as follows:
1) and carrying out target detection on the picture, and identifying the categories of all commodities in the picture.
2) And obtaining the distribution of each layer of the commodities in the freezer according to the category and the position of each identified commodity.
3) And calculating all other related display indexes such as saturation, purity and the like according to the distribution result.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
1. the invention utilizes an algorithm for automatically analyzing the best shooting position to judge the position of the refrigerator door in real time, so that the camera can shoot all the commodities in the refrigerator at the best position.
2. The front of the commodity can be clearly shot, and the commodity display identification is facilitated.
3. The power supply module in the intelligent camera device can use a large-capacity mobile power supply, so that the rapid installation and use can be realized under the condition that the main body of the refrigerator is not changed aiming at the placed refrigerator.
4. The system replaces the salesman to collect the photos of the freezer, greatly improves the collection frequency and efficiency, and saves the cost
5. The automatic analysis algorithm can be suitable for various types of freezers, and has wide application range.
A second aspect.
Referring to fig. 10-14, an embodiment of the present invention provides a shooting method for refrigerator display identification, including:
s10, receiving a cabinet door opening signal, and starting the shooting device to shoot to obtain first image data.
And S20, processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data.
In a specific embodiment, the step S20 includes:
s21, performing primary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing.
In a specific embodiment, the step S21 includes:
s211, preprocessing the first image data to obtain fifth image data; wherein the pre-processing comprises: image scaling and image rotation.
S212, carrying out gray level processing on the fifth image data, and converting the fifth image data into a binary image to obtain sixth image data;
s213, filtering the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the horizontal direction contour of the sixth image data and removing the region with the region area smaller than the threshold value in the sixth image data.
S22, searching the item contour image in the fourth image data, and calculating the centroid point of each item contour image to obtain the coordinate set of the centroid point.
S23, establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid point of each item outline image and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data.
S24, respectively sorting coordinates of centroid points in the first centroid point coordinate list and coordinates of centroid points in the second centroid point coordinate list in an ascending sorting mode according to the distance relationship between the centroid point of each article contour image and the central line of the fourth image data to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data, the earlier the coordinate-sorted position of the center of mass point.
And S25, judging whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, if so, judging whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset rule, and if so, taking the fourth image data as second image data.
And S30, carrying out deformity correction on the second image data to obtain third image data without distortion.
In a specific embodiment, the step S30 includes:
and S31, acquiring the internal parameters and the distortion matrix of the shooting device.
And S32, mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain distortion-free third image data.
And S40, processing the third image data through an image recognition technology to obtain a commodity display result, and finishing the refrigerator display recognition.
In a specific embodiment, the step S40 includes:
and S41, carrying out target detection on the third image data to obtain the commodity type information and the commodity position information in the third image data.
And S42, obtaining the refrigerator commodity display information according to the commodity type information and the commodity position information.
S43, calculating display indexes according to the commodity display information of the refrigerator; wherein the display metrics include: saturation and purity.
In a third aspect.
Referring to fig. 15-19, one embodiment of the pump provides a camera system for freezer display identification, comprising:
and the shooting module 10 is used for receiving the cabinet door opening signal and starting the shooting device to shoot to obtain first image data.
And the optimal shooting position algorithm processing module 20 is configured to process the first image data according to an automatic optimal shooting position analysis algorithm to obtain second image data.
In a specific embodiment, the optimal shooting position algorithm processing module 20 includes:
an image preliminary processing submodule 21, configured to perform preliminary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing.
In a specific embodiment, the image preliminary processing sub-module 21 includes:
the image preprocessing submodule 210 is configured to preprocess the first image data to obtain fifth image data. Wherein the pre-processing comprises: image scaling and image rotation.
And the image gray processing sub-module 220 is configured to perform gray processing on the fifth image data, and convert the fifth image data into a binary image to obtain sixth image data.
The image filtering processing sub-module 230 is configured to perform filtering processing on the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the horizontal direction contour of the sixth image data and removing the region with the region area smaller than the threshold value in the sixth image data.
And the centroid point calculation submodule 22 is configured to search the item contour images in the fourth image data, and calculate a centroid point of each item contour image to obtain a coordinate set of the centroid point.
The coordinate list establishing submodule 23 of the centroid point is used for establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid point of each article outline image and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data.
A centroid point coordinate list optimization submodule 24, configured to sort, according to a distance relationship between a centroid point of each item profile image and a central line of the fourth image data, coordinates of centroid points in the first centroid point coordinate list and coordinates of centroid points in the second centroid point coordinate list in an ascending sorting manner, respectively, so as to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data, the earlier the coordinate-sorted position of the center of mass point.
And a determining submodule 25, configured to determine whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, if yes, determine whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset rule, and if yes, take the fourth image data as the second image data.
And a malformation correcting module 30, configured to perform malformation correction on the second image data to obtain third image data without distortion.
In one embodiment, the deformity correction module 30 is further configured to:
acquiring internal parameters and a distortion matrix of the shooting device;
and mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain third image data without distortion.
And the pattern recognition module 40 is used for processing the third image data through an image recognition technology to obtain a commodity display result, so that the refrigerator display recognition is completed.
In a specific embodiment, the pattern recognition module 40 includes:
and a target identification submodule 41, configured to perform target detection on the third image data, so as to obtain commodity category information and commodity position information in the third image data.
And the freezer commodity display information acquisition submodule 42 is used for acquiring freezer commodity display information according to the commodity category information and the commodity position information.
A display index calculation submodule 43 for calculating a display index from said ice chest merchandise display information; wherein the display metrics include: saturation and purity.
A fourth aspect.
The present invention provides an electronic device, including:
a processor, a memory, and a bus;
the bus is used for connecting the processor and the memory;
the memory is used for storing operation instructions;
the processor is used for calling the operation instructions, and the executable instructions enable the processor to execute the operation corresponding to the shooting method for the freezer display identification in the second aspect of the application.
In an alternative embodiment, there is provided an electronic apparatus, as shown in fig. 19, an electronic apparatus 5000 shown in fig. 19 including: a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are coupled, such as via a bus 5002. Optionally, the electronic device 5000 may also include a transceiver 5004. It should be noted that the transceiver 5004 is not limited to one in practical application, and the structure of the electronic device 5000 is not limited to the embodiment of the present application.
The processor 5001 may be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 5001 may also be a combination of processors implementing computing functionality, e.g., a combination comprising one or more microprocessors, a combination of DSPs and microprocessors, or the like.
The memory 5003 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 5003 is used for storing application program codes for executing the present solution, and the execution is controlled by the processor 5001. The processor 5001 is configured to execute application program code stored in the memory 5003 to implement the teachings of any of the foregoing method embodiments.
Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like.
A fifth aspect.
The present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a photographing method for ice chest display identification as set forth in the first aspect of the present application.
Yet another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, which, when run on a computer, enables the computer to perform the corresponding content in the aforementioned method embodiments.
Claims (8)
1. A shooting method for freezer display identification, comprising:
receiving a cabinet door opening signal, and starting a shooting device to shoot to obtain first image data;
processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data;
performing deformity correction on the second image data to obtain third image data without distortion;
processing the third image data through an image recognition technology to obtain a commodity display result, and finishing the refrigerator display recognition;
wherein, the processing the first image data according to the algorithm for automatically analyzing the best shooting position to obtain the second image data comprises:
performing primary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing;
searching the object outline images in the fourth image data, and calculating the centroid point of each object outline image to obtain the coordinate set of the centroid point;
establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid point of each article outline image and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data;
sorting coordinates of centroid points in the first centroid point coordinate list and coordinates of centroid points in the second centroid point coordinate list respectively in an ascending sorting mode according to the distance relationship between the centroid point of each article outline image and a central line of the fourth image data to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data is, the more forward the coordinate sorting position of the center of mass point is;
and judging whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, if so, judging whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset range, and if so, taking the fourth image data as second image data.
2. The capture method for freezer display identification of claim 1, wherein the preliminary processing of the first image data to obtain fourth image data comprises:
preprocessing the first image data to obtain fifth image data; wherein the pre-processing comprises: zooming and rotating the image;
carrying out gray level processing on the fifth image data, and converting the fifth image data into a binary image to obtain sixth image data;
filtering the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the region of the sixth image data with the horizontal direction and the area smaller than the threshold value through a vertical direction filter and a normal filter.
3. The capture method for ice chest display identification of claim 1, wherein said correcting deformities in said second image data to obtain a third image data without distortion comprises:
acquiring internal parameters and a distortion matrix of the shooting device;
and mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain third image data without distortion.
4. The capture method for freezer display identification of claim 1, wherein said processing said third image data by image recognition techniques to obtain product display results comprises:
performing target detection on the third image data to obtain commodity category information and commodity position information in the third image data;
obtaining freezer commodity display information according to the commodity category information and the commodity position information;
calculating a display index according to the ice chest commodity display information; wherein the display metrics include: saturation and purity.
5. A camera system for freezer display identification, comprising:
the shooting module is used for receiving a cabinet door opening signal and starting the shooting device to shoot to obtain first image data;
the optimal shooting position algorithm processing module is used for processing the first image data according to an automatic analysis optimal shooting position algorithm to obtain second image data;
the deformity correction module is used for carrying out deformity correction on the second image data to obtain third image data without distortion;
the pattern recognition module is used for processing the third image data through an image recognition technology to obtain a commodity display result and finish the refrigerator display recognition;
wherein, the best shooting position algorithm processing module comprises:
the image preliminary processing submodule is used for carrying out preliminary processing on the first image data to obtain fourth image data; wherein the preliminary processing comprises: image scaling, image rotation, gray level processing and filtering processing;
the centroid point calculation submodule is used for searching the article outline images in the fourth image data and calculating the centroid point of each article outline image to obtain a coordinate set of the centroid point;
the coordinate list establishing submodule of the centroid points is used for establishing a first centroid point coordinate list and a second centroid point coordinate list according to the position relation between the centroid points of the outline images of the objects and the central line of the fourth image data; wherein the first centroid point coordinate list stores coordinates of centroid points located on a left side of a center line of the fourth image data, and the second centroid point coordinate list stores coordinates of centroid points located on a right side of the center line of the fourth image data;
the coordinate list optimization submodule of the centroid points is used for sorting the coordinates of the centroid points in the first centroid point coordinate list and the coordinates of the centroid points in the second centroid point coordinate list respectively in an ascending sorting mode according to the distance relationship between the centroid points of each article contour image and the central line of the fourth image data to obtain a third centroid point coordinate list and a fourth centroid point coordinate list; wherein the smaller the distance from the center of mass point to the center line of the fourth image data is, the more forward the coordinate sorting position of the center of mass point is;
and the judging submodule is used for judging whether the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, judging whether a first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets a preset range if the third centroid point coordinate list and the fourth centroid point coordinate list are non-empty, and taking the fourth image data as second image data if the first centroid point coordinate in the third centroid point coordinate list and the fourth centroid point coordinate list meets the preset range.
6. The camera system for ice chest display identification of claim 5, wherein said image preliminary processing sub-module comprises:
the image preprocessing submodule is used for preprocessing the first image data to obtain fifth image data; wherein the pre-processing comprises: zooming and rotating the image;
the image gray processing submodule is used for carrying out gray processing on the fifth image data and converting the fifth image data into a binary image to obtain sixth image data;
the image filtering processing submodule is used for carrying out filtering processing on the sixth image data through a filter to obtain fourth image data; wherein the filtering process includes: and removing the region of the sixth image data with the horizontal direction and the area smaller than the threshold value through a vertical direction filter and a normal filter.
7. The camera system for ice chest display identification of claim 5, wherein said deformity correction module is further configured to:
acquiring internal parameters and a distortion matrix of the shooting device;
and mapping the second image data according to the internal parameters of the shooting device and the distortion matrix to obtain third image data without distortion.
8. The camera system for ice chest display identification of claim 5, wherein said pattern recognition module comprises:
the target identification submodule is used for carrying out target detection on the third image data to obtain commodity category information and commodity position information in the third image data;
the freezer commodity display information acquisition submodule is used for acquiring freezer commodity display information according to the commodity category information and the commodity position information;
the display index calculation submodule is used for calculating a display index according to the commodity display information of the freezer; wherein the display metrics include: saturation and purity.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110427523.4A CN112990115B (en) | 2021-04-21 | 2021-04-21 | Shooting method and system for freezer display identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110427523.4A CN112990115B (en) | 2021-04-21 | 2021-04-21 | Shooting method and system for freezer display identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990115A CN112990115A (en) | 2021-06-18 |
| CN112990115B true CN112990115B (en) | 2021-09-14 |
Family
ID=76341443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110427523.4A Active CN112990115B (en) | 2021-04-21 | 2021-04-21 | Shooting method and system for freezer display identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990115B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114372993B (en) * | 2021-12-20 | 2022-10-28 | 广州市玄武无线科技股份有限公司 | Layered detection method and system for oblique-shooting shelf based on image correction |
| CN116957979A (en) * | 2023-07-29 | 2023-10-27 | 京品高科信息科技（北京）有限公司 | Goods display supervision method, container and server |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2003217534A2 (en) * | 2002-02-21 | 2003-09-09 | Promega Corporation | RF point of sale and delivery method and system using communication with remote computer and having features to read a large number of RF tags |
| KR20070053510A (en) * | 2005-11-21 | 2007-05-25 | 주식회사 씨브이네트 | System for management household goods using rfid-tag |
| CN103458219A (en) * | 2013-09-02 | 2013-12-18 | 小米科技有限责任公司 | Method, device and terminal device for adjusting face in video call |
| CN103597515A (en) * | 2011-06-06 | 2014-02-19 | 微软公司 | System for recognizing an open or closed hand |
| CN105096329A (en) * | 2015-08-20 | 2015-11-25 | 厦门雅迅网络股份有限公司 | Method for accurately correcting ultra-wide-angle lens |
| CN106052575A (en) * | 2016-08-02 | 2016-10-26 | 易讯科技股份有限公司 | Pantograph carbon slide plate wear online detection method based on train high speed running |
| CN108875831A (en) * | 2018-06-22 | 2018-11-23 | 广州图匠数据科技有限公司 | A kind of refrigerator-freezer commodity information identification method, system and device based on image recognition |
| CN109784193A (en) * | 2018-12-20 | 2019-05-21 | 上海扩博智能技术有限公司 | Terminal data acquisition device and system for intelligent refrigerator |
| CN111340845A (en) * | 2020-02-25 | 2020-06-26 | 上海黑眸智能科技有限责任公司 | Automatic tracking method, system, terminal and medium based on depth vision sensor |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090056181A1 (en) * | 2007-09-05 | 2009-03-05 | David Meyer | Device for displaying lenticular images |
| CN102930264B (en) * | 2012-09-29 | 2015-10-28 | 李炳华 | Based on commodity display information acquisition and analysis system and the method for image recognition technology |
| CN109461132B (en) * | 2018-10-31 | 2021-04-27 | 中国人民解放军国防科技大学 | SAR image automatic registration method based on feature point geometric topological relation |
| CN112651339A (en) * | 2020-12-28 | 2021-04-13 | 江苏苏豪蚕种有限公司 | Silkworm pupa gender feature identification system and identification method based on image processing |
-
2021
- 2021-04-21 CN CN202110427523.4A patent/CN112990115B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2003217534A2 (en) * | 2002-02-21 | 2003-09-09 | Promega Corporation | RF point of sale and delivery method and system using communication with remote computer and having features to read a large number of RF tags |
| KR20070053510A (en) * | 2005-11-21 | 2007-05-25 | 주식회사 씨브이네트 | System for management household goods using rfid-tag |
| CN103597515A (en) * | 2011-06-06 | 2014-02-19 | 微软公司 | System for recognizing an open or closed hand |
| CN103458219A (en) * | 2013-09-02 | 2013-12-18 | 小米科技有限责任公司 | Method, device and terminal device for adjusting face in video call |
| CN105096329A (en) * | 2015-08-20 | 2015-11-25 | 厦门雅迅网络股份有限公司 | Method for accurately correcting ultra-wide-angle lens |
| CN106052575A (en) * | 2016-08-02 | 2016-10-26 | 易讯科技股份有限公司 | Pantograph carbon slide plate wear online detection method based on train high speed running |
| CN108875831A (en) * | 2018-06-22 | 2018-11-23 | 广州图匠数据科技有限公司 | A kind of refrigerator-freezer commodity information identification method, system and device based on image recognition |
| CN109784193A (en) * | 2018-12-20 | 2019-05-21 | 上海扩博智能技术有限公司 | Terminal data acquisition device and system for intelligent refrigerator |
| CN111340845A (en) * | 2020-02-25 | 2020-06-26 | 上海黑眸智能科技有限责任公司 | Automatic tracking method, system, terminal and medium based on depth vision sensor |
Non-Patent Citations (4)
| Title |
|---|
| A method for detection ofrandomly placed objects for robotic handling;Panagiota Tsarouchi 等;《CIRP Journal of Manufacturing Science and Technology》;20160423;20-27 * |
| Image Recognition For Product And Shelf Monitoring And Analysis;Amit Jnagal;《https://www.infrrd.ai/blog/image-recognition-for-product-and-shelf-monitoring-and-analysis》;20181212;1 * |
| 基于深度学习的商品检测和识别研究;张文强;《中国优秀硕士学位论文全文数据库 信息科技辑》;20200115(第01期);I138-1796 * |
| 无人售货柜营销场景下的人机信息传播研究;高鑫;《中国优秀硕士学位论文全文数据库 信息科技辑》;20190915(第09期);I141-44 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990115A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109188457B (en) | Object detection frame generation method, device, equipment, storage medium and vehicle | |
| CN112990115B (en) | Shooting method and system for freezer display identification | |
| CN110033386B (en) | Vehicle accident identification method and device and electronic equipment | |
| CN110363076B (en) | Personnel information association method and device and terminal equipment | |
| CN110793607B (en) | Self-service weighing method, system and computer readable storage medium | |
| CN108985199A (en) | Detection method, device and the storage medium of commodity loading or unloading operation | |
| CN103500517B (en) | A kind of parking position detection method and device | |
| CN110660102B (en) | Speaker recognition method, device and system based on artificial intelligence | |
| JP2004354257A (en) | Calibration slippage correction device, and stereo camera and stereo camera system equipped with the device | |
| CN103119627A (en) | An adaptable framework for cloud assisted augmented reality | |
| CN112100425B (en) | Label labeling method and device based on artificial intelligence, electronic equipment and medium | |
| EP3410353A1 (en) | Method for estimating a timestamp in a video stream and method of augmenting a video stream with information | |
| CN112348686B (en) | Claim settlement picture acquisition method and device and communication equipment | |
| CN109711427A (en) | Object detection method and Related product | |
| CN114898249B (en) | Method, system and storage medium for confirming number of articles in shopping cart | |
| CN112330715A (en) | Tracking method, tracking device, terminal equipment and readable storage medium | |
| CN111008954A (en) | Information processing method and device, electronic equipment and storage medium | |
| CN107644332B (en) | Method, device and system for detecting whether user steals articles and intelligent equipment | |
| CN111428743B (en) | Commodity identification method, commodity processing device and electronic equipment | |
| CN112106113A (en) | Method and device for determining pose information of image in three-dimensional reconstruction | |
| CN111126288B (en) | Target object attention calculation method, target object attention calculation device, storage medium and server | |
| CN110084187B (en) | Position identification method, device, equipment and storage medium based on computer vision | |
| CN112308915A (en) | Method and device for positioning express package | |
| CN114140751B (en) | Examination room monitoring method and system | |
| CN112380938B (en) | Face recognition and temperature measurement method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: 510000 room 23bd, No. 109, TIYU West Road, Tianhe District, Guangzhou City, Guangdong Province Patentee after: GUANGZHOU XUANWU WIRELESS TECHNOLOGY Co.,Ltd. Address before: 32B, no.103b, TianLiHe Road, Guangzhou, 510000 Patentee before: GUANGZHOU XUANWU WIRELESS TECHNOLOGY Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |