CN108447061B - Commodity information processing method and device, computer equipment and storage medium - Google Patents
Commodity information processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN108447061B CN108447061B CN201810097478.9A CN201810097478A CN108447061B CN 108447061 B CN108447061 B CN 108447061B CN 201810097478 A CN201810097478 A CN 201810097478A CN 108447061 B CN108447061 B CN 108447061B
- Authority
- CN
- China
- Prior art keywords
- commodity
- neural network
- deep learning
- network model
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a commodity information processing method and device, computer equipment and a storage medium. Acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified; inputting the color depth picture into a trained deep learning neural network model, and identifying to obtain a commodity category and a commodity position corresponding to the commodity to be identified; and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position. And identifying the color picture containing the depth information through a deep learning neural network to obtain more accurate commodity category and commodity position, and determining commodity value information through the commodity category and the commodity position.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing commodity information, a computer device, and a storage medium.
Background
With the development of computer technology, the application based on computer technology is more and more extensive. In a supermarket or a warehouse, when the commodity is sold and settled and stored, commodity information needs to be recorded. Which is the need to determine the category and value of the goods when they are sold. With the continuous development of computer technology, the commodity identification technology is continuously intelligent. In the traditional market settlement, goods are identified by scanning codes, on one hand, a lot of manpower is needed, and meanwhile, the condition that the bar codes are lost or unavailable exists, so that the identification speed is slow.
Disclosure of Invention
In view of the above, it is necessary to provide a commodity information processing method, apparatus, computer device and storage medium for quickly identifying commodity information corresponding to a commodity in a color depth picture through a deep learning neural network model.
A merchandise information processing method, the method comprising: acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified; inputting the color depth picture into a trained deep learning neural network model, and identifying to obtain a commodity category and a commodity position corresponding to the commodity to be identified; and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
In one embodiment, after the step of obtaining a color depth picture composed of a color picture and a depth picture corresponding to a commodity to be identified, the method further includes: when the condition that the ratio of a commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value is detected, detecting a human body limb position area in the color depth picture; determining a first target area according to the human body limb position area; and determining a second target area according to the first target area, wherein the area of the second target area is smaller than that of the first target area.
In one embodiment, the deep learning neural network model includes a convolutional layer, and the step of inputting the color depth picture into the trained deep learning neural network model to identify the commodity category and the commodity position corresponding to the commodity includes: acquiring a standard commodity suggestion frame, and inputting the commodity suggestion frame into the deep learning neural network model; acquiring the normalized color depth picture, and performing downsampling on the normalized color depth picture through the convolution layer to obtain a downsampled convolution characteristic diagram; performing sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to the color depth image to obtain the depth sampling image; and carrying out area division on the depth sampling image according to the commodity standard suggestion frame to obtain a corresponding segmentation area, and identifying the segmentation area to obtain a corresponding commodity type and a corresponding commodity position.
In one embodiment, the step of generating the trained deep learning neural network model includes: acquiring a standard commodity picture set and a standard commodity label set, wherein the label set comprises commodity category information and position information, the standard commodity picture set is divided into a training data set and a test data set, and the standard commodity picture is a color depth picture containing depth information; training the training data set to obtain a trained deep learning neural network model; testing the trained deep learning neural network model by adopting the test data set to obtain an identification result set; determining a test identification rate according to the identification result set and the standard commodity label set; and when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
In one embodiment, the deep learning neural network model includes parameters, and the step of obtaining the trained deep learning neural network model by training the training data set includes: updating parameters of the deep learning neural network model according to each standard commodity picture in the training data set; and when the deep learning neural network has learned all the standard commodity pictures in the training data set and stops updating the parameters of the deep learning neural network model, obtaining the trained deep learning neural network model.
An article information processing apparatus, the apparatus comprising:
the data acquisition module is used for acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to the commodity to be identified;
the commodity identification module is used for inputting the color depth pictures into the trained deep learning neural network model and identifying to obtain commodity categories and commodity positions corresponding to the commodities;
and the commodity value information module is used for determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
In one embodiment, a commodity information processing apparatus includes:
the limb position detection module is used for detecting a human body limb position area in the color depth picture when detecting that the ratio of a commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value;
the first target area determining module is used for determining a first target area according to the human body limb position area;
and the second target area determining module is used for determining a second target area according to the first target area, wherein the area of the second target area is smaller than that of the first target area.
In one embodiment, the article identification module comprises:
a commodity suggestion frame acquisition unit, configured to acquire a standard commodity suggestion frame and input the commodity suggestion frame into the deep learning neural network model;
the convolution feature map obtaining unit is used for obtaining the normalized color depth picture, and performing downsampling on the normalized color depth picture through the convolution layer to obtain a downsampled convolution feature map;
the depth sampling image acquisition unit is used for performing sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to the depth image to obtain the depth sampling image;
and the commodity identification unit is used for carrying out area division on the depth sampling image according to the commodity standard suggestion frame to obtain a corresponding segmentation area, and identifying the segmentation area to obtain a corresponding commodity type and a corresponding commodity position.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the merchandise information processing method when executing the computer program.
A computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described article information processing method.
According to the commodity information processing method, the commodity information processing device, the computer equipment and the storage medium, the color depth picture which is composed of the color picture and the depth picture and corresponds to the commodity to be identified is obtained; inputting the color depth picture into a trained deep learning neural network model, and identifying to obtain a commodity category and a commodity position corresponding to the commodity to be identified; and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position. And identifying the color depth picture by adopting the trained deep learning neural network to obtain the corresponding commodity category and commodity position in the commodity picture, so that the identification accuracy is improved, and the commodity value information corresponding to the commodity category is more accurate.
Drawings
FIG. 1 is a diagram of an exemplary embodiment of a method for processing merchandise information;
FIG. 2 is a flowchart illustrating a method for processing merchandise information according to an embodiment;
FIG. 3 is a flowchart illustrating a merchandise information processing method according to another embodiment;
FIG. 4 is a diagram of a scenario illustrating positioning of an item in one embodiment;
FIG. 5 is a flowchart illustrating an article identification step according to another embodiment;
FIG. 6 is a schematic flow chart diagram illustrating the steps for generating a neural network in one embodiment;
FIG. 7 is a schematic flow chart diagram illustrating the steps for training a neural network in one embodiment;
FIG. 8 is a block diagram showing the structure of an article recognition apparatus according to an embodiment;
fig. 9 is a block diagram showing the construction of an article recognition apparatus according to another embodiment;
FIG. 10 is a block diagram showing the structure of a product identifying module according to one embodiment;
fig. 11 is a block diagram showing the structure of an article recognition apparatus in still another embodiment;
FIG. 12 is a block diagram of the structure of a training module in one embodiment;
FIG. 13 is a block diagram showing an internal configuration of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The commodity information processing method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 acquires a color depth picture which is composed of a color picture and a depth picture and corresponds to the commodity to be recognized, the color depth picture is recognized through a trained deep learning neural network model, the commodity category and the commodity position which correspond to the commodity to be recognized are obtained, and the commodity value information of the commodity is determined according to the commodity category and the commodity position which are obtained through recognition. The server 104 may also receive a color depth picture corresponding to the to-be-identified commodity sent by the terminal, identify the color depth picture through the deep learning neural network model in the server 104 to obtain a commodity category and a commodity position corresponding to the to-be-identified commodity, and determine commodity value information of the commodity according to the commodity category and the commodity position obtained through identification. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in fig. 2, a method for processing commodity information is provided, which is described by taking the method as an example applied to the terminal in fig. 1, and includes the following steps:
step S202, a color depth picture which is composed of a color picture and a depth picture and corresponds to the commodity to be identified is obtained.
Specifically, a color depth picture refers to an image including a color image and a depth image, and the depth image refers to an image in which the distance (depth) from an image pickup to each point in a scene is taken as a pixel value, and the geometric shape of a visible surface of a scene is directly reflected by distance information. The method comprises the steps of obtaining a color Depth picture shot by a shooting device, wherein the color Depth picture (Red Green Blue-Depth, RGB-D) comprises a commodity area, the commodity area is used for describing commodity features of a commodity to be identified, and the commodity features comprise features such as image shapes and textures.
And step S204, inputting the color depth picture into the trained deep learning neural network model, and identifying to obtain the commodity category and the commodity position corresponding to the commodity to be identified.
The trained deep learning neural network model is obtained by learning and training a large number of color deep pictures carrying commodity information. The characteristics of various commodities can be learned by learning a large number of color depth pictures. The trained deep learning neural network model can accurately and quickly extract the features in the image.
Specifically, the acquired color depth picture corresponding to the commodity to be identified is input into a trained deep learning neural network model, commodity features of the color depth picture are extracted through the network, the commodity category is determined according to the commodity features, and the commodity position is determined according to the corresponding relation between the commodity category and the commodity position.
In one embodiment, the color depth picture corresponding to the commodity to be recognized is subjected to multiple segmentation according to a preset segmentation rule, the segmented images are recognized to obtain corresponding recognition results for segmentation areas corresponding to multiple segmentation modes, the best recognition result is determined from the recognition results to serve as a target recognition result, and the commodity category and the commodity position are determined according to the target recognition result.
And step S206, determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
Specifically, the commodity value information may be information indicating the value of the commodity, such as the selling price of the commodity, the production cost, and the like. Different commodity types and commodity positions correspond to different commodity value information, and after the commodity types and the commodity positions are determined, the commodity value information is determined according to the corresponding relation between the commodity types and the commodity positions and the commodity value information.
According to the commodity information processing method, the obtained color depth picture corresponding to the commodity to be identified is input into the trained deep learning neural network model, the color depth picture is identified through the trained deep learning neural network model, the commodity type and the commodity position are obtained, and commodity value information is determined according to the commodity type and the commodity position. The color depth picture comprises depth information and color information, the commodity described jointly according to the depth information and the color information is more accurate, the deep learning neural network model can extract more accurate commodity characteristics, the commodity category obtained through rich and accurate information and rapid and accurate network model identification is more accurate, and the commodity value information is more accurate according to more accurate commodity category and commodity position.
As shown in fig. 3, in an embodiment, after step S202, the method further includes:
step S208, when the ratio of the commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value, detecting the human body limb position area in the color depth picture.
Specifically, the acquired color depth picture corresponding to the to-be-identified image is preprocessed, and the ratio of the commodity area in the shot image to the whole image is inconsistent due to inconsistency of the shooting angle, the shooting distance and the like, so that positioning processing is performed before the color depth picture is identified. Calculating the ratio of the commodity area in the obtained color depth picture in the whole image, and detecting the human body limb in the color depth picture when the ratio of the commodity area obtained by calculation is smaller than a preset threshold value, thereby determining the position area of the human body limb.
Step S210, a first target area is determined according to the position area of the human body limb.
Step S212, a second target region is determined according to the first target region, and the region area of the second target region is smaller than that of the first target region.
Specifically, the first target region is an initial positioning region having a region area larger than the area of the commodity region, which includes the commodity region. And determining a first target area according to the position area of the limb of the human body, and determining a second target area according to the first target area. The second target region is a relocation region including the commodity region, and the region area of the second target region is smaller than the region area of the first target region. The commodity is positioned through the position of the limbs of the human body, and positioning errors are reduced.
In one embodiment, the area after the first positioning may be enlarged, and the second target area may be determined by the first target area after the enlargement.
As shown in fig. 4, the first target area is an area a, the second target area is an area b, and the product area is an area c. The area of the region a is larger than that of the region b, and the area of the region c is smaller than that of the region b.
As shown in fig. 5, in one embodiment, step S204 includes:
step S2042, a standard commodity suggestion frame is obtained, and the commodity suggestion frame is input into the deep learning neural network model.
Specifically, at least one preset standard commodity suggestion frame is obtained, and the standard commodity suggestion frame is input into the deep learning neural network model. The standard commodity suggestion box is a frame with a plurality of different aspect ratios and is set by a user. The standard commodity suggestion frame can determine the aspect ratio of the standard commodity suggestion frame according to the resolution of the depth image, and can also determine the aspect ratio according to the resolution of the sampling image.
Step S2044, acquiring the normalized color depth image, and performing downsampling on the normalized color depth image through the convolution layer to obtain a downsampled convolution feature map.
Specifically, normalization refers to processing all color depth pictures to the same resolution. And performing downsampling on the color depth picture through the convolution layer, namely performing feature extraction on the color depth picture through convolution operation to obtain a convolution feature image. For example, the images are all adjusted to a resolution of 416 × 416, and then down-sampled by 32 times by the convolutional layer, resulting in a convolutional feature map of 13 × 13 size.
Step S2046, performing sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to a color depth picture to obtain a depth sampling image.
Specifically, sliding window sampling refers to setting a window, and sampling is performed through the sliding window on an image. And setting a sliding window, and sliding the sliding window on the convolution characteristic diagram for sampling to obtain a sampled image after sampling. Mapping the sampling image into a color depth picture to obtain a depth sampling image
And step S2048, performing area division on the depth sampling image according to the commodity standard suggestion frame to obtain corresponding divided areas, and identifying the divided areas to obtain corresponding commodity categories and commodity positions.
Specifically, a trained deep learning neural network model is used for identifying a commodity area which is obtained by segmenting each standard commodity suggestion frame in a deep sampling image and corresponds to each standard commodity suggestion frame to obtain a commodity identification result corresponding to each standard commodity suggestion frame, one standard commodity suggestion frame is selected as a target commodity suggestion frame from a plurality of commodity identification results corresponding to the standard commodity suggestion frames according to a custom algorithm, the identification result corresponding to the target trademark suggestion frame is used as a final commodity category, and a commodity position is determined according to the final commodity category. The recognition results of the commodity areas can be recognized and obtained by arranging the standard commodity suggestion frames, the optimal recognition result is selected from the recognition results to serve as the final recognition result, and the recognition accuracy can be improved.
In one embodiment, the commodity category and the commodity position are determined according to the identification probability corresponding to the commodity identification result corresponding to each standard commodity suggestion frame. And if so, selecting the commodity identification result corresponding to the standard commodity suggestion frame with the highest identification probability as the identified commodity category.
As shown in fig. 6, in an embodiment, the commodity information processing method further includes:
step S214, a standard commodity picture set and a standard commodity label set are obtained, the standard commodity picture set is divided into a training data set and a testing data set, and the standard commodity picture is a color depth picture containing depth information.
Specifically, the standard commercial picture set is a set composed of a plurality of color depth pictures containing depth information. The standard commodity picture set comprises a picture data set crawled from the network, and can also be an image data set obtained in at least one mode of the picture data set obtained directly from the shooting device. And dividing the standard commodity picture set into a training data set and a test data set, wherein the training data set is used for training the deep learning neural network model, and the test data set is used for testing the trained deep learning neural network model.
And step S216, training the training data set to obtain a trained deep learning neural network model.
Specifically, the training data set is input into the deep learning neural network model, and the network model automatically trains each picture in the training data set to obtain the trained deep learning neural network model.
And step S218, testing the trained deep learning neural network model by using the test data set to obtain a recognition result set.
Specifically, the trained deep learning neural network model is tested by adopting a test data set, namely, each picture of the test data set is input into the trained deep learning neural network model, identification results corresponding to each picture are obtained by identifying through the model, and the identification results corresponding to each picture form an identification result set.
And step S220, determining a test identification rate according to the identification result set and the standard commodity label set.
Specifically, whether the identification result is correct or not is judged according to the corresponding relation between the standard commodity label and the picture, and the picture number of the total test data set is closed to the picture number of the correct identification result to obtain the test identification rate. And the test recognition rate is used for measuring the recognition capability of the trained deep learning neural network model.
Step S222, when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
Specifically, if the obtained test recognition rate reaches a preset threshold value, it indicates that the recognition capability of the trained deep learning neural network model meets an expected result, and the trained deep learning neural network model is directly used as the trained deep learning neural network model. If the obtained test recognition rate does not reach the preset threshold value, the recognition capability of the trained deep learning neural network model is not in line with the expected result, the parameters of the deep learning neural network model need to be adjusted, and the network model is trained again.
The neural network model obtained by learning a large number of color depth pictures with labels can quickly and accurately extract the feature set in the pictures, and the commodity category determined according to the feature set is accurate.
As shown in fig. 7, in one embodiment, step S216 includes:
step S2162, according to each standard commodity picture in the training data set, the parameters of the deep learning neural network model are updated.
Specifically, in the process of training the deep learning neural network, the features of each standard commodity picture in the training data set are learned, and in the learning process, because the picture contents of different pictures are not completely consistent, when the features are extracted, the features extracted from different pictures are not consistent, and the extracted features need to be weighted, so that the final recognition result meets the expected result. Therefore, the feature weights are continuously adjusted in the learning process, that is, the parameters of the deep learning neural network model are updated.
Step S2164, when the deep learning neural network has learned each standard commodity picture in the training data set and stops updating the parameters of the deep learning neural network model, the trained deep learning neural network model is obtained.
Specifically, when the deep learning neural network has learned all the standard commodity pictures in the training data set, the parameters of the deep learning neural network model are not updated any more, and the parameters of the deep learning neural network model are determined to indicate that the training is completed, so that the trained deep learning neural network model is obtained. The parameter adjustment of the deep learning neural network is used for better identifying the commodities to be identified in the color depth pictures corresponding to the commodities to be identified and shot in various scenes, and the identification accuracy of the deep learning neural network model is improved.
As shown in fig. 8, in one embodiment, a commodity information processing apparatus 200 includes:
and the data acquisition module 202 is configured to acquire a color depth picture composed of a color picture and a depth picture corresponding to the commodity to be identified.
And the commodity identification module 204 is used for inputting the color depth pictures into the trained deep learning neural network model, and identifying to obtain the commodity category and the commodity position corresponding to the commodity.
And the commodity value information module 206 is used for determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
As shown in fig. 9, in one embodiment, the commodity information processing apparatus 200 includes:
the limb position detection module 208 is configured to detect a human limb position region in the color depth picture when it is detected that a ratio of a commodity region included in the color depth picture to the color depth picture is smaller than a preset threshold.
The first target area determining module 210 determines a first target area according to the body limb position area.
A second target region determining module 212, configured to determine a second target region according to the first target region, where a region area of the second target region is smaller than a region area of the first target region.
As shown in FIG. 10, in one embodiment, the item identification module 204 includes:
the commodity suggestion frame obtaining unit 2042 is configured to obtain a standard commodity suggestion frame, and input the commodity suggestion frame into the deep learning neural network model.
The convolution feature map obtaining unit 2044 is configured to obtain the normalized color depth image, and perform downsampling on the normalized color depth image through the convolution layer to obtain a downsampled convolution feature map.
And the depth sampling image obtaining unit 2046 is configured to perform sliding window sampling on the convolution feature map to obtain a sampling image, and map the sampling image to the depth map to obtain a depth sampling image.
The commodity identification unit 2048 is configured to perform area division on the depth sampling image according to the commodity standard suggestion frame to obtain corresponding divided areas, and identify the divided areas to obtain corresponding commodity categories and commodity positions.
As shown in fig. 11, in one embodiment, the product information processing apparatus 200 further includes:
the image set acquisition module 214 is configured to acquire a standard commodity image set and a standard commodity label set in a standard manner, where the label set includes commodity category information and position information, and the standard commodity image set is divided into a training data set and a test data set, and the standard commodity image is a color depth image including depth information.
And the training module 216 is configured to obtain a trained deep learning neural network model by training the training data set.
And the testing module 218 is configured to test the trained deep learning neural network model by using a test data set to obtain a recognition result set.
And the identification rate calculating module 220 is used for determining the test identification rate according to the identification result set and the standard commodity label set.
And the model determining module 222 is configured to, when the test recognition rate of the test data set reaches a preset threshold, take the trained deep learning neural network model as the trained deep learning neural network model.
As shown in FIG. 12, in one embodiment, training module 216 includes:
a parameter updating unit 2162, configured to update the parameters of the deep learning neural network model according to each standard commodity picture in the training data set.
The training unit 2164 is used for obtaining the trained deep learning neural network model when the deep learning neural network has learned each standard commodity picture in the training data set and stops updating the parameters of the deep learning neural network model.
For specific limitations of the product information processing device, reference may be made to the above limitations of the product information processing method, which are not described herein again. Each module in the above-described commodity information processing apparatus may be entirely or partially implemented by software, hardware, or a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 13. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a commodity information processing method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 13 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified; inputting the color depth picture into the trained deep learning neural network model, and identifying to obtain the commodity category and the commodity position corresponding to the commodity to be identified; and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
In one embodiment, the processor, when executing the computer program, further performs the steps of: when the ratio of the commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value, detecting a human body limb position area in the color depth picture; determining a first target area according to the position area of the limb of the human body; and determining a second target region according to the first target region, wherein the region area of the second target region is smaller than that of the first target region.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring a standard commodity suggestion frame, and inputting the commodity suggestion frame into a deep learning neural network model; acquiring a normalized color depth picture, and performing downsampling on the normalized color depth picture through a convolution layer to obtain a downsampled convolution characteristic diagram; carrying out sliding window sampling on the convolution characteristic graph to obtain a sampling image, and mapping the sampling image to a depth map to obtain a depth sampling image; and carrying out area division on the depth sampling image according to the commodity standard suggestion frame to obtain a corresponding segmentation area, and identifying the segmentation area to obtain a corresponding commodity type and a corresponding commodity position.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring a standard commodity picture set and a standard commodity label set, wherein the label set comprises commodity category information and position information, dividing the standard commodity picture set into a training data set and a test data set, and the standard commodity picture is a color depth picture containing depth information; training a training data set to obtain a trained deep learning neural network model; testing the trained deep learning neural network model by adopting a test data set to obtain an identification result set; determining a test identification rate according to the identification result set and the standard commodity label set; and when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
In one embodiment, the processor, when executing the computer program, further performs the steps of: updating parameters of the deep learning neural network model according to each standard commodity picture in the training data set; and when the deep learning neural network has learned all the standard commodity pictures in the training data set and stops updating the parameters of the deep learning neural network model, obtaining the trained deep learning neural network model.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified; inputting the color depth picture into the trained deep learning neural network model, and identifying to obtain the commodity category and the commodity position corresponding to the commodity to be identified; and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
In one embodiment, the computer program when executed by the processor further performs the steps of: when the ratio of the commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value, detecting a human body limb position area in the color depth picture; determining a first target area according to the position area of the limb of the human body; and determining a second target region according to the first target region, wherein the region area of the second target region is smaller than that of the first target region.
In one embodiment, the computer program when executed by the processor further performs the steps of: the deep learning neural network model comprises a convolution layer, color deep pictures are input into the trained deep learning neural network model, and the commodity category and the commodity position corresponding to the commodity are identified and obtained, and the deep learning neural network model comprises the following steps: acquiring a standard commodity suggestion frame, and inputting the commodity suggestion frame into a deep learning neural network model; acquiring a normalized color depth picture, and performing downsampling on the normalized color depth picture through a convolution layer to obtain a downsampled convolution characteristic diagram; carrying out sliding window sampling on the convolution characteristic graph to obtain a sampling image, and mapping the sampling image to a depth map to obtain a depth sampling image; and carrying out area division on the depth sampling image according to the commodity standard suggestion frame to obtain a corresponding segmentation area, and identifying the segmentation area to obtain a corresponding commodity type and a corresponding commodity position.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a standard commodity picture set and a standard commodity label set, and dividing the standard commodity picture set into a training data set and a testing data set, wherein the standard commodity picture is a color depth picture containing depth information; training a training data set to obtain a trained deep learning neural network model; testing the trained deep learning neural network model by adopting a test data set to obtain an identification result set; determining a test identification rate according to the identification result set and the standard commodity label set; and when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
In one embodiment, the computer program when executed by the processor further performs the steps of: updating parameters of the deep learning neural network model according to each standard commodity picture in the training data set; and when the deep learning neural network has learned all the standard commodity pictures in the training data set and stops updating the parameters of the deep learning neural network model, obtaining the trained deep learning neural network model.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (8)
1. A merchandise information processing method, the method comprising:
acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified;
when detecting that the ratio of a commodity region contained in the color depth picture to the color depth picture is smaller than a preset threshold value, detecting a human body limb position region in the color depth picture, determining a first target region according to the human body limb position region, and determining a second target region according to the first target region, wherein the region area of the second target region is smaller than that of the first target region, and the second target region is a re-positioning region containing the commodity region;
inputting the color depth picture into a trained deep learning neural network model, wherein the trained deep learning neural network model is obtained by learning and training a color depth training picture carrying commodity information, the deep learning neural network model comprises a convolutional layer, and commodity categories and commodity positions corresponding to commodities to be recognized are obtained by recognition, and the method comprises the following steps: acquiring at least one preset standard commodity suggestion frame, and inputting the standard commodity suggestion frame into the deep learning neural network model, wherein the standard commodity suggestion frame is a self-defined frame with different aspect ratios; acquiring the normalized color depth picture, and performing downsampling on the normalized color depth picture through the convolution layer to obtain a downsampled convolution characteristic diagram; performing sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to the color depth image to obtain a depth sampling image; carrying out region division on the depth sampling image according to the standard commodity suggestion frame to obtain corresponding segmentation regions, and identifying the segmentation regions to obtain corresponding commodity types and commodity positions;
and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
2. The method of claim 1, wherein the trained deep learning neural network model comprises the steps of:
acquiring a standard commodity picture set and a standard commodity label set, wherein the label set comprises commodity category information and position information, the standard commodity picture set is divided into a training data set and a test data set, and the standard commodity picture is a color depth picture containing depth information;
training the training data set to obtain a trained deep learning neural network model;
testing the trained deep learning neural network model by adopting the test data set to obtain an identification result set;
determining a test identification rate according to the identification result set and the standard commodity label set;
and when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
3. The method of claim 2, wherein the deep learning neural network model comprises parameters, and the step of obtaining the trained deep learning neural network model by training the training data set comprises:
updating parameters of the deep learning neural network model according to each standard commodity picture in the training data set;
and when the deep learning neural network has learned all the standard commodity pictures in the training data set and stops updating the parameters of the deep learning neural network model, obtaining the trained deep learning neural network model.
4. An article information processing apparatus characterized by comprising:
the data acquisition module is used for acquiring a color depth picture corresponding to a commodity to be identified;
the limb position detection module is used for detecting a human body limb position area in the color depth picture when the ratio of a commodity area contained in the color depth picture to the color depth picture is smaller than a preset threshold value;
the first target area determining module is used for determining a first target area according to the limb position;
a second target region determination module configured to determine the second target region based on the first target region, where a region area of the second target region is smaller than a region area of the first target region, and the second target region is a relocation region including the commodity region;
the commodity identification module is used for inputting the color depth pictures into a trained deep learning neural network model, the trained deep learning neural network model is obtained by learning and training color depth training pictures carrying commodity information, and commodity categories and commodity positions corresponding to commodities are obtained through identification; the convolution feature map acquisition unit is used for acquiring the normalized color depth picture, and performing down-sampling on the normalized color depth picture through a convolution layer to obtain a down-sampled convolution feature map; the depth sampling image acquisition unit is used for carrying out sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to the depth image to obtain a depth sampling image; the commodity identification unit is used for carrying out area division on the depth sampling image according to the standard commodity suggestion frame to obtain a corresponding segmentation area, and identifying the segmentation area to obtain a corresponding commodity type and a corresponding commodity position;
and the commodity value information module is used for determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
5. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of:
acquiring a color depth picture which is composed of a color picture and a depth picture and corresponds to a commodity to be identified;
when detecting that the ratio of a commodity region contained in the color depth picture to the color depth picture is smaller than a preset threshold value, detecting a human body limb position region in the color depth picture, determining a first target region according to the human body limb position region, and determining a second target region according to the first target region, wherein the region area of the second target region is smaller than that of the first target region, and the second target region is a re-positioning region containing the commodity region;
inputting the color depth picture into a trained deep learning neural network model, wherein the trained deep learning neural network model is obtained by learning and training a color depth training picture carrying commodity information, the deep learning neural network model comprises a convolutional layer, and commodity categories and commodity positions corresponding to commodities to be recognized are obtained by recognition, and the method comprises the following steps: acquiring at least one preset standard commodity suggestion frame, and inputting the standard commodity suggestion frame into the deep learning neural network model, wherein the standard commodity suggestion frame is a self-defined frame with different aspect ratios; acquiring the normalized color depth picture, and performing downsampling on the normalized color depth picture through the convolution layer to obtain a downsampled convolution characteristic diagram; performing sliding window sampling on the convolution characteristic image to obtain a sampling image, and mapping the sampling image to the color depth image to obtain a depth sampling image; carrying out region division on the depth sampling image according to the standard commodity suggestion frame to obtain corresponding segmentation regions, and identifying the segmentation regions to obtain corresponding commodity types and commodity positions;
and determining the commodity value information of the commodity to be identified according to the commodity category and the commodity position.
6. The computer device of claim 5, wherein the trained deep learning neural network model comprises the steps of:
acquiring a standard commodity picture set and a standard commodity label set, wherein the label set comprises commodity category information and position information, the standard commodity picture set is divided into a training data set and a test data set, and the standard commodity picture is a color depth picture containing depth information;
training the training data set to obtain a trained deep learning neural network model;
testing the trained deep learning neural network model by adopting the test data set to obtain an identification result set;
determining a test identification rate according to the identification result set and the standard commodity label set;
and when the test recognition rate of the test data set reaches a preset threshold value, taking the trained deep learning neural network model as the trained deep learning neural network model.
7. The computer device of claim 6, wherein the deep-learning neural network model comprises parameters, and wherein the training of the training data set to obtain the trained deep-learning neural network model comprises:
updating parameters of the deep learning neural network model according to each standard commodity picture in the training data set;
and when the deep learning neural network has learned all the standard commodity pictures in the training data set and stops updating the parameters of the deep learning neural network model, obtaining the trained deep learning neural network model.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097478.9A CN108447061B (en) | 2018-01-31 | 2018-01-31 | Commodity information processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097478.9A CN108447061B (en) | 2018-01-31 | 2018-01-31 | Commodity information processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108447061A CN108447061A (en) | 2018-08-24 |
| CN108447061B true CN108447061B (en) | 2020-12-08 |
Family
ID=63191535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810097478.9A Expired - Fee Related CN108447061B (en) | 2018-01-31 | 2018-01-31 | Commodity information processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108447061B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109472205B (en) * | 2018-10-09 | 2021-07-30 | 深兰科技（上海）有限公司 | Commodity identification method, commodity identification device, and storage medium |
| CN111126110B (en) * | 2018-10-31 | 2024-01-05 | 杭州海康威视数字技术股份有限公司 | Commodity information identification method, settlement method, device and unmanned retail system |
| CN109784385A (en) * | 2018-12-29 | 2019-05-21 | 广州海昇计算机科技有限公司 | A kind of commodity automatic identifying method, system, device and storage medium |
| CN110210334B (en) * | 2019-05-15 | 2022-01-04 | 广州影子科技有限公司 | Pig spot check method and device, pig spot check system and computer storage medium |
| CN110689005A (en) * | 2019-09-05 | 2020-01-14 | 上海零眸智能科技有限公司 | Commodity identification method based on deep learning fusion position and shape information |
| CN110991372A (en) * | 2019-12-09 | 2020-04-10 | 河南中烟工业有限责任公司 | Method for identifying cigarette brand display condition of retail merchant |
| CN111191551B (en) * | 2019-12-23 | 2023-07-14 | 达闼机器人股份有限公司 | Commodity detection method and device |
| CN111310706B (en) * | 2020-02-28 | 2022-10-21 | 创新奇智(上海)科技有限公司 | Commodity price tag identification method and device, electronic equipment and storage medium |
| CN112653900A (en) * | 2020-12-21 | 2021-04-13 | Oppo广东移动通信有限公司 | Target display method, device and equipment in video live broadcast |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825494B (en) * | 2015-08-31 | 2019-01-29 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
-
2018
- 2018-01-31 CN CN201810097478.9A patent/CN108447061B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN108447061A (en) | 2018-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108447061B (en) | Commodity information processing method and device, computer equipment and storage medium | |
| US10410292B2 (en) | Method, system, apparatus, and storage medium for realizing antifraud in insurance claim based on consistency of multiple images | |
| US11244435B2 (en) | Method and apparatus for generating vehicle damage information | |
| CN110852285B (en) | Object detection method and device, computer equipment and storage medium | |
| CN111340126B (en) | Article identification method, apparatus, computer device, and storage medium | |
| CN111259889A (en) | Image text recognition method and device, computer equipment and computer storage medium | |
| CN109376631B (en) | Loop detection method and device based on neural network | |
| CN109670437B (en) | Age estimation model training method, facial image recognition method and device | |
| CN111656358A (en) | Analyzing captured images to determine test outcomes | |
| CN110991261A (en) | Interactive behavior recognition method and device, computer equipment and storage medium | |
| CN111144398A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| CN111429482A (en) | Target tracking method and device, computer equipment and storage medium | |
| CN111144372A (en) | Vehicle detection method, device, computer equipment and storage medium | |
| CN111832561B (en) | Character sequence recognition method, device, equipment and medium based on computer vision | |
| CN112183307A (en) | Text recognition method, computer device, and storage medium | |
| CN113780145A (en) | Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium | |
| CN113298078A (en) | Equipment screen fragmentation detection model training method and equipment screen fragmentation detection method | |
| CN112307989A (en) | Method and device for identifying road surface object, computer equipment and storage medium | |
| CN116964588A (en) | Target detection method, target detection model training method and device | |
| CN114821551A (en) | Method, apparatus and storage medium for legacy detection and model training | |
| CN111652245B (en) | Vehicle contour detection method, device, computer equipment and storage medium | |
| CN112651315A (en) | Information extraction method and device of line graph, computer equipment and storage medium | |
| CN115775386A (en) | User interface component identification method and device, computer equipment and storage medium | |
| CN111428553B (en) | Face pigment spot recognition method and device, computer equipment and storage medium | |
| CN112950568A (en) | Scale length calculation method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201208 Termination date: 20220131 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |