WO2020107951A1 - 一种基于图像的商品结算方法、装置、介质及电子设备 - Google Patents

一种基于图像的商品结算方法、装置、介质及电子设备 Download PDF

Info

Publication number
WO2020107951A1
WO2020107951A1 PCT/CN2019/101113 CN2019101113W WO2020107951A1 WO 2020107951 A1 WO2020107951 A1 WO 2020107951A1 CN 2019101113 W CN2019101113 W CN 2019101113W WO 2020107951 A1 WO2020107951 A1 WO 2020107951A1
Authority
WO
WIPO (PCT)
Prior art keywords
commodity
image
settled
feature vector
present disclosure
Prior art date
Application number
PCT/CN2019/101113
Other languages
English (en)
French (fr)
Inventor
梅涛
吴楠
赵何
刘武
徐迎庆
张雷
周伯文
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Publication of WO2020107951A1 publication Critical patent/WO2020107951A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0283—Price estimation or determination

Definitions

  • the present disclosure relates to the technical field of commodity identification and settlement, in particular, to an image-based commodity settlement method and an image-based commodity settlement device.
  • the current commodity settlement methods mainly include the following methods: commodity identification technology based on barcode recognition, commodity identification technology based on radio frequency identification and image recognition technology.
  • These technologies have the following shortcomings: in the retail settlement scenario based on the barcode recognition method, complex operations of the settlement personnel are required, first of all, it is necessary to find the position of the barcode on the product packaging, handheld barcode recognition equipment, device scanner and barcode alignment scanning, etc., and It needs to scan multiple products one by one, which consumes a lot of manpower and prolongs the settlement time; because the identification method based on radio frequency needs to install RFID electronic tags for the products in advance, the high deployment and maintenance costs of RFID electronic tags are very high, and it is difficult to be widely used in retail Scene; image-based recognition method.
  • the general deep neural network model still has difficulty in achieving high accuracy.
  • the use of a deeper network structure will increase computing time on the one hand, and on the other hand Model training is difficult.
  • the purpose of the embodiments of the present disclosure is to provide an image-based commodity settlement method and an image-based commodity settlement device, and to overcome, at least to a certain extent, the long settlement time and complicated settlement operation due to the limitations and defects of related technologies.
  • One or more problems such as high cost of electronic tags and low practicality of existing image recognition technologies.
  • an image-based commodity settlement method including:
  • Detect the acquired image to be settled to determine the area image of the commodity to be settled in the image to be settled;
  • the price information corresponding to the feature vector of the commodity to be settled is identified in a preset feature vector recognition database
  • the identified price information is summed to output the total settlement price of the image to be settled.
  • the above detection of the acquired image to be settled to determine the area image of the commodity to be settled in the image to be settled includes:
  • Commodity detection is performed on the normalized image to be settled through the pre-trained product detection model to determine the coordinate information of the bounding box that surrounds the commodity to be settled in the normalized image to be settled, and to intercept the image in the coordinate information area of the bounding box ;
  • An area image of each commodity to be settled is cut out from the image in the bounding box coordinate information area.
  • the above pre-trained commodity detection model includes:
  • the commodity detection model is trained by a stochastic gradient descent algorithm, and the pre-trained commodity detection model is output.
  • the above-mentioned low-dimensional mapping of the area image of the commodity to be settled to obtain a feature vector corresponding to the commodity to be settled includes:
  • the region image of the commodity to be settled is mapped to a low-dimensional feature vector.
  • the pre-trained feature embedding neural network model includes:
  • the normalized image and product category of the training image are used as the training samples of the feature embedded neural network model
  • the above feature embedded neural network model is trained by a stochastic gradient descent algorithm, and the above pre-trained feature embedded neural network model is output.
  • the price information corresponding to the feature vector of the commodity to be settled in the preset feature vector recognition database based on the feature vector of the commodity to be settled includes:
  • the above price information is determined as the price information of the above-mentioned commodity to be settled.
  • the feature vectors of the preset feature vector recognition database that have been included in the database include:
  • the forward-propagation operation is performed on the image of the warehoused commodity in the preset standard format to obtain a low-dimensional feature vector of the image of the warehoused commodity;
  • a feature vector recognition database is established based on the low-dimensional feature vectors of the images of the commodities that have been put in storage.
  • an image-based commodity settlement device including:
  • a determining module configured to detect the acquired image to be settled, and determine the area image of the commodity to be settled in the image to be settled;
  • the feature vector acquisition module is used to perform a low-dimensional mapping on the area image of the commodity to be settled to obtain a feature vector corresponding to the commodity to be settled;
  • An identification module used to identify the price information corresponding to the feature vector of the commodity to be settled in the preset feature vector recognition database based on the feature vector of the commodity to be settled;
  • the output module is used to sum the identified price information and output the total settlement price of the image to be settled.
  • a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the image-based commodity settlement method as described in the first aspect of the above embodiments is implemented .
  • an electronic device including: one or more processors; a storage device for storing one or more programs, when the one or more programs are When executed by each processor, the above one or more processors implement the image-based commodity settlement method as described in the first aspect of the above embodiment.
  • Embodiments of the present disclosure provide an image-based commodity settlement method, device, medium, and electronic equipment, including: detecting the acquired image to be settled, and determining an area image of the commodity to be settled in the image to be settled; Perform a low-dimensional mapping on the area image of the commodity to be settled to obtain the feature vector corresponding to the commodity to be settled; based on the feature vector of the commodity to be settled, identify the feature vector of the commodity to be settled in the preset feature vector recognition database Corresponding price information; sum the identified price information and output the total settlement price of the image to be settled.
  • the technical solution of the embodiment of the present disclosure does not require the operation of a clearer to identify the goods placed by the user on the clearing desk, and realizes automatic, rapid and accurate identification of the type and quantity of goods to calculate the settlement amount and reduce costs .
  • FIG. 1 schematically shows a flowchart of an image-based commodity settlement method according to an embodiment of the present disclosure.
  • FIG. 2 schematically shows a flow chart of warehousing and scanning through a self-checkout counter according to an embodiment of the present disclosure
  • FIG. 3 schematically shows a flowchart of determining an area image of each commodity to be settled according to an embodiment of the present disclosure
  • FIG. 4 schematically shows a training flowchart of a commodity detection model according to an embodiment of the present disclosure
  • FIG. 5 schematically shows a training schematic diagram of a feature embedding neural network model according to an embodiment of the present disclosure
  • FIG. 6 schematically shows a schematic diagram of establishing a feature vector recognition database according to an embodiment of the present disclosure
  • FIG. 7 schematically shows a block diagram of an image-based commodity settlement system according to an embodiment of the present disclosure
  • FIG. 8 schematically shows a block diagram of an image-based commodity settlement device according to an embodiment of the present disclosure
  • FIG. 9 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the drawings.
  • the example embodiments can be implemented in various forms, and should not be construed as being limited to the examples set forth herein; on the contrary, providing these embodiments makes the present disclosure more comprehensive and complete, and fully conveys the idea of the example embodiments For those skilled in the art.
  • FIG. 1 schematically shows a flowchart of an image-based commodity settlement method according to an embodiment of the present disclosure.
  • an image-based commodity settlement method includes the following steps:
  • Step S110 Detect the acquired image to be settled to determine the area image of the commodity to be settled in the image to be settled;
  • Step S120 Perform a low-dimensional mapping on the area image of the commodity to be settled to obtain a feature vector corresponding to the commodity to be settled;
  • Step S130 based on the feature vector of the commodity to be settled, the price information corresponding to the feature vector of the commodity to be settled is identified in a preset feature vector recognition database;
  • step S140 the identified price information is summed to output the total settlement price of the image to be settled.
  • the technical solution of the embodiment shown in FIG. 1 does not require the operation of a clearer in the retail settlement scenario, and recognizes the goods placed on the checkout counter by the user, which realizes automatic, fast and accurate identification of the type and quantity of goods. Related costs.
  • step S110 the acquired image to be settled is detected to determine the area image of the commodity to be settled in the image to be settled.
  • the method further includes: acquiring images to be settled through a self-settlement self-settlement device pre-erected.
  • the self-settlement self-settlement device pre-erected mainly includes: a storage table (a storage basket, a surrounding table, etc.) ), lighting equipment (LED lights, etc.), and video recording equipment (cameras, cameras, mobile phones, etc.), through which the video recording equipment can also provide corresponding training images for intelligent image recognition algorithms and provide product images for product storage.
  • the erection of the above-mentioned recording device can be adjusted according to actual needs, and the corresponding parameters of the recording device are selected and erected according to the reasons for identifying the required image quality, video quality, and light .
  • the appearance information of the product can be photographed by using two color cameras, a top view and a side view. For each camera, the product collects an image every X degrees of rotation, and Y images are collected in total.
  • step S110 is to perform validity detection on the acquired image to be settled and identify whether the product is complete. Therefore, during the process of setting up the recording device before step S110, multiple Take pictures of animals from an angle to collect information about the product, such as the bar code, price, and name of the product, to provide a data basis for later accurate identification of the product.
  • the recording device provided in the self-checkout station is an infrastructure for providing images, and the video recorded by the recording device can be saved according to different storage methods, and the captured video can be Stored in the cloud, or stored in local external device storage.
  • FIG. 2 schematically shows a flow chart of goods entering and scanning through the self-checkout station according to an embodiment of the present disclosure.
  • the process of determining the area image of each commodity to be settled includes the following steps:
  • Step S210 acquiring a product
  • Step S220 input the barcode, price, name and other information of the commodity by scanning
  • Step S230 photographing the product
  • step S240 it is determined whether the number of captured images is the preset Y sheets. If not, return to step S230 to take another shot; if yes, perform the subsequent steps;
  • step S250 the above-mentioned captured images are stored in the commodity database and the training picture database, respectively.
  • the above step S110 specifically includes: normalizing the image to be settled, determining the invariant in the image to be settled, and converting it to a normalized Settlement image; perform commodity detection on the normalized image to be settled through a pre-trained product detection model, determine the coordinate information of the bounding box surrounding the product to be settled in the normalized image to be settled, and intercept the area of the coordinate information The image in the area; the area image of each commodity to be settled is cut out from the image in the coordinate information area of the bounding box.
  • FIG. 3 schematically shows a flowchart of determining an area image of each commodity to be settled according to an embodiment of the present disclosure.
  • the process of determining the area image of each product to be settled includes the following steps:
  • Step S310 the user places the commodity to be settled on the self-checkout counter, and the camera automatically shoots all the images of the commodity to be settled;
  • Step S320 the normalized image to be settled is used as the input of the pre-trained commodity detection model
  • Step S330 initialize the pre-trained commodity detection model with the trained detection model
  • Step S340 using the pre-trained commodity detection model to detect the commodity to be settled in the image to be settled, to obtain the bounding box coordinates of the commodity to be settled in the image to be settled;
  • the bounding box coordinates include at least the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) of the bounding box.
  • step S350 the area images of the commodities to be settled are intercepted according to the coordinates of the commodity bounding box.
  • the above determination of the area images of each commodity to be settled is an online processing process.
  • the above pre-trained commodity detection model may be obtained in the following manner:
  • the commodity detection model is trained by a stochastic gradient descent algorithm, and the pre-trained commodity detection model is output.
  • FIG. 4 schematically shows a training flowchart of a commodity detection model according to an embodiment of the present disclosure.
  • the training process of a commodity detection model includes the following steps:
  • Step S410 Perform pre-processing such as normalization and color balance on the batch of in-stock commodity images
  • Step S420 using the training image and the corresponding product bounding box coordinates and product category as the training samples of the lightweight product detection model;
  • Step S3430 use the pre-trained model to initialize the commodity detection model
  • Step S440 Use the sum of the bounding box regression loss function and the commodity classification loss function as the total loss function of the commodity detection model, and use the stochastic gradient descent algorithm to train the commodity detection model;
  • Step S450 save the commodity detection model.
  • the training process of the aforementioned commodity detection model is an offline process.
  • step S120 perform a low-dimensional mapping on the area image of the commodity to be settled to obtain a feature vector corresponding to the commodity to be settled.
  • the low-dimensional mapping of the area image of the commodity to be settled to obtain the feature vector corresponding to the commodity to be settled includes: inputting the area image of the commodity to be settled into a pre-trained feature embedding After the neural network model, the area image of the commodity to be settled is mapped to a low-dimensional feature vector.
  • the pre-trained feature embedding neural network model is trained in the following manner: the training image is subjected to the normalized image and the commodity category as the feature embedding neural network model training sample ;
  • the preset normalized exponential loss function is determined as the loss function of the above feature embedded neural network model; based on the training sample and the above loss function, the above feature embedded neural network model is trained by a random gradient descent algorithm, and the above pre The trained features are embedded in the neural network model.
  • FIG. 5 schematically shows a training schematic diagram of a feature embedding neural network model according to an embodiment of the present disclosure.
  • the training process of feature embedding neural network model includes the following steps:
  • Step S510 Perform normalization, color balance and other preprocessing on batches of training commodity images
  • Step S520 embedding the training images and commodity categories as features in the training samples of the neural network model
  • Step S530 using the pre-trained model to initialize the feature embedding neural network model
  • Step S540 Use the normalized exponential loss function to calculate the network loss of the feature embedded neural network model, and use the stochastic gradient descent method to train the feature embedded neural network model;
  • Step S550 save the feature embedded neural network model.
  • the establishment of the feature vector recognition database described above is an offline process.
  • step S130 based on the feature vector of the commodity to be settled, the price information corresponding to the feature vector of the commodity to be settled is identified in a preset feature vector recognition database.
  • the price information corresponding to the feature vector of the commodity to be settled identified in the preset feature vector recognition database specifically includes: calculating the feature vector of the commodity to be settled and the preset feature Recognize the similarity between the feature vectors of the warehousing products in the database to determine the feature vectors of the warehousing products with the highest similarity; According to the product category corresponding to the feature vectors of the warehousing products with the highest similarity, Find out the corresponding price information; determine the above price information as the price information of the goods to be settled.
  • the feature vector of the preset feature vector recognition database in the warehoused commodity includes: normalizing the image of the warehoused commodity to obtain a format conforming to the preset standard The image of the stocked goods; the pre-trained feature embedding neural network model performs the forward propagation operation on the image of the stocked goods in the above standard format to obtain the low-dimensional features of the image of the stocked goods Vector; based on the low-dimensional feature vectors of the images of the commodities that have been included in the database, a feature vector recognition database is established.
  • FIG. 6 schematically shows a schematic diagram of establishing a feature vector recognition database according to an embodiment of the present disclosure.
  • establishing a feature vector recognition database includes the following steps:
  • Step S610 Perform normalization and other processing on the acquired merchandise pictures that have been stored in the warehouse;
  • Step S620 using the trained recognition model to initialize the feature embedding neural network model
  • Step S630 using the feature embedding neural network model to perform a forward propagation operation on the stored commodity pictures to extract the feature vector of the stored commodity pictures;
  • step S640 a feature vector recognition database is established for the feature vectors of all the stored commodity pictures
  • Step S650 saving and establishing a feature vector recognition database.
  • the establishment of the feature vector recognition database described above is an offline process.
  • step S140 the identified price information is summed to output the total settlement price of the image to be settled.
  • the total settlement amount of the image to be identified can be calculated, which realizes automatic, fast and accurate identification of the product category and quantity and Calculate the total settlement amount.
  • summing the price information identified above and outputting the total settlement price of the image to be settled is an online process.
  • FIG. 7 schematically shows a block diagram of an image-based commodity settlement system according to one embodiment of the present disclosure.
  • an image-based commodity settlement system 700 includes: a self-service settlement station 701, a data line 702, and a self-service settlement device 703; wherein,
  • Self-service checkout station 701 used to provide functions such as placing goods, lighting, shooting, etc.;
  • the data line 702 is used to connect the self-checkout station 701 and the self-checkout device 703 to provide a data channel to transmit data;
  • the self-service settlement device 703 is used to determine the total settlement amount by acquiring and analyzing the image of the commodity to be settled from the data line 702.
  • FIG. 8 schematically shows a block diagram of an image-based commodity settlement apparatus according to an embodiment of the present disclosure.
  • an image-based commodity settlement apparatus 800 includes:
  • the determining module 801 is configured to detect the acquired image to be settled and determine the area image of the commodity to be settled in the image to be settled;
  • the feature vector acquisition module 802 is used to perform a low-dimensional mapping on the area image of the commodity to be settled to obtain a feature vector corresponding to the commodity to be settled;
  • the identification module 803 is configured to identify the price information corresponding to the feature vector of the commodity to be settled in a preset feature vector recognition database based on the feature vector of the commodity to be settled;
  • the output module 804 is configured to sum the identified price information and output the total settlement price of the image to be settled.
  • each function module of the image-based commodity settlement apparatus of the exemplary embodiment of the present disclosure corresponds to the steps of the above-described exemplary embodiment of the image-based commodity settlement method, for details not disclosed in the embodiment of the disclosed apparatus, please refer to this An embodiment of the aforementioned image-based commodity settlement method is disclosed.
  • FIG. 9 shows a schematic structural diagram of a computer system 900 suitable for implementing an electronic device of an embodiment of the present disclosure.
  • the computer system 900 of the electronic device shown in FIG. 9 is only an example, and should not bring any limitation to the functions and use scope of the embodiments of the present disclosure.
  • the computer system 900 includes a central processing unit (CPU) 901 that can be loaded into a random access memory (RAM) 903 from a program stored in a read-only memory (ROM) 902 or from the storage section 908 Instead, perform various appropriate actions and processing.
  • RAM 903 random access memory
  • ROM 902 read-only memory
  • RAM 903 various programs and data necessary for system operation are also stored.
  • the CPU 901, ROM 902, and RAM 903 are connected to each other through a bus 904.
  • An input/output (I/O) interface 905 is also connected to the bus 904.
  • the following components are connected to the I/O interface 905: an input section 1206 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 908 including a hard disk, etc. ; And a communication section 909 including a network interface card such as a LAN card, a modem, etc.
  • the communication section 909 performs communication processing via a network such as the Internet.
  • the drive 910 is also connected to the I/O interface 905 as needed.
  • a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as necessary, so that the computer program read out therefrom is installed into the storage section 908 as necessary.
  • the process described above with reference to the flowchart may be implemented as a computer software program.
  • embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication section 909, and/or installed from the removable medium 911.
  • CPU central processing unit
  • the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable removable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
  • the computer-readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. .
  • the program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.
  • each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more for implementing a prescribed logical function Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks represented in succession may actually be executed in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
  • each block in the block diagram or flowchart, and a combination of blocks in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be used It is realized by a combination of dedicated hardware and computer instructions.
  • the units described in the embodiments of the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the present application also provides a computer-readable medium, which may be included in the electronic device described in the foregoing embodiment; or may exist alone without being assembled into the electronic device in.
  • the computer-readable medium carries one or more programs.
  • the electronic device is enabled to implement the screen control implementation and display method as in the foregoing embodiments.
  • Step S110 the acquired image to be settled is detected to determine the area image of the commodity to be settled in the image to be settled;
  • Step S120 to the settled Perform a low-dimensional mapping on the regional image of the commodity to obtain a feature vector corresponding to the commodity to be settled;
  • Step S130 based on the feature vector of the commodity to be settled, identify the feature vector of the commodity to be settled in a preset feature vector recognition database Corresponding price information;
  • Step S140 sum the identified price information, and output the total settlement price of the image to be settled.
  • the above-described electronic device can implement various steps shown in FIG. 2.
  • the electronic device described above can implement various steps shown in FIG. 3.
  • the electronic device described above can implement various steps shown in FIG. 4.
  • the electronic device described above can implement various steps shown in FIG. 5.
  • the electronic device described above can implement various steps shown in FIG. 6.
  • the example embodiments described here can be implemented by software, or can be implemented by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, U disk, mobile hard disk, etc.) or on a network , Including several instructions to enable a computing device (which may be a personal computer, server, touch terminal, or network device, etc.) to perform the method according to the embodiments of the present disclosure.
  • a non-volatile storage medium which may be a CD-ROM, U disk, mobile hard disk, etc.
  • a computing device which may be a personal computer, server, touch terminal, or network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Development Economics (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • General Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Data Mining & Analysis (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

本公开实施例提供了一种基于图像的商品结算方法、装置、介质及电子设备，包括：对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；将所识别出的价格信息求和，输出上述待结算图像的结算总价格。本公开实施例的技术方案在零售结算场景中，无需结算员操作，对用户放置在结算台上的商品进行识别，实现了自动、快速、准确地识别商品类别与数量计算结算金额，降低了成本。 (图1)

Description

一种基于图像的商品结算方法、装置、介质及电子设备 技术领域
本公开涉及基于商品识别及结算技术领域，具体而言，涉及一种基于图像的商品结算方法及一种基于图像的商品结算装置。
背景技术
目前的商品结算方式主要包括以下几种方式：基于条码识别的商品识别技术、基于射频识别的商品识别技术和图像识别技术。而这些技术存在以下缺点：基于条形码识别方式在零售结算场景中需要结算人员复杂的操作，首先需寻找商品包装上条码的位置、手持条码识别设备、设备扫描器与条码对准扫描等操作，并且需要对多件商品进行逐一扫描操作，耗费大量人力，延长结算时间；基于射频的识别方式由于需要提前为商品安装RFID电子标签，RFID电子标签高昂的部署和维护成本很高，难以广泛应用于零售场景；基于图像的识别方式由于目前针对细粒度的对象识别如商品识别，一般的深度神经网络模型仍难以达到较高准确率，而采用更深的网络结构一方面会增加计算时间，另一方面导致模型训练困难。此外，由于商品外观的相似性，仅通过一般的分类损失函数进行网络训练难以达到较好的区分效果。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本公开的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本公开实施例的目的在于提供一种基于图像的商品结算方法及一种基于图像的商品结算装置，进而至少在一定程度上克服由于相关技术的限制和缺陷而导致的结算时间长、结算操作复杂、电子标签成本高以及现有图像识别技术实用性低等一个或者多个问题。
本公开实施例的其他特性和优点将通过下面的详细描述变得显然，或部分地通过本公开的实践而习得。
根据本公开实施例的第一方面，提供一种基于图像的商品结算方法，包括：
对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；
对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向 量；
基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；
将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
在本公开的一个实施例中，上述对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像，包括：
将上述待结算图像进行归一化处理，确定出上述待结算图像中的不变量并转换为符合预设标准形式的归一化待结算图像；
通过预训练的商品检测模型对上述归一化待结算图像进行商品检测，确定出上述归一化待结算图像中包围待结算商品的包围盒坐标信息，并截取上述包围盒坐标信息区域内的图像；
从上述包围盒坐标信息区域内的图像中截取出上述各待结算商品的区域图像。
在本公开的一个实施例中，上述预训练的商品检测模型包括：
将训练图像、待结算商品包围盒坐标信息以及商品类别作为商品检测模型的训练样本输入上述商品检测模型；
将预设的包围盒回归损失函数和预设的商品分类损失函数求和确定为上述商品检测模型的总损失函数；
基于上述训练样本和上述总损失函数，通过随机梯度下降算法对上述商品检测模型进行训练，输出上述预训练的商品检测模型。
在本公开的一个实施例中，上述对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量，包括：
将上述待结算商品的区域图像输入预训练的特征嵌入神经网络模型后，将上述待结算商品的区域图像映射为低维度的特征向量。
在本公开的一个实施例中，上述预训练的特征嵌入神经网络模型包括：
将训练图像进行归一化处理后的图像以及商品类别作为特征嵌入神经网络模型的训练样本；
将预设的归一化指数损失函数确定为上述特征嵌入神经网络模型的损失函数；
基于上述训练样本和上述损失函数，通过随机梯度下降算法对上述特征嵌入神经网络模型进行训练，输出上述预训练的特征嵌入神经网络模型。
在本公开的一个实施例中，上述基于上述待结算商品的特征向量，在预设的特征向 量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息，包括：
计算上述待结算商品的特征向量与预设的特征向量识别数据库中已入库商品的特征向量之间的相似度，确定出相似度最高的已入库商品的特征向量；
根据上述相似度最高的已入库商品的特征向量所对应的商品类别，查询出对应的价格信息；
将上述价格信息确定为上述待结算商品的价格信息。
在本公开的一个实施例中，上述预设的特征向量识别数据库中已入库商品的特征向量包括：
对已入库商品的图像进行归一化处理，获得符合预设标准格式的已入库商品的图像；
通过预训练的特征嵌入神经网络模型对上述预设标准格式的已入库商品的图像作前向传播运算，获得出上述已入库商品的图像的低维度的特征向量；
基于上述已入库商品的图像的低维度的特征向量建立特征向量识别数据库。
根据本公开实施例的第二方面，提供一种基于图像的商品结算装置，包括：
确定模块，用于对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；
特征向量获取模块，用于对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；
识别模块，用于基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；
输出模块，用于将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
根据本公开实施例的第三方面，提供了一种计算机可读介质，其上存储有计算机程序，上述程序被处理器执行时实现如上述实施例中第一方面上述的基于图像的商品结算方法。
根据本公开实施例的第四方面，提供了一种电子设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，当上述一个或多个程序被上述一个或多个处理器执行时，使得上述一个或多个处理器实现如上述实施例中第一方面上述的基于图像的商品结算方法。
本公开实施例提供的技术方案可以包括以下有益效果：
本公开实施例提供了一种基于图像的商品结算方法、装置、介质及电子设备，包括：对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；对 所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；将所识别出的价格信息求和，输出上述待结算图像的结算总价格。本公开实施例的技术方案在零售结算场景中，无需结算员操作，对用户放置在结算台上的商品进行识别，实现了自动、快速、准确地识别商品类别与数量计算结算金额，降低了成本。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1示意性示出了根据本公开的一个实施例的基于图像的商品结算方法的流程图。
图2示意性的示出了根据本公开的一个实施例的商品通过自助结算台进行入库扫描的流程图；
图3示意性的示出了根据本公开的一个实施例的确定出各待结算商品的区域图像的流程图；
图4示意性的示出了根据本公开的一个实施例的商品检测模型的训练流程图；
图5示意性示出了根据本公开的一个实施例的特征嵌入神经网络模型的训练示意图；
图6示意性示出了根据本公开的一个实施例的建立特征向量识别数据库的示意图；
图7示意性示出了根据本公开的一个实施例的基于图像的商品结算***的框图；
图8示意性示出了根据本公开的一个实施例的基于图像的商品结算装置的框图；
图9示出了适于用来实现本公开实施例的电子设备的计算机***的结构示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本公开将更 加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。
此外，所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中，提供许多具体细节从而给出对本公开的实施例的充分理解。然而，本领域技术人员将意识到，可以实践本公开的技术方案而没有特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知方法、装置、实现或者操作以避免模糊本公开的各方面。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
图1示意性示出了根据本公开的一个实施例的基于图像的商品结算方法的流程图。
参照图1所示，根据本公开的一个实施例的基于图像的商品结算方法，包括以下步骤：
步骤S110，对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；
步骤S120，对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；
步骤S130，基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；
步骤S140，将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
图1所示实施例的技术方案在零售结算场景中，无需结算员操作，对用户放置在结算台上的商品进行识别，实现了自动、快速、准确地识别商品类别与数量计算结算金额，降低了相关成本。
以下对图1中所示的各个步骤的实现细节进行详细阐述：
在步骤S110中，对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像。
在本公开的一个实施例中，上述步骤S110之前，还包括：通过预先架设的自助结算设备获取待结算图像，具体的，预先架设的自助结算设备主要包括：置物台(置物筐、 包围台等)、照明设备(LED灯等)、摄录设备(摄像头、相机、手机等)，通过该摄录设备也可以为智能图像识别算法提供相应的训练图像以及为商品入库时提供商品图像。
在本公开的一个实施例中，基于前述方案，上述摄录设备的架设可以根据实际需求进行调整，根据识别所需求的图像质量、视频质量、光线等原因，选取相应参数的摄录设备进行架设。
在本公开的一个实施例中，可以采用俯视和侧视两个彩色摄像头对商品的外观信息进行拍摄，对于每个摄像头，商品每旋转X度采集一次图像，共采集Y张图像。
在本公开的一个实施例中，基于前述方案，步骤S110是对所获取的待结算图像进行有效性检测以及识别商品是否完整，因此，在步骤S110之前架设摄录设备的过程中需要从多个角度对动物进行拍摄，以在采集商品的相关信息，例如：商品的条形码、价格、名称等信息，为后期能够准确的识别出商品提供数据基础。
在本公开的一个实施例中，基于前述方案，上述自助结算台所设置摄录设备是提供图像的基础设施，可以根据不同的存储方式将摄录设备所摄录视频进行保存，所拍摄的视频可以存储在云端，或存储在本地外接的设备存储中。
图2示意性的示出了根据本公开的一个实施例的商品通过自助结算台进行入库扫描的流程图。
参照图2所示，根据本公开的一个实施例的确定出各待结算商品的区域图像的流程，包括以下步骤：
步骤S210，获取一件商品；
步骤S220，通过扫描输入该商品的条形码、价格、名称等信息；
步骤S230，对商品进行拍摄；
步骤S240，判断拍摄的图像数量是否为预设的Y张，如过不是，返回步骤S230重新拍摄；如果是，执行后续步骤；
步骤S250，将上述拍摄的图像分别存入商品数据库和训练图片库。
在本公开的一个实施例中，上述步骤S110中具体包括：将上述待结算图像进行归一化处理，确定出上述待结算图像中的不变量并转换为符合预设标准形式的归一化待结算图像；通过预训练的商品检测模型对上述归一化待结算图像进行商品检测，确定出上述归一化待结算图像中包围待结算商品的包围盒坐标信息，并截取上述包围盒坐标信息区域内的图像；从上述包围盒坐标信息区域内的图像中截取出上述各待结算商品的区域 图像。
图3示意性的示出了根据本公开的一个实施例的确定出各待结算商品的区域图像的流程图。
参照图3所示，根据本公开的一个实施例的确定出各待结算商品的区域图像的流程，包括以下步骤：
步骤S310，用户将待结算商品放置于自助结算台上，摄录设备自动拍摄全部待结算商品图像；
步骤S320，对待结算图像进行归一化处理，作为预训练的商品检测模型的输入；
步骤S330，用训练好的检测模型初始化预训练的商品检测模型；
步骤S340，使用预训练的商品检测模型检测待结算图像中的待结算商品，得到待结算商品在待结算图像中的包围盒坐标；
在本公开的一个实施例中，上述包围盒坐标至少包括包围盒的左上角坐标(x1，y1)和右下角坐标(x2，y2)。
步骤S350，按照商品包围盒坐标截取上述各待结算商品的区域图像。
在本公开的一个实施例中，上述确定出各待结算商品的区域图像为在线处理过程。
在本公开的一个实施例中，基于前述方案，上述预训练的商品检测模型可以通过以下方式获得：
将训练图像、待结算商品包围盒坐标信息以及商品类别作为商品检测模型的训练样本输入上述商品检测模型；
将预设的包围盒回归损失函数和预设的商品分类损失函数求和确定为上述商品检测模型的总损失函数；
基于上述训练样本和上述总损失函数，通过随机梯度下降算法对上述商品检测模型进行训练，输出上述预训练的商品检测模型。
图4示意性的示出了根据本公开的一个实施例的商品检测模型的训练流程图。
参照图4所示，根据本公开的一个实施例的商品检测模型的训练流程，包括以下步骤：
步骤S410，将批量的入库商品图像进行归一化、颜色平衡等预处理；
步骤S420，将训练图像与对应的商品包围盒坐标及商品类别作为轻量级商品检测模型的训练样本；
步骤S3430，使用预训练模型初始化商品检测模型；
步骤S440，用包围盒回归损失函数和商品分类损失函数求和作为商品检测模型的总损失函数，采用随机梯度下降算法训练商品检测模型；
步骤S450，保存商品检测模型。
在本公开的一个实施例中，上述商品检测模型的训练流程为离线处理过程。
在步骤S120中，对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量。
在本公开的一个实施例中，上述对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量，包括：将上述待结算商品的区域图像输入预训练的特征嵌入神经网络模型后，将上述待结算商品的区域图像映射为低维度的特征向量。
在本公开的一个实施例中，基于前述方案，预训练的特征嵌入神经网络模型通过以下方式进行训练：将训练图像进行归一化处理后的图像以及商品类别作为特征嵌入神经网络模型的训练样本；将预设的归一化指数损失函数确定为上述特征嵌入神经网络模型的损失函数；基于上述训练样本和上述损失函数，通过随机梯度下降算法对上述特征嵌入神经网络模型进行训练，输出上述预训练的特征嵌入神经网络模型。
图5示意性示出了根据本公开的一个实施例的特征嵌入神经网络模型的训练示意图。
参照图5所示，根据本公开的一个实施例的特征嵌入神经网络模型的训练流程，包括以下步骤：
步骤S510，将批量的训练商品图像进行归一化、颜色平衡等预处理；
步骤S520，将训练图像与商品类别作为特征嵌入神经网络模型的训练样本；
步骤S530，使用预训练模型初始化特征嵌入神经网络模型；
步骤S540，使用归一化指数损失函数计算特征嵌入神经网络模型的网络损失，采用随机梯度下降法训练特征嵌入神经网络模型；
步骤S550，保存特征嵌入神经网络模型。
在本公开的一个实施例中，上述建立特征向量识别数据库为离线处理过程。
在步骤S130中，基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息。
在本公开的一个实施例中，上述在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息，具体包括：计算上述待结算商品的特征向量与预设的特征向量识别数据库中已入库商品的特征向量之间的相似度，确定出相似度最高的已 入库商品的特征向量；根据上述相似度最高的已入库商品的特征向量所对应的商品类别，查询出对应的价格信息；将上述价格信息确定为上述待结算商品的价格信息。
在本公开的一个实施例中，基于前述方案，上述预设的特征向量识别数据库中已入库商品的特征向量包括：对已入库商品的图像进行归一化处理，获得符合预设标准格式的已入库商品的图像；通过预训练的特征嵌入神经网络模型对上述预设标准格式的已入库商品的图像作前向传播运算，获得出上述已入库商品的图像的低维度的特征向量；基于上述已入库商品的图像的低维度的特征向量建立特征向量识别数据库。
图6示意性示出了根据本公开的一个实施例的建立特征向量识别数据库的示意图。
参照图6所示，根据本公开的一个实施例的建立特征向量识别数据库，包括以下步骤：
步骤S610，对获取的已入库商品图片进行归一化等处理；
步骤S620，使用训练好的识别模型初始化特征嵌入神经网络模型；
步骤S630，使用特征嵌入神经网络模型对已入库商品图片进行前向传播运算，提取已入库商品图片的特征向量；
步骤S640，对于全部已入库商品图片的特征向量建立特征向量识别数据库；
步骤S650，保存建立特征向量识别数据库。
在本公开的一个实施例中，上述建立特征向量识别数据库为离线处理过程。
步骤S140，将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
在本公开的一个实施例中，对识别出的待识别商品的价格信息进行求和后，即可计算出待识别图像的总结算金额，实现了自动、快速、准确地识别商品类别与数量并计算总结算金额。
在本公开的一个实施例中，上述识别出的价格信息求和，输出上述待结算图像的结算总价格为在线处理过程。
需要说明的是，上述内容，仅为本公开的较佳实施例而已，并非用于限定本公开的保护范围。
以下介绍本公开的装置实施例，可以用于执行本公开上述的基于图像的商品结算方法。
图7示意性示出了根据本公开的一个实施例的基于图像的商品结算***的框图。
参照图7所示，根据本公开的一个实施例的基于图像的商品结算***700，包括：自助结算台701、数据线702、自助结算装置703；其中，
自助结算台701，用于提供放置商品、照明、拍摄等功能；
数据线702，用于连接自助结算台701和自助结算装置703，提供数据通道传输数据；
自助结算装置703，用于通过从数据线702获取待结算商品图像并进行分析，确定出总结算金额。
图8示意性示出了根据本公开的一个实施例的基于图像的商品结算装置的框图。
参照图8所示，根据本公开的一个实施例的基于图像的商品结算装置800，包括：
确定模块801，用于对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；
特征向量获取模块802，用于对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；
识别模块803，用于基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；
输出模块804，用于将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
由于本公开的示例实施例的基于图像的商品结算装置的各个功能模块与上述基于图像的商品结算方法的示例实施例的步骤对应，因此对于本公开装置实施例中未披露的细节，请参照本公开上述的基于图像的商品结算方法的实施例。
下面参考图9，其示出了适于用来实现本公开实施例的电子设备的计算机***900的结构示意图。图9示出的电子设备的计算机***900仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图9所示，计算机***900包括中央处理单元(CPU)901，其可以根据存储在只读存储器(ROM)902中的程序或者从存储部分908加载到随机访问存储器(RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中，还存储有***操作所需的各种程序和数据。CPU 901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。
以下部件连接至I/O接口905：包括键盘、鼠标等的输入部分1206；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分907；包括硬盘等的存储部分908；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分909。通信部分909经由诸如因特网的网络执行通信处理。驱动器910也根据需要连接至I/O接口905。可拆卸介质911，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在 驱动器910上，以便于从其上读出的计算机程序根据需要被安装入存储部分908。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分909从网络上被下载和安装，和/或从可拆卸介质911被安装。在该计算机程序被中央处理单元(CPU)901执行时，执行本申请的***中限定的上述功能。
需要说明的是，本公开所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的***、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行***、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行***、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本公开各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的***来实现，或者可以用 专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现，所描述的单元也可以设置在处理器中。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被一个该电子设备执行时，使得该电子设备实现如上述实施例中的屏幕控制实现及展示方法。
例如，上述的电子设备可以实现如图1中所示的：步骤S110，对所获取的待结算图像进行检测，确定出上述待结算图像中待结算商品的区域图像；步骤S120，对所待结算商品的区域图像进行低维度映射，获得对应于上述待结算商品的特征向量；步骤S130，基于上述待结算商品的特征向量，在预设的特征向量识别数据库中识别出上述待结算商品的特征向量所对应的价格信息；步骤S140，将所识别出的价格信息求和，输出上述待结算图像的结算总价格。
又如，上述的电子设备可以实现如图2所示的各个步骤。
又如，上述的电子设备可以实现如图3所示的各个步骤。
又如，上述的电子设备可以实现如图4所示的各个步骤。
又如，上述的电子设备可以实现如图5所示的各个步骤。
又如，上述的电子设备可以实现如图6所示的各个步骤。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本公开实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、触控终端、或者网络设备等)执行根据本公开实施方式的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它 实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (10)

  1. 一种基于图像的商品结算方法，其特征在于，包括：
    对所获取的待结算图像进行检测，确定出所述待结算图像中待结算商品的区域图像；
    对所待结算商品的区域图像进行低维度映射，获得对应于所述待结算商品的特征向量；
    基于所述待结算商品的特征向量，在预设的特征向量识别数据库中识别出所述待结算商品的特征向量所对应的价格信息；
    将所识别出的价格信息求和，输出所述待结算图像的结算总价格。
  2. 根据权利要求1所述的基于图像的商品结算方法，其特征在于，所述对所获取的待结算图像进行检测，确定出所述待结算图像中待结算商品的区域图像，包括：
    将所述待结算图像进行归一化处理，确定出所述待结算图像中的不变量并转换为符合预设标准形式的归一化待结算图像；
    通过预训练的商品检测模型对所述归一化待结算图像进行商品检测，确定出所述归一化待结算图像中包围待结算商品的包围盒坐标信息，并截取所述包围盒坐标信息区域内的图像；
    从所述包围盒坐标信息区域内的图像中截取出所述各待结算商品的区域图像。
  3. 根据权利要求2所述的基于图像的商品结算方法，其特征在于，所述预训练的商品检测模型包括：
    将训练图像、待结算商品包围盒坐标信息以及商品类别作为商品检测模型的训练样本输入所述商品检测模型；
    将预设的包围盒回归损失函数和预设的商品分类损失函数求和确定为所述商品检测模型的总损失函数；
    基于所述训练样本和所述总损失函数，通过随机梯度下降算法对所述商品检测模型进行训练，输出所述预训练的商品检测模型。
  4. 根据权利要求1所述的基于图像的商品结算方法，其特征在于，所述对所待结算商品的区域图像进行低维度映射，获得对应于所述待结算商品的特征向量，包括：
    将所述待结算商品的区域图像输入预训练的特征嵌入神经网络模型后，将所述待结算商品的区域图像映射为低维度的特征向量。
  5. 根据权利要求4所述的基于图像的商品结算方法，其特征在于，所述预训练的 特征嵌入神经网络模型包括：
    将训练图像进行归一化处理后的图像以及商品类别作为特征嵌入神经网络模型的训练样本；
    将预设的归一化指数损失函数确定为所述特征嵌入神经网络模型的损失函数；
    基于所述训练样本和所述损失函数，通过随机梯度下降算法对所述特征嵌入神经网络模型进行训练，输出所述预训练的特征嵌入神经网络模型。
  6. 根据权利要求1所述的基于图像的商品结算方法，其特征在于，所述基于所述待结算商品的特征向量，在预设的特征向量识别数据库中识别出所述待结算商品的特征向量所对应的价格信息，包括：
    计算所述待结算商品的特征向量与预设的特征向量识别数据库中已入库商品的特征向量之间的相似度，确定出相似度最高的已入库商品的特征向量；
    根据所述相似度最高的已入库商品的特征向量所对应的商品类别，查询出对应的价格信息；
    将所述价格信息确定为所述待结算商品的价格信息。
  7. 根据权利要求6所述的基于图像的商品结算方法，其特征在于，所述预设的特征向量识别数据库中已入库商品的特征向量包括：
    对已入库商品的图像进行归一化处理，获得符合预设标准格式的已入库商品的图像；
    通过预训练的特征嵌入神经网络模型对所述预设标准格式的已入库商品的图像作前向传播运算，获得出所述已入库商品的图像的低维度的特征向量；
    基于所述已入库商品的图像的低维度的特征向量建立特征向量识别数据库。
  8. 一种基于图像的商品结算装置，其特征在于，包括：
    确定模块，用于对所获取的待结算图像进行检测，确定出所述待结算图像中待结算商品的区域图像；
    特征向量获取模块，用于对所待结算商品的区域图像进行低维度映射，获得对应于所述待结算商品的特征向量；
    识别模块，用于基于所述待结算商品的特征向量，在预设的特征向量识别数据库中识别出所述待结算商品的特征向量所对应的价格信息；
    输出模块，用于将所识别出的价格信息求和，输出所述待结算图像的结算总价格。
  9. 一种计算机可读介质，其上存储有计算机程序，其特征在于，所述程序被处理器执行时实现权利要求1至7中任一项所述的基于图像的商品结算方法。
  10. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行时，使得所述一个或多个处理器实现权利要求1至7中任一所述的基于图像的商品结算方法。
PCT/CN2019/101113 2018-11-27 2019-08-16 一种基于图像的商品结算方法、装置、介质及电子设备 WO2020107951A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811426518.6 2018-11-27
CN201811426518.6A CN111222382A (zh) 2018-11-27 2018-11-27 一种基于图像的商品结算方法、装置、介质及电子设备

Publications (1)

Publication Number Publication Date
WO2020107951A1 true WO2020107951A1 (zh) 2020-06-04

Family

ID=70805741

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/101113 WO2020107951A1 (zh) 2018-11-27 2019-08-16 一种基于图像的商品结算方法、装置、介质及电子设备

Country Status (2)

Country Link
CN (1) CN111222382A (zh)
WO (1) WO2020107951A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114419540A (zh) * 2021-12-28 2022-04-29 达闼机器人有限公司 用于视觉识别货柜商品包装的方法、装置及包装盒
CN114429556A (zh) * 2020-10-15 2022-05-03 中移动信息技术有限公司 一种图片稽核方法及装置

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111967304A (zh) * 2020-06-30 2020-11-20 北京百度网讯科技有限公司 基于边缘计算的获取物品信息方法、装置和结算台
CN112132060A (zh) * 2020-09-25 2020-12-25 广州市派客朴食信息科技有限责任公司 一种智能识别结算食物的方法
CN113903042A (zh) * 2021-09-13 2022-01-07 同盾科技有限公司 商标识别方法、装置、计算机设备和存储介质
CN114299395A (zh) * 2021-12-30 2022-04-08 百富计算机技术（深圳）有限公司 商品结算处理方法、装置、终端设备及存储介质
CN118302802A (zh) * 2022-01-10 2024-07-05 烟台创迹软件有限公司 物体识别方法、物体识别装置以及模型学习方法
CN114463217A (zh) * 2022-02-08 2022-05-10 口碑(上海)信息技术有限公司 一种图像处理方法以及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063616A (zh) * 2010-12-30 2011-05-18 上海电机学院 一种基于图像特征匹配的商品自动识别***及方法
CN107679850A (zh) * 2017-09-15 2018-02-09 苏衍杰 一种商品结算方法、装置及***
CN108537994A (zh) * 2018-03-12 2018-09-14 深兰科技（上海）有限公司 基于视觉识别及重量感应技术的智能商品结算***及方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003067643A (ja) * 2001-08-27 2003-03-07 Nec Corp 取引を促進するためのウェブサイト
CN106951484B (zh) * 2017-03-10 2020-10-30 百度在线网络技术（北京）有限公司 图片检索方法及装置、计算机设备及计算机可读介质
CN108389316B (zh) * 2018-03-02 2021-07-13 北京京东尚科信息技术有限公司 自动售货方法、装置和计算机可读存储介质
CN108520273A (zh) * 2018-03-26 2018-09-11 天津大学 一种基于目标检测的稠密小商品快速检测识别方法
CN108776770A (zh) * 2018-04-24 2018-11-09 深圳奥比中光科技有限公司 一种智能购物车的信息处理方法及智能购物车

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063616A (zh) * 2010-12-30 2011-05-18 上海电机学院 一种基于图像特征匹配的商品自动识别***及方法
CN107679850A (zh) * 2017-09-15 2018-02-09 苏衍杰 一种商品结算方法、装置及***
CN108537994A (zh) * 2018-03-12 2018-09-14 深兰科技（上海）有限公司 基于视觉识别及重量感应技术的智能商品结算***及方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114429556A (zh) * 2020-10-15 2022-05-03 中移动信息技术有限公司 一种图片稽核方法及装置
CN114419540A (zh) * 2021-12-28 2022-04-29 达闼机器人有限公司 用于视觉识别货柜商品包装的方法、装置及包装盒

Also Published As

Publication number Publication date
CN111222382A (zh) 2020-06-02

Similar Documents

Publication Publication Date Title
WO2020107951A1 (zh) 一种基于图像的商品结算方法、装置、介质及电子设备
US10853699B2 (en) Method and apparatus for vehicle damage identification
WO2019080674A1 (zh) 自助结算设备、方法、装置、介质及电子设备
WO2021000418A1 (zh) 图像数据处理方法和图像数据处理设备
WO2021179137A1 (zh) 结算方法、装置和***
WO2019128362A1 (zh) 人脸识别方法、装置、***及介质
CN115797736B (zh) 目标检测模型的训练和目标检测方法、装置、设备和介质
CN111563398A (zh) 用于确定目标物的信息的方法和装置
CN114663871A (zh) 图像识别方法、训练方法、装置、***及存储介质
WO2020083283A1 (zh) 一种自动识别支付金额的支付方法
EP3376438A1 (en) A system and method for detecting change using ontology based saliency
CN115861400A (zh) 目标对象检测方法、训练方法、装置以及电子设备
US20220414900A1 (en) Item identification using multiple cameras
US20220414899A1 (en) Item location detection using homographies
US20220414379A1 (en) Hand detection trigger for item identification
CN114255377A (zh) 一种智能货柜的差异商品检测分类方法
KR20200101024A (ko) 반품상품의 사진에 포함된 위치정보를 활용하여 픽업요청이 가능한 반품서비스 시스템
US20220414374A1 (en) Reducing a search space for item identification using machine learning
CN113269730B (zh) 图像处理方法、装置、计算机设备及存储介质
Pietrini et al. Embedded Vision System for Real-Time Shelves Rows Detection for Planogram Compliance Check
US20240029405A1 (en) System and method for selecting an item from a plurality of identified items by filtering out back images of the items
US20240020333A1 (en) System and method for selecting an item from a plurality of identified items based on a similarity value
US20240020858A1 (en) System and method for search space reduction for identifying an item
US20240020978A1 (en) System and method for space search reduction in identifying items from images via item height
US20240029275A1 (en) System and method for identifying unmoved items on a platform during item identification

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19888636

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27/09/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19888636

Country of ref document: EP

Kind code of ref document: A1