TWI807605B - Image identifying shopping system - Google Patents

Image identifying shopping system Download PDF

Info

Publication number
TWI807605B
TWI807605B TW111102591A TW111102591A TWI807605B TW I807605 B TWI807605 B TW I807605B TW 111102591 A TW111102591 A TW 111102591A TW 111102591 A TW111102591 A TW 111102591A TW I807605 B TWI807605 B TW I807605B
Authority
TW
Taiwan
Prior art keywords
image
product
commodity
module
information
Prior art date
Application number
TW111102591A
Other languages
Chinese (zh)
Other versions
TW202331620A (en
Inventor
吳世光
Original Assignee
國立勤益科技大學
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 國立勤益科技大學 filed Critical 國立勤益科技大學
Priority to TW111102591A priority Critical patent/TWI807605B/en
Application granted granted Critical
Publication of TWI807605B publication Critical patent/TWI807605B/en
Publication of TW202331620A publication Critical patent/TW202331620A/en

Links

Images

Landscapes

  • Control Of Vending Devices And Auxiliary Devices For Vending Devices (AREA)
  • Devices For Checking Fares Or Tickets At Control Points (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

An image identifying shopping system includes an entrance identifying device, an application program, and a payment device. The entrance identifying device captures a face image and obtains corresponding member information. The application program is applied for the user to select a commodity option and assess a guiding interface, so that the user is guided to the location of the commodity. The payment device captures a payment image and a commodity image, so as to accordingly obtain the member information and the commodity information, thereby executing a payment process to obtain a payment amount and a consumption detail information, whereby the user accordingly pays the bill. Therefore, the payment efficiency is increased, and the shopping experience is improved.

Description

影像辨識購物系統Image recognition shopping system

本發明係關於一種智慧商店相關領域，尤指一種影像辨識購物系統。The invention relates to the related field of a smart store, in particular to an image recognition shopping system.

隨著智慧科技進步及發展，目前零售業逐漸地由傳統人工管理商店轉型成自助化智慧商店，以降低店內人事成本，而釋放人力到更高價值需求的適用，智慧無人商店成為科技發展的主要趨勢。With the advancement and development of smart technology, the current retail industry is gradually transforming from traditional manual management stores to self-service smart stores to reduce personnel costs in stores and release manpower to meet higher value requirements. Smart unmanned stores have become the main trend of technological development.

然而，現有大多的智慧商店，還是以消費者自行利用條碼掃描器掃描商品條碼進行結帳，條碼掃描器無法一次對多種商品進行掃描操作，因此消費者自助結帳時，必須逐一對商品進行掃描，而延長結帳時間並降低結帳的效率。However, in most existing smart stores, consumers still use barcode scanners to scan commodity barcodes for checkout. Barcode scanners cannot scan multiple commodities at one time. Therefore, consumers must scan the commodities one by one when self-checkout, which prolongs the checkout time and reduces the efficiency of checkout.

此外，當發生消費者在智慧商店內找不到商品的情況，由於店內沒有店員可以即時詢問，消費者只能花費更多時間尋找商品，而導致消費者的購物體驗降低。In addition, when a consumer cannot find a product in a smart store, since there is no clerk in the store to inquire immediately, the consumer can only spend more time looking for the product, resulting in a reduced shopping experience for the consumer.

為解決上述課題，本發明提供一種影像辨識購物系統，其能夠快速辨識欲購買的商品，並提供消費者商品擺放位置的導引，以有效改善結帳的效率並提升消費者購物體驗。In order to solve the above problems, the present invention provides an image recognition shopping system, which can quickly identify products to be purchased, and provide guidance for consumers to place the products, so as to effectively improve the efficiency of checkout and enhance the shopping experience of consumers.

本發明之一項實施例提供一種影像辨識購物系統，其架設於一商場，商場具有一入口及一出口；影像辨識購物系統包含：一入場辨識裝置、一應用程式以及一結帳裝置；入場辨識裝置設置於商場之入口，入場辨識裝置具有一會員資料庫、一攝影模組及一分析模組，會員資料庫存有複數會員資訊，攝影模組用以拍攝一使用者以產生一臉部影像，分析模組取得臉部影像，並對臉部影像進行影像處理與特徵比對，以自會員資料庫取得匹配之所述會員資訊；應用程式架設於一伺服器並搭載於一終端裝置，應用程式與入場辨識裝置耦接，應用程式具有一導引模組，導引模組具有一記憶單元及一指引單元，記憶單元存有一商場地圖及複數商品之一商品位置資訊，指引單元於終端裝置顯示商場地圖以及複數商品選項，並供使用者選取其中一商品選項而跳轉至一導引介面，指引單元取得所述商品選項對應之商品位置資訊，以導引使用者前往所述商品所在位置；結帳裝置設置於商場之出口並與入場辨識裝置以及應用程式耦接，結帳裝置具有一商品資料庫、一攝像模組、一處理模組以及一金融模組，商品資料庫存有複數商品資訊，攝像模組拍攝使用者之一結帳影像以及一商品影像，處理模組接收結帳影像以及商品影像，處理模組對結帳影像進行影像處理與特徵比對，以自入場辨識裝置取得匹配之所述會員資訊，並產生一認證資訊，處理模組根據認證資訊對商品影像進行影像處理與特徵比對，以自商品資料庫取得匹配之所述商品資訊，金融模組根據所述商品資訊而執行一結算處理，以取得一付款金額以及一消費明細，金融模組提供使用者依據付款金額以一支付工具進行支付，並將消費明細傳輸至應用程式並顯示於終端裝置。One embodiment of the present invention provides an image recognition shopping system, which is set up in a shopping mall, and the shopping mall has an entrance and an exit; the image recognition shopping system includes: an entry identification device, an application program, and a checkout device; the entry identification device is installed at the entrance of the mall, and the entry identification device has a member database, a camera module and an analysis module. Compare with the characteristics to obtain the matching member information from the member database; the application program is set up on a server and loaded on a terminal device, the application program is coupled with the entry identification device, the application program has a guide module, the guide module has a memory unit and a guide unit, the memory unit stores a shopping mall map and product location information of a plurality of commodities, the guide unit displays the mall map and multiple commodity options on the terminal device, and allows the user to select one of the commodity options to jump to a guide interface, and the guide unit obtains the product option corresponding Commodity location information to guide the user to the location of the product; the checkout device is installed at the exit of the mall and is coupled with the entry identification device and the application program. The checkout device has a product database, a camera module, a processing module and a financial module. The product database has a plurality of product information. The camera module shoots a checkout image of the user and a product image, and the processing module receives the checkout image and the product image. And generate an authentication information, the processing module performs image processing and feature comparison on the product image according to the authentication information, so as to obtain the matching product information from the product database, and the financial module executes a settlement process according to the product information to obtain a payment amount and a consumption detail, the financial module provides the user with a payment tool to pay according to the payment amount, and transmits the consumption detail to the application program and displays it on the terminal device.

藉此，本發明利用入場辨識裝置以及結帳裝置，能夠對使用者的臉部以及欲購買的商品進行辨識與特徵萃取，而取得匹配的會員身分以及商品資訊，進而加速使用者結帳流程，並有效改善結帳效率。In this way, the present invention utilizes the entry recognition device and the checkout device to identify and extract features of the user's face and the product to be purchased, and obtain matching membership and product information, thereby speeding up the user's checkout process and effectively improving checkout efficiency.

再者，本發明透過導引模組，能夠根據使用者所選取的商品而取得對應的商品位置資訊，並引導使用者前往商品擺放的區域，進而提升使用者的購物體驗。Furthermore, through the guide module, the present invention can obtain corresponding product location information according to the product selected by the user, and guide the user to the area where the product is placed, thereby improving the user's shopping experience.

為便於說明本發明於上述發明內容一欄中所表示的中心思想，茲以具體實施例表達。實施例中各種不同物件係按適於說明之比例、尺寸、變形量或位移量而描繪，而非按實際元件的比例予以繪製，合先敘明。In order to illustrate the central idea of the present invention expressed in the column of the above-mentioned summary of the invention, it is expressed in specific embodiments. Various objects in the embodiments are drawn according to proportions, sizes, deformations or displacements suitable for illustration, rather than drawn according to the proportions of actual components, which are described first.

請參閱圖1至圖8所示，本發明提供一種影像辨識購物系統100，其架設於一商場，商場具有一入口A及一出口B；影像辨識購物系統100包含相互耦接之一入場辨識裝置10、一應用程式20以及一結帳裝置30，其中，入場辨識裝置10設於商場之入口A，結帳裝置30設於商場之出口B，應用程式20架設於一伺服器2並搭載於一使用者1之一終端裝置3。1 to 8, the present invention provides an image recognition shopping system 100, which is set up in a shopping mall. The shopping mall has an entrance A and an exit B; the image recognition shopping system 100 includes an entrance identification device 10 coupled to each other, an application program 20 and a checkout device 30, wherein the entrance identification device 10 is set at the entrance A of the mall, the checkout device 30 is set at the exit B of the mall, and the application program 20 is set up on a server 2 and mounted on a terminal of a user 1 device 3.

入場辨識裝置10具有相互連接之一會員資料庫11、一攝影模組12、一分析模組13以及一暫存模組14。The entrance identification device 10 has a member database 11 , a camera module 12 , an analysis module 13 and a temporary storage module 14 connected to each other.

會員資料庫11存有複數會員資訊111，所述會員資訊111包括對應各使用者1之一身分影像資料以及一會員基本資料，其中，身分影像資料包含使用者1之臉部照片以及一臉部樣本特徵，會員基本資料包含使用者1之姓名、會員編號、連絡電話以及住址。The member database 11 stores a plurality of member information 111, and the member information 111 includes an identity image data corresponding to each user 1 and a member basic data, wherein the identity image data includes a facial photo of the user 1 and a facial sample feature, and the basic member information includes the name, membership number, contact number and address of the user 1.

攝影模組12用以拍攝進入商場之使用者1的臉部，以產生一臉部影像121。The camera module 12 is used to photograph the face of the user 1 entering the mall to generate a facial image 121 .

分析模組13取得臉部影像121，並對臉部影像121進行影像處理與計算轉換而取得一入場人物特徵並暫存至暫存模組14，分析模組13將臉部影像121之入場人物特徵與會員資訊111之臉部樣本特徵進行特徵相似度比對，以自會員資料庫11取得匹配的會員資訊111，而確定使用者1之身份；於本實施例中，臉部樣本特徵及入場人物特徵為根據人臉部之各項如鼻子的長度與寬度、額頭寬度、眼睛形狀等變數，經由計算與轉換而取得。The analysis module 13 obtains the facial image 121, and performs image processing and calculation conversion on the facial image 121 to obtain a feature of the person entering the venue and temporarily stores it in the temporary storage module 14. The analysis module 13 compares the feature of the person entering the venue with the facial sample feature of the member information 111 to obtain the matching member information 111 from the member database 11 to determine the identity of the user 1; Variables such as length and width, forehead width, and eye shape are obtained through calculation and conversion.

於本實施例中，分析模組13採用深度神經網路(DNN, Deep Neural Network)技術進行臉部辨識與特徵比對。In this embodiment, the analysis module 13 uses Deep Neural Network (DNN, Deep Neural Network) technology for face recognition and feature comparison.

進一步說明，使用者1必須註冊為會員才得以進入商場，此外，使用者1在進入商場前皆須透過入場辨識裝置10進行身分辨識，以利系統進行人員控管(如圖3所示)。To further explain, user 1 must be registered as a member to enter the mall. In addition, user 1 must perform identification through the entrance identification device 10 before entering the mall, so as to facilitate the system to control personnel (as shown in FIG. 3 ).

應用程式20具有一會員模組21以及一導引模組22，會員模組21供使用者1經由網路而註冊為會員，導引模組22能夠指引使用者1取得欲購買之一商品4。The application program 20 has a member module 21 and a guide module 22. The member module 21 allows the user 1 to register as a member via the Internet. The guide module 22 can guide the user 1 to obtain a product 4 to be purchased.

會員模組21提供使用者1透過終端裝置3建立會員資訊111，使用者1首先輸入其姓名或聯絡方式等基本資料，並經由臉部活體辨識以建立其對應的會員資訊111(如圖4所示)，會員模組21透過網路將建立完成之會員資訊111經由伺服器2傳輸至入場辨識裝置10之會員資料庫11。Member module 21 provides user 1 to create member information 111 through terminal device 3. User 1 first enters basic information such as his name or contact information, and establishes corresponding member information 111 through facial biometric recognition (as shown in FIG. 4 ).

於本實施例中，會員模組21透過如點頭、搖頭等頭部指令，或眨眼、張嘴等臉部表情指令之互動方式對使用者1進行活體辨識與判別，進而產生所述身分影像資料。In this embodiment, the member module 21 performs biometric identification and discrimination on the user 1 through interactive methods such as head commands such as nodding and shaking the head, or facial expression commands such as blinking and opening the mouth, and then generates the identity image data.

導引模組22具有相互連接之一記憶單元221及一指引單元222，記憶單元221存有一商場地圖M以及複數商品4之一商品位置資訊2211，其中，所述商品位置資訊2211具有一區域標示Z，區域標示Z包含各商品4在商場地圖M中所擺放之區域以及所擺放的貨架；在其他實施例中，記憶單元221存有不同分店的商場地圖M以及對應的商品位置資訊2211， 使用者1能夠選取其所在分店的商場地圖M以檢視對應的商品位置資訊2211。The guidance module 22 has a memory unit 221 and a guide unit 222 connected to each other. The memory unit 221 stores a shopping mall map M and commodity location information 2211 of a plurality of commodities 4, wherein the commodity location information 2211 has a region mark Z, and the region mark Z includes the area where each commodity 4 is placed in the mall map M and the shelf on which it is placed; 1 can select the shopping mall map M of its branch to view the corresponding commodity location information 2211.

如圖2、圖5及圖6所示，指引單元222於終端裝置3顯示商場地圖M以及複數商品選項S，指引單元222供使用者1選取其中一商品選項S而跳轉至一導引介面GI，指引單元222根據使用者1選取之商品選項S，而自記憶單元221取得所述商品選項S對應之區域標示Z，以導引使用者1前往所述商品4擺放的位置。As shown in FIG. 2, FIG. 5 and FIG. 6, the guide unit 222 displays a shopping mall map M and multiple product options S on the terminal device 3. The guide unit 222 allows the user 1 to select one of the product options S to jump to a guide interface GI. The guide unit 222 obtains the area mark Z corresponding to the product option S from the memory unit 221 according to the product option S selected by the user 1, so as to guide the user 1 to the position where the product 4 is placed.

進一步說明，指引單元222由終端裝置3取得使用者1之一當前位置L，自記憶單元221取得所述商品4對應之區域標示Z，並在當前位置L以及區域標示Z之間產生一導引軌跡T，並將當前位置L、區域標示Z、導引軌跡T以及商場地圖M整合並顯示於導引介面GI，隨著使用者1的移動，當前位置L也會隨之移動，使用者1可透過其當前位置L而確認自己是否按照導引軌跡T行進。To further illustrate, the guidance unit 222 obtains the current location L of the user 1 from the terminal device 3, obtains the area mark Z corresponding to the commodity 4 from the memory unit 221, and generates a guide track T between the current position L and the area mark Z, and integrates the current position L, the area mark Z, the guide track T, and the shopping mall map M and displays them on the guidance interface GI.

結帳裝置30具有相互連接之一商品資料庫31、訓練模組32、一攝像模組33、一處理模組34以及一金融模組35。The checkout device 30 has a commodity database 31 , a training module 32 , a camera module 33 , a processing module 34 and a financial module 35 which are connected to each other.

商品資料庫31存有複數商品資訊311，所述商品資訊311包括對應各商品4之一圖像組合、一商品模型以及一商品價格，其中，圖像組合為複數張大小、角度、數量不同或變形的同一商品4的圖像；商品模型包括商品4之一商品樣本特徵。The commodity database 31 stores a plurality of commodity information 311, and the commodity information 311 includes an image combination corresponding to each commodity 4, a commodity model and a commodity price, wherein the image combination is a plurality of images of the same commodity 4 with different sizes, angles, quantities or deformations; the commodity model includes a commodity sample feature of the commodity 4.

訓練模組32對各圖像組合以一訓練程式321進行特徵萃取以及演算學習，以產生各對應之所述商品模型與其所對應之商品樣本特徵，並儲存於商品資料庫31；於本實施例中，訓練模組32採用之訓練程式321為卷積神經網路(Convolutional Neural Network, CNN)處理技術。The training module 32 performs feature extraction and calculation learning with a training program 321 for each image combination, so as to generate the corresponding product models and corresponding product sample features, and store them in the product database 31; in this embodiment, the training program 321 used by the training module 32 is Convolutional Neural Network (CNN) processing technology.

藉由訓練模組32事先學習商品4的形狀、顏色、花紋等特徵，訓練模組32能不受生鮮食品或便當等商品4之個體外觀差異的影響，以達到正確辨識商品4之功效。By learning the shape, color, pattern and other characteristics of the product 4 in advance, the training module 32 can not be affected by the individual appearance differences of the product 4 such as fresh food or bento, so as to achieve the effect of correctly identifying the product 4 .

攝像模組33拍攝使用者1進行結帳認證之使用者1臉部以產生一結帳影像331，並拍攝使用者1放置於一結帳載台之商品4以產生一商品影像332；於本實施例中，攝像模組33包含四台攝像機，其中一台用於拍攝結帳之使用者1臉部，而另外三台設置於結帳載台，並分別以不同角度拍攝使用者1放置於結帳載台之商品4。The camera module 33 photographs the face of the user 1 who is authenticated by the user 1 to generate a checkout image 331, and photographs the product 4 placed by the user 1 on a checkout platform to generate a product image 332;

處理模組34接收所述結帳影像331以及商品影像332，並對結帳影像331進行影像處理而取得一結帳人物特徵；於本實施例中，結帳人物特徵為根據人臉部如鼻子的長度與寬度、額頭寬度、眼睛形狀等各項變數，經由計算與轉換而取得。The processing module 34 receives the checkout image 331 and the product image 332, and performs image processing on the checkout image 331 to obtain a checkout character feature; in this embodiment, the checkout character feature is obtained through calculation and conversion according to various variables such as the length and width of the nose, forehead width, and eye shape.

處理模組34自入場辨識裝置10之暫存模組14取得入場人物特徵，處理模組34將結帳人物特徵以及入場人物特徵進行特徵相似度比對，而取得對應入場的使用者1之會員資訊111，進而確定進行付款之使用者1身份，並產生一認證資訊。The processing module 34 obtains the characteristics of the entry person from the temporary storage module 14 of the entry identification device 10. The processing module 34 compares the characteristics of the checkout person and the entry person for feature similarity, and obtains the membership information 111 of the user 1 corresponding to the entry, and then determines the identity of the user 1 who made the payment, and generates authentication information.

處理模組34根據所述認證資訊對商品影像332進行影像處理而取得一商品特徵，進一步說明，處理模組34只有在根據認證資訊以確保使用者1為會員且有進入商場的情況下，才會進行商品4的辨識以及後續結帳流程；於本實施例中，商品樣本特徵及商品特徵為根據商品4之形狀、顏色、紋理及空間關係等變數，經由計算與轉換而取得。The processing module 34 performs image processing on the product image 332 according to the authentication information to obtain a product feature. Further explanation, the processing module 34 will only carry out the identification of the product 4 and the subsequent checkout process only when the user 1 is a member and has entered the mall according to the authentication information.

處理模組34自商品資料庫31取得複數商品樣本特徵，處理模組34將商品特徵與商品樣本特徵進行特徵相似度比對，而取得匹配之所述商品資訊311。The processing module 34 obtains a plurality of product sample features from the product database 31 , and compares the feature similarity between the product features and the product sample features to obtain the matching product information 311 .

於本實施例中，處理模組34採用深度神經網路(DNN, Deep Neural Network)技術進行使用者1之臉部以及商品4的辨識與特徵比對。In this embodiment, the processing module 34 uses Deep Neural Network (DNN, Deep Neural Network) technology to identify and compare the face of the user 1 and the product 4 .

此外，若遇到處理模組34無法成功辨識商品4的情況，由於各商品4上標示有一商品條碼C，處理模組34亦能由商品影像332讀取商品條碼C，以取得所述商品資訊311。In addition, if the processing module 34 cannot identify the commodity 4 successfully, since each commodity 4 is marked with a commodity barcode C, the processing module 34 can also read the commodity barcode C from the commodity image 332 to obtain the commodity information 311 .

在使用者1完成所有欲購買商品4的辨識後，處理模組34提供使用者1點選一確認選項，金融模組35接收所述確認選項並根據商品資訊311而執行一結算處理，以取得一付款金額351以及一消費明細352，其中，金融模組35具有一結算介面CI，金融模組35將付款金額351以及消費明細352顯示於結算介面CI，並提供使用者1依據付款金額351以一支付工具進行支付，並將消費明細352傳輸至應用程式20之並顯示於終端裝置3；於本實施例中，支付工具包含一第三方支付、一電子支付、一行動支付、一***支付或一儲值卡支付。After the user 1 completes the identification of all commodities 4 to be purchased, the processing module 34 provides the user 1 with a confirmation option to click. The financial module 35 receives the confirmation option and executes a settlement process according to the product information 311 to obtain a payment amount 351 and a consumption details 352. The financial module 35 has a settlement interface CI, and the financial module 35 displays the payment amount 351 and consumption details 352 on the settlement interface CI, and provides the user 1 with a payment tool to pay according to the payment amount 351 , and transmit the consumption details 352 to the application program 20 and display it on the terminal device 3; in this embodiment, the payment tool includes a third-party payment, an electronic payment, a mobile payment, a credit card payment or a stored value card payment.

再者，在其他可能的實施例中，結帳裝置30亦能提供使用者1以手勢控制的方式進行結帳；其中，結帳裝置30更具有一手勢資料庫，所述手勢資料庫存有複數手勢模型以及所述手勢模型對應之指令；攝像模組33能夠拍攝使用者1的手勢動作以產生一手部影像；處理模組34能夠接收所述手部影像並與手勢資料庫中的手勢模型匹配，進而執行對應的指令而完成結帳動作。Furthermore, in other possible embodiments, the checkout device 30 can also provide the user 1 with gesture control to checkout; wherein, the checkout device 30 further has a gesture database, the gesture database has a plurality of gesture models and instructions corresponding to the gesture models; the camera module 33 can capture the gesture movements of the user 1 to generate a hand image; the processing module 34 can receive the hand image and match it with the gesture model in the gesture database, and then execute the corresponding instructions to complete the checkout action.

綜合上述，本發明影像辨識購物系統100能夠達成之功效如下：Based on the above, the functions that the image recognition shopping system 100 of the present invention can achieve are as follows:

一、本發明利用入場辨識裝置10以及結帳裝置30，能夠對使用者1的臉部以及欲購買的商品4進行影像處理以及特徵萃取，而自會員資料庫11以及商品資料庫31取得匹配的會員資訊111以及商品資訊311，進而加速使用者1的結帳流程，並有效改善結帳效率。1. The present invention utilizes the entrance recognition device 10 and the checkout device 30 to perform image processing and feature extraction on the face of the user 1 and the product 4 to be purchased, and obtain matching member information 111 and product information 311 from the member database 11 and the product database 31, thereby speeding up the checkout process of the user 1 and effectively improving the checkout efficiency.

二、本發明利用應用程式20之導引模組22，能夠根據使用者1所選取的商品選項S而取得對應的商品位置資訊2211，以生成區域標示Z以及當前位置L之間的導引軌跡T，以引導使用者1前往商品4擺放的位置，而幫助使用者1快速取得商品4，進而提升使用者1的購物體驗。2. The present invention uses the guide module 22 of the application 20 to obtain the corresponding product location information 2211 according to the product option S selected by the user 1, so as to generate a guide track T between the area mark Z and the current location L, so as to guide the user 1 to the position where the product 4 is placed, and help the user 1 quickly obtain the product 4, thereby improving the shopping experience of the user 1.

以上所舉實施例僅用以說明本發明而已，非用以限制本發明之範圍。舉凡不違本發明精神所從事的種種修改或變化，俱屬本發明意欲保護之範疇。The above-mentioned embodiments are only used to illustrate the present invention, and are not intended to limit the scope of the present invention. All modifications or changes that do not violate the spirit of the present invention belong to the intended protection category of the present invention.

1:使用者1: user

2:伺服器2: Server

3:終端裝置3: Terminal device

4:商品4:Commodity

100:影像辨識購物系統100: Image recognition shopping system

10:入場辨識裝置10: Admission identification device

11:會員資料庫11: Member database

111:會員資訊111:Member Information

12:攝影模組12: Photography module

121:臉部影像121: facial image

13:分析模組13: Analysis module

14:暫存模組14: Temporary storage module

20:應用程式20: Apps

21:會員模組21:Member module

22:導引模組22: Guidance module

221:記憶單元221: memory unit

2211:商品位置資訊2211: Commodity location information

222:指引單元222: Guidance Unit

30:結帳裝置30: checkout device

31:商品資料庫31:Commodity database

311:商品資訊311: Commodity information

32:訓練模組32:Training module

321:訓練程式321: Training program

33:攝像模組33: Camera module

331:結帳影像331: checkout image

332:商品影像332:Commodity image

34:處理模組34: Processing module

35:金融模組35: Financial Module

351:付款金額351:Payment amount

352:消費明細352: Consumption details

A:入口A: Entrance

B:出口B: export

M:商場地圖M:Mall map

Z:區域標示Z: area mark

S:商品選項S: Commodity options

GI:導引介面GI: Guidance Interface

T:導引軌跡T: guide track

L:當前位置L: current location

C:商品條碼C: Commodity barcode

CI:結算介面CI: settlement interface

圖1係本發明系統連結示意圖。 圖2係本發明系統方塊圖。 圖3係本發明實施例之實施狀態示意圖(一)，用於表示入場辨識裝置辨識入場會員身份。 圖4係本發明實施例之實施狀態示意圖(二)，用於表示使用者透過會員模組建立會員基本資料示意圖。 圖5係本發明實施例之實施狀態示意圖(三)，用於表示使用者透過導引模組選取欲找尋的商品。 圖6係本發明實施例之實施狀態示意圖(四)，用於表示導引模組以導引軌跡指示使用者前往所述商品所在位置。 圖7係本發明實施例之實施狀態示意圖(五)，用於表示結帳裝置辨識結帳會員與商品。 圖8係本發明實施例之實施狀態示意圖(六)，用於表示結帳裝置顯示付款金額及消費明細。Fig. 1 is a schematic diagram of the system connection of the present invention. Fig. 2 is the system block diagram of the present invention. Fig. 3 is a schematic view (1) of the implementation state of the embodiment of the present invention, which is used to show that the entry identification device identifies the identity of the entry member. FIG. 4 is a schematic diagram (2) of the implementation state of the embodiment of the present invention, which is used to show a schematic diagram of the user creating basic member information through the membership module. FIG. 5 is a schematic diagram (3) of the implementation state of the embodiment of the present invention, which is used to show that the user selects the desired product through the guidance module. FIG. 6 is a schematic view (4) of the implementation state of the embodiment of the present invention, which is used to show that the guidance module directs the user to the location of the commodity by the guidance track. FIG. 7 is a schematic view (5) of the implementation state of the embodiment of the present invention, which is used to show that the checkout device recognizes checkout members and commodities. FIG. 8 is a schematic diagram (6) of the implementation state of the embodiment of the present invention, which is used to show that the checkout device displays payment amount and consumption details.

2:伺服器 2: Server

10:入場辨識裝置 10: Admission identification device

20:應用程式 20: Apps

30:結帳裝置 30: checkout device

Claims (8)

一種影像辨識購物系統，其架設於一商場，該商場具有一入口及一出口；該影像辨識購物系統包含：一入場辨識裝置，其設置於該商場之入口，該入場辨識裝置具有一會員資料庫、一攝影模組及一分析模組，該會員資料庫存有複數會員資訊，該攝影模組用以拍攝一使用者以產生一臉部影像，該分析模組取得該臉部影像，並對該臉部影像進行影像處理與特徵比對，以自該會員資料庫取得匹配之所述會員資訊；一應用程式，其架設於一伺服器並搭載於一終端裝置，該應用程式與該入場辨識裝置耦接，該應用程式具有一導引模組及一會員模組，該會員模組提供該使用者透過該終端裝置建立該會員資訊，該會員模組將該會員資訊傳輸至該會員資料庫，該導引模組具有一記憶單元及一指引單元，該記憶單元存有一商場地圖及複數商品之一商品位置資訊，該商品位置資訊具有一區域標示，該指引單元於該終端裝置顯示該商場地圖以及複數商品選項，並供該使用者選取其中一商品選項而跳轉至一導引介面，該指引單元由該終端裝置取得一當前位置，該指引單元取得所述商品選項對應之商品位置資訊，並由該記憶單元取得區域標示，該指引單元在該導引介面上，在該區域標示以及該當前位置之間產生一導引軌跡，以導引該使用者前往所述商品所在位置，並將該當前位置、該區域標、該導引軌跡以及該商場地圖整合並顯示於該導引介面，隨著該使用者的移動，該當前位置也會隨之移動，該使用者可透過該當前位置而確認是否按照該導引軌跡行進；以及一結帳裝置，其設置於該商場之出口並與該入場辨識裝置以及該應用程式耦接，該結帳裝置具有一商品資料庫、一攝像模組、一處理模組以及一金融模組，該商品資料庫存有複數商品資訊，該攝像模組拍攝該使用者之一結帳影像以及 一商品影像，該處理模組接收該結帳影像以及該商品影像，該處理模組對該結帳影像進行影像處理與特徵比對，以自該入場辨識裝置取得匹配之所述會員資訊，並產生一認證資訊，該處理模組根據該認證資訊對該商品影像進行影像處理與特徵比對，以自該商品資料庫取得匹配之所述商品資訊，該金融模組根據所述商品資訊而執行一結算處理，以取得一付款金額以及一消費明細，該金融模組提供該使用者依據該付款金額以一支付工具進行支付，並將該消費明細傳輸至該應用程式並顯示於該終端裝置。 An image recognition shopping system, which is set up in a shopping mall, the shopping mall has an entrance and an exit; the image recognition shopping system includes: an entry recognition device, which is installed at the entrance of the mall, the entry recognition device has a member database, a camera module and an analysis module, the member database has a plurality of member information, the camera module is used to photograph a user to generate a facial image, the analysis module obtains the facial image, and performs image processing and feature comparison on the facial image to obtain from the member database The matching member information; an application program, which is set up on a server and mounted on a terminal device, the application program is coupled with the entry identification device, the application program has a guide module and a member module, the member module provides the user with the terminal device to create the member information, the member module transmits the member information to the member database, the guide module has a memory unit and a guide unit, the memory unit stores a shopping mall map and commodity location information of multiple commodities, and the commodity location information has an area label The guidance unit displays the mall map and multiple commodity options on the terminal device, and allows the user to select one of the commodity options to jump to a guidance interface. The guidance unit obtains a current location from the terminal device, the guidance unit obtains the product location information corresponding to the commodity options, and obtains the area mark from the memory unit. And the shopping mall map is integrated and displayed on the guidance interface. As the user moves, the current location will also move accordingly. The user can confirm whether to follow the guidance track through the current location; and a checkout device, which is installed at the exit of the shopping mall and coupled with the entry identification device and the application program. The checkout device has a product database, a camera module, a processing module and a financial module. A product image. The processing module receives the checkout image and the product image. The processing module performs image processing and feature comparison on the checkout image to obtain matching member information from the entry recognition device and generates authentication information. The processing module performs image processing and feature comparison on the product image according to the authentication information to obtain matching product information from the product database. The financial module executes a settlement process based on the product information to obtain a payment amount and consumption details. The payment amount is paid with a payment tool, and the consumption details are transmitted to the application program and displayed on the terminal device. 如請求項1所述之影像辨識購物系統，其中，所述會員資訊包括對應各使用者之一身分影像資料以及一會員基本資料，所述商品資訊包括對應各商品之一圖像組合、一商品模型以及一商品價格。 The image recognition shopping system as described in Claim 1, wherein the member information includes an identity image data corresponding to each user and a member basic data, and the product information includes an image combination corresponding to each product, a product model and a product price. 如請求項2所述之影像辨識購物系統，其中，該分析模組對該臉部影像進行影像處理而取得一入場人物特徵，該分析模組依據該入場人物特徵與該身分影像資料進行比對，以取得匹配之所述會員資訊。 The image recognition shopping system as described in Claim 2, wherein the analysis module performs image processing on the facial image to obtain a feature of a person entering the venue, and the analysis module compares the feature of the person entering the venue with the identity image data to obtain the matching member information. 如請求項2所述之影像辨識購物系統，其中，該結帳裝置更具有一訓練模組，其與該商品資料庫耦接，該訓練模組對各圖像組合以一訓練程式進行特徵萃取以及演算學習，以產生各對應之所述商品模型並儲存於該商品資料庫。 The image recognition shopping system as described in Claim 2, wherein the checkout device further has a training module, which is coupled to the commodity database, and the training module performs feature extraction and algorithm learning on each image combination with a training program, so as to generate each corresponding commodity model and store it in the commodity database. 如請求項4所述之影像辨識購物系統，其中，該圖像組合為複數張大小、角度、數量不同或變形的同一商品的圖像。 The image recognition shopping system as described in Claim 4, wherein the images are combined into a plurality of images of the same commodity with different sizes, angles, quantities or deformations. 如請求項4所述之影像辨識購物系統，其中，該訓練模組採用之訓練程式為卷積神經網路(Convolutional Neural Network,CNN)處理技術。 The image recognition shopping system as described in Claim 4, wherein the training program adopted by the training module is a convolutional neural network (Convolutional Neural Network, CNN) processing technology. 如請求項3所述之影像辨識購物系統，其中，該處理模組對該結 帳影像進行影像處理而取得一結帳人物特徵，並對該商品影像進行影像處理而取得一商品特徵，該處理模組依據該結帳人物特徵與該入場人物特徵進行特徵比對，以取得所述會員資訊，該處理模組依據該商品特徵與該商品模型進行特徵比對，以取得所述商品資訊。 The image recognition shopping system as described in claim 3, wherein, the processing module Image processing is performed on the account image to obtain a checkout character feature, and image processing is performed on the product image to obtain a product feature. The processing module performs feature comparison based on the checkout character feature and the entry character feature to obtain the member information. The processing module performs feature comparison based on the product feature and the product model to obtain the product information. 如請求項1所述之影像辨識購物系統，其中，各該商品標示有一商品條碼，該處理模組由該商品影像讀取該商品條碼，以取得所述商品資訊。The image recognition shopping system as described in Claim 1, wherein each commodity is marked with a commodity barcode, and the processing module reads the commodity barcode from the commodity image to obtain the commodity information.
TW111102591A 2022-01-21 2022-01-21 Image identifying shopping system TWI807605B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW111102591A TWI807605B (en) 2022-01-21 2022-01-21 Image identifying shopping system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW111102591A TWI807605B (en) 2022-01-21 2022-01-21 Image identifying shopping system

Publications (2)

Publication Number Publication Date
TWI807605B true TWI807605B (en) 2023-07-01
TW202331620A TW202331620A (en) 2023-08-01

Family

ID=88149103

Family Applications (1)

Application Number Title Priority Date Filing Date
TW111102591A TWI807605B (en) 2022-01-21 2022-01-21 Image identifying shopping system

Country Status (1)

Country Link
TW (1) TWI807605B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106663251A (en) * 2014-07-01 2017-05-10 罗尼·哈伊 Computer-controlled, unattended, automated checkout store outlet and related method
CN108537992A (en) * 2017-03-06 2018-09-14 东芝泰格有限公司 Verifying attachment, information processing unit and control method
TWI675337B (en) * 2018-03-13 2019-10-21 新漢股份有限公司 Unmanned goods management system and unmanned goods management method
CN110689389A (en) * 2018-07-04 2020-01-14 上海霖罕信息科技有限公司 Computer vision-based shopping list automatic maintenance method and device, storage medium and terminal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106663251A (en) * 2014-07-01 2017-05-10 罗尼·哈伊 Computer-controlled, unattended, automated checkout store outlet and related method
CN108537992A (en) * 2017-03-06 2018-09-14 东芝泰格有限公司 Verifying attachment, information processing unit and control method
TWI675337B (en) * 2018-03-13 2019-10-21 新漢股份有限公司 Unmanned goods management system and unmanned goods management method
CN110689389A (en) * 2018-07-04 2020-01-14 上海霖罕信息科技有限公司 Computer vision-based shopping list automatic maintenance method and device, storage medium and terminal

Also Published As

Publication number Publication date
TW202331620A (en) 2023-08-01

Similar Documents

Publication Publication Date Title
CN110998626B (en) Providing hands-free data for interaction
CN105741451B (en) Information processing system and information processing method
JP7003136B2 (en) Cash register system
CN105247541B (en) Financial account information is extracted in the improved digital picture from card
US20200065791A1 (en) Self-service product return using computer vision and artificial intelligence
CN109643417A (en) Automatic hand-free service request
WO2014210576A2 (en) Hierarchical classification in credit card data extraction
KR102334095B1 (en) Payment method and payment system using user terminal
KR101216115B1 (en) Method and device for generating personal information of consumer, computer-readable recording medium for the same, and pos system
JP2004326208A (en) Customer managing system, program for realizing system, and recording medium
JP2006221515A (en) Pos system
US20150235217A1 (en) Photos to detect fraud at point of sale method and apparatus
US20160125404A1 (en) Face recognition business model and method for identifying perpetrators of atm fraud
KR102441562B1 (en) Smart vending machine with AI-based adult authentication function
CN113222582B (en) Face payment retail terminal
Alzamel et al. Point of Sale (POS) Network with Embedded Fingerprint Biometric Authentication
CN108389053A (en) Method of payment, device, electronic equipment and readable storage medium storing program for executing
TWM560067U (en) Smart shop system
US20220277311A1 (en) A transaction processing system and a transaction method based on facial recognition
TWI807605B (en) Image identifying shopping system
TWM570489U (en) Smart store shopping system
JP2023162229A (en) Monitoring device and program
CN109598871A (en) Unattended Supermarket system
TW202006628A (en) Smart store shopping system and purchasing method using thereof
CN109544169A (en) A kind of method of ordering based on recognition of face, storage medium and terminal device