TWI759875B - Object warehouse management system - Google Patents

Object warehouse management system Download PDF

Info

Publication number
TWI759875B
TWI759875B TW109132736A TW109132736A TWI759875B TW I759875 B TWI759875 B TW I759875B TW 109132736 A TW109132736 A TW 109132736A TW 109132736 A TW109132736 A TW 109132736A TW I759875 B TWI759875 B TW I759875B
Authority
TW
Taiwan
Prior art keywords
images
image
group
processing unit
storage box
Prior art date
Application number
TW109132736A
Other languages
Chinese (zh)
Other versions
TW202213205A (en
Inventor
游德榮
侯嘉德
何國維
Original Assignee
台灣松下電器股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 台灣松下電器股份有限公司 filed Critical 台灣松下電器股份有限公司
Priority to TW109132736A priority Critical patent/TWI759875B/en
Application granted granted Critical
Publication of TW202213205A publication Critical patent/TW202213205A/en
Publication of TWI759875B publication Critical patent/TWI759875B/en

Links

Images

Landscapes

  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

一種物件倉儲管理系統，其在一儲物箱的一開口被開放期間，令一影像擷取單元持續擷取該開口附近的複數個影像，並令一影像處理單元根據該等影像判斷有物體在該開口附近移動時，在各個影像中設定位於該開口處的一基準線以及將影像中的物體標示一標誌，並將該等影像依拍攝時序前後排列後，根據該標誌與該基準線之距離變化，從該等影像中找出一臨界影像，以將該等影像分成包含該臨界影像和該臨界影像之前的該等影像的第一組影像，以及在該臨界影像之後的第二組影像，並比對該第一組影像和該第二組影像，以判斷是否有物件出入該儲物箱。An object storage management system, which enables an image capturing unit to continuously capture a plurality of images near the opening while an opening of a storage box is opened, and enables an image processing unit to determine that there is an object in the vicinity of the opening according to the images When moving near the opening, set a reference line at the opening in each image and mark the object in the image with a mark, and arrange the images in front and back according to the shooting sequence, according to the distance between the mark and the reference line changing, finding a critical image from the images to divide the images into a first group of images comprising the critical image and the images preceding the critical image, and a second group of images following the critical image, The first group of images and the second group of images are compared to determine whether there is an object entering or leaving the storage box.

Description

物件倉儲管理系統Object warehouse management system

本發明是有關於一種物件管理系統，特別是指一種物件倉儲管理系統。The present invention relates to an object management system, in particular to an object warehouse management system.

現有一種具有庫內物品管理功能的冰箱是透過冰箱庫內的攝影機拍攝一庫內影像，再以影像辨識技術對該庫內影像中的物品進行識別後，進行物品出入庫管理。但此做法需要事先提供冰箱可能會存放的各式各樣物品或食材的大量資料來訓練一影像辨識模型，使具有人工智慧，才能對出入庫的物品進行有效的辨識。In the existing refrigerator with the function of managing items in the warehouse, an image in the warehouse is captured by a camera in the refrigerator warehouse, and then the items in the image in the warehouse are identified by the image recognition technology, and then the items in and out of the warehouse are managed. However, this method needs to provide a large amount of data of various items or ingredients that may be stored in the refrigerator in advance to train an image recognition model, so that artificial intelligence can be used to effectively identify the items in and out of the warehouse.

因此，本發明的目的，即在提供一種物件倉儲管理系統，其不須對物件進行影像辨識，只須比較影像差異，即可進行物件的出入庫管理，並簡化物件出入庫管理的複雜度。Therefore, the purpose of the present invention is to provide an object storage management system, which does not need to perform image recognition on the objects, only needs to compare the image differences, and can carry out the in-out management of the objects, and simplify the complexity of the in-out management of the objects.

於是，本發明一種物件倉儲管理系統，包括一具有一可被開放或封閉的開口的儲物箱，一設於該儲物箱的記憶體單元，一設於該儲物箱並與該記憶體單元電連接的影像擷取單元以及一影像處理單元；該影像擷取單元會在該開口被開放至該開口被封閉的期間，持續地擷取該開口附近的複數個影像，並記錄每一個影像及其被擷取的時間於該記憶體單元；該影像處理單元與該記憶體單元電連接，以取得該等影像，並根據該等影像判斷一物體在該開口附近移動時，於各該影像設定一基準線，該基準線的一第一側是該影像中該儲物箱的內部，該基準線的一與該第一側相反的第二側是該影像中該儲物箱的外部；且該影像處理單元將各該影像中的該物體標示一標誌，並依時序前後排列該等影像後，從該等影像的第一張開始找出該標誌位於該基準線的該第一側且與該基準線距離最大的該張影像做為一臨界影像，並以該臨界影像為準，將依時序前後排列的該等影像分成包含該臨界影像和該臨界影像之前的該等影像的一第一組影像，以及在該臨界影像之後的一第二組影像，並比對該第一組影像和該第二組影像，發現該第一組影像中的該物體包含有該第二組影像中的該物體所沒有的物件時，判定有物件被放入該儲物箱，或者，發現該第二組影像中的該物體包含有該第一組影像中的該物體所沒有的物件時，判定有物件被從該儲物箱取出。Therefore, an object storage management system of the present invention includes a storage box with an opening that can be opened or closed, a memory unit disposed in the storage box, a storage box disposed in the storage box and connected with the memory An image capture unit and an image processing unit electrically connected to the unit; the image capture unit will continuously capture a plurality of images near the opening during the period from when the opening is opened to when the opening is closed, and record each image and the time when it was captured in the memory unit; the image processing unit is electrically connected with the memory unit to obtain the images, and according to the images to determine when an object is moving near the opening, in each of the images setting a reference line, a first side of the reference line is the inside of the storage box in the image, and a second side of the reference line opposite the first side is the outside of the storage box in the image; And the image processing unit marks the object in each of the images with a mark, and after arranging the images in sequence, finds out that the mark is located on the first side of the reference line starting from the first sheet of the images. The image with the largest distance from the reference line is used as a critical image, and based on the critical image, the images arranged before and after the time sequence are divided into a first image including the critical image and the images before the critical image. A set of images, and a second set of images after the critical image, and comparing the first set of images with the second set of images, it is found that the object in the first set of images is included in the second set of images When it is found that the object in the second group of images contains an object that is not in the object in the first group of images, it is determined that an object is put into the storage box An item was taken out of the storage box.

在本發明的一些實施態樣中，該儲物箱具有一儲物櫃體及一與該儲物櫃體的該開口對應的門板，該門板可相對該儲物櫃體開合以開放或封閉該開口。In some embodiments of the present invention, the storage box has a storage cabinet body and a door panel corresponding to the opening of the storage cabinet body, and the door panel can be opened and closed relative to the storage cabinet body to open or close the opening.

在本發明的一些實施態樣中，該影像擷取單元包括至少一攝影鏡頭及一控制器，該攝影鏡頭設於該開口的一側以拍攝該開口附近影像，該控制器會偵測該開口被開放或封閉以控制該攝影鏡頭動作與否。In some embodiments of the present invention, the image capturing unit includes at least one camera lens and a controller, the camera lens is disposed on one side of the opening to capture images near the opening, and the controller detects the opening is opened or closed to control whether the camera lens moves or not.

在本發明的一些實施態樣中，當該影像處理單元判定有物件被放入該儲物箱後，該影像處理單元將該第一組影像儲存於該記憶體單元的一入庫儲存區；而當該影像處理單元判定有物件被從該儲物箱取出後，該影像處理單元將該第二組影像與儲存於該入庫儲存區中的至少一第一組影像進行比對，以於該入庫儲存區中尋找與該第二組影像相似的該第一組影像，並於找到與該第二組影像相似的該第一組影像時，刪除該第二組影像以及與該第二組影像相似的該第一組影像。In some embodiments of the present invention, after the image processing unit determines that an object is placed in the storage box, the image processing unit stores the first set of images in a storage area of the memory unit; and After the image processing unit determines that an object has been taken out from the storage box, the image processing unit compares the second set of images with at least one first set of images stored in the storage area, so that the storage Searching for the first group of images similar to the second group of images in the storage area, and when finding the first group of images similar to the second group of images, delete the second group of images and the images similar to the second group of images of the first set of images.

在本發明的一些實施態樣中，該影像處理單元判定有物件被放入該儲物箱後，該影像處理單元以該臨界影像做為一物件入庫影像，並將該物件入庫影像及其被擷取的時間儲存於該記憶體單元的一出入庫物件管理表單；且該影像處理單元判定有物件被從該儲物箱取出後，該影像處理單元以該第二組影像中該標誌位於該基準線的該第二側且與該基準線距離最大的該張影像做為一物件出庫影像，並將該物件出庫影像及其被擷取的時間儲存於該出入庫物件管理表單。In some embodiments of the present invention, after the image processing unit determines that an object has been put into the storage box, the image processing unit uses the critical image as an object storage image, and uses the object storage image and its storage box. The captured time is stored in an inbound and outbound object management form of the memory unit; and after the image processing unit determines that an object is taken out from the storage box, the image processing unit uses the flag in the second set of images to be located in the storage box. The image on the second side of the reference line and with the largest distance from the reference line is used as an object outbound image, and the object outbound image and the captured time are stored in the inbound and outbound object management form.

在本發明的一些實施態樣中，該物件倉儲管理系統還包括一設於該門板外表面的顯示器，且該影像處理單元將該出入庫物件管理表單輸出至該顯示器，使該顯示器顯示該出入庫物件管理表單。In some embodiments of the present invention, the object storage management system further includes a display disposed on the outer surface of the door panel, and the image processing unit outputs the inbound and outbound object management form to the display, so that the display displays the display. Inbound object management form.

在本發明的一些實施態樣中，該物件倉儲管理系統還包括一與該影像處理單元電連接的無線傳輸模組，且該影像處理單元透過該無線傳輸模組將該出入庫物件管理表單輸出至一電子裝置。In some implementation aspects of the present invention, the object storage management system further includes a wireless transmission module electrically connected to the image processing unit, and the image processing unit outputs the in-warehouse object management form through the wireless transmission module to an electronic device.

在本發明的一些實施態樣中，該標誌是一矩形框，其涵蓋該物體的一部分或全部，且該影像處理單元是以該等影像其中在時序上第一張標示該矩形框的該影像中，該矩形框之與該基準線平行且距離該基準線較近的一側邊代表該標誌。In some embodiments of the present invention, the marker is a rectangular frame covering a part or all of the object, and the image processing unit is the first image of the images that marks the rectangular frame in time sequence , the side of the rectangular frame that is parallel to the reference line and closer to the reference line represents the mark.

在本發明的一些實施態樣中，該儲物箱是一冰箱。In some embodiments of the present invention, the storage box is a refrigerator.

本發明之功效在於：藉由拍攝儲物箱的開口被開放期間的複數個影像，並藉由在各個影像中設定位於開口處的基準線以及將影像中的物體標示一標誌，並將該等影像依拍攝時序前後排列後，根據該標誌與該基準線之距離變化，從該等影像中找出該臨界影像，再根據該臨界影像將該等影像分成包含該臨界影像以及該臨界影像之前的該等影像的第一組影像，以及在該臨界影像之後的第二組影像，並比對該第一組影像和該第二組影像，即可判斷是否有物件出入該儲物箱。藉此，無需事先搜集儲物箱可能會存放的各種物品的大量資料來訓練一影像辨識模型，即可判斷是否有物件出入庫，並對出入庫物件進行有效的管理，並且簡化物件出入庫管理的複雜度。The effect of the present invention is: by shooting a plurality of images when the opening of the storage box is opened, and by setting the reference line at the opening in each image and marking the object in the image with a sign, After the images are arranged before and after the shooting sequence, according to the change of the distance between the mark and the reference line, the critical image is found from the images, and then the images are divided into the critical image and the images before the critical image according to the critical image. The first group of images of the images and the second group of images after the critical image are compared with the first group of images and the second group of images to determine whether there is an object entering or leaving the storage box. In this way, it is not necessary to collect a large amount of data of various items that may be stored in the storage box to train an image recognition model, so as to determine whether there is an object in and out of the warehouse, and effectively manage the in-out and out of the warehouse, and simplify the management of the in-out and out of the warehouse. complexity.

在本發明被詳細描述之前，應當注意在以下的說明內容中，類似的元件是以相同的編號來表示。Before the present invention is described in detail, it should be noted that in the following description, similar elements are designated by the same reference numerals.

參閱圖1和圖2，本發明物件倉儲管理系統的一實施例主要包括，一儲物箱1、一設於該儲物箱1的記憶體單元2、一設於該儲物箱1並與該記憶體單元2電連接的影像擷取單元3，以及一與該記憶體單元2電連接的影像處理單元4。在本實施例中，該儲物箱1是以冰箱為例，但不以此為限，亦即該儲物箱1也可以是應用在倉儲場域中用以儲藏物品的儲物櫃或貨櫃等。1 and 2, an embodiment of the object storage management system of the present invention mainly includes a storage box 1, a memory unit 2 disposed in the storage box 1, a memory unit 2 disposed in the storage box 1 and connected to the storage box 1. The image capturing unit 3 is electrically connected to the memory unit 2 , and an image processing unit 4 is electrically connected to the memory unit 2 . In this embodiment, the storage box 1 is a refrigerator as an example, but not limited thereto, that is, the storage box 1 can also be a storage cabinet or a container used in a storage field to store items Wait.

該儲物箱1具有一儲物櫃體11及一與該儲物櫃體11的一開口111對應的門板12，該門板12可相對該儲物櫃體11開合以開放或封閉該開口111。該記憶體單元2設於該儲物箱1的該儲物櫃體11，其是由記憶體構成的儲存媒體；該影像擷取單元3設於該開口111的一側1111並與該記憶體單元2電連接，其主要包含至少一個用以拍攝該開口111附近影像的攝影鏡頭31以及控制該攝影鏡頭31拍攝的控制器32，該控制器32會偵測該開口111被開放或封閉，亦即偵測該門板12是否相對該儲物櫃體11開合，以控制該攝影鏡頭3動作與否。值得一提的是，本實施例也可以根據該攝影鏡頭31的拍攝範圍，在該儲物櫃體11設置一個以上的攝影鏡頭31以拍攝該儲物櫃體11中不同儲藏區域的影像。The storage box 1 has a storage cabinet body 11 and a door panel 12 corresponding to an opening 111 of the storage cabinet body 11 . The door panel 12 can be opened and closed relative to the storage cabinet body 11 to open or close the opening 111 . . The memory unit 2 is disposed in the storage cabinet 11 of the storage box 1, which is a storage medium composed of memory; the image capture unit 3 is disposed on one side 1111 of the opening 111 and is connected with the memory The unit 2 is electrically connected, and mainly includes at least one photographic lens 31 for capturing images near the opening 111 and a controller 32 for controlling the photographing of the photographing lens 31. The controller 32 detects whether the opening 111 is opened or closed, and also That is, it is detected whether the door panel 12 opens or closes relative to the storage cabinet 11, so as to control whether the photographing lens 3 operates or not. It is worth mentioning that, in this embodiment, according to the shooting range of the photographing lens 31 , more than one photographing lens 31 may be arranged in the locker body 11 to capture images of different storage areas in the locker body 11 .

該影像處理單元4與該記憶體單元2電連接以存取該記憶體單元2。此外，該記憶體單元2、該影像擷取單元3及該影像處理單元4通常是全部或部分整合於同一電路板上。The image processing unit 4 is electrically connected to the memory unit 2 for accessing the memory unit 2 . In addition, the memory unit 2 , the image capturing unit 3 and the image processing unit 4 are usually all or partially integrated on the same circuit board.

藉此，當該門板12從封閉該儲物櫃體11的位置相對該儲物櫃體11打開而使該開口111開放時，該控制器31會在該開口111被該門板12開放至該開口被該門板12封閉的期間，控制該攝影鏡頭32每間隔一預設時間，例如0.1秒，持續地拍攝該開口111附近的影像，並將每一個影像及其被擷取的時間記錄於該記憶體單元2。因此，該攝影鏡頭32通常會拍攝到兩種影像，一種是當有人將一物件放入該儲物櫃體11內時，如圖3所示的複數張影像；反之，另一種是當有人從該儲物櫃體11內取出一物件時，如圖4所示的複數張影像。當然，還有另外一種是沒有放入或取出任何物件於該儲物櫃體11。Thereby, when the door panel 12 is opened relative to the locker body 11 from the position closing the locker body 11 to open the opening 111 , the controller 31 will be opened to the opening by the door panel 12 at the opening 111 . During the period of being closed by the door panel 12 , the camera lens 32 is controlled to continuously capture images near the opening 111 at a preset time interval, such as 0.1 second, and record each image and the captured time in the memory body unit 2. Therefore, the photographic lens 32 usually captures two kinds of images, one is when someone puts an object into the locker body 11 , as shown in FIG. 3 ; When an object is taken out from the locker body 11 , a plurality of images are shown in FIG. 4 . Of course, there is another case where no objects are put into or taken out of the locker body 11 .

然後，當該門板12再次封閉該儲物櫃體11的該開口111後，該控制器34會通知該影像處理單元4，使該影像處理單元4從該記憶體單元2中讀取該攝影鏡頭32方才拍攝的該等影像，且該影像處理單元4根據該等影像判斷(發現)該開口111附近有一物體移動時，該影像處理單元4於各該影像中設定一基準線L，該基準線L是位於該開口111處的一縱向直線，並定義該基準線L的相反兩側為第一側L1和第二側L2，該第一側L1是指各該影像中該儲物櫃體11的內部，該第二側L2是指各該影像中該儲物櫃體11的外部。Then, when the door panel 12 closes the opening 111 of the locker body 11 again, the controller 34 notifies the image processing unit 4 so that the image processing unit 4 reads the camera lens from the memory unit 2 32 the images taken just now, and the image processing unit 4 determines (finds) an object moving near the opening 111 according to the images, the image processing unit 4 sets a reference line L in each of the images, the reference line L is a longitudinal straight line located at the opening 111, and defines the opposite sides of the reference line L as a first side L1 and a second side L2, the first side L1 refers to the locker body 11 in each image , the second side L2 refers to the outside of the locker body 11 in each image.

此外，該影像處理單元4還將出現在各該影像中的該物體S標示一會隨著該物體S移動的標誌M，該標誌M是一動態視窗，它會隨著物體S移動並動態地調整其範圍以全部涵蓋或至少部分涵蓋物體S，且由於上述的動態視窗已是一習知影像處理技術，且非本案技術重點故在此不予詳述。In addition, the image processing unit 4 will also mark the object S appearing in each of the images with a mark M that will move with the object S. The mark M is a dynamic window, which will move with the object S and dynamically The range is adjusted to fully cover or at least partially cover the object S, and since the above-mentioned dynamic window is a conventional image processing technology and is not the technical focus of the present application, it will not be described in detail here.

而且，該影像處理單元4依影像拍攝的時序前後排列該等影像後，從該等影像的第一張開始找出該標誌M位於該基準線L1的該第一側且與該基準線L1距離最大的該張影像做為一臨界影像；以圖3為例，可以看到第一張影像中的該標誌M是一個方框，且在本實施例中，是以第一張影像中的方框的最靠近該基準線L1的一側邊代表該標誌M；因此，該影像處理單元4會發現在第一張影像中，該標誌M位於該基準線L1的第二側(左側)，在第二張影像中，該標誌M隨著物體S移動到位於該基準線L1的第一側(右側)，在第三張影像中，該標誌M位於該基準線L1的第一側(右側)並距離該基準線L1更遠，在第四張影像中，該標誌M位於該基準線L1的第一側(右側)但距離該基準線L1較近，在第五張影像中，該標誌M已移動到位於該基準線L1的第二側(左側)且距離該基準線L1較近，在第六張影像中，該標誌M位於該基準線L1的第二側(右側)並距離該基準線L1更遠。Moreover, after the image processing unit 4 arranges the images one after the other according to the image shooting sequence, starting from the first one of the images, it finds that the marker M is located on the first side of the reference line L1 and is at a distance from the reference line L1 The largest image is used as a critical image; taking FIG. 3 as an example, it can be seen that the mark M in the first image is a box, and in this embodiment, it is the square in the first image. The side of the frame closest to the reference line L1 represents the mark M; therefore, the image processing unit 4 will find that in the first image, the mark M is located on the second side (left side) of the reference line L1, at In the second image, the marker M moves with the object S to the first side (right side) of the reference line L1, and in the third image, the marker M is located on the first side (right side) of the reference line L1 And farther from the reference line L1, in the fourth image, the marker M is located on the first side (right side) of the reference line L1 but closer to the reference line L1, in the fifth image, the marker M It has been moved to the second side (left side) of the reference line L1 and is closer to the reference line L1. In the sixth image, the marker M is located on the second side (right side) of the reference line L1 and is far from the reference line L1. Line L1 is further away.

因此，由於第三張影像中的該標誌M位於該基準線L1的第一側並且是該標誌M與該基準線L1的距離最遠(即該標誌M的邊線已與影像的右邊緣重疊)的第一張影像(因為第三張影像之後的影像中的該標誌M與該基準線L1的距離也有可能和第三張影像一樣)，該影像處理單元4會選擇上述的第三張影像做為該臨界影像，並以該臨界影像為準，將依拍攝時序前後排列的該等影像分成包含該臨界影像以及該臨界影像之前的該等影像的一第一組影像(即第一~第三張影像)，以及在該臨界影像之後的一第二組影像(即第四~第六張影像)；然後，該影像處理單元4比對該第一組影像和該第二組影像；且如圖3所示，當該影像處理單元4發現該第一組影像中的該物體S1包含有該第二組影像中的該物體S2所沒有的物件時，該影像處理單元4則判定有物件被放入該儲物櫃體11。Therefore, since the mark M in the third image is located on the first side of the reference line L1 and the distance between the mark M and the reference line L1 is the farthest (that is, the edge of the mark M has overlapped with the right edge of the image) (because the distance between the mark M and the reference line L1 in the images after the third image may also be the same as the third image), the image processing unit 4 will select the above third image as is the critical image, and based on the critical image, the images arranged before and after the shooting sequence are divided into a first group of images (ie the first to third images including the critical image and the images before the critical image) images), and a second group of images (ie, the fourth to sixth images) after the critical image; then, the image processing unit 4 compares the first group of images with the second group of images; and as in As shown in FIG. 3 , when the image processing unit 4 finds that the object S1 in the first group of images includes an object that the object S2 in the second group of images does not have, the image processing unit 4 determines that an object is Put into the locker body 11 .

反之，在圖4中，根據上述找出臨界影像的方式，該影像處理單元4將會以圖4中的第三張影像當做該臨界影像，並以第一~第三張影像為第一組影像，第四~第六張影像為第二組影像，且該影像處理單元4將發現該第二組影像中的該物體S1包含有該第一組影像中的該物體S2所沒有的物件，因此判定有物件被從該儲物櫃體11取出。On the contrary, in FIG. 4 , according to the above method of finding the critical image, the image processing unit 4 will take the third image in FIG. 4 as the critical image, and take the first to third images as the first group images, the fourth to sixth images are the second group of images, and the image processing unit 4 will find that the object S1 in the second group of images includes objects that the object S2 in the first group of images does not have, Therefore, it is determined that an object is taken out from the locker body 11 .

而且，當該影像處理單元4判定有物件被放入該儲物櫃體11後，該影像處理單元4會將該第一組影像儲存於該記憶體單元2的一入庫儲存區21，並以該第一組影像中的該臨界影像做為一物件入庫影像，並將該物件入庫影像及其被擷取的時間儲存於該記憶體單元2的一出入庫物件管理表單22；而當該影像處理單元4判定有物件被從該儲物櫃體11取出後，該影像處理單元4以該第二組影像中該標誌M位於該基準線L的該第二側L2且與該基準線L距離最大的該張影像，例如圖4中的第六張影像，做為一物件出庫影像，並將該物件出庫影像及其被擷取的時間儲存於該出入庫物件管理表單22；然後，該影像處理單元4將該第二組影像與儲存於該入庫儲存區21中的至少一第一組影像進行比對，以於該入庫儲存區21中尋找與該第二組影像相似的該第一組影像，並於該入庫儲存區21中找到與該第二組影像相似的該第一組影像時，刪除該第二組影像以及該入庫儲存區21中與該第二組影像相似的該第一組影像。Moreover, when the image processing unit 4 determines that an object is put into the locker body 11, the image processing unit 4 stores the first set of images in a storage area 21 of the memory unit 2, and uses The critical image in the first group of images is used as an object storage image, and the storage image of the object and the time when the object is captured is stored in a storage and exit object management table 22 of the memory unit 2; and when the image is stored After the processing unit 4 determines that an object is taken out from the locker body 11 , the image processing unit 4 uses the mark M in the second set of images to be located on the second side L2 of the reference line L and at a distance from the reference line L. The largest image, such as the sixth image in FIG. 4, is used as an object outbound image, and the object outbound image and the time when it was captured are stored in the inbound and outbound object management form 22; then, the image The processing unit 4 compares the second group of images with at least one first group of images stored in the inbound storage area 21 to find the first group of images similar to the second group of images in the inbound storage area 21 images, and when the first group of images similar to the second group of images is found in the storage area 21, the second group of images and the first group of images similar to the second group of images in the storage area 21 are deleted group images.

此外，本實施例的物件倉儲管理系統還包括一設於該儲物箱1上，例如該門板12外表面的顯示器5，該影像處理單元4與該顯示器5電連接，且該影像處理單元4會自動地或根據使用者透過設於該儲物箱1上的一輸入介面輸入的操作指令，將該出入庫物件管理表單22輸出至該顯示器5，使該顯示器5顯示該出入庫物件管理表單22。藉此，該儲物箱1的使用者即可透過該顯示器5顯示的該出入庫物件管理表單22內容，得知該儲物箱1中儲藏有哪些物品和該些物品被放入該儲物箱1的時間，以及哪些物品已被從該儲物箱1中取出和取出的時間。In addition, the object storage management system of this embodiment further includes a display 5 disposed on the storage box 1, such as the outer surface of the door panel 12, the image processing unit 4 is electrically connected to the display 5, and the image processing unit 4 The inbound and outbound object management form 22 will be output to the display 5 automatically or according to an operation command input by the user through an input interface provided on the storage box 1, so that the display 5 will display the inbound and outbound object management form twenty two. Thereby, the user of the storage box 1 can know which items are stored in the storage box 1 and which items are put into the storage box 1 through the content of the in-out item management form 22 displayed on the display 5 The time of box 1, and which items have been taken out of and out of that storage box 1.

此外，本實施例還可包括一設於該儲物箱1並與該影像處理單元4電連接的無線傳輸模組6，例如短距離無線通訊模組、無線網路通訊模組等，且該影像處理單元4會定時地或者隨時地(例如當有物件被放入該儲物箱1或者有物件被從該儲物箱1取出時)透過該無線傳輸模組6將該出入庫物件管理表單22輸出至一電子裝置7，該電子裝置7可以是一使用者的手機或平板電腦等可攜式電子裝置，或者是設於遠端或雲端的一伺服器，再由該伺服器將該出入庫物件管理表單22傳送至該使用者的手機或平板電腦等可攜式電子裝置。藉此，使用者即可透過所使用的可攜式電子裝置得知該儲物箱1中的物件的出入庫資訊。In addition, this embodiment may further include a wireless transmission module 6 disposed in the storage box 1 and electrically connected to the image processing unit 4, such as a short-range wireless communication module, a wireless network communication module, etc., and the The image processing unit 4 will periodically or at any time (for example, when an object is put into the storage box 1 or an object is taken out from the storage box 1 ) through the wireless transmission module 6 to manage the inbound and outbound objects management list. 22 is output to an electronic device 7, the electronic device 7 can be a portable electronic device such as a user's mobile phone or a tablet computer, or a server located in the remote or cloud, and then the server will output the output The inbound object management form 22 is sent to the user's portable electronic device such as a mobile phone or a tablet computer. In this way, the user can know the in-out information of the objects in the storage box 1 through the used portable electronic device.

綜上所述，上述實施例藉由拍攝儲物箱1的開口111被開放期間的複數個影像，並根據該等影像判斷有物體在開口111附近移動時，藉由在各個影像中設定位於開口111處的基準線L以及將影像中的物體標示一標誌M，並將該等影像依拍攝時序前後排列後，根據該標誌M與該基準線L之距離變化，從該等影像中找出該臨界影像，再根據該臨界影像將該等影像分成包含該臨界影像以及該臨界影像之前的該等影像的第一組影像，以及在該臨界影像之後的第二組影像，並比對該第一組影像和該第二組影像，即可判斷是否有物件被放入該儲物箱1中或者是否有物件被從該儲物箱1中取出，藉此，本實施例無需事先搜集儲物箱，例如冰箱可能會存放的各式各樣的物品或食材的大量資料來訓練一影像辨識模型，即可判斷是否有物件出入庫並對出入庫物件進行有效的管理，確實達到本發明的功效與目的。To sum up, in the above-mentioned embodiment, a plurality of images are taken when the opening 111 of the storage box 1 is opened, and when an object is determined to move near the opening 111 according to the images, the position at the opening 111 is set in each image. The reference line L at 111 and the object in the image are marked with a mark M, and after arranging the images before and after the shooting sequence, according to the change of the distance between the mark M and the reference line L, find out the mark M from the images. a critical image, and then divide the images into a first group of images including the critical image and the images before the critical image and a second group of images after the critical image according to the critical image, and compare the first group of images One group of images and the second group of images can be used to determine whether an object is put into the storage box 1 or whether an object is taken out of the storage box 1 . Therefore, in this embodiment, there is no need to collect storage boxes in advance. For example, a large amount of data of various items or ingredients that may be stored in the refrigerator can be used to train an image recognition model, which can determine whether there are objects in and out of the warehouse and effectively manage the in-out objects, which truly achieves the effects of the present invention. Purpose.

惟以上所述者，僅為本發明的實施例而已，當不能以此限定本發明實施的範圍，凡是依本發明申請專利範圍及專利說明書內容所作的簡單的等效變化與修飾，皆仍屬本發明專利涵蓋的範圍內。However, the above are only examples of the present invention, and should not limit the scope of implementation of the present invention. Any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the patent specification are still included in the scope of the present invention. within the scope of the invention patent.

1       儲物箱 11     儲物櫃體 12     門板 111   開口 1111側邊 2        記憶體單元 21     入庫儲存區 22     出入庫物件管理表單 3       記憶體單元 31     攝影鏡頭 32     控制器 4       影像處理單元 5       顯示器 6       無線傳輸模組 7       電子裝置 L      基準線 L1    第一側 L2    第二側 M     標誌 S、S1、S2   物體 1 storage box 11 locker body 12 door panels 111 Opening 1111 Side 2 memory cells 21 Inbound storage area 22 Inbound and outbound object management form 3 memory cells 31 camera lens 32 Controller 4 Image processing unit 5 Displays 6 Wireless transmission module 7 Electronic devices L Baseline L1 first side L2 second side M logo S, S1, S2 objects

本發明的其他的特徵及功效，將於參照圖式的實施方式中清楚地呈現，其中： 圖1是一示意圖，說明本發明物件倉儲管理系統的一實施例包括的儲物箱構造； 圖2是一示意圖，說明本實施例設置於儲物箱的硬體電路方塊； 圖3是一示意圖，說明本實施例的儲物箱的開口開放時，有物件放入儲物箱時拍攝到的複數張影像；及 圖4是一示意圖，說明本實施例的儲物箱的開口開放時，從儲物箱取出物件時拍攝到的複數張影像。 Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: 1 is a schematic diagram illustrating the structure of a storage box included in an embodiment of the object storage management system of the present invention; FIG. 2 is a schematic diagram illustrating a hardware circuit block disposed in the storage box according to the present embodiment; 3 is a schematic diagram illustrating a plurality of images captured when an object is put into the storage box when the opening of the storage box of the present embodiment is open; and FIG. 4 is a schematic diagram illustrating a plurality of images captured when objects are taken out from the storage box when the opening of the storage box of the present embodiment is open.

2    記憶體單元 21  入庫儲存區 22  出入庫物件管理表單 3    記憶體單元 31  攝影鏡頭 32  控制器 4    影像處理單元 5    顯示器 6    無線傳輸模組 7    電子裝置 2 memory cells 21 Inbound storage area 22 Inbound and outbound object management form 3 memory cells 31 camera lens 32 Controller 4 Image processing unit 5 monitors 6 Wireless transmission module 7 Electronic devices

Claims (8)

一種物件倉儲管理系統，包括：一儲物箱，具有一可被開放或封閉的開口；一記憶體單元，設於該儲物箱；一影像擷取單元，設於該儲物箱並與該記憶體單元電連接，該影像擷取單元會在該開口被開放至該開口被封閉的期間，持續地擷取該開口附近的複數個影像，並記錄每一個影像及其被擷取的時間於該記憶體單元；及一影像處理單元，與該記憶體單元電連接，以取得該等影像，並根據該等影像判斷一物體在該開口附近移動時，於各該影像設定一基準線，該基準線的一第一側是該影像中該儲物箱的內部，該基準線的一與該第一側相反的第二側是該影像中該儲物箱的外部；且該影像處理單元將各該影像中的該物體標示一標誌，並依時序前後排列該等影像後，從該等影像的第一張開始找出該標誌位於該基準線的該第一側且與該基準線距離最大的該張影像做為一臨界影像，並以該臨界影像為準，將依時序前後排列的該等影像分成包含該臨界影像和該臨界影像之前的該等影像的一第一組影像，以及在該臨界影像之後的一第二組影像，並比對該第一組影像和該第二組影像，發現該第一組影像中的該物體包含有該第二組影像中的該物體所沒有的物件時，判定有物件被放入該儲物箱，或者，發現該第二組影像中的該物體包含有該第一組影像中的該物體所沒有的物件時，判定有物件被從該儲物箱取出；且該影像處理單元判定有物件被放入該儲物箱後，該影像處理單元以該臨界影像做為一物件入庫影像，並將該物件入庫影像及其被擷取的時間 儲存於該記憶體單元的一出入庫物件管理表單；而該影像處理單元判定有物件被從該儲物箱取出後，該影像處理單元以該第二組影像中該標誌位於該基準線的該第二側且與該基準線距離最大的該張影像做為一物件出庫影像，並將該物件出庫影像及其被擷取的時間儲存於該出入庫物件管理表單。 An object storage management system, comprising: a storage box with an opening that can be opened or closed; a memory unit arranged in the storage box; an image capture unit arranged in the storage box and connected with the storage box The memory unit is electrically connected, and the image capture unit continuously captures a plurality of images near the opening during the period from when the opening is opened to when the opening is closed, and records each image and its captured time in the memory unit; and an image processing unit electrically connected to the memory unit to obtain the images, and to determine according to the images when an object moves near the opening, set a reference line in each of the images, the A first side of the reference line is the inside of the storage box in the image, and a second side of the reference line opposite to the first side is the outside of the storage box in the image; and the image processing unit will The object in each of the images is marked with a mark, and after arranging the images in sequence, starting from the first of the images, find that the mark is located on the first side of the reference line and has the largest distance from the reference line The image is regarded as a critical image, and based on the critical image, the images arranged before and after the time sequence are divided into a first group of images including the critical image and the images before the critical image, and the A second group of images after the critical image, and comparing the first group of images with the second group of images, it is found that the object in the first group of images includes the object in the second group of images that does not exist When an object is found, it is determined that an object has been put into the storage box, or when it is found that the object in the second group of images contains an object that the object in the first group of images does not have, it is determined that an object has been removed from the storage box. After the image processing unit determines that an object is put into the storage box, the image processing unit uses the critical image as a storage image of the object, and records the storage image of the object and the time when it was captured an inbound and outbound object management form stored in the memory unit; and after the image processing unit determines that an object is taken out from the storage box, the image processing unit uses the second set of images with the mark located on the reference line The image on the second side and the largest distance from the reference line is used as an object outbound image, and the object outbound image and the captured time are stored in the inbound and outbound object management form. 如請求項1所述的物件倉儲管理系統，其中該儲物箱具有一儲物櫃體及一與該儲物櫃體的該開口對應的門板，該門板可相對該儲物櫃體開合以開放或封閉該開口。 The object storage management system according to claim 1, wherein the storage box has a storage cabinet body and a door panel corresponding to the opening of the storage cabinet body, and the door panel can be opened and closed relative to the storage cabinet body to Open or close the opening. 如請求項1所述的物件倉儲管理系統，其中，該影像擷取單元包括至少一攝影鏡頭及一控制器，該攝影鏡頭設於該開口的一側以拍攝該開口附近影像，該控制器會偵測該開口被開放或封閉以控制該攝影鏡頭動作與否。 The object storage management system according to claim 1, wherein the image capturing unit comprises at least one camera lens and a controller, the camera lens is disposed on one side of the opening to capture images near the opening, and the controller will Detecting whether the opening is opened or closed to control the action of the camera lens. 如請求項1所述的物件倉儲管理系統，其中，當該影像處理單元判定有物件被放入該儲物箱後，該影像處理單元將該第一組影像儲存於該記憶體單元的一入庫儲存區；而當該影像處理單元判定有物件被從該儲物箱取出後，該影像處理單元將該第二組影像與儲存於該入庫儲存區中的至少一第一組影像進行比對，以於該入庫儲存區中尋找與該第二組影像相似的該第一組影像，並於找到與該第二組影像相似的該第一組影像時，刪除該第二組影像以及與該第二組影像相似的該第一組影像。 The object storage management system according to claim 1, wherein, after the image processing unit determines that an object is put into the storage box, the image processing unit stores the first set of images in a warehouse of the memory unit a storage area; and when the image processing unit determines that an object is taken out of the storage box, the image processing unit compares the second group of images with at least one first group of images stored in the inbound storage area, so as to search for the first group of images similar to the second group of images in the storage area, and delete the second group of images and the second group of images when the first group of images similar to the second group of images is found. The first set of images in which the two sets of images are similar. 如請求項1所述的物件倉儲管理系統，還包括一設於該門板外表面的顯示器，且該影像處理單元將該出入庫物件管理表單輸出至該顯示器，使該顯示器顯示該出入庫物件管理表單。 The object storage management system according to claim 1, further comprising a display disposed on the outer surface of the door panel, and the image processing unit outputs the inbound and outbound objects management form to the display, so that the display can display the inbound and outbound objects management form. 如請求項1所述的物件倉儲管理系統，還包括一與該影像處理單元 電連接的無線傳輸模組，且該影像處理單元透過該無線傳輸模組將該出入庫物件管理表單輸出至一電子裝置。 The object storage management system according to claim 1, further comprising an image processing unit connected to the image processing unit The wireless transmission module is electrically connected, and the image processing unit outputs the inbound and outbound object management form to an electronic device through the wireless transmission module. 如請求項1所述的物件倉儲管理系統，其中該標誌是一矩形框，其涵蓋該物體的一部分或全部，且該影像處理單元是以該等影像其中在時序上第一張標示該矩形框的該影像中，該矩形框之與該基準線平行且距離該基準線較近的一側邊代表該標誌。 The object storage management system according to claim 1, wherein the mark is a rectangular frame covering a part or the whole of the object, and the image processing unit marks the rectangular frame with the first frame of the images in time sequence In the image of , the side of the rectangular frame that is parallel to the reference line and closer to the reference line represents the mark. 如請求項1所述的物件倉儲管理系統，其中該儲物箱是一冰箱。The object storage management system of claim 1, wherein the storage box is a refrigerator.
TW109132736A 2020-09-22 2020-09-22 Object warehouse management system TWI759875B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW109132736A TWI759875B (en) 2020-09-22 2020-09-22 Object warehouse management system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW109132736A TWI759875B (en) 2020-09-22 2020-09-22 Object warehouse management system

Publications (2)

Publication Number Publication Date
TW202213205A TW202213205A (en) 2022-04-01
TWI759875B true TWI759875B (en) 2022-04-01

Family

ID=82197410

Family Applications (1)

Application Number Title Priority Date Filing Date
TW109132736A TWI759875B (en) 2020-09-22 2020-09-22 Object warehouse management system

Country Status (1)

Country Link
TW (1) TWI759875B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105164481A (en) * 2013-04-10 2015-12-16 Lg电子株式会社 Method for managing storage product in refrigerator using image recognition, and refrigerator for same
JP6095551B2 (en) * 2013-10-24 2017-03-15 三菱電機株式会社 refrigerator
CN107403332A (en) * 2016-05-18 2017-11-28 中华电信股份有限公司 Goods shelf fetching detection system and method
TWI668644B (en) * 2018-09-03 2019-08-11 財團法人工業技術研究院 Picking system and picking method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105164481A (en) * 2013-04-10 2015-12-16 Lg电子株式会社 Method for managing storage product in refrigerator using image recognition, and refrigerator for same
JP6095551B2 (en) * 2013-10-24 2017-03-15 三菱電機株式会社 refrigerator
CN107403332A (en) * 2016-05-18 2017-11-28 中华电信股份有限公司 Goods shelf fetching detection system and method
TWI668644B (en) * 2018-09-03 2019-08-11 財團法人工業技術研究院 Picking system and picking method

Also Published As

Publication number Publication date
TW202213205A (en) 2022-04-01

Similar Documents

Publication Publication Date Title
CN106060470B (en) Video monitoring method and system
CN108108114B (en) A kind of thumbnail display control method and mobile terminal
US20160021342A1 (en) Device and method for determination
US20140354768A1 (en) Socialized Mobile Photography
CN106570110A (en) De-overlapping processing method and apparatus of image
RU2643464C2 (en) Method and apparatus for classification of images
CN109325731B (en) Intelligent standardized law enforcement and case handling center management system
CN106295499A (en) Age estimation method and device
JP2017106671A (en) Refrigerator, image management system in refrigerator and program
CN107566742A (en) Image pickup method, device, storage medium and electronic equipment
US20210144197A1 (en) Method for Presenting Schedule Reminder Information, Terminal Device, and Cloud Server
CN107168536A (en) Examination question searching method, examination question searcher and electric terminal
CN205883437U (en) Video monitoring device
TWI759875B (en) Object warehouse management system
CN106021573A (en) Image classification method and device and electronic device
CN114322409B (en) Refrigerator and method for displaying indoor scenery pictures
CN102298651A (en) Card information management device
CN205582019U (en) Formula of shooing attendance machine of preventing practising fraud
CN115408544A (en) Image database construction method, device, equipment, storage medium and product
KR20140035541A (en) System and method for photo management
JP2014026408A (en) Information processing device and program
KR102192682B1 (en) NFC Tagging Memory System for Tagging of NFC Tag for document and storage closet and the method using it
CN102750759A (en) Gate inhibition control system and method
CN106778500B (en) A kind of method and apparatus obtaining personage face phase information
CN114294886B (en) Refrigerator and food material input method