CN113538775B - Method and device for monitoring display equipment, monitoring equipment and storage medium - Google Patents
Method and device for monitoring display equipment, monitoring equipment and storage medium Download PDFInfo
- Publication number
- CN113538775B CN113538775B CN202110632626.4A CN202110632626A CN113538775B CN 113538775 B CN113538775 B CN 113538775B CN 202110632626 A CN202110632626 A CN 202110632626A CN 113538775 B CN113538775 B CN 113538775B
- Authority
- CN
- China
- Prior art keywords
- display
- user
- display device
- object information
- image sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07F—COIN-FREED OR LIKE APPARATUS
- G07F9/00—Details other than those peculiar to special kinds or types of apparatus
- G07F9/02—Devices for alarm or indication, e.g. when empty; Advertising arrangements in coin-freed apparatus
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Alarm Systems (AREA)
- Closed-Circuit Television Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The embodiment of the application discloses a monitoring method of display equipment, which comprises the following steps: acquiring first object information corresponding to a first display object in display equipment at a first moment; if the distance between the display equipment and the first user is within the distance threshold range and the display equipment does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment; if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by display equipment and is shot by a first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user. The embodiment of the application also discloses a monitoring device, monitoring equipment and a storage medium of the display equipment.
Description
Technical Field
The present application relates to the field of intelligent container technology, and in particular, to a monitoring method for a display device, a monitoring apparatus for a display device, a monitoring device, and a storage medium.
Background
Along with the development of science and technology and the popularization of internet of things, the unmanned retail cabinet is slowly integrated into each field of people's life, and the unmanned retail cabinet mostly appears in the crowded places of personnel such as park, subway mouth and shopping mall, sells bottled beverage, bagged food, articles for daily use etc. and has brought very big convenience for people's life.
In the related art, no matter a purchaser or a non-purchaser user exists near the unmanned retail cabinet, the user and/or the purchasing behavior of the user are/is shot through a camera arranged in the unmanned retail cabinet, and shot video information is analyzed, so that the monitoring of the unmanned retail cabinet is realized. However, this method has at least problems of a large processing amount for analyzing the video and a complicated processing algorithm for the video.
Content of application
Embodiments of the present application provide a monitoring method for a display device, a monitoring apparatus for a display device, a monitoring device, and a storage medium, so as to solve at least the problems in the related art that the processing amount of analyzing a video is large, and the processing algorithm of the video is complex.
The technical scheme of the application is realized as follows:
a method of monitoring display equipment, the method comprising:
acquiring first object information corresponding to a first display object in the display equipment at a first moment;
if the distance between the display equipment and a first user is within a distance threshold range and the display equipment does not detect a shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment;
if the first object information and the second object information accord with an abnormal monitoring condition, acquiring a first image sequence which is sent by the display equipment and is obtained by shooting the first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period includes at least a period in which object information of the display apparatus is changed, and the first image sequence is used to identify the display apparatus and the first user.
A monitoring device for a display apparatus, the device comprising:
the display equipment comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring first object information corresponding to a first display object in the display equipment at a first moment;
the obtaining module is further configured to obtain second object information corresponding to a second display object in the display device at a second time, where the second time is after the first time, if the distance between the display device and the first user is within a distance threshold range and the display device does not detect a shopping request of the first user;
the acquisition module is further configured to acquire a first image sequence, which is sent by the display device and is obtained by shooting the first user within a target time period, if the first object information and the second object information meet an abnormal monitoring condition;
the processing module is used for marking the first image sequence as an abnormal image sequence; wherein the target period includes at least a period in which object information of the display apparatus is changed, and the first image sequence is used to identify the display apparatus and the first user.
A monitoring device, the monitoring device comprising:
a memory for storing executable instructions;
and the processor is used for executing the executable instructions stored in the memory so as to realize the monitoring method of the display equipment.
A computer readable storage medium storing one or more programs which are executable by one or more processors to implement the above-described monitoring method of a display device.
According to the monitoring method of the display equipment, the monitoring device of the display equipment, the monitoring equipment and the storage medium, first object information corresponding to a first display object in the display equipment at a first moment is obtained; if the distance between the display equipment and the first user is within the distance threshold range and the display equipment does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment; if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by display equipment and is obtained by shooting a first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user; that is, if the distance between the display device and the first user is within the distance threshold range and the display device does not detect the shopping request of the first user, the monitoring device determines that a first image sequence, which is sent by the display device and is shot by the first user within the target time period, is an abnormal image sequence by judging whether first object information corresponding to a first display object at a first time and second object information corresponding to a second display object at a second time meet an abnormal monitoring condition. Therefore, by the method, the monitoring equipment does not need to analyze the obtained first image sequence by using a video processing algorithm, and determines that the abnormal operation of the behavior of the first user on the display equipment exists; therefore, by extracting the key time, the abnormal information can be effectively collected, and the data transmission quantity can be reduced; meanwhile, the accuracy of monitoring abnormal behaviors by the monitoring equipment is improved, the processing speed of the monitoring equipment is also improved, and the operating pressure of the monitoring equipment is reduced.
Drawings
FIG. 1 is a schematic diagram of a display device interacting with a monitoring device provided by an embodiment of the present application;
FIG. 2 is a schematic flow chart of an alternative method for monitoring a display device provided in an embodiment of the present application;
FIG. 3 is a schematic flow chart of an alternative method for monitoring a display device provided in an embodiment of the present application;
FIG. 4 is a schematic flow chart of an alternative method for monitoring a display device provided in an embodiment of the present application;
FIG. 5 is a schematic flow chart of an alternative method for monitoring a display device provided in an embodiment of the present application;
FIG. 6 is a schematic view of an alternative configuration of a monitoring device of the display apparatus according to the embodiment of the present application;
fig. 7 is an alternative structural schematic diagram of a monitoring device provided in an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture of a monitoring method of a display device provided in the present application, where the network architecture at least includes a display device 100, a monitoring device 200, and a network 300; wherein the display device 100 and the monitoring device 200 are connected via a network 300. Illustratively, the display device 100 is a container device with retail functionality; display device 100 includes, but is not limited to, a refrigerator, a wine chest, a retail cabinet, a merchandiser, a freezer, a walk-in cabinet. The monitoring device 200 may be referred to as a remote device connected to the display device 100, the monitoring device 200 may be any one of a smartphone, a tablet computer, a Personal Digital Assistant (PDA), a camera, a wearable device, a smart television, a smart camera, a smart projector, a laptop portable computer, a desktop computer, and the like, and the monitoring device 200 may also be a server; network 300 includes, but is not limited to, local area networks, metropolitan area networks, and wide area networks.
Here, referring to fig. 1, the display apparatus 100 includes a cabinet 101 and a cabinet door 102. The display device 100 is generally enclosed, such as having a door with a lock. The display apparatus 100 stores therein one or more display objects to be sold. When a user wishes to purchase a display object in the display device 100, shopping is accomplished by unlocking the cabinet door 102 of the display device 100. For example, the user may use the mobile terminal device to scan a graphic identification code, such as a two-dimensional code, a barcode, etc., on the display device 100; then, a door opening request message is sent to the monitoring device 200 by using the mobile terminal, the monitoring device determines that the user meets the door opening condition, and the monitoring device 200 instructs the display device 100 to unlock the cabinet door 102. It should be understood by those skilled in the art that the above is only one way to unlock the cabinet door 102, and the cabinet door 102 may also be unlocked in other ways, for example, the display device 100 may unlock the cabinet door 102 by scanning an image identification code on the mobile terminal of the user, scanning a fingerprint of the user, scanning a palm print of the user, scanning an iris of the user, or the display device may unlock the cabinet door 102 by scanning a face of the user, and then correspondingly generate a door opening signal, and the display device 100 unlocks the cabinet door 102 based on the door opening signal. After the door 102 of the display apparatus 100 is unlocked, the user may open the door 102 by pulling the door 102 to access the display object in the display apparatus 100. After the user finishes shopping, for example, after the consumer closes the cabinet door 102, a corresponding settlement order is generated according to the display object taken by the consumer. When the user pays the amount of money in the settlement order, one-time shopping using the display apparatus 100 is completed.
An embodiment of the present application provides a monitoring method of a display device, which is applied to a monitoring device, and as shown in fig. 2, the monitoring method of the display device includes the steps of:
In the embodiment of the application, the first time may be a preset time that the display device is placed at the current position for the first time and is connected with the monitoring device, and the preset time is reached at intervals of a first time length along with the lapse of time; the first moment may also be the moment when the display device monitors a door open signal or a door close signal for a door of the display device. Here, the first time period may be, for example, 30 minutes.
In the embodiment of the present application, the first display object is an article to be sold stored in the display apparatus at the first time, where the first display object may be one, and the first display object may be a plurality of display objects.
In the embodiment of the present application, the first object information includes, but is not limited to, weight information or quantity information corresponding to the first display object stored at the first moment of the display apparatus.
In the embodiment of the present application, if the first object information is weight information corresponding to the first display object stored in the display device at the first time. Referring to fig. 1, the display apparatus 100 includes a plurality of display layers 1021 and a plurality of weight sensing apparatuses (not shown), the plurality of display layers 1021 being spaced apart along the height of the display apparatus 100. At least one weight sensing device is correspondingly arranged in each accommodating layer 1021. Illustratively, when a plurality of weight sensing devices are correspondingly arranged in each of the plurality of storage layers 1021, the plurality of weight sensing devices are arranged on each of the plurality of storage layers 1021 at equal intervals, and each of the plurality of storage layers 1021 is provided with a weight sensing device for acquiring gravity information of a first display object on the storage layer 1021. It should be noted that the weight sensing device includes a gravity sensor.
In the embodiment of the present application, if the first object information is the quantity information corresponding to the first display object stored in the display device at the first time. Referring to fig. 1, the display apparatus 100 includes a plurality of display layers 1021 and a plurality of near field communication identification devices (not shown), and the plurality of display layers 1021 are arranged at intervals in a height direction of the display apparatus 100. At least one near field communication identification device is correspondingly arranged in each accommodating layer 1021. Optionally, when a plurality of near field communication identification devices are correspondingly arranged in each of the storage layers 1021, the plurality of near field communication identification devices are arranged on each of the storage layers 1021 at equal intervals, and each of the storage layers 1021 is provided with a near field communication identification device for acquiring information of the number of the first display objects on the storage layer 1021. It should be noted that the near field communication Identification device includes a Radio Frequency Identification (RFID) identifier.
In this embodiment of the application, the monitoring device may be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a camera, a wearable device, a smart television, a smart camera, an intelligent projector, a laptop portable computer, a desktop computer, and the like, and the monitoring device may also be a server, which is not limited in this application.
Wherein the second time is after the first time.
In this embodiment, the distance threshold range may be a preset distance range based on the monitoring range of the display device.
In this embodiment of the application, the second time may be a starting time when the display device is placed at the current position for the first time and connected with the monitoring device, and as time goes by, the second time reaches a time preset by the monitoring device and the starting time at intervals of a first time length; the second moment may also be the moment when the display device detects a door open signal or a door close signal for a door of the display device. Note that the second time is after the first time.
In the embodiment of the present application, the second display object is a commodity to be sold stored in the display device at the second time, and here, the second display object may be one or a plurality of display objects. It should be noted that the number of the second display objects is smaller than or equal to the number of the first display objects, regardless of restocking of the display apparatus.
In the embodiment of the present application, the second object information includes, but is not limited to, weight information or quantity information corresponding to the second display object stored at the second time of the display apparatus.
In the embodiment of the application, after the monitoring device acquires first object information corresponding to a first display object in the display device at a first moment, if the distance between the display device and a first user is within a distance threshold range and the display device does not detect a shopping request of the first user, the monitoring device acquires second object information corresponding to a second display object in the display device at a second moment. Here, when the distance between the display device and the first user is within a distance threshold, an image capturing device provided in the display device is triggered to capture a first image sequence of the first user and to transmit the first image sequence to the monitoring device.
In the embodiment of the present application, referring to fig. 1, the display apparatus 100 includes an image capturing device (not shown), which may be disposed inside the cabinet 101 or outside the cabinet 101. When the image pickup device is disposed inside the cabinet 101, the image pickup device is used to pick up an image of a display object currently stored in the display device, and the image pickup device is also used to pick up an image of a display object, which is moved out of the display device, among all display objects currently stored in the display device. When the image pickup device is disposed outside the cabinet 101, the image pickup device is used to pick up an image of the vicinity around the display device and an image of the display object moved out of the display device. Here, the image pickup device may be a camera. The camera can be a 360-degree panoramic camera or a wide-angle camera, and can also be a common camera. Here, in order to ensure the clarity of the image, an illumination device may be provided on the image capture device.
And 203, if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by the display equipment and is obtained by shooting a first user in a target time period, and marking the first image sequence as an abnormal image sequence.
Wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user. Here, the target period may be a period in which the weight of the display objects stored in the display apparatus is changed, and the target period may be a period in which the number of the display objects stored in the display apparatus is changed; the target period may also be a period between the first time and the second time.
In the embodiment of the present application, the abnormality monitoring condition includes, but is not limited to, a weight variation threshold range of the display object in the display apparatus, and a number variation threshold range of the display object in the display apparatus.
In the embodiment of the present application, the abnormal image sequence represents that the user behavior included in the image belongs to an abnormal purchasing behavior, for example, a stored display object is taken out by destroying a cabinet door of the display device.
In this embodiment, if the first object information is weight information corresponding to a first display object stored in the display device at a first time, and the second object information is weight information corresponding to a second display object stored in the display device at a second time, after the monitoring device acquires the first object information and the second object information, it is determined whether a difference value between weight data in the first object information and weight data in the second object information is within a weight change threshold range, and if the difference value between the weight data in the first object information and the weight data in the second object information is within the weight change threshold range, a first image sequence shot by the display device for the first user in a target time period, that is, a time period in which the object information of the display device changes, is acquired, and the first image sequence is marked as an abnormal image sequence.
In this embodiment of the application, if the first object information is quantity information corresponding to a first display object stored in the display device at a first time, and the second object information is quantity information corresponding to a second display object stored in the display device at a second time, after the monitoring device acquires the first object information and the second object information, it is determined whether a difference value between quantity data corresponding to the first display object and quantity data corresponding to the second display object is within a quantity change threshold range, and if the difference value between the quantity data corresponding to the first display object and the quantity data corresponding to the second display object is within the quantity change threshold range, a first image sequence shot by the display device for the first user in a target time period, that is, a time period in which object information of the display device changes is acquired, and the first image sequence is marked as an abnormal image sequence.
According to the monitoring method of the display equipment provided by the embodiment of the application, the first object information corresponding to the first display object in the display equipment at the first moment is obtained; if the distance between the display equipment and the first user is within the distance threshold range and the display equipment does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment; if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by display equipment and is obtained by shooting a first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device changes, and the first sequence of images is used to identify the display device and the first user; that is, if the distance between the display device and the first user is within the distance threshold range and the display device does not detect the shopping request of the first user, the monitoring device determines that a first image sequence, which is sent by the display device and is shot by the first user within the target time period, is an abnormal image sequence by judging whether first object information corresponding to a first display object at a first time and second object information corresponding to a second display object at a second time meet an abnormal monitoring condition. Therefore, by the method, the monitoring equipment does not need to analyze the obtained first image sequence by using a video processing algorithm, and determines that the abnormal operation of the behavior of the first user on the display equipment exists; therefore, by extracting the key time, the abnormal information can be effectively collected, and the data transmission quantity can be reduced; meanwhile, the accuracy of monitoring abnormal behaviors by the monitoring equipment is improved, the processing speed of the monitoring equipment is also improved, and the operating pressure of the monitoring equipment is reduced.
An embodiment of the present application provides a monitoring method of a display device, which is applied to a monitoring device, and as shown in fig. 3, the monitoring method of the display device includes the steps of:
In an embodiment of the application, the shopping request of the second user characterizes a shopping request generated by the second user by using a door of the display device which is normally unlocked. Here, the shopping request of the second user may be a request that the second user may generate by scanning a graphic identification code on the display device using the mobile terminal device; the second user's shopping request may also be a request generated by the display device by scanning an image identification code on the user's mobile terminal; the second user's shopping request may in turn be a request generated by the display device by scanning the user's fingerprint. This is not limited by the present application.
In the embodiment of the application, if the distance between the display device and the second user is within the distance threshold range and the display device detects the shopping request of the second user, after the user takes out the display object from the display device, the cabinet door is closed in a mode of pushing the cabinet door, and at the moment, the monitoring device determines that the moment when the second user closes the cabinet door of the display device is the first moment.
Wherein the second image sequence comprises a frame in which the second user takes the display object out of the display apparatus and closes the display apparatus at the first moment.
In the embodiment of the application, the normal image sequence represents that the user behavior included in the image belongs to normal purchasing behavior.
In the embodiment of the application, the monitoring device acquires a second image sequence which is sent by the display device and obtained by shooting a second user, and marks the second image sequence as a normal image sequence.
And step 303, if the distance between the display device and the first user is within the distance threshold range and the display device does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display device at a second moment.
Wherein the second time is after the first time.
In other embodiments of the present application, before the second object information corresponding to the second display object in the display apparatus at the second time is obtained in step 303, the following procedure may be further performed,
the monitoring device determines that a first user is present within a distance threshold range of the display device and determines a sampling interval of object information of the display object. The monitoring device determines the second time based on the first time and a sampling interval of object information of the display object.
In this embodiment of the application, the sampling interval of the object information of the displayed object may be a sampling interval preset by a user, or the sampling interval of the object information of the displayed object may also be a sampling interval set in real time by an operator at the monitoring device side, which is not specifically limited in this application. For example, the sampling interval of the object information of the display object may be 2 minutes.
In the embodiment of the application, the monitoring device determines that the first user appears in the distance threshold range of the display device, determines the sampling interval of the object information of the display object, meanwhile, can also obtain the moment when the first user appears, and determines the second moment based on the first moment, the moment when the first user appears and the sampling interval of the object information of the display object.
And 304, if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by the display equipment and is shot by the first user in a target time period, and marking the first image sequence as an abnormal image sequence.
Wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
As described above, first, the monitoring apparatus acquires first object information of the first display object at a first time when the second user takes out the display object from the display apparatus and closes the display apparatus; secondly, the display device determines a first user whose distance to the display device is within a distance threshold range, and the display device does not detect a shopping request of the first user, the monitoring device acquires a sampling interval of object information of a display object, and determines a second time based on the first time and the sampling interval. Thirdly, the monitoring equipment acquires second object information corresponding to a second display object in the display equipment at a second moment; and finally, the monitoring equipment determines that a first image sequence which is sent by the display equipment and is obtained by shooting a first user in a target time period is an abnormal image sequence by judging whether first object information corresponding to a first display object at a first moment and second object information corresponding to a second display object at a second moment meet abnormal monitoring conditions. Therefore, by extracting the key time, the abnormal information can be effectively collected, and the data transmission quantity can be reduced; meanwhile, the accuracy of monitoring abnormal behaviors by the monitoring equipment is improved, the processing speed of the monitoring equipment is also improved, and the operating pressure of the monitoring equipment is reduced.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
An embodiment of the present application provides a monitoring method of a display device, which is applied to a monitoring device, and as shown in fig. 4, the monitoring method of the display device includes the steps of:
In the embodiment of the application, the first time is a preset time which is reached at a first time interval along with the lapse of time at the starting time when the display device is placed at the current position for the first time and is connected with the monitoring device. Here, the first time period may be, for example, 30 minutes.
In an implementable scenario, the display device records first object information corresponding to a first display object in the display device at a time that arrives at a first time interval, e.g. 30 minutes, over time, with a start time at which the display device is first placed at a current location and connected to the monitoring device.
Wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
Wherein the second time is after the first time.
In an embodiment of the application, the shopping request of the third user characterizes a shopping request generated by the third user by using a door of the display device which is normally unlocked. Here, the shopping request of the third user may be a request that the third user may generate by scanning a graphic identification code on the display device using the mobile terminal device; the shopping request of the third user may also be a request generated by the display device by scanning an image identification code on the mobile terminal of the user; the shopping request of the third user may in turn be a request generated by the display device by scanning the fingerprint of the user. This is not limited by the present application.
In the embodiment of the application, if the distance between the display device and the third user is within the distance threshold range, and the display device detects the shopping request of the third user, when the user opens the cabinet door by pulling the cabinet door, the monitoring device determines that the moment when the third user punches the card to display the cabinet door of the device is the second moment.
And step 404, acquiring second object information corresponding to a second display object in the display equipment at a second moment.
As can be seen from the above, first, the monitoring device acquires first object information of a first display object at a preset time, that is, a first time, and then, the display device determines a first user whose distance from the display device is within a distance threshold range, and the display device does not detect a shopping request of the first user, and acquires a first image sequence which is transmitted by the display device and is captured by the first user within a target time period. And thirdly, the monitoring device determines that the distance between the third user and the display device is within the distance threshold range, and if the display device detects a shopping request of the third user, determines second object information corresponding to a second display object at a second moment when the display device is opened by the third user. And finally, the monitoring equipment determines that a first image sequence which is sent by the display equipment and is obtained by shooting a first user in a target time period is an abnormal image sequence by judging whether first object information corresponding to a first display object at a first moment and second object information corresponding to a second display object at a second moment meet abnormal monitoring conditions. Therefore, by extracting the key time, the abnormal information can be effectively collected, and the data transmission quantity can be reduced; meanwhile, the accuracy of monitoring abnormal behaviors by the monitoring equipment is improved, the processing speed of the monitoring equipment is also improved, and the operating pressure of the monitoring equipment is reduced.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
An embodiment of the present application provides a monitoring method of a display device, which is applied to a monitoring device, and as shown in fig. 5, the monitoring method of a display device includes the steps of:
Wherein the second image sequence comprises a frame in which the second user takes the display object out of the display apparatus and closes the display apparatus at the first moment.
Wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
Wherein the second time is after the first time.
And step 505, acquiring second object information corresponding to a second display object in the display equipment at a second moment.
And step 507, generating alarm information and outputting the alarm information.
The alarm information is used for alarming for the abnormal operation event of the display equipment by the first user.
In the embodiment of the application, after the monitoring device marks the first image sequence as an abnormal image sequence, alarm information for alarming the abnormal operation event of the display device by the first user is generated, and the alarm information is output.
As described above, first, the monitoring apparatus acquires first object information of the first display object at a first time when the second user takes out the display object from the display apparatus and closes the display apparatus; secondly, the display device determines a first user with a distance between the first user and the display device within a distance threshold range, the display device does not detect a shopping request of the first user, and a first image sequence sent by the display device and obtained by shooting the first user within a target time period is obtained. And thirdly, the monitoring device determines that the distance between the third user and the display device is within the distance threshold range, and if the display device detects a shopping request of the third user, determines second object information corresponding to a second display object at a second moment when the display device is opened by the third user. And finally, the monitoring equipment determines that a first image sequence which is sent by the display equipment and is obtained by shooting a first user in a target time period is an abnormal image sequence by judging whether first object information corresponding to a first display object at a first moment and second object information corresponding to a second display object at a second moment meet abnormal monitoring conditions. Therefore, by extracting the key time, the abnormal information can be effectively collected, and the data transmission quantity can be reduced; meanwhile, the accuracy of monitoring abnormal behaviors by the monitoring equipment is improved, the processing speed of the monitoring equipment is also improved, and the operating pressure of the monitoring equipment is reduced.
It should be noted that, for the descriptions of the same steps and the same contents in this embodiment as those in other embodiments, reference may be made to the descriptions in other embodiments, which are not described herein again.
An embodiment of the present application provides a monitoring apparatus for a display device, which may be applied to a monitoring method for a display device provided in an embodiment corresponding to fig. 2 to 5, and as shown in fig. 6, the monitoring apparatus 6 for a display device includes:
an obtaining module 61, configured to obtain first object information corresponding to a first display object in a first time display apparatus;
the obtaining module 61 is further configured to obtain second object information corresponding to a second display object in the display device at a second time, where the second time is after the first time, if the distance between the display device and the first user is within the distance threshold range and the display device does not detect the shopping request of the first user;
the obtaining module 61 is further configured to obtain a first image sequence, sent by the display device, obtained by shooting the first user within a target time period if the first object information and the second object information meet the abnormality monitoring condition;
a processing module 62, configured to mark the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
In other embodiments of the present application, the processing module 62 is further configured to determine that the time when the second user takes out the display object from the display apparatus and closes the display apparatus is the first time if the distance between the display apparatus and the second user is within the distance threshold range and the display apparatus detects the shopping request of the second user.
In other embodiments of the present application, the obtaining module 61 is further configured to obtain a second image sequence sent by the display apparatus and obtained by shooting the second user, and the processing module 62 is further configured to mark the second image sequence as a normal image sequence, where the second image sequence includes a picture that the second user takes the display object out of the display apparatus and closes the display apparatus at the first time.
In other embodiments of the present application, the processing module 62 is further configured to determine that a distance between the third user and the display device is within a distance threshold; and if the display device detects a shopping request of a third user, determining the moment when the display device is opened by the third user as a second moment.
In other embodiments of the present application, the processing module 62 is further configured to determine that a first user is present within a distance threshold of the display apparatus, determine a sampling interval of object information for displaying the object; the second time is determined based on the first time and a sampling interval of object information of the display object.
In other embodiments of the present application, the processing module 62 is further configured to generate an alarm message, and the output module is configured to output the alarm message, where the alarm message is used to alarm the first user about the abnormal operation event of the display device.
Embodiments of the present application provide a monitoring device, which may be used to implement a monitoring method of a display device provided by the embodiments corresponding to fig. 2 to 5, and as shown in fig. 7, the monitoring device 200 includes (the monitoring device 200 in fig. 7 corresponds to the monitoring apparatus 6 of the display device in fig. 6) including: a memory 71 and a processor 72, wherein; the processor 72 is adapted to execute a monitoring program for the display device stored in the memory 71, the monitoring device 7 being adapted to perform the following steps by means of the processor 72:
acquiring first object information corresponding to a first display object in display equipment at a first moment;
if the distance between the display equipment and the first user is within the distance threshold range and the display equipment does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment;
if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by display equipment and is obtained by shooting a first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
In other embodiments of the present application, the processor 72 is configured to execute a monitoring program for a display device stored in the memory 71 to perform the following steps:
if the distance between the display device and the second user is within the distance threshold range and the display device detects the shopping request of the second user, the time when the second user takes the display object out of the display device and closes the display device is determined as the first time.
In other embodiments of the present application, the processor 72 is configured to execute a monitoring program for a display device stored in the memory 71 to perform the following steps:
and acquiring a second image sequence which is sent by the display equipment and is obtained by shooting a second user, and marking the second image sequence as a normal image sequence, wherein the second image sequence comprises a picture that the second user takes out the display object from the display equipment and closes the display equipment at the first moment.
In other embodiments of the present application, the processor 72 is configured to execute a monitoring program for a display device stored in the memory 71 to perform the following steps:
determining that a distance between the third user and the display device is within a distance threshold range; and if the display device detects a shopping request of a third user, determining the moment when the display device is opened by the third user as a second moment.
In other embodiments of the present application, the processor 72 is configured to execute a monitoring program for a display device stored in the memory 71 to perform the following steps:
determining that a first user appears within a distance threshold range of the display device, and determining a sampling interval of object information of a display object; the second time is determined based on the first time and a sampling interval of object information of the display object.
In other embodiments of the present application, the processor 72 is configured to execute a monitoring program for a display device stored in the memory 71 to perform the following steps:
and generating alarm information and outputting the alarm information, wherein the alarm information is used for alarming the abnormal operation event of the display equipment for the first user.
Embodiments of the application provide a computer readable storage medium storing one or more programs executable by one or more processors to perform the steps of:
acquiring first object information corresponding to a first display object in display equipment at a first moment;
if the distance between the display equipment and the first user is within the distance threshold range and the display equipment does not detect the shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment;
if the first object information and the second object information accord with the abnormal monitoring condition, acquiring a first image sequence which is sent by display equipment and is obtained by shooting a first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period comprises at least a period in which object information of the display device is changed, and the first sequence of images is used to identify the display device and the first user.
In other embodiments of the present application, the one or more programs are executable by the one or more processors to perform the steps of:
if the distance between the display device and the second user is within the distance threshold range and the display device detects a shopping request of the second user, the time when the second user takes out the display object from the display device and closes the display device is determined as the first time.
In other embodiments of the present application, the one or more programs are executable by the one or more processors to perform the steps of:
and acquiring a second image sequence which is sent by the display equipment and is obtained by shooting a second user, and marking the second image sequence as a normal image sequence, wherein the second image sequence comprises a picture that the second user takes out the display object from the display equipment and closes the display equipment at the first moment.
In other embodiments of the present application, the one or more programs are executable by the one or more processors to perform the steps of:
determining that a distance between the third user and the display device is within a distance threshold range; and if the display device detects a shopping request of a third user, determining the moment when the display device is opened by the third user as a second moment.
In other embodiments of the present application, the one or more programs are executable by the one or more processors to perform the steps of:
determining that a first user appears within a distance threshold range of a display device, and determining a sampling interval of object information of a display object; the second time is determined based on the first time and a sampling interval of object information of the display object.
In other embodiments of the application, the one or more programs are executable by the one or more processors to perform the steps of:
and generating alarm information and outputting the alarm information, wherein the alarm information is used for alarming the abnormal operation event of the display equipment for the first user.
Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.
The computer storage medium/Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM), and the like; but may also be various terminals such as mobile phones, computers, tablet devices, personal digital assistants, etc., that include one or any combination of the above-mentioned memories.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment of the present application" or "a previous embodiment" or "some embodiments" or "some implementations" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "an embodiment of the present application" or "the preceding embodiments" or "some implementations" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.
Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.
The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the related art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
It should be noted that the drawings in the embodiments of the present application are only for illustrating schematic positions of the respective devices on the terminal device, and do not represent actual positions in the terminal device, actual positions of the respective devices or the respective areas may be changed or shifted according to actual conditions (for example, a structure of the terminal device), and a scale of different parts in the terminal device in the drawings does not represent an actual scale.
The above description is only for the embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of monitoring display equipment, for use with monitoring equipment, the method comprising:
acquiring first object information corresponding to a first display object in the display equipment at a first moment, wherein the object information comprises weight data and/or quantity data of the display object;
if the distance between the display equipment and a first user is within a distance threshold range and the display equipment does not detect a shopping request of the first user, acquiring second object information corresponding to a second display object in the display equipment at a second moment, wherein the second moment is after the first moment;
if the first object information and the second object information accord with an abnormal monitoring condition, acquiring a first image sequence which is sent by the display equipment and is shot by the first user in a target time period, and marking the first image sequence as an abnormal image sequence; wherein the target period includes at least a period in which object information of the display apparatus is changed, the first image sequence is used to identify the display apparatus and the first user, and the abnormality monitoring condition includes: a first difference of the weight data of the first display object minus the weight data of the second display object in the display apparatus is within a weight variation threshold range and/or a second difference of the quantity data of the first display object minus the quantity data of the second display object in the display apparatus is within a quantity variation threshold range.
2. The method of claim 1, wherein prior to obtaining first object information corresponding to a first display object in the display apparatus at a first time, the method further comprises:
and if the distance between the display equipment and the second user is within the distance threshold range and the display equipment detects a shopping request of the second user, determining that the moment when the second user takes out the display object from the display equipment and closes the display equipment is the first moment.
3. The method of claim 2, further comprising:
and acquiring a second image sequence which is sent by the display equipment and obtained by shooting the second user, and marking the second image sequence as a normal image sequence, wherein the second image sequence comprises a picture that the second user takes out a display object from the display equipment and closes the display equipment at the first moment.
4. The method according to any of claims 1 to 3, wherein prior to said obtaining second object information corresponding to a second display object in the display apparatus at a second time, the method further comprises:
determining that a distance between a third user and the display device is within the distance threshold range;
and if the display equipment detects the shopping request of the third user, determining the moment when the display equipment is opened by the third user as the second moment.
5. The method of claim 1, wherein prior to obtaining second object information corresponding to a second display object in the display apparatus at a second time, the method further comprises:
determining that the first user appears within a distance threshold range of the display apparatus, and determining a sampling interval of object information of a display object;
the second time is determined based on the first time and a sampling interval of object information of the display object.
6. The method according to any one of claims 1 to 3, wherein said marking of said first image sequence as being after an abnormal image sequence comprises:
and generating alarm information and outputting the alarm information, wherein the alarm information is used for alarming the abnormal operation event of the display equipment aiming at the first user.
7. A monitoring device for a display apparatus, characterized in that the monitoring device for a display apparatus comprises:
the display equipment comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring first object information corresponding to a first display object in the display equipment at a first moment, and the object information comprises weight data and/or quantity data of the display object;
the obtaining module is further configured to obtain second object information corresponding to a second display object in the display device at a second time, where the second time is after the first time, if the distance between the display device and the first user is within a distance threshold range and the display device does not detect a shopping request of the first user;
the obtaining module is further configured to obtain a first image sequence, which is sent by the display device and obtained by shooting the first user within a target time period, if an abnormality monitoring condition is met between the first object information and the second object information, where the abnormality monitoring condition includes: a first difference in weight data of the first display object minus weight data of the second display object in the display apparatus is within a weight change threshold range and/or a second difference in quantity data of the first display object minus quantity data of the second display object in the display apparatus is within a quantity change threshold range;
the processing module is used for marking the first image sequence as an abnormal image sequence; wherein the target period includes at least a period in which object information of the display apparatus is changed, and the first image sequence is used to identify the display apparatus and the first user.
8. The apparatus of claim 7, wherein the apparatus comprises:
the processing module is further configured to determine that a time when the second user takes out the display object from the display apparatus and closes the display apparatus is the first time if the distance between the display apparatus and the second user is within the distance threshold range and the display apparatus detects a shopping request of the second user.
9. A monitoring device, comprising:
a memory for storing executable instructions;
a processor for executing executable instructions stored in the memory to implement a method of monitoring a display device as claimed in any one of claims 1 to 6.
10. A computer readable storage medium storing one or more programs which are executable by one or more processors to implement a method of monitoring a display device as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110632626.4A CN113538775B (en) | 2021-06-07 | 2021-06-07 | Method and device for monitoring display equipment, monitoring equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110632626.4A CN113538775B (en) | 2021-06-07 | 2021-06-07 | Method and device for monitoring display equipment, monitoring equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113538775A CN113538775A (en) | 2021-10-22 |
| CN113538775B true CN113538775B (en) | 2022-07-05 |
Family
ID=78124654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110632626.4A Active CN113538775B (en) | 2021-06-07 | 2021-06-07 | Method and device for monitoring display equipment, monitoring equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113538775B (en) |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107920224B (en) * | 2016-10-08 | 2020-02-18 | 杭州海康威视***技术有限公司 | Abnormity warning method, abnormity warning equipment and video monitoring system |
| CN108921048A (en) * | 2018-06-14 | 2018-11-30 | 深圳码隆科技有限公司 | A kind of shopping settlement method, device and user terminal |
| CN109035629A (en) * | 2018-07-09 | 2018-12-18 | 深圳码隆科技有限公司 | A kind of shopping settlement method and device based on open automatic vending machine |
| CN109523428A (en) * | 2018-09-13 | 2019-03-26 | 上海交通大学 | Intelligent food and drink settlement method, system, device and medium based on image recognition technology |
| CN109949074B (en) * | 2019-02-20 | 2024-07-19 | 京东科技信息技术有限公司 | Article display management method, apparatus, device, server, and storage medium |
| CN110084582A (en) * | 2019-04-03 | 2019-08-02 | 北京百度网讯科技有限公司 | Automatic accounting method, device, electronic equipment and storage medium |
| CN110276892A (en) * | 2019-06-26 | 2019-09-24 | 深圳市腾讯计算机***有限公司 | Self-service method, apparatus, equipment and storage medium |
| CN110322249B (en) * | 2019-07-08 | 2022-04-29 | 湖南共睹互联网科技有限责任公司 | Process monitoring method based on guarantee transaction, terminal and storage medium |
| CN110705424B (en) * | 2019-09-25 | 2020-10-02 | 广州市玄武无线科技股份有限公司 | Method and device for positioning commodity display position and storage medium |
| CN112801579B (en) * | 2021-01-21 | 2023-12-08 | 北京顺达同行科技有限公司 | Distribution task abnormity monitoring method, distribution task abnormity monitoring device, computer equipment and storage medium |
-
2021
- 2021-06-07 CN CN202110632626.4A patent/CN113538775B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113538775A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10192408B2 (en) | Registry verification for a mechanized store using radio frequency tags | |
| US11881091B1 (en) | Registry verification with authentication using a mobile device | |
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| CN107862775B (en) | Supermarket commodity anti-theft early warning system and method based on artificial intelligence | |
| CN107464116B (en) | Order settlement method and system | |
| CN103733633B (en) | Video analytic system | |
| CN109243096B (en) | Intelligent container and method for determining commodity taking | |
| US20210407267A1 (en) | Theft prediction and tracking system | |
| US20080319835A1 (en) | Information system and information processing apparatus | |
| EP3510571A1 (en) | Order information determination method and apparatus | |
| CN105518755A (en) | Security system, security method, and non-temporary computer-readable medium | |
| CN109034778A (en) | Automatic selling method, vending machine and computer readable storage medium | |
| CN109353397B (en) | Commodity management method, device and system, storage medium and shopping cart | |
| CN109166000B (en) | Intelligent container and method for creating order | |
| CA3090092A1 (en) | Systems and methods for verifying machine-readable label associated withmerchandise | |
| CN108960132B (en) | Method and device for purchasing commodities in open type vending machine | |
| CN110210866A (en) | Commodity purchasing method and device based on recognition of face | |
| CN109934569B (en) | Settlement method, device and system | |
| CN108171286B (en) | Unmanned selling method and system | |
| CN109671211B (en) | Intelligent container tearing and signing behavior monitoring method, intelligent container and computer storage medium | |
| CN107945392A (en) | Automatic vending machine and the method and storage medium for gathering evidence | |
| CN113538775B (en) | Method and device for monitoring display equipment, monitoring equipment and storage medium | |
| CN109671227B (en) | Intelligent container consumption behavior identification method and device, storage medium and electronic equipment | |
| CN113378601A (en) | Method for preventing goods loss, self-service equipment and storage medium | |
| CN111178265A (en) | Commodity identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |