CN112036303A - Method and device for reminding left-over article, electronic equipment and storage medium - Google Patents

Method and device for reminding left-over article, electronic equipment and storage medium Download PDF

Info

Publication number
CN112036303A
CN112036303A CN202010896526.8A CN202010896526A CN112036303A CN 112036303 A CN112036303 A CN 112036303A CN 202010896526 A CN202010896526 A CN 202010896526A CN 112036303 A CN112036303 A CN 112036303A
Authority
CN
China
Prior art keywords
information
reminding
identity information
target
item
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010896526.8A
Other languages
Chinese (zh)
Inventor
祁凯悦
伍俊
范亦卿
陶莹
许亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Original Assignee
Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Sensetime Lingang Intelligent Technology Co Ltd filed Critical Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Priority to CN202010896526.8A priority Critical patent/CN112036303A/en
Publication of CN112036303A publication Critical patent/CN112036303A/en
Priority to PCT/CN2021/078681 priority patent/WO2022041669A1/en
Priority to JP2023513402A priority patent/JP2023539250A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Traffic Control Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The disclosure relates to a method and a device for reminding a legacy article, an electronic device and a storage medium. The method comprises the following steps: detecting left-over articles in a cabin of a vehicle; in response to detecting a legacy item, determining identity information of a target occupant to which the legacy item belongs; and generating reminding information according to the identity information.

Description

Method and device for reminding left-over article, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of vehicle and computer vision technologies, and in particular, to a method and an apparatus for reminding a left-over object, an electronic device, and a storage medium.
Background
People sometimes leave items (e.g., purses, keys, etc.) in a vehicle while riding the vehicle, causing many unnecessary troubles and losses. Therefore, the method has great significance for detecting and reminding the left articles of the vehicles.
Disclosure of Invention
The present disclosure provides a reminding technical solution for a left-over article.
Detecting left-over articles in a cabin of a vehicle;
in response to detecting a legacy item, determining identity information of a target occupant to which the legacy item belongs;
and generating reminding information according to the identity information.
The identity information of the target passenger belonging to the left-over object is determined in response to the detection of the left-over object by detecting the left-over object in the cabin of the vehicle, and the reminding information is generated according to the identity information, so that the reminding of the left-over object can be performed in a targeted manner based on the identity information of the passenger leaving over the object in the cabin of the vehicle, the reminding effect of the left-over object can be improved, and the guarantee of the property safety of the passenger can be enhanced.
In one possible implementation, the determining, in response to detecting the legacy article, identity information of a target occupant to which the legacy article belongs includes:
in response to detecting an carryover item, determining a location of the carryover item within the bin;
and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin.
In this implementation, by determining the position of the left-over article in the cabin in response to the detection of the left-over article, and determining the identity information of the target passenger to which the left-over article belongs according to the position of the left-over article in the cabin, the accuracy of the determined identity information of the target passenger left over the left-over article in the cabin can be improved by using the position association relationship between the left-over article and the passenger.
In one possible implementation, the determining the identity information of the target occupant to which the carry-over item belongs according to the position of the carry-over item in the cabin includes:
determining a relevant area of the carry-over item within the compartment according to the location of the carry-over item within the compartment;
based on the video stream in the cabin, carrying out face recognition on the related area to obtain a face recognition result of the related area;
and determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area.
The method comprises the steps of determining a relevant area of the left article in the cabin according to the position of the left article in the cabin, carrying out face recognition on the relevant area based on a video stream in the cabin to obtain a face recognition result of the relevant area, and determining the identity information of a target passenger to which the left article belongs based on the face recognition result of the relevant area, so that the accuracy of the identity information of the determined target passenger can be improved by further utilizing a face recognition technology, and the calculation complexity of determining the target passenger can be reduced.
In a possible implementation manner, the performing face recognition on the relevant area based on the video stream in the cabin to obtain a face recognition result of the relevant area includes:
determining a target video frame from a video stream of the capsule according to a first occurrence time of the legacy article in the video stream;
and carrying out face recognition on the relevant area in the target video frame to obtain a face recognition result of the relevant area.
According to the first occurrence time of the left-over article in the video stream in the cabin, a target video frame is determined from the video stream, and a face recognition is carried out on a relevant area in the target video frame to obtain a face recognition result of the relevant area, so that the efficiency of determining the identity information of a target passenger is improved, and the accuracy of the determined identity information of the target passenger is improved.
In one possible implementation manner, the determining, based on the face recognition result of the relevant area, the identity information of the target occupant to which the carry-over article belongs includes:
determining candidate occupants related to the left-behind item based on the face recognition result of the related area;
and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream, of the candidate passengers.
The accuracy of the identity information of the determined target passenger can be further improved by determining candidate passengers related to the carry-over article based on the face recognition result of the related area, and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers of the candidate passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream.
In one possible implementation, the determining, in response to detecting the legacy article, identity information of a target occupant to which the legacy article belongs includes:
in response to detecting a carryover, identity information of a target occupant to which the carryover is affiliated is determined based on a first time of occurrence of the carryover in a video stream within the cabin.
According to the implementation mode, the identity information of the target passenger to which the left-over article belongs can be determined quickly and conveniently.
In one possible implementation manner, the determining, according to a first occurrence time of the legacy article in a video stream in the cabin, identity information of a target passenger to which the legacy article belongs includes:
acquiring a use record of the vehicle, wherein the use record comprises identity information of a user and corresponding use time;
according to the usage record, determining the identity information of the target user, wherein the corresponding usage time of the target user is matched with the first occurrence time of the legacy article in the video stream;
and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs.
By acquiring the use record of the vehicle, determining the identity information of the target user, of which the corresponding use time is matched with the first occurrence time of the legacy article in the video stream, according to the use record, and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs, the identity information of the target passenger can be accurately determined based on the use record of the vehicle.
In one possible implementation form of the method,
prior to the detecting of the carry-over item within the cabin of the vehicle, the method further comprises: acquiring images and/or videos of passengers entering the cabin;
detecting information of items carried by the occupant based on images and/or video of the occupant when entering the cabin;
binding information of an article carried by the occupant with identity information of the occupant;
the determining identity information of a target occupant to which the legacy item belongs in response to detecting the legacy item includes:
in response to detecting a legacy item, determining identity information of an occupant bound to the legacy item as identity information of a target occupant to which the legacy item belongs.
In this implementation, by acquiring an image and/or a video of a passenger entering the cabin, detecting information of an article carried by the passenger based on the image and/or the video of the passenger entering the cabin, binding the information of the article carried by the passenger with identity information of the passenger, and determining that the identity information of the passenger bound with a legacy article is identity information of a target passenger to which the legacy article belongs in response to detection of the legacy article, the identity information of the target passenger can be accurately determined.
In one possible implementation form of the method,
the generating of the reminding information according to the identity information includes: generating first reminding information according to the identity information;
after the generating of the reminder information according to the identity information, the method further includes: and sending the first reminding information to a target terminal corresponding to the identity information.
In the implementation mode, the first reminding information is sent to the target terminal corresponding to the identity information, so that the target passenger can timely know the condition of leaving articles in the vehicle.
In one possible implementation form of the method,
the generating of the reminding information according to the identity information includes: generating second reminding information according to the identity information and the information of the left-over article;
after the generating of the reminder information according to the identity information, the method further includes: and sending the second reminding information to a management terminal connected with the vehicle and/or a management server connected with the vehicle.
In this implementation, by sending the second warning information to a management terminal connected to the vehicle and/or a management server connected to the vehicle, it is possible for a management party (e.g., an operation company) to timely notify a target passenger and keep the legacy article of the target passenger in storage in a case where the target passenger does not timely retrieve the legacy article.
In a possible implementation manner, the generating second reminding information according to the identity information and the information of the legacy article includes:
and generating second reminding information according to the identification information of the vehicle, at least one of the boarding place of the target passenger and the alighting place of the target passenger, the identity information and the information of the left-over articles.
In this implementation, by generating the second reminding information according to the identification information of the vehicle, at least one of the boarding location of the target passenger and the alighting location of the target passenger, and the identity information and the information of the legacy article, the management party (e.g., an operating company) of the vehicle is facilitated to record more relevant information of the legacy article, thereby facilitating finding of the target passenger and facilitating legacy article management.
In one possible implementation form of the method,
the generating of the reminding information according to the identity information includes: when the target passenger is not the driver of the vehicle, generating third reminding information according to the identity information and the information of the left-over articles;
the method further comprises the following steps: sending the third reminding information to a terminal of a driver of the vehicle;
and/or the presence of a gas in the gas,
the generating of the reminding information according to the identity information includes: generating fourth reminding information according to the identity information;
the method further comprises the following steps: and generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information.
In this implementation, when the target passenger is not the driver of the vehicle, third reminding information is generated according to the identity information and the information of the left-over article, and the third reminding information is sent to the terminal of the driver of the vehicle, so that the driver of the vehicle can be reminded to help the target passenger to temporarily store the left-over article. The fourth reminding information is generated according to the identity information, and the control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information is generated, so that passengers in the cabin can be reminded to take corresponding safety keeping measures in time on the basis, and the risk of losing articles is reduced.
In a possible implementation manner, after the sending the first reminding information to the target terminal corresponding to the identity information, the method further includes:
and generating repeated reminding information in response to receiving the repeated reminding request returned by the target terminal.
In the implementation mode, the repeated reminding information is generated in response to the received repeated reminding request returned by the target terminal, so that the target passenger can be reminded again when the target passenger cannot return to the cabin to take back the left-over article in a short time, and the protection of the property safety of the target passenger is further promoted.
In one possible implementation, after the generating the repeat reminder information, the method further includes at least one of:
responding to a time interval between the current time and the time for sending the first reminding information reaching a first preset time length, and sending the repeated reminding information to the target terminal;
responding to the time interval between the current time and the time of receiving the repeated reminding request reaching a second preset time length, and sending the repeated reminding information to the target terminal;
responding to that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length and the fact that the left-over article is not taken out of the cabin is detected, and sending the repeated reminding information to the target terminal;
and sending the repeated reminding information to the target terminal in response to that the time interval between the current time and the time of receiving the repeated reminding request reaches a second preset time length and that the left-over article is not taken out of the cabin.
According to the implementation mode, when the target passenger cannot return to the cabin in time to take back the left-over articles, the repeated reminding information can be sent to the target terminal, so that the guarantee of property safety of the target passenger is further enhanced.
In one possible implementation manner, after the generating the repeated reminding information, the method further includes:
and generating control information for controlling vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information in response to the detection that the target passenger returns to the cabin.
By responding to the detection that the target passenger returns to the cabin, control information for controlling vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information is generated, so that the target passenger can be reminded in time after the target passenger returns to the cabin.
In one possible implementation, the method further includes:
receiving a first reminding setting request through a vehicle machine of the vehicle;
according to the first reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
According to the implementation mode, the user can flexibly set the types of the left-over articles which are not reminded, need to be reminded and need to be reminded repeatedly through the vehicle machine according to the requirements of the user.
In one possible implementation, the method further includes:
receiving a second reminding setting request returned by the target terminal;
according to the second reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
According to the implementation mode, the user can flexibly set the types of the left-over articles which are not reminded, need to be reminded and need to be reminded repeatedly through the target terminal according to the requirements of the user.
According to an aspect of the present disclosure, there is provided a reminder device for an article left, including:
the first detection module is used for detecting the left articles in the cabin of the vehicle;
a determination module to determine, in response to detecting a legacy item, identity information of a target occupant to which the legacy item belongs;
and the first generating module is used for generating reminding information according to the identity information.
In one possible implementation, the determining module is configured to:
in response to detecting an carryover item, determining a location of the carryover item within the bin;
and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin.
In one possible implementation, the determining module is configured to:
determining a relevant area of the carry-over item within the compartment according to the location of the carry-over item within the compartment;
based on the video stream in the cabin, carrying out face recognition on the related area to obtain a face recognition result of the related area;
and determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area.
In one possible implementation, the determining module is configured to:
determining a target video frame from a video stream of the capsule according to a first occurrence time of the legacy article in the video stream;
and carrying out face recognition on the relevant area in the target video frame to obtain a face recognition result of the relevant area.
In one possible implementation, the determining module is configured to:
determining candidate occupants related to the left-behind item based on the face recognition result of the related area;
and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream, of the candidate passengers.
In one possible implementation, the determining module is configured to:
in response to detecting a carryover, identity information of a target occupant to which the carryover is affiliated is determined based on a first time of occurrence of the carryover in a video stream within the cabin.
In one possible implementation, the determining module is configured to:
acquiring a use record of the vehicle, wherein the use record comprises identity information of a user and corresponding use time;
according to the usage record, determining the identity information of the target user, wherein the corresponding usage time of the target user is matched with the first occurrence time of the legacy article in the video stream;
and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs.
In one possible implementation form of the method,
the device further comprises: the acquisition module is used for acquiring images and/or videos of passengers entering the cabin;
the second detection module is used for detecting information of articles carried by the passenger based on images and/or videos when the passenger enters the cabin;
the binding module is used for binding the information of the articles carried by the passenger with the identity information of the passenger;
the determination module is to:
in response to detecting a legacy item, determining identity information of an occupant bound to the legacy item as identity information of a target occupant to which the legacy item belongs.
In one possible implementation form of the method,
the first generation module is used for: generating first reminding information according to the identity information;
the device further comprises: and the first sending module is used for sending the first reminding information to a target terminal corresponding to the identity information.
In one possible implementation form of the method,
the first generation module is used for: generating second reminding information according to the identity information and the information of the left-over article;
the device further comprises: and the second sending module is used for sending the second reminding information to a management terminal connected with the vehicle and/or a management server connected with the vehicle.
In one possible implementation, the first generating module is configured to:
and generating second reminding information according to the identification information of the vehicle, at least one of the boarding place of the target passenger and the alighting place of the target passenger, the identity information and the information of the left-over articles.
In one possible implementation form of the method,
the first generation module is used for: when the target passenger is not the driver of the vehicle, generating third reminding information according to the identity information and the information of the left-over articles;
the device further comprises: the third sending module is used for sending the third reminding information to a terminal of a driver of the vehicle;
and/or the presence of a gas in the gas,
the first generation module is used for: generating fourth reminding information according to the identity information;
the device further comprises: and the first control module is used for generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information.
In one possible implementation, the apparatus further includes:
and the second generation module is used for responding to the received repeated reminding request returned by the target terminal and generating repeated reminding information.
In a possible implementation manner, the apparatus further includes a fourth sending module, where the fourth sending module is configured to at least one of:
responding to a time interval between the current time and the time for sending the first reminding information reaching a first preset time length, and sending the repeated reminding information to the target terminal;
responding to the time interval between the current time and the time of receiving the repeated reminding request reaching a second preset time length, and sending the repeated reminding information to the target terminal;
responding to that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length and the fact that the left-over article is not taken out of the cabin is detected, and sending the repeated reminding information to the target terminal;
and sending the repeated reminding information to the target terminal in response to that the time interval between the current time and the time of receiving the repeated reminding request reaches a second preset time length and that the left-over article is not taken out of the cabin.
In one possible implementation, the apparatus further includes:
and the second control module is used for responding to the detection that the target passenger returns to the cabin and generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information.
In one possible implementation, the apparatus further includes:
the first receiving module is used for receiving a first reminding setting request through a vehicle machine of the vehicle;
the first setting module is used for setting at least one of the following according to the first reminding setting request: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
In one possible implementation, the apparatus further includes:
the second receiving module is used for receiving a second reminding setting request returned by the target terminal;
the second setting module is used for setting at least one of the following according to the second reminding setting request: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
According to an aspect of the present disclosure, there is provided an electronic device including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.
According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.
In the embodiment of the disclosure, the identity information of the target passenger to which the left-over object belongs is determined in response to the detection of the left-over object by detecting the left-over object in the cabin of the vehicle, and the reminding information is generated according to the identity information, so that the reminding of the left-over object can be performed in a targeted manner based on the identity information of the passenger who has left-over object in the cabin of the vehicle, the reminding effect of the left-over object can be improved, and the property safety of the passenger can be guaranteed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.
Fig. 1 shows a flowchart of a method for reminding a legacy article according to an embodiment of the present disclosure.
Fig. 2 shows a schematic diagram of an application scenario of a private car.
Fig. 3 shows a schematic diagram of an application scenario for sharing a car.
Fig. 4 shows a schematic diagram of an application scenario of a network appointment vehicle.
Fig. 5 shows a block diagram of a reminder device for a legacy item according to an embodiment of the present disclosure.
Fig. 6 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure.
Fig. 7 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
In the embodiment of the disclosure, the identity information of the target passenger to which the left-over object belongs is determined in response to the detection of the left-over object by detecting the left-over object in the cabin of the vehicle, and the reminding information is generated according to the identity information, so that the reminding of the left-over object can be performed in a targeted manner based on the identity information of the passenger who has left-over object in the cabin of the vehicle, the reminding effect of the left-over object can be improved, and the property safety of the passenger can be guaranteed.
Fig. 1 shows a flowchart of a method for reminding a legacy article according to an embodiment of the present disclosure. The execution subject of the reminding method of the left-over article can be a reminding device of the left-over article. In a possible implementation manner, the reminding method of the legacy article may be executed by a terminal device or a server or other processing device. The terminal device may be a vehicle-mounted device, a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, or a wearable device. The on-board device may be a vehicle or a domain controller in a vehicle cabin, or may be a device host for performing data processing operations such as images in an OMS (Occupant Monitoring System) or a DMS (Driver Monitoring System). In some possible implementations, the method for reminding the legacy article may be implemented by a processor calling a computer readable instruction stored in a memory. As shown in fig. 1, the method for reminding the legacy articles includes steps S11 to S13.
In step S11, a carry-over detection is performed on the inside of the cabin of the vehicle.
In the embodiment of the present disclosure, the vehicle may be a vehicle, and may also be a vehicle for carrying people or cargo, such as a ship or an airplane. Accordingly, the cabin of the vehicle may be a car cabin, a ship cabin, a cabin or the like. The vehicle may be a private car or a commercial vehicle, for example, the commercial vehicle may be a shared automobile, a network appointment car, a taxi, a bus, a school bus, a long-short-distance truck, a long-short-distance passenger car, a train, a subway, a tram, or the like.
In one possible implementation, the detecting the carry-over object in the cabin of the vehicle may include: obtaining a video stream within a cabin of the vehicle; and detecting the left-over articles in the cabin based on the video stream in the cabin.
In this implementation, the video stream in the cabin may be captured by a camera disposed in the cabin. For example, the cameras disposed within the chamber may include OMS cameras, DMS cameras, and the like. The OMS may also perform other occupant detection functions, including detection of occupant attributes (e.g., age, gender, emotional state, physical state, etc.), detection of speaking actions of the occupant, and so forth. As an example of this implementation, the processor may obtain a video stream of the cabin of the vehicle from a camera disposed in the cabin and perform the carry-over detection of the carry-over based on the video stream of the cabin.
In this implementation, the detection of the carry-over within the compartment may include detecting at least one of the presence of a carry-over, a carry-over category detection, a carry-over location detection, and the like.
As an example of this implementation, the performing the detection of the carry-over object in the cabin based on the video stream in the cabin may include: detecting whether a legacy item is present in the chamber based on the video stream in the chamber.
In a possible implementation manner, an image of all passengers before entering the cabin may be acquired as a reference image, and the image of all passengers after leaving the cabin is compared with the reference image, so as to determine the left-over articles in the cabin.
In another possible implementation manner, for each passenger, an image before the passenger enters the cabin is acquired as a reference image corresponding to the passenger, and an image after the passenger leaves the cabin is compared with the reference image corresponding to the passenger, so as to determine the carry-over object appearing in the cabin of the vehicle during the passenger takes the vehicle. The image before the passenger enters the cabin may include a video frame before the video frame in which the passenger is first detected in the video stream in the cabin. The image of the occupant after leaving the cabin may include a video frame subsequent to a last detected video frame of the occupant in the video stream in the cabin.
In embodiments of the present disclosure, the detection of the carry-over within the compartment may include detecting at least one of the presence of a carry-over, a carry-over category detection, a carry-over location detection, and the like.
In a possible implementation manner, the image of the passenger leaving the cabin may be compared with a reference image, and whether an article that does not exist in the reference image exists in the image of the passenger leaving the cabin may be determined. If so, determining that the left-over article exists; if not, it may be determined that no legacy items exist. In another possible implementation manner, the image of the passenger leaving the cabin may be compared with a reference image, and whether an article that is not present in the reference image and belongs to a preset category exists in the image of the passenger leaving the cabin may be determined. If so, determining that the left-over article exists; if not, it may be determined that no legacy items exist.
In one possible implementation, the type of the legacy item may be detected at the same time as the presence or absence of the legacy item is detected. In another possible implementation, in response to detecting the presence of the legacy item, a category identification may be performed on the detected legacy item, determining a category of the legacy item.
In one possible implementation, the location of the legacy item may be detected while detecting the presence of the legacy item and/or detecting the category of the legacy item. In another possible implementation, the location of the legacy item may be detected in response to detecting the presence of the legacy item.
In the embodiment of the disclosure, by detecting the left-over article in the cabin of the vehicle, the detection result of the left-over article in the cabin can be obtained. If the legacy item detection result includes detecting a legacy item, the legacy item detection result may further include information of the legacy item.
In one possible implementation, the information of the legacy item may include a category of the legacy item. For example, the category of the left-behind item may be a purse, a key, a cell phone, an umbrella, a briefcase, a bag, a luggage case, or a water bottle, etc.
In another possible implementation, the information of the legacy article may include a location of the legacy article within the compartment.
As one example of this implementation, the location of the carry-over item within the compartment may include the approximate location of the carry-over item within the compartment. For example, the location of the carry-over item within the cabin may be a primary driver's seat area, a secondary driver's seat area, a rear seating area, a trunk area, or the like. For another example, the position of the carry-over article in the cabin may be on the main driver seat, on the passenger seat, on the rear seat, in the trunk, below the main driver seat, below the passenger seat, below the rear seat, a floor in front of the main driver seat, a floor in front of the passenger seat, a floor of the rear seat, and the like.
As another example of this implementation, the location of the carry-over item within the compartment may include a range of locations or an exact location to which the carry-over item belongs within the compartment. In one example, the location of the legacy article within the chamber may include image coordinates in an image coordinate system corresponding to an image of the legacy article in a video stream within the chamber. In another example, the location of the legacy article within the compartment may include spatial coordinates of the legacy article in a spatial coordinate system within the compartment.
In another possible implementation, the information of the legacy item may include an image of the legacy item. As an example of this implementation, the region where the legacy article is located may be intercepted from one or more video frames containing the legacy article in the video stream in the cabin, so as to obtain an image of the legacy article. As another example of this implementation, one or more video frames in the video stream within the capsule that contain a carryover may be used as the image of the carryover. As another example of this implementation, the legacy article may be marked according to a position of the legacy article in the video frame containing the legacy article, and the video frame marked with the legacy article may be used as an image of the legacy article. In this example, the marking of the legacy item may be presented in the video frame containing the legacy item by marking a bounding box of the legacy item or by pointing at the legacy item with an arrow, or the like.
In step S12, in response to detecting the carry-over item, identity information of a target occupant to which the carry-over item belongs is determined.
In the disclosed embodiments, the occupants of the vehicle may include any person riding the vehicle. For example, the occupants of the vehicle may include a driver and/or a non-driver in the vehicle. For example, in the application scenario of a private car, the non-driver may be a family or friend of the car owner, etc.; in the application scenario of the net appointment, the non-driver may be a passenger or the like.
In the disclosed embodiment, the target occupant to which the carry-over item belongs represents an occupant who carries over the carry-over item in the cabin. The identity information of the target occupant may be information related to the identity of the target occupant. For example, the identity information of the target occupant may include at least one of a name, an identification number, an account number, a serial number, a telephone number, a face image, and the like of the target occupant.
In the embodiment of the disclosure, in response to the detection of the left-behind object, a target passenger to which the left-behind object belongs may be determined according to the association relationship between the position of the left-behind object in the cabin and the position of the passenger in the cabin, and the target passenger may be subjected to face recognition to obtain the identity information of the target passenger. That is, the identity information of the target occupant to which the carry-over item belongs may be determined from the correlation between the carry-over item and the position of the occupant within the cabin. Alternatively, in embodiments of the present disclosure, in response to detecting the legacy item, identity information of a target occupant to which the legacy item belongs may be determined based on the time of occurrence of the legacy item within the cabin and the time the occupant uses the vehicle. That is, the identity information of the target occupant to which the carry-over item belongs may be determined from the correlation between the time of occurrence of the carry-over item in the cabin and the time when the occupant uses the vehicle. Alternatively, the attributes of the legacy article, such as its color, category, etc., associated with the user population may be identified in response to detecting the legacy article, and then the identity information of the target occupant to whom the legacy article belongs may be determined based on the degree of correlation between the person present in the cabin and the attributes of the legacy article, e.g., when the legacy article is detected as a female carrying bag and there is only one woman present in the cabin within a corresponding time period, it may be determined that the legacy woman carrying bag belongs to the woman.
In step S13, a reminder is generated according to the identity information.
In one possible implementation, the identity information may be included in the reminder information. For example, if the identity information includes the name "zhang san", a reminder message such as "mr. zhang, your mobile phone left in the car, please get back in time" may be generated. In another possible implementation, the alert information for sending to the terminal device of the target occupant may be generated according to the identity information of the target occupant. For example, a reminder message "there is an article left on the vehicle" and the configuration item of the message is "send to the mobile phone number 133 ×".
In another possible implementation manner, the forms of the reminding information corresponding to different identity information may be different, for example, the form of the reminding information may be a text reminding information, a voice reminding information, a picture reminding information, an animation reminding information, and the like. As an example of the implementation manner, the reminding information may be generated according to a form of the reminding information corresponding to the identity information. For example, if the form of the reminding information corresponding to the identity information is voice reminding information, the voice reminding information may be generated. As another example of this implementation, a picture of the items left behind in the cabin may be intercepted, and a picture reminder message may be generated for sending to the cell phone of the target occupant. As another example of this implementation manner, according to a form of the reminding information corresponding to the identity information, reminding information for reminding a target passenger corresponding to the identity information may be generated. For example, if the form of the reminding information corresponding to the identity information is animation reminding information, and the left article is a mobile phone, the animation reminding information of the mobile phone left in the vehicle by the animation character corresponding to the identity information of the target passenger can be generated according to the identity information of the target passenger. For example, if the target occupant is a middle-aged man, the animation character corresponding to the identification information of the target occupant may be an animation character of the middle-aged man.
In the embodiment of the disclosure, the identity information of the target passenger to which the left-over object belongs is determined in response to the detection of the left-over object by detecting the left-over object in the cabin of the vehicle, and the reminding information is generated according to the identity information, so that the reminding of the left-over object can be performed in a targeted manner based on the identity information of the passenger who has left-over object in the cabin of the vehicle, the reminding effect of the left-over object can be improved, and the property safety of the passenger can be guaranteed.
An application scenario of the embodiment of the present disclosure is explained below. In this application scenario, a video stream in the cabin may be captured by a camera disposed in the cabin of the vehicle. The camera may send the video stream within the chamber to a processor. The processor may perform carryover detection on the capsule based on a video stream within the capsule. The processor may determine, in response to detecting a carry-over item, identity information of a target occupant to which the carry-over item belongs. For example, if the identity information includes the name "zhang san", and the left-over article is a mobile phone, the processor may generate a text reminding message "mr. zhang, your mobile phone is left in the vehicle, please retrieve in time", and send the text reminding message to the mobile phone of the target passenger. Meanwhile, the lamp of the vehicle can be controlled to flash to remind the target passenger just leaving the vehicle to return to the vehicle to retrieve the article.
In one possible implementation, the determining, in response to detecting the legacy article, identity information of a target occupant to which the legacy article belongs includes: in response to detecting an carryover item, determining a location of the carryover item within the bin; and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin.
As one example of this implementation, the detecting the carry-over item within the cabin of the vehicle may include detecting a location of the carry-over item within the cabin; the legacy item information may include a location of the legacy item within the compartment. According to this example, the location of the carry-over item within a cabin of a vehicle may be obtained by carrying out a carry-over item detection within the cabin. As another example of this implementation, if the location of the legacy article within the compartment is not detected in the above step S11, the location of the legacy article within the compartment may continue to be detected based on the video stream within the compartment after step S11 in response to detecting the legacy article.
As an example of this implementation, the location of the carry-over item within the compartment may include the location where the carry-over item first appeared within the compartment. As another example of this implementation, the location of the carry-over item within the compartment may include the last occurrence of the carry-over item within the compartment. As another example of this implementation, the location of the carry-over item within the compartment may include a plurality of locations where the carry-over item has occurred within the compartment. For example, the location of the carry-over item within the compartment may include various locations where the carry-over item has occurred within the compartment.
In this implementation manner, a target passenger to which the legacy article belongs may be determined according to the association relationship between the position of the legacy article in the cabin and the position of the passenger in the cabin, and the target passenger is subjected to face recognition to determine the identity information of the target passenger.
In this implementation, by determining the position of the legacy article in the cabin in response to the detection of the legacy article, and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin, since the legacy article and the target passenger to which the legacy article belongs generally have a strong correlation in position, the accuracy of the determined identity information of the target passenger to which the legacy article is left in the cabin can be improved by using the position correlation between the legacy article and the passenger.
As an example of this implementation, the determining identity information of a target occupant to which the carry-over item belongs according to the location of the carry-over item within the cabin includes: determining a relevant area of the carry-over item within the compartment according to the location of the carry-over item within the compartment; based on the video stream in the cabin, carrying out face recognition on the related area to obtain a face recognition result of the related area; and determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area. Wherein the region of interest may comprise a spatial region within a distance of the perimeter of the legacy article, such as a three-dimensional spatial coordinate region that diverges outward, centered on the location coordinate of the legacy article. Alternatively, the relevant area may be a preset space area of the vehicle cabin to which the position of the left-behind object belongs, where the preset space area of the vehicle cabin may be obtained by dividing areas in the vehicle cabin, such as a driver area, a passenger area, a left-side rear area, a right-side rear area, and the like.
In one example, the location of the carry-over item within the compartment may include: image coordinates of the legacy article in an image coordinate system corresponding to an image in the video stream within the chamber; the relevant area may include: a relative image coordinate region of the legacy article in the image coordinate system; the performing of the human face recognition on the video stream in the cabin based on the relevant area may include: and carrying out face recognition on the relevant area in the video stream in the cabin.
In another example, the location of the carry-over item within the compartment may include: spatial coordinates of the left-behind item in a spatial coordinate system within the compartment; the relevant area may include: an associated spatial coordinate region of the carry-over item within the in-cabin spatial coordinate region; the performing of the human face recognition on the video stream in the cabin based on the relevant area may include: determining an image coordinate area corresponding to the relevant space coordinate area; and carrying out face recognition on an image coordinate area corresponding to the relevant space coordinate area in the video stream in the cabin.
In one example, the area of interest of the item left behind within the compartment may be a predetermined size area defined by geometric centers of coordinates of the item left behind within the compartment. For example, the relevant area of the legacy article in the chamber may be a relevant image coordinate area of a preset size determined by taking image coordinates in an image coordinate system corresponding to an image of the legacy article in a video stream in the chamber as a geometric center. For another example, the relevant area of the legacy article in the chamber may be a relevant spatial coordinate area of a preset size determined by taking the spatial coordinates of the legacy article in the spatial coordinate system in the chamber as the geometric center.
In another example, a correspondence between the location of the item within the compartment and the associated zone of the item within the compartment may be pre-established, such that the associated zone of the item within the compartment may be determined based on the pre-established correspondence and the location of the item within the compartment. For example, if the position of the legacy article in the cabin is within a first preset coordinate interval, the relevant area of the legacy article in the cabin may be a main driver seat area; if the position of the left article in the cabin is within a second preset coordinate interval, the relevant area of the left article in the cabin can be a passenger seat area; the position of the article left in the cabin is within a third preset coordinate interval, and then the relevant area of the article left in the cabin can be a rear seat area.
In one example, face recognition may be performed on an occupant in the relevant area, at least one body part of which has contacted the left-behind object, to obtain a face recognition result. In another example, face recognition may be performed on all the occupants in the relevant area to obtain a face recognition result.
In this example, by determining a relevant region of the legacy article in the cabin according to the position of the legacy article in the cabin, performing face recognition on the relevant region based on a video stream in the cabin to obtain a face recognition result of the relevant region, and determining the identity information of a target passenger to which the legacy article belongs based on the face recognition result of the relevant region, it is possible to extract an image of a region of a video frame included in the video stream, which is positionally related to the legacy article, improve the accuracy of the determined identity information of the target passenger by using a face recognition technology, and reduce the computational complexity of determining the target passenger.
In one example, the performing face recognition on the relevant area based on the video stream in the cabin to obtain a face recognition result of the relevant area includes: determining a target video frame from a video stream of the capsule according to a first occurrence time of the legacy article in the video stream; and carrying out face recognition on the relevant area in the target video frame to obtain a face recognition result of the relevant area.
In this example, the first time that the legacy article appears in the video stream within the chamber may represent the time in the video stream within the chamber that the first video frame containing the legacy article was in the video stream within the chamber. The number of the target video frames may be one or more. For example, the target video frame may include a first video frame containing the legacy article in a video stream within the chamber. As another example, the target video frame may also include a number of video frames after the first video frame containing the legacy article. As another example, the target video frame may also include a number of video frames prior to the first video frame containing the legacy article.
In this example, by determining a target video frame from the video stream according to the first occurrence time of the legacy article in the video stream in the cabin, performing face recognition on a relevant area in the target video frame to obtain a face recognition result of the relevant area, and further extracting the video frame relevant to the legacy article from the time dimension, the number of video frames to be analyzed is reduced, thereby contributing to improving the efficiency of determining the identity information of the target occupant and improving the accuracy of the determined identity information of the target occupant.
In one example, the determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area includes: determining candidate occupants related to the left-behind item based on the face recognition result of the related area; and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream, of the candidate passengers.
In this example, the face recognition result of the relevant area may include the identity information of each detected face. The occupants corresponding to the identity information of each face included in the face recognition result may be respectively regarded as the candidate occupants, or the occupants having at least one body part contacting the article left behind among the occupants corresponding to the identity information of each face included in the face recognition result may be regarded as the candidate occupants. The number of candidate occupants may be one or more. If the number of the candidate occupants is one, the identity information of the candidate occupants may be determined as the identity information of the target occupant to which the carry-over item belongs, or further, whether the candidate occupants are the target occupants may be determined by a matching result of the occurrence times of the candidate occupants and the carry-over item. If the number of the candidate passengers is multiple, the multiple candidate passengers can be screened according to the time when the candidate passengers enter the cabin and the time when the candidate passengers leave the cabin, and the identity information of the target passengers to which the left-over articles belong is determined. In this example, the time of the occupant entering the cabin and the time of the occupant leaving the cabin may be determined by performing occupant detection on a video stream within the cabin.
In this example, by determining candidate occupants related to the left-behind article based on the face recognition result of the related region, and determining the identity information of the target occupant to which the left-behind article belongs according to the occupants, which enter the cabin at a time earlier than or equal to the first occurrence time of the left-behind article in the video stream and leave the cabin at a time later than the first occurrence time of the left-behind article in the video stream, the degree of engagement between the left-behind article and the occurrence time of the target occupant in the video stream in the vehicle cabin can be further improved, and the accuracy of the identity information of the determined target occupant can be improved.
In one example, if there are multiple occupants with body parts that have contacted the carry-over item among the candidate occupants that have entered the cabin at a time earlier than or equal to the first occurrence of the carry-over item in the video stream and exited the cabin at a time later than the first occurrence of the carry-over item in the video stream, the identity information of the candidate occupant that has contacted the carry-over item earliest may be determined as the identity information of the target occupant to which the carry-over item belongs. In another example, if there are multiple occupants with body parts in contact with the carry-over item among the candidate occupants who enter the cabin at a time earlier than or equal to the first occurrence time of the carry-over item in the video stream and leave the cabin at a time later than the first occurrence time of the carry-over item in the video stream, the identification information of the candidate occupant having the longest contact time with the carry-over item may be determined as the identification information of the target occupant to which the carry-over item belongs. Alternatively, if there are a plurality of occupants whose body parts have been in contact with the carry-over article among candidate occupants whose entry time into the cabin is earlier than or equal to the first appearance time of the carry-over article in the video stream and whose exit time from the cabin is later than the first appearance time of the carry-over article in the video stream, the target occupant may be further discriminated from other attribute information relating to characteristics such as sex, age, etc. of the person to which the object article has been in contact, and for example, when it is detected that the object article is a lady bag, the probability that the object article is assigned to a female occupant is higher than the probability that the object article is assigned to a male occupant among male occupants and female occupants whose body parts have been in contact with the object article.
As another example of this implementation, a target occupant to which the carry-over item belongs may be determined based on a location of the carry-over item within the compartment and a location of at least one body part of the occupant within the compartment; and carrying out face recognition on the target passenger to obtain the identity information of the target passenger. For example, if the occupant a carries the carry-over article B when entering the cabin (e.g., the occupant a holds the carry-over article B in his hand when getting on the vehicle, etc.), it may be determined that the target occupant to which the carry-over article B belongs is the occupant a.
In one possible implementation, the determining, in response to detecting the legacy article, identity information of a target occupant to which the legacy article belongs includes: in response to detecting a carryover, identity information of a target occupant to which the carryover is affiliated is determined based on a first time of occurrence of the carryover in a video stream within the cabin.
In this implementation manner, for an application scenario in which a private car, a network appointment car, a shared car, and the like have a small number of passengers appearing in the cabin at the same time, only the first appearance time of the legacy article in the video stream in the cabin may be considered, and the identity information of the target passenger to which the legacy article belongs may be determined according to the first appearance time of the legacy article in the video stream in the cabin. According to the implementation mode, the identity information of the target passenger to which the left-over article belongs can be determined quickly and conveniently.
Of course, in other implementations, for application scenarios such as private car, online car appointment, shared car, etc., the first time of occurrence of the legacy article in the video stream in the cabin may also be considered, while the position of the legacy article in the cabin is also considered. For example, if there are multiple occupants in a private car, the target occupant to whom the carryover item belongs may be accurately determined by considering both the time of first occurrence of the carryover item in the video stream of the cabin and the location of the carryover item within the cabin. As another example, in a ride share scenario of a network appointment, a target occupant (i.e., a ride share or driver) to which a legacy item belongs may be accurately determined by considering both a time of first occurrence of the legacy item in a video stream within the cabin and a location of the legacy item within the cabin.
As an example of this implementation, the determining, according to a first occurrence time of the carry-over item in a video stream within the cabin, identity information of a target occupant to which the carry-over item belongs includes: acquiring a use record of the vehicle, wherein the use record comprises identity information of a user and corresponding use time; according to the usage record, determining the identity information of the target user, wherein the corresponding usage time of the target user is matched with the first occurrence time of the legacy article in the video stream; and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs.
For example, in an application scenario of a network appointment, the vehicle may be a network appointment, the usage record of the vehicle may be an order record of the network appointment, the user may be a passenger, and the corresponding usage time may be a time when the passenger takes the network appointment. By obtaining the order record of the network appointment, according to the order record of the network appointment, the identity information of the target passenger, of which the time for taking the network appointment is matched with the first occurrence time of the legacy article in the video stream, is determined, and the identity information of the target passenger is determined as the identity information of the target passenger to which the legacy article belongs, so that the identity information of the passenger who leaves the legacy article in the network appointment can be accurately determined. Of course, in a net appointment application scenario, the target occupant may also be a driver of the net appointment.
For another example, in an application scenario of a shared automobile, the vehicle may be the shared automobile, the usage record of the vehicle may be a lease record of the shared automobile, the user may be a leaser, and the corresponding usage time may be a time when the leaser leases the shared automobile. By acquiring the lease record of the shared automobile, determining the identity information of the target leaser, the time of leasing the shared automobile is matched with the first occurrence time of the legacy article in the video stream, and determining the identity information of the target leaser as the identity information of the target passenger to which the legacy article belongs, the identity information of the leaser who leaves the legacy article in the shared automobile can be accurately determined. Of course, in an application scenario of a shared automobile, the target occupant may also be a worker or the like of an operator of the shared automobile.
As another example, in an application scenario of a private car, a car owner of the private car may allow family or friends, etc. to drive. By acquiring the use record of the private car, the identity information of the target user matched with the time of using the private car and the first occurrence time of the left-over article in the video stream can be determined, so that the identity information of the family or the friend who leave the left-over article in the private car can be accurately determined.
In this example, by acquiring the usage record of the vehicle, determining, according to the usage record, the identity information of the target user whose corresponding usage time matches the first occurrence time of the legacy article in the video stream, and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs, the usage record of the vehicle can be effectively utilized to determine the identity information of the target passenger more quickly and accurately, so that the reminding information of the target passenger of the legacy article can be generated timely and accurately in a scene in which the vehicle is registered for use.
In one possible implementation, prior to the detecting of the carry-over item within the cabin of the vehicle, the method further comprises: acquiring images and/or videos of passengers entering the cabin; detecting information of items carried by the occupant based on images and/or video of the occupant when entering the cabin; binding information of an article carried by the occupant with identity information of the occupant; the determining identity information of a target occupant to which the legacy item belongs in response to detecting the legacy item includes: in response to detecting a legacy item, determining identity information of an occupant bound to the legacy item as identity information of a target occupant to which the legacy item belongs.
In this implementation, the image and/or video of the passenger entering the cabin can be obtained by a camera arranged in the cabin, and the image and/or video of the passenger entering the cabin can also be obtained by a camera arranged outside the cabin of the vehicle.
In this implementation, the information of the article carried by the passenger may include at least one of an image of the article carried by the passenger, feature information extracted from the image of the article carried by the passenger, a category of the article carried by the passenger, an attribute (e.g., one or more of a color, a size, a pattern) of the article carried by the passenger, and the like.
In this implementation, the information of the occupant may include at least one of identity information of the occupant and/or face information of the occupant, and the like. The face information of the passenger may include a face image and/or a face feature of the passenger.
In this implementation, by acquiring an image and/or a video of a passenger entering the cabin, detecting information of an article carried by the passenger based on the image and/or the video of the passenger entering the cabin, binding the information of the article carried by the passenger with identity information of the passenger, and determining that the identity information of the passenger bound with the legacy article is identity information of a target passenger to which the legacy article belongs in response to the detection of the legacy article, the identity information of the target passenger to which the legacy article belongs can be determined quickly and accurately when the article is left over.
In a possible implementation manner, the generating of the reminding information according to the identity information includes: generating first reminding information according to the identity information; after the generating of the reminder information according to the identity information, the method further includes: and sending the first reminding information to a target terminal corresponding to the identity information.
In this implementation, the target terminal corresponding to the identity information may represent the terminal of the target occupant. For example, the target terminal may be a cell phone, a tablet, a wearable smart device, etc. of the target occupant. In the implementation mode, the first reminding information is sent to the target terminal corresponding to the identity information, so that the target passenger can timely know the condition of leaving articles in the vehicle.
As an example of this implementation, the first reminder information may be generated in combination with information of the legacy item. For example, the first reminder information may be generated in conjunction with at least one of a category, an image, a location, etc. of the item left over, thereby enabling the target occupant to obtain intuitive information of the item left over, thereby facilitating the target occupant's quick determination as to whether the item left over needs to be retrieved back into the vehicle.
In a possible implementation manner, the generating of the reminding information according to the identity information includes: generating second reminding information according to the identity information and the information of the left-over article; after the generating of the reminder information according to the identity information, the method further includes: and sending the second reminding information to a management terminal connected with the vehicle and/or a management server connected with the vehicle.
In this implementation, the information of the legacy item may include at least one of a category, an image, a location, etc. of the legacy item. In this implementation, the management terminal connected to the vehicle may be a terminal related to an operating company of the vehicle (e.g., a terminal of a worker of the operating company) or the like, and the management server connected to the vehicle may be a server of an operating company platform of the vehicle or the like.
In this implementation, by sending the second warning information to a management terminal connected to the vehicle and/or a management server connected to the vehicle, the target passenger can be notified or the left-over items of the target passenger can be kept in storage by a management party (e.g., an operation company) in a timely manner if the target passenger does not retrieve the left-over items in a timely manner.
As an example of this implementation manner, the generating, according to the identity information and the information of the legacy article, second reminding information includes: and generating second reminding information according to the identification information of the vehicle, at least one of the boarding place of the target passenger and the alighting place of the target passenger, the identity information and the information of the left-over articles.
In this implementation, the identification information of the vehicle may represent information capable of uniquely identifying the vehicle. For example, the identification information of the vehicle may include one of a license plate number of the vehicle, a number of the vehicle, an identification number of a driver of the vehicle, a phone number of the driver of the vehicle, and the like.
In this implementation, the video frame in which the target passenger first appears and the video frame in which the target passenger last appears may be determined from the video stream in the cabin, and the boarding time of the target passenger may be determined according to the time of the video frame in which the target passenger first appears, and the disembarking time of the target passenger may be determined according to the time of the video frame in which the target passenger last appears. According to the geographical position information of the vehicle at the boarding time, the boarding place of the target passenger can be determined. According to the geographical position information of the vehicle at the getting-off time, the getting-off position of the target passenger can be determined.
In this implementation, by generating the second reminding information according to the identification information of the vehicle, at least one of the boarding location of the target passenger and the alighting location of the target passenger, and the identity information and the information of the legacy article, the management party (e.g., an operating company) of the vehicle is facilitated to record more relevant information of the legacy article, thereby facilitating finding of the target passenger and facilitating legacy article management.
In a possible implementation manner, the generating of the reminding information according to the identity information includes: when the target passenger is not the driver of the vehicle, generating third reminding information according to the identity information and the information of the left-over articles; the method further comprises the following steps: sending the third reminding information to a terminal of a driver of the vehicle; and/or, the generating of the reminding information according to the identity information comprises: generating fourth reminding information according to the identity information; the method further comprises the following steps: and generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information.
As one example of this implementation, the in-vehicle device may include a speaker; the fourth reminder message may include a voice for reminding the target occupant to retrieve the left-over item, and the fourth reminder message may further include an alert tone. For example, the voice for reminding the target passenger to retrieve the left-behind article may be a voice of "XX ladies leave mobile phones in the car, please retrieve in time", or the like.
As an example of this implementation, the in-vehicle device may include a display screen; after receiving the control information, the display screen can send the fourth reminding information by flashing, playing animation, playing characters and the like.
In this implementation, when the target passenger is not the driver of the vehicle, third reminding information is generated according to the identity information and the information of the left-over article, and the third reminding information is sent to the terminal of the driver of the vehicle, so that the driver of the vehicle can be reminded to help the target passenger to temporarily store the left-over article. The fourth reminding information is generated according to the identity information, and the control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information is generated, so that passengers in the cabin can be reminded to take corresponding safety keeping measures in time on the basis, and the risk of losing articles is reduced.
In one possible implementation, the method further includes: in response to detecting that the target occupant has exited the cabin, generating control information that controls flashing of an indicator light of the vehicle to cause the indicator light to flash in response to the control information. According to the implementation mode, the target passenger just leaving the cabin can be reminded to return to the cabin to retrieve the goods.
In a possible implementation manner, after the sending the first reminding information to the target terminal corresponding to the identity information, the method further includes: and generating repeated reminding information in response to receiving the repeated reminding request returned by the target terminal. In the implementation mode, the repeated reminding information is generated in response to the received repeated reminding request returned by the target terminal, so that the target passenger can be reminded again when the target passenger cannot return to the cabin to take back the left-over article in a short time, and the protection of the property safety of the passenger is further promoted.
As an example of this implementation, after the generating the repeat reminder information, the method further includes at least one of: responding to a time interval between the current time and the time for sending the first reminding information reaching a first preset time length, and sending the repeated reminding information to the target terminal; responding to the time interval between the current time and the time of receiving the repeated reminding request reaching a second preset time length, and sending the repeated reminding information to the target terminal; responding to that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length and the fact that the left-over article is not taken out of the cabin is detected, and sending the repeated reminding information to the target terminal; and sending the repeated reminding information to the target terminal in response to that the time interval between the current time and the time of receiving the repeated reminding request reaches a second preset time length and that the left-over article is not taken out of the cabin.
In this example, the first and second predetermined durations may be default durations or durations set by the target occupant. For example, the first and second preset time periods may be 30 minutes, 1 hour, or the like. In this example, the repeated reminding may be cancelled in response to detecting that the legacy article has been removed from the compartment, so that the reminding efficiency can be improved and unnecessary repeated reminding can be reduced.
As an example of this implementation manner, after the generating the repeated reminding information, the method further includes: and generating control information for controlling vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information in response to the detection that the target passenger returns to the cabin. In this example, by generating control information for controlling the in-vehicle device provided in the cabin to issue the repeated reminding information in response to the detection of the return of the target passenger into the cabin, the target passenger can be reminded in time after the target passenger returns into the cabin.
In one example, the generating control information for controlling the vehicle-mounted device arranged in the cabin to send out the repeated reminding information in response to the detection that the target passenger returns to the cabin may include: and generating control information for controlling vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information according to the type and/or position of the left-over articles in response to the detection that the target passenger returns to the cabin. For example, if the category of the left-behind article is a wallet and the location of the left-behind article is the floor in front of the rear seat, a voice message "leave a wallet on the floor in front of the rear seat" can be issued by the in-vehicle apparatus.
In one possible implementation, the method further includes: receiving a first reminding setting request through a vehicle machine of the vehicle; according to the first reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly. For example, the user can set paper towels and water bottles as the article types which are not reminded any more, set purses, mobile phones and briefcases as the article types which need to be reminded, and set mobile phones as the article types which need to be reminded repeatedly through the vehicle machine. According to the implementation mode, the user can flexibly set the types of the left-over articles which are not reminded, need to be reminded and need to be reminded repeatedly through the vehicle machine according to the requirements of the user.
As an example of the implementation manner, after at least one of the item type that is not reminded any more, the item type that needs to be reminded, and the item type that needs to be reminded repeatedly is set according to the first reminder setting request, the user may change the setting again through the in-vehicle device or the target terminal according to the own requirement. For example, after the user sets the water bottle to the article type which is not reminded any more through the car machine, if the idea is changed, the water bottle can be set to the article type which needs to be reminded again through the car machine or the mobile phone.
In one possible implementation, the method further includes: receiving a second reminding setting request returned by the target terminal; according to the second reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly. According to the implementation mode, the user can flexibly set the types of the left-over articles which are not reminded, need to be reminded and need to be reminded repeatedly through the target terminal according to the requirements of the user.
As an example of the implementation manner, after at least one of the item type that is not reminded any more, the item type that needs to be reminded, and the item type that needs to be reminded repeatedly is set according to the second reminder setting request, the user may change the setting again through the in-vehicle device or the target terminal according to the self-requirement. For example, after the user sets the umbrella to the article type needing reminding through the mobile phone, if the idea is changed, the umbrella can be set to the article type no longer needing reminding again through the car machine or the mobile phone.
Fig. 2 shows a schematic diagram of an application scenario of a private car. In the example shown in fig. 2, a camera provided in the cabin of a private car may be used to capture a video stream in the cabin. The camera may send the video stream within the chamber to a processor. The processor may perform a carry-over detection of the item within the chamber based on the video stream within the chamber. The processor may determine, in response to detecting a carry-over item, identity information of a target occupant to which the carry-over item belongs. For example, the target occupant is a driver, and the processor may generate first reminding information according to the identity information and send the first reminding information to a driver mobile phone. For example, after the driver of a private car receives the first reminding information through a mobile phone, the driver can return to the car to retrieve the left-over article (such as a wallet). The processor can also generate fourth reminding information according to the identity information and generate control information for controlling vehicle-mounted equipment arranged in the cabin to send the fourth reminding information. The processor can respond to the received repeated reminding request returned by the target terminal to generate repeated reminding information, respond to the fact that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length, detect that the left-over article is not taken out of the cabin, and send the repeated reminding information to the mobile phone of the driver. The driver's cell phone may also feed back a second reminder set-up request to the processor. The processor may set at least one of the following according to the second reminder setting request: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly. For example, the first reminding information reminds the categories of the left-over articles to comprise a tissue box and a water bottle, and after the driver receives the first reminding information, the driver can feed back a second reminding setting request through a mobile phone to set the tissue box and the water bottle into the categories of the articles which are not reminded any more.
Fig. 3 shows a schematic diagram of an application scenario for sharing a car. In the example shown in fig. 3, a camera disposed in a cabin of a shared automobile may be used to capture a video stream in the cabin. The camera may send the video stream within the chamber to a processor. The processor may perform a carry-over detection of the item within the chamber based on the video stream within the chamber. The processor may determine, in response to detecting a carry-over item, identity information of a target occupant to which the carry-over item belongs. For example, the target passenger is a driver, and the processor may generate first reminding information according to the identity information and send the first reminding information to a mobile phone of the driver. The processor can also generate fourth reminding information according to the identity information and generate control information for controlling vehicle-mounted equipment arranged in the cabin to send the fourth reminding information. The processor can also generate second reminding information according to the identity information and the information of the left-over articles, and send the second reminding information to a server of an operation company platform connected with the shared automobile. And staff of an operation company sharing the automobile can help target passengers who cannot timely retrieve the left-over articles to keep the left-over articles according to the second reminding information so that the target passengers can conveniently retrieve the left-over articles.
Fig. 4 shows a schematic diagram of an application scenario of a network appointment vehicle. In the example shown in fig. 4, a camera provided in a cabin of the net appointment car may be used to capture a video stream in the cabin. The camera may send the video stream within the chamber to a processor. The processor may perform a carry-over detection of the item within the chamber based on the video stream within the chamber. The processor may determine, in response to detecting a carry-over item, identity information of a target occupant to which the carry-over item belongs. If the target passenger is a passenger, the processor can generate first reminding information according to the identity information and send the first reminding information to a mobile phone of the passenger. The processor can also generate third reminding information according to the identity information and the information of the left-over articles, and send the third reminding information to a mobile phone of a driver of the networked car appointment so as to remind the driver of the networked car appointment to help passengers to keep the left-over articles. The processor can also generate fourth reminding information according to the identity information and generate control information for controlling vehicle-mounted equipment arranged in the cabin to send the fourth reminding information. The processor can also generate second reminding information according to the identity information and the information of the left-over articles, and send the second reminding information to a server of an operation company platform connected with the network car booking.
It should be noted that, besides private cars, shared cars, and network appointments, the embodiments of the present disclosure may also be applied to other vehicles, such as public transportation vehicles (including but not limited to rail public transportation vehicles), ships, etc., and the present application is not limited thereto.
It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above methods of the specific embodiments, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.
In addition, the present disclosure also provides a reminding device, an electronic device, a computer-readable storage medium, and a program for a legacy article, which can be used to implement any one of the reminding methods for a legacy article provided by the present disclosure, and corresponding technical solutions and technical effects can be referred to in corresponding descriptions of the method sections, and are not described again.
Fig. 5 shows a block diagram of a reminder device for a legacy item according to an embodiment of the present disclosure. As shown in fig. 5, the reminding device for the left-behind object comprises: a first detection module 51 for detecting an article left in the cabin of the vehicle; a determination module 52 for determining, in response to detecting a legacy item, identity information of a target occupant to which the legacy item belongs; and the first generating module 53 is configured to generate the reminding information according to the identity information.
In one possible implementation, the determining module 52 is configured to: in response to detecting an carryover item, determining a location of the carryover item within the bin; and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin.
In one possible implementation, the determining module 52 is configured to: determining a relevant area of the carry-over item within the compartment according to the location of the carry-over item within the compartment; based on the video stream in the cabin, carrying out face recognition on the related area to obtain a face recognition result of the related area; and determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area.
In one possible implementation, the determining module 52 is configured to: determining a target video frame from a video stream of the capsule according to a first occurrence time of the legacy article in the video stream; and carrying out face recognition on the relevant area in the target video frame to obtain a face recognition result of the relevant area.
In one possible implementation, the determining module 52 is configured to: determining candidate occupants related to the left-behind item based on the face recognition result of the related area; and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream, of the candidate passengers.
In one possible implementation, the determining module 52 is configured to: in response to detecting a carryover, identity information of a target occupant to which the carryover is affiliated is determined based on a first time of occurrence of the carryover in a video stream within the cabin.
In one possible implementation, the determining module 52 is configured to: acquiring a use record of the vehicle, wherein the use record comprises identity information of a user and corresponding use time; according to the usage record, determining the identity information of the target user, wherein the corresponding usage time of the target user is matched with the first occurrence time of the legacy article in the video stream; and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs.
In one possible implementation, the apparatus further includes: the acquisition module is used for acquiring images and/or videos of passengers entering the cabin; the second detection module is used for detecting information of articles carried by the passenger based on images and/or videos when the passenger enters the cabin; the binding module is used for binding the information of the articles carried by the passenger with the identity information of the passenger; the determination module 52 is configured to: in response to detecting a legacy item, determining identity information of an occupant bound to the legacy item as identity information of a target occupant to which the legacy item belongs.
In a possible implementation manner, the first generating module 53 is configured to: generating first reminding information according to the identity information; the device further comprises: and the first sending module is used for sending the first reminding information to a target terminal corresponding to the identity information.
In a possible implementation manner, the first generating module 53 is configured to: generating second reminding information according to the identity information and the information of the left-over article; the device further comprises: and the second sending module is used for sending the second reminding information to a management terminal connected with the vehicle and/or a management server connected with the vehicle.
In a possible implementation manner, the first generating module 53 is configured to: and generating second reminding information according to the identification information of the vehicle, at least one of the boarding place of the target passenger and the alighting place of the target passenger, the identity information and the information of the left-over articles.
In a possible implementation manner, the first generating module 53 is configured to: when the target passenger is not the driver of the vehicle, generating third reminding information according to the identity information and the information of the left-over articles; the device further comprises: the third sending module is used for sending the third reminding information to a terminal of a driver of the vehicle; and/or, the first generating module 53 is configured to: generating fourth reminding information according to the identity information; the device further comprises: and the first control module is used for generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information.
In one possible implementation, the apparatus further includes: and the second generation module is used for responding to the received repeated reminding request returned by the target terminal and generating repeated reminding information.
In a possible implementation manner, the apparatus further includes a fourth sending module, where the fourth sending module is configured to at least one of: responding to a time interval between the current time and the time for sending the first reminding information reaching a first preset time length, and sending the repeated reminding information to the target terminal; responding to the time interval between the current time and the time of receiving the repeated reminding request reaching a second preset time length, and sending the repeated reminding information to the target terminal; responding to that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length and the fact that the left-over article is not taken out of the cabin is detected, and sending the repeated reminding information to the target terminal; and sending the repeated reminding information to the target terminal in response to that the time interval between the current time and the time of receiving the repeated reminding request reaches a second preset time length and that the left-over article is not taken out of the cabin.
In one possible implementation, the apparatus further includes: and the second control module is used for responding to the detection that the target passenger returns to the cabin and generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information.
In one possible implementation, the apparatus further includes: the first receiving module is used for receiving a first reminding setting request through a vehicle machine of the vehicle; the first setting module is used for setting at least one of the following according to the first reminding setting request: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
In one possible implementation, the apparatus further includes: the second receiving module is used for receiving a second reminding setting request returned by the target terminal; the second setting module is used for setting at least one of the following according to the second reminding setting request: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
In the embodiment of the disclosure, the identity information of the target passenger to which the left-over object belongs is determined in response to the detection of the left-over object by detecting the left-over object in the cabin of the vehicle, and the reminding information is generated according to the identity information, so that the reminding of the left-over object can be performed in a targeted manner based on the identity information of the passenger who has left-over object in the cabin of the vehicle, the reminding effect of the left-over object can be improved, and the property safety of the passenger can be guaranteed.
In some embodiments, functions or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementations and technical effects thereof may refer to the description of the above method embodiments, which are not described herein again for brevity.
Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.
The embodiment of the present disclosure also provides a computer program, which includes computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device executes the computer program to implement the method described above.
The embodiments of the present disclosure also provide another computer program product for storing computer readable instructions, which when executed, cause a computer to perform the operations of the method for reminding a legacy article provided in any of the above embodiments.
An embodiment of the present disclosure further provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the above-described method.
The electronic device may be provided as a terminal, server, or other form of device. For example, the electronic device may be a vehicle or a domain controller in a vehicle cabin, or may be a device host for performing data processing operations such as image processing in the OMS or the DMS.
Fig. 6 illustrates a block diagram of an electronic device 800 provided by an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.
Referring to fig. 6, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.
The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (Wi-Fi), a second generation mobile communication technology (2G), a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G)/long term evolution of universal mobile communication technology (LTE), a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.
Fig. 7 shows a block diagram of an electronic device 1900 provided by an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 7, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932TM) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X)TM) Multi-user, multi-process computer operating system (Unix)TM) Free and open native code Unix-like operating System (Linux)TM) Open native code Unix-like operating System (FreeBSD)TM) Or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (20)

1. A method for reminding a legacy article, comprising:
detecting left-over articles in a cabin of a vehicle;
in response to detecting a legacy item, determining identity information of a target occupant to which the legacy item belongs;
and generating reminding information according to the identity information.
2. The method of claim 1, wherein the determining identity information of a target occupant to which the carry-over item belongs in response to detecting the carry-over item comprises:
in response to detecting an carryover item, determining a location of the carryover item within the bin;
and determining the identity information of the target passenger to which the legacy article belongs according to the position of the legacy article in the cabin.
3. The method of claim 2, wherein determining identity information of a target occupant to which the carry-over item belongs based on a location of the carry-over item within the compartment comprises:
determining a relevant area of the carry-over item within the compartment according to the location of the carry-over item within the compartment;
based on the video stream in the cabin, carrying out face recognition on the related area to obtain a face recognition result of the related area;
and determining the identity information of the target passenger to which the legacy article belongs based on the face recognition result of the relevant area.
4. The method according to claim 3, wherein the performing face recognition on the relevant area based on the video stream in the cabin to obtain a face recognition result of the relevant area comprises:
determining a target video frame from a video stream of the capsule according to a first occurrence time of the legacy article in the video stream;
and carrying out face recognition on the relevant area in the target video frame to obtain a face recognition result of the relevant area.
5. The method of claim 4, wherein determining identity information of a target occupant to which the carry-over item belongs based on the face recognition result of the relevant region comprises:
determining candidate occupants related to the left-behind item based on the face recognition result of the related area;
and determining the identity information of the target passenger to which the carry-over article belongs according to the passengers, wherein the passengers enter the cabin at a time earlier than or equal to the first appearance time of the carry-over article in the video stream and leave the cabin at a time later than the first appearance time of the carry-over article in the video stream, of the candidate passengers.
6. The method of claim 1, wherein the determining identity information of a target occupant to which the carry-over item belongs in response to detecting the carry-over item comprises:
in response to detecting a carryover, identity information of a target occupant to which the carryover is affiliated is determined based on a first time of occurrence of the carryover in a video stream within the cabin.
7. The method of claim 6, wherein determining identity information of a target occupant to which the carry-over item belongs based on a time of first occurrence of the carry-over item in a video stream within the cabin comprises:
acquiring a use record of the vehicle, wherein the use record comprises identity information of a user and corresponding use time;
according to the usage record, determining the identity information of the target user, wherein the corresponding usage time of the target user is matched with the first occurrence time of the legacy article in the video stream;
and determining the identity information of the target user as the identity information of the target passenger to which the legacy article belongs.
8. The method according to any one of claims 1 to 7,
the method further comprises the following steps: acquiring images and/or videos of passengers entering the cabin;
detecting information of items carried by the occupant based on images and/or video of the occupant when entering the cabin;
binding information of an article carried by the occupant with identity information of the occupant;
the determining identity information of a target occupant to which the legacy item belongs in response to detecting the legacy item includes:
in response to detecting a legacy item, determining identity information of an occupant bound to the legacy item as identity information of a target occupant to which the legacy item belongs.
9. The method according to any one of claims 1 to 8,
the generating of the reminding information according to the identity information includes: generating first reminding information according to the identity information;
after the generating of the reminder information according to the identity information, the method further includes: and sending the first reminding information to a target terminal corresponding to the identity information.
10. The method according to any one of claims 1 to 9,
the generating of the reminding information according to the identity information includes: generating second reminding information according to the identity information and the information of the left-over article;
after the generating of the reminder information according to the identity information, the method further includes: and sending the second reminding information to a management terminal connected with the vehicle and/or a management server connected with the vehicle.
11. The method of claim 10, wherein generating second reminder information based on the identity information and the information of the legacy item comprises:
and generating second reminding information according to the identification information of the vehicle, at least one of the boarding place of the target passenger and the alighting place of the target passenger, the identity information and the information of the left-over articles.
12. The method according to any one of claims 1 to 11,
the generating of the reminding information according to the identity information includes: when the target passenger is not the driver of the vehicle, generating third reminding information according to the identity information and the information of the left-over articles;
the method further comprises the following steps: sending the third reminding information to a terminal of a driver of the vehicle;
and/or the presence of a gas in the gas,
the generating of the reminding information according to the identity information includes: generating fourth reminding information according to the identity information;
the method further comprises the following steps: and generating control information for controlling the vehicle-mounted equipment arranged in the cabin to send the fourth reminding information.
13. The method according to claim 9, wherein after the sending the first reminding information to the target terminal corresponding to the identity information, the method further comprises:
and generating repeated reminding information in response to receiving the repeated reminding request returned by the target terminal.
14. The method of claim 13, wherein after the generating of the repeat alert message, the method further comprises at least one of:
responding to a time interval between the current time and the time for sending the first reminding information reaching a first preset time length, and sending the repeated reminding information to the target terminal;
responding to the time interval between the current time and the time of receiving the repeated reminding request reaching a second preset time length, and sending the repeated reminding information to the target terminal;
responding to that the time interval between the current time and the time for sending the first reminding information reaches a first preset time length and the fact that the left-over article is not taken out of the cabin is detected, and sending the repeated reminding information to the target terminal;
and sending the repeated reminding information to the target terminal in response to that the time interval between the current time and the time of receiving the repeated reminding request reaches a second preset time length and that the left-over article is not taken out of the cabin.
15. The method of claim 13 or 14, wherein after the generating of the repeat alert message, the method further comprises:
and generating control information for controlling vehicle-mounted equipment arranged in the cabin to send out the repeated reminding information in response to the detection that the target passenger returns to the cabin.
16. The method according to any one of claims 1 to 15, further comprising:
receiving a first reminding setting request through a vehicle machine of the vehicle;
according to the first reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
17. The method according to any one of claims 1 to 16, further comprising:
receiving a second reminding setting request returned by the target terminal;
according to the second reminding setting request, setting at least one of the following: the item type needing reminding, the item type needing reminding and the item type needing reminding repeatedly.
18. A reminder device for an item left over, comprising:
the first detection module is used for detecting the left articles in the cabin of the vehicle;
a determination module to determine, in response to detecting a legacy item, identity information of a target occupant to which the legacy item belongs;
and the first generating module is used for generating reminding information according to the identity information.
19. An electronic device, comprising:
one or more processors;
a memory for storing executable instructions;
wherein the one or more processors are configured to invoke the memory-stored executable instructions to perform the method of any one of claims 1 to 17.
20. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 17.
CN202010896526.8A 2020-08-31 2020-08-31 Method and device for reminding left-over article, electronic equipment and storage medium Pending CN112036303A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202010896526.8A CN112036303A (en) 2020-08-31 2020-08-31 Method and device for reminding left-over article, electronic equipment and storage medium
PCT/CN2021/078681 WO2022041669A1 (en) 2020-08-31 2021-03-02 Method and apparatus for providing reminder of item which is left behind, and device and storage medium
JP2023513402A JP2023539250A (en) 2020-08-31 2021-03-02 Lost item notification methods, devices, electronic devices, storage media and computer programs

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010896526.8A CN112036303A (en) 2020-08-31 2020-08-31 Method and device for reminding left-over article, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112036303A true CN112036303A (en) 2020-12-04

Family

ID=73587434

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010896526.8A Pending CN112036303A (en) 2020-08-31 2020-08-31 Method and device for reminding left-over article, electronic equipment and storage medium

Country Status (3)

Country Link
JP (1) JP2023539250A (en)
CN (1) CN112036303A (en)
WO (1) WO2022041669A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112733611A (en) * 2020-12-15 2021-04-30 建信金融科技有限责任公司 Legacy article detection method and device, electronic equipment and storage medium
CN113537117A (en) * 2021-07-27 2021-10-22 广东机电职业技术学院 Vehicle-mounted legacy monitoring and alarming method and 5G system thereof
WO2022041669A1 (en) * 2020-08-31 2022-03-03 上海商汤临港智能科技有限公司 Method and apparatus for providing reminder of item which is left behind, and device and storage medium
CN115690046A (en) * 2022-10-31 2023-02-03 江苏慧眼数据科技股份有限公司 Article legacy detection and tracing method and system based on monocular depth estimation

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863669B (en) * 2022-04-29 2023-04-25 湖南喜宝达信息科技有限公司 Monitoring method for shared vehicle and basket left article and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4419672B2 (en) * 2003-09-16 2010-02-24 株式会社デンソー Vehicle left behind prevention device
KR101841948B1 (en) * 2015-10-02 2018-03-26 엘지전자 주식회사 Apparatus, Method and Mobile Terminal for providing service to prevent lost article on inner Vehicle
CN107599971B (en) * 2017-09-04 2020-08-28 驭势（上海）汽车科技有限公司 Method and device for prompting getting-off in pertinence
JP6662935B2 (en) * 2018-03-09 2020-03-11 本田技研工業株式会社 Vehicle control device, vehicle control method, and program
US10311704B1 (en) * 2018-03-29 2019-06-04 Futurewei Technologies, Inc. Passenger-related item loss mitigation
CN112036303A (en) * 2020-08-31 2020-12-04 上海商汤临港智能科技有限公司 Method and device for reminding left-over article, electronic equipment and storage medium

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022041669A1 (en) * 2020-08-31 2022-03-03 上海商汤临港智能科技有限公司 Method and apparatus for providing reminder of item which is left behind, and device and storage medium
CN112733611A (en) * 2020-12-15 2021-04-30 建信金融科技有限责任公司 Legacy article detection method and device, electronic equipment and storage medium
CN113537117A (en) * 2021-07-27 2021-10-22 广东机电职业技术学院 Vehicle-mounted legacy monitoring and alarming method and 5G system thereof
CN113537117B (en) * 2021-07-27 2022-05-06 广东机电职业技术学院 Vehicle-mounted legacy monitoring and alarming method and 5G system thereof
CN115690046A (en) * 2022-10-31 2023-02-03 江苏慧眼数据科技股份有限公司 Article legacy detection and tracing method and system based on monocular depth estimation
CN115690046B (en) * 2022-10-31 2024-02-23 江苏慧眼数据科技股份有限公司 Article carry-over detection and tracing method and system based on monocular depth estimation

Also Published As

Publication number Publication date
WO2022041669A1 (en) 2022-03-03
JP2023539250A (en) 2023-09-13

Similar Documents

Publication Publication Date Title
CN112036303A (en) Method and device for reminding left-over article, electronic equipment and storage medium
CN109636368B (en) Internet taxi taking method and device, taxi taking equipment and computer readable storage medium
CN111937050B (en) Passenger related item loss reduction
WO2021159630A1 (en) Vehicle commuting control method and apparatus, electronic device, medium, and vehicle
CN112001348A (en) Method and device for detecting passenger in vehicle cabin, electronic device and storage medium
CN112037380A (en) Vehicle control method and device, electronic equipment, storage medium and vehicle
CN111332252A (en) Vehicle door unlocking method, device, system, electronic equipment and storage medium
CN112096222B (en) Trunk control method and device, vehicle, electronic device and storage medium
JP7403546B2 (en) Remaining object detection
US10769930B2 (en) Forgetting-to-carry prevention assistance method, terminal device, and forgetting-to-carry prevention assistance system
CN112036314A (en) Steering wheel hands-off detection method and device, electronic equipment and storage medium
CN113486760A (en) Object speaking detection method and device, electronic equipment and storage medium
CN113763670A (en) Alarm method and device, electronic equipment and storage medium
CN112667084B (en) Control method and device for vehicle-mounted display screen, electronic equipment and storage medium
US20190294867A1 (en) Information provision device, and moving body
CN113486759A (en) Dangerous action recognition method and device, electronic equipment and storage medium
CN112100445A (en) Image information processing method and device, electronic equipment and storage medium
CN113766479A (en) Method and device for transmitting passenger information to rescue call center for vehicle
US20230169467A1 (en) Reminding Method and Related Apparatus
CN114407630A (en) Vehicle door control method and device, electronic equipment and storage medium
US20160379172A1 (en) Method and device for creating event reminder
WO2023071175A1 (en) Method and apparatus for associating person with object in vehicle, and electronic device and storage medium
CN115035500A (en) Vehicle door control method and device, electronic equipment and storage medium
CN111475233B (en) Information acquisition method, graphic code generation method and device
CN112937479A (en) Vehicle control method and device, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination