GB2568670A

GB2568670A - Vehicle controller

Info

Publication number: GB2568670A
Application number: GB1719066.1A
Authority: GB
Inventors: Hasedzic Elvir
Original assignee: Jaguar Land Rover Ltd
Current assignee: Jaguar Land Rover Ltd
Priority date: 2017-11-17
Filing date: 2017-11-17
Publication date: 2019-05-29
Anticipated expiration: 2037-11-17
Also published as: GB201719066D0; GB2568670B

Abstract

A method is provided for controlling operation of a receptacle 12 located within a vehicle cabin (1 see fig 1), wherein the receptacle 12 has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle 12 is removable therefrom, and a closed state in which the object such as a cup is retained within the receptacle 12. A camera (7 see fig 1) detects a hand 17 of a vehicle occupant within a volume of space within the vehicle cabin (1). A controller (19 see fig 3) determines the relative position of the hand 17 with respect to the receptacle and controls operation of the receptacle 12 in dependence on a distance between at least a portion of the hand and the receptacle 12 being less than or equal to a predefined proximity threshold. The receptacle 12 may be a cup holder. The invention addresses a risk of beverage spillages.

Description

The present disclosure relates to a vehicle controller. Aspects of the invention relate to a controller for controlling operation of a receptacle located within a vehicle cabin, to a method of controlling operation of a receptacle located within a vehicle cabin, to a vehicle comprising the controller, to a vehicle arranged in use to carry out the method, to a system comprising the controller, to a computer program product comprising instructions for carrying out the method, and to a computer-readable data carrier having stored therein instructions for carrying out the method.

BACKGROUND

Current vehicles, in particular automobiles, often comprise holders for objects that may be used by one or more of the vehicle occupants. For example, current vehicles often comprise holders to retain beverage containers, such as coffee cups or water bottles, so that the beverage container is secure at times when it is not required by the vehicle occupant. Normally each holder is of a standard size and shape with the aim of enabling vehicle occupants to use the holder for a variety of different sized beverage containers.

A disadvantage associated with providing holders of a standard size in this way is that the beverage container may not be suitably fitted in the holder. For example, the holder may be slightly too small for the beverage container, resulting in the container being squeezed into the holder. This induces pressure on the container and thus increases the likelihood of the beverage being spilled as the vehicle occupant removes the container from the holder. On the other hand, the holder may be too large for the beverage container, resulting in the container being too loose in the holder and thereby resulting in the beverage being prone to spillages when the vehicle is in motion. It follows that in order to avoid spillages, vehicle occupants are required to limit the use of the holder in the vehicle to beverage containers of a particular size and shape.

At least in certain embodiments, the present invention seeks to mitigate or overcome at least some of the above-mentioned problems.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a method, a controller, a system, a vehicle, a computer program product and a computer readable data carrier as claimed in the appended claims.

In accordance with an aspect of the present invention there is provided a method for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. The method may comprise: detecting a hand of a vehicle occupant within a volume of space within the vehicle cabin; determining the relative position of the hand with respect to the receptacle; and controlling operation of the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance.

The method is beneficial in that it enables the receptacle to be operated without requiring specific user input, which is particularly convenient whilst driving a vehicle. The operation of the receptacle is controlled in dependence on a distance between the vehicle occupant’s hand and the operation is carried out so that the object is either retained in the receptacle, preventing the object from disengaging from the receptacle, or removable from the receptacle, allowing the object to disengage from the receptacle quickly and conveniently. The dependence of the operation of the receptacle on the position of the vehicle occupant’s hand enables the receptacle to be operated only when it is required by the vehicle occupant. For example, at a moment in which the vehicle occupant wishes for the receptacle to transition from a closed state to an open state, the vehicle occupant moves their hand towards the receptacle and the receptacle transitions from a closed state to an open state in which the object within the receptacle is removable, in dependence on the distance between the receptacle and the vehicle occupant’s hand being less than or equal to the predefined threshold distance. Since the receptacle is only operated when required, battery power is managed efficiently.

Optionally, the method may comprise determining the relative position of the hand with respect to a control proximity boundary associated with the receptacle, the control proximity boundary defining a boundary offset from the receptacle by the predefined threshold distance; and controlling operation of the receptacle in dependence on the position of at least a portion of the hand intersecting the control proximity boundary.

The method may comprise transitioning the receptacle, when it is in the open state, from the open state to the closed state in dependence on the distance between at least a portion of the hand and the receptacle being greater than the predefined threshold distance.

The method may comprise transitioning the receptacle, when it is in the closed state, from the closed state to the open state in dependence on the distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

Optionally, the method may comprise determining if the hand is the vehicle occupant’s left or right hand; and controlling operation of the receptacle in dependence on whether the hand is the vehicle occupant’s left or right hand.

Optionally, the method may comprise determining if the hand is oriented palm upwards or downwards within the volume of space relative to the receptacle; and determining if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand. Analysis of the orientation of the hand provides a convenient way of determining whether the hand is a vehicle occupant’s left or right hand.

The method may comprise determining which vehicle occupant the hand belongs to, more specifically determining if the hand is a hand of a first vehicle occupant or of a second vehicle occupant; and controlling operation of the receptacle in dependence on which vehicle occupant the hand belongs to.

The first vehicle occupant may be a driver of the vehicle, and the method may comprise controlling operation of the receptacle in dependence on the hand belonging to the driver of the vehicle.

Optionally, the method may comprise determining a direction of entry of the hand into the volume of space relative to the receptacle.

Advantageously, it is possible to selectively restrict control of the operation of the receptacle in dependence on which vehicle occupant the hand belongs to. This may be particularly advantageous where operation of the receptacle needs to be restricted to specific vehicle occupants, such as the driver or the front passenger of the vehicle. For example, if a driver is attempting to access an object comprised within the receptacle, selectively controlling the operation of the receptacle in dependence on the on the hand belonging to the driver may prevent unwanted interference from other vehicle occupants and thus minimise interruptions to the driver of the vehicle.

Optionally, the method may comprise obtaining image data of the hand within the volume of space; receiving a reflectance signal reflected from the hand; determining a distance of the hand from a designated origin in dependence on the reflectance signal; and determining the relative position of the detected hand with respect to the receptacle in dependence on the distance of the hand relative to the designated origin, the obtained image data, and a known distance of the receptacle relative to the designated origin.

Optionally, the designated origin may be coincident with the position of an image capture device.

The method may comprise determining the presence of an object within the receptacle, and transitioning the receptacle from the open state to the closed state in dependence on the presence of the object within the receptacle.

Advantageously, it is possible to selectively restrict control of the operation of the receptacle in dependence on the presence of an object within the receptacle. This may prevent unwanted operation of the receptacle, for example when there is no object within the receptacle and the operation of the receptacle is not required. Since the receptacle is only operated when required, battery power is managed efficiently.

Optionally, the receptacle may comprise a fastener having a first state associated with the open state of the receptacle and to a second state associated with the closed state of the receptacle. In the second state the fastener may be arranged to grip an object located within the receptacle, and the method may comprise varying the state of the fastener in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

Optionally, the fastener may be configured to grip an object located within the receptacle with a gripping force when the fastener is in the second state, and the method may comprise identifying a material of the object and selecting the value of the gripping force in dependence on the identified material.

Advantageously, when the fastener is in the second state, the gripping force can be controlled in dependence on the identified material so to ensure that the object is sufficiently stabilised within the receptacle whilst maintaining the shape and structure of the object itself.

Optionally, the receptacle may comprise a movable surface arranged to lower an object into the volume of the receptacle, and the method may comprise determining if the object is in contact with the surface and lowering the object into the volume of the receptacle in dependence on the object being in contact with the surface.

Advantageously, the lowering of an object into the volume of the receptacle provides a more secure way of storing the object within the vehicle. A further advantage is that the movement of the movable surface is in dependence on the contact of the object with the surface, thereby securing an object within the receptacle without requiring any input from the user.

Optionally, the method may comprise identifying a shape of an object located within the receptacle, determining if the shape is consistent with an expected shape, and varying the state of the receptacle from the open state to the closed state in dependence on the shape being consistent with the expected shape.

Advantageously, it is possible to selectively restrict control of the operation of the receptacle in dependence on the shape of an object within the receptacle. This may prevent unwanted operation of the receptacle. For example, the intention of the vehicle occupant may be for the receptacle to be in a closed state when a beverage container is placed in the receptacle, yet remain in an open state when a mobile phone is placed in the receptacle.

The receptacle may be a cup holder, the object may be a beverage container, and the receptacle’s volume may be arranged to receive the beverage container.

In accordance with a further aspect of the present invention there is provided a controller for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. The controller may comprise: an input configured to receive image data obtained by an image capture device; a processor arranged in use to: recognise a hand of a vehicle occupant from the image data, the hand being located within a volume of space within the vehicle cabin within which image data is obtained by the image capture device; and determine a position of the hand with respect to the receptacle; and an output arranged in use to output a control signal to the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance. The controller and its associated embodiments benefit from the same advantages as set out in relation to the previous aspect and its embodiments of the invention.

Optionally, the processor may be arranged in use to determine the position of the hand with respect to a control proximity boundary associated with the control device, the control proximity boundary defining a boundary offset from the control device by the predefined threshold distance; and the output may be arranged in use to output the control signal in dependence on the position of at least a portion of the hand intersecting the control proximity boundary.

Optionally, the output may be configured in use to output a control signal comprising instructions for transitioning the receptacle, when it is in the open state, from the open state to the closed state, in dependence on the distance between at least a portion of the hand and the receptacle being greater than to the predefined threshold distance.

Optionally, the output may be configured in use to output a control signal comprising instructions for transitioning the receptacle, when it is in the closed state, from the closed state to the open state in dependence on the distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

Optionally, the processor may be arranged in use to determine if the hand is the vehicle occupant’s left or right hand; and the output may be arranged in use to output the control signal in dependence on whether the hand is the vehicle occupant’s left or right hand.

The processor may be arranged in use to determine if the hand is oriented palm upwards or downwards within the volume of space relative to the receptacle, and to determine if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand.

Optionally, the processor may be arranged in use to determine which vehicle occupant the hand belongs to, more specifically to determine if the hand is a hand of a first vehicle occupant or of a second vehicle occupant; and the output may be arranged in use to output the control signal in dependence on which vehicle occupant the hand belongs to.

Optionally, the first vehicle occupant may be a driver of the vehicle, and the output may be arranged in use to output the control signal in dependence on the hand belonging to the driver of the vehicle.

Optionally, the processor may be arranged in use to determine a direction of entry of the hand into the volume of space relative to the receptacle.

Optionally, the input may be configured to receive data from a time-of-flight (ToF) image capture device comprising a sensor, the ToF image capture device being arranged in use to obtain image data of the hand, to illuminate the hand, and to measure a time of return of a reflected illumination signal, the time of return of the reflected illumination signal being proportional to a distance of the hand from the sensor, the data comprising the image of the hand and the time of return of the reflected illumination signal; and the processor may be arranged in use to determine the relative position of the hand with respect to the receptacle in dependence on the determined distance of the hand from the sensor, the obtained image data, and a known distance of the receptacle relative to the sensor, wherein the distance of the hand from the sensor may be determined in dependence on the time of return of the reflected illumination signal.

Optionally, the processor may be arranged in use to determine the presence of any object within the receptacle; and the output may be arranged in use to output a control signal comprising instructions for transitioning the receptacle from the open state to the closed state in dependence on the presence of the object within the receptacle.

The receptacle may comprise a fastener having a first state associated with the open state of the receptacle and a second state associated with the closed state of the receptacle, in the second state the fastener being arranged to grip an object located within the receptacle; and the output may be arranged in use to output a control signal for varying the state of the fastener in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

Optionally, the fastener may be configured to grip an object located within the receptacle with a gripping force when the fastener is in the second state, the processor may be arranged in use to identify a material of the object located within the receptacle from the image data; and the output may be arranged in use to output a control signal defining the value of the gripping force in dependence on the identified material.

Optionally, the receptacle may comprise a movable surface arranged to lower an object into the volume of the receptacle, the processor may be arranged in use to determine if the object is in contact with the surface; and the output may be arranged in use to output a control signal for lowering the object into the volume of the receptacle in dependence on the object being in contact with the surface.

The processor may be arranged in use to: identify a shape of any object located within the receptacle; determine if the shape is consistent with an expected shape; and the output may be arranged in use to: output a control signal comprising instructions for varying the state of the receptacle from the open state to the closed state in dependence on the shape being consistent with the expected shape.

Optionally, the receptacle may be a cup holder, and the receptacle’s volume may be arranged to receive a beverage container.

In accordance with yet a further aspect of the invention, there is provided a system comprising the aforementioned controller and a time-of-flight (ToF) image capture device.

In accordance with yet a further aspect of the invention, there is provided a vehicle arranged in use to carry out the previously described method for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. The method may comprise: detecting a hand of a vehicle occupant within a volume of space within the vehicle cabin; determining the relative position of the hand with respect to the receptacle; and controlling operation of the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance. This aspect of the invention benefits from the same advantages as set out previously in relation to the other aspects and embodiments of the invention.

In accordance with yet a further aspect of the invention there is provided a vehicle comprising the previously described controller for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. The controller may comprise: an input configured to receive image data obtained by an image capture device; a processor arranged in use to: recognise a hand of a vehicle occupant from the image data, the hand being located within a volume of space within the vehicle cabin within which image objects are obtained by the image capture device; and determine a position of the hand with respect to the receptacle; and an output arranged in use to output a control signal to the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance. This aspect of the invention benefits from the same advantages as set out previously in relation to the other aspects and embodiments of the invention.

In accordance with yet a further aspect of the invention there is provided a computer program product comprising instructions for carrying out the previously described method. In particular, the computer program product may comprise instructions for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. When executed on a processor, the instructions may configure the processor to: detect a hand of a vehicle occupant within a volume of space within the vehicle cabin; determine the relative position of the hand with respect to the receptacle; and control operation of the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance.

In accordance with yet a further aspect of the invention there is provided a computerreadable data carrier having stored thereon instructions for carrying out the previously described method. In particular, the computer-readable data carrier may comprise instructions for controlling operation of a receptacle located within a vehicle cabin, wherein the receptacle has a volume for receiving an object and is arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle. When executed on a processor, the instructions may configure the processor to: detect a hand of a vehicle occupant within a volume of space within the vehicle cabin; determine the relative position of the hand with respect to the receptacle; and control operation of the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance. This aspect of the invention benefits from the same advantages as set out previously in relation to the other aspects and embodiments of the invention.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 is a schematic cut-away illustration of a front portion of a vehicle cabin having a camera having a field of view arranged to obtain image data of a vehicle occupant’s hand within a volume of space within a vehicle cabin;

Figure 2a is a schematic illustration of a portion of the control panel of the vehicle cabin of Figure 1, showing two receptacles in a closed state;

Figure 2b is a schematic illustration of a portion of the control panel of the vehicle cabin of Figure 1, showing two receptacles in an open state;

Figure 2c is a schematic illustration of two receptacles, each in an open state, showing a fastener arranged to grip an object located within the receptacle;

Figure 2d is a schematic illustration of two receptacles, each in an open state, and showing the fasteners and a movable surface which would make contact with the object when placed within the receptacle;

Figure 3 is a schematic illustration of the functional components of a controller configured to control operation of a receptacle located within the vehicle cabin of Figure 1;

Figure 4 is a process flow chart outlining a method for controlling operation of a receptacle located within a vehicle cabin, in dependence on the proximity of a hand to the receptacle, using the camera of Figure 1;

Figure 5 is a schematic illustration highlighting the principle of operation of a Time-ofFlight (ToF) camera, which may be used to determine the position of a vehicle occupant’s hand within the vehicle cabin of Figure 1;

Figures 6a and 6b are schematic illustrations showing a three-dimensional point cloud of a vehicle occupant’s hand generated using the ToF camera of Figure 5; and

Figure 7 is a schematic illustration of a vehicle comprising the camera of Figure 1, and the controller of Figure 3.

DETAILED DESCRIPTION

Figure 1 is a cut-away perspective view of a portion of the vehicle cabin 1, and in particular shows the driver 3 sitting in the driver’s seat 5. An image capture device in the form of a camera 7, having a field of view 9 delineated in Figure 1 by lines 11, is shown located in the cabin roof. Optionally, the camera 7 may comprise a ToF camera. The camera 7 is arranged to image objects located within the camera’s field of view 9. The field of view defines a volume of space within the vehicle cabin 1 within which objects are imaged by the camera 7. The camera 7 is arranged such that a control panel 13 of the vehicle 1 lies within the camera’s field of view 9. The control panel 13 comprises a plurality of different user operated control devices 15, which may relate to, but are not limited to: air ventilation switches; air conditioning switches; vehicle infotainment system; air circulation switches; and any other user operated control device configured to operate a control system of the vehicle. The vehicle cabin 1 may be comprised in the vehicle 43 of Figure 7.

Figures 2a to 2d describe receptacles in accordance with embodiments of the present invention.

Figure 2a is a schematic illustration of a portion of the control panel 13 of the vehicle cabin 1 of Figure 1, showing two receptacles 12. Each receptacle 12 may be arranged to transition between an open state in which an object 10 located within the receptacle 12 is removable therefrom, and a closed state in which the object 10 is retained within the receptacle 12. Each receptacle 12 shown in Figure 2a is in a closed state with an object 10 located therein. The vehicle occupant’s hand 17 is detected within a volume of space within the vehicle cabin 1. The relative position of the vehicle occupant’s hand 17 is determined with respect to the receptacle 12. The operation of the receptacle 12 is controlled in dependence on a distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being less than or equal to a predefined threshold distance. In Figure 2a, the receptacle 12 remains in a closed state to retain the object 10 within the receptacle 12. This is explained in further detail in the ensuing description.

Figure 2b is a schematic illustration of a portion of the control panel 13 of the vehicle cabin 1 of Figure 1, showing two receptacles 12. Each receptacle 12 may be arranged to transition between an open state in which an object located within the receptacle 12 is removable therefrom, and a closed state in which the object is retained within the receptacle 12. In Figure 2b, the vehicle occupant’s hand 17 is shown within a predefined threshold distance from the receptacle 12, and thus, the receptacle 12 is transitioning from a closed state to an open state in which an object located within the receptacle 12 is removable therefrom. Once the receptacle 12 is in an open state, the receptacle 12 may transition from an open state to a closed state in dependence on the distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being less than or equal to the predefined threshold distance.

A magnified view of each of the two receptacles 12 in an open state is shown in Figures 2c and 2d. Each receptacle 12 may comprise a fastener 14 (see Figure 2c), wherein each fastener may have a first state, associated with the open state of the receptacle 12, or a second state, associated with the closed state of the receptacle 12. When the fastener 14 is in the second state and the receptacle 12 is in a closed state, as shown in Figure 2a, the fastener 14 may be arranged to grip an object 10 located within the receptacle 12.

When the fastener 14 is in the first state and the receptacle 12 is in an open state, as shown in Figure 2c, the fastener 14 may be in a retracted position enabling an object 10 to be placed within the receptacle 12 without any obstruction, or enabling an object 10 that is already in place within the receptacle 12 to be removed from the receptacle 12 without any obstruction. The state of the fastener 14 may be varied in dependence on a distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being less than or equal to the predefined threshold distance.

The receptacle 12 may be configured to detect the presence of an object 10 using, for example, a pressure sensor 20, as shown in Figure 2d. If the presence of an object 10 is determined, the receptacle 12 may transition from an open state to a closed state in dependence on the presence of the object 10 within the receptacle 12.

The receptacle 12 may comprise a movable surface 22 (see Figure 2c). The movable surface 22 may comprise one or more pressure sensors 24 (Figure 2d) to determine if an object 10 is in contact with the movable surface 22. The movable surface 22 may be arranged to lower an object 10 into the volume of the receptacle 12 if the object 10 is in contact with the movable surface 22. Advantageously, this can ensure that the object 10 located within the receptacle 12 is maintained securely. Accordingly, the movable surface 22 enables objects of any material to be held securely within the receptacle 12, which could result in fewer spillages and fewer distractions for the driver of the vehicle.

The receptacle 12 may be, for example a cup holder, the object 10 to be located within the receptacle may be a beverage container, and the receptacle’s volume may be arranged to receive the beverage container. Accordingly, the beverage container may be, for example, a water bottle, a coffee cup or a flask, and the holder may be of an appropriate size and shape to receive the beverage container.

In certain embodiments, the camera 7 may be operatively coupled to a controller 19 (shown in Figure 3), and configured to receive image data obtained by the camera 7, and to output a control signal to the receptacle 12 in dependence on an analysis of the received image data.

Figure 3 provides a functional overview of the controller 19 in accordance with an embodiment of the present invention. The controller 19 may be functionally embedded into an existing electronic control unit of the vehicle 1. The controller 19 may be provided with an input 21 and an output 23. The input 21 may be configured to receive image data obtained by the camera 7, and the output 23 may be configured to output a control signal to the receptacle 12. The control signal may be output in dependence on a distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being less than or equal to a predefined threshold. In addition, the controller 19 may comprise a processor 25 arranged to analyse image data received from the camera 7, to identify image objects such as the hand 17 of a vehicle occupant within the obtained image data, and to generate control signals for controlling operation of a receptacle 12 located within a vehicle cabin 1 in dependence on the relative position of the hand 17 with respect to the receptacle 12.

In use, as an image of a vehicle occupant’s hand 17 is obtained by the camera 7, and its position relative to a receptacle 12 is determined by the controller 19, typically by the processor 25 of the controller 19, the receptacle 12 may be transitioned, when it is in the open state, from the open state to the closed state, or, when it is in the closed state, from the closed state to the open state.

In certain embodiments, the controller 19 may be configured in use to output the control signal in dependence on a distance between at least a portion of a vehicle occupant’s hand 17 and the receptacle 12 being less than or equal to a predefined threshold distance. For example, as the camera 7 obtains image data of a vehicle occupant’s hand 17, such as the driver’s hand, the controller 19 may be configured to identify the image of the hand within the received obtained image data. The relative position of the imaged hand with respect to a desired receptacle may then be determined, from the obtained image data. In order to identify an image of a hand, the controller 19, and specifically the processor 25 may be configured with image recognition software configured to identify a vehicle occupant’s hand 17 located within the camera’s field of view 9, from obtained image data. From the relative position of the imaged hand with respect to the receptacle 12, the distance between the vehicle occupant’s hand 17 and the receptacle 12 may be calculated and compared with a predefined threshold distance. If the calculated distance is less than or equal to the predefined threshold distance, the controller 19 may output a control signal to the receptacle 12 to control the operation of the receptacle 12.

In certain embodiments, the controller 19 may be configured in use to identify the shape of an object 10 using, for example, the pressure sensor 20. If the shape of the object 10 is consistent with the expected shape, the controller 19 may output a control signal to the receptacle 12 to perform a transition from an open state to a closed state in dependence on the shape being consistent with the expected shape. The expected shape may be defined at any time, for example when the vehicle is first acquired or at the start of each journey.

In certain embodiments, the controller 19 may be configured in use to determine the material of the object 10, by using a sensor to detect material types. The sensor may be located anywhere on the receptacle such as, for example, on the fastener 14, and may be either combined with or separate from the pressure sensor 20 used to detect the presence of an object 10. The controller 19 may detect whether the material of the object 10 is mainly composed of metal, plastic, glass or paper using a sensor. In certain embodiments, the controller 19 may be configured in use to detect other material properties of the object 10 such as, for example, if the material is magnetic or non-magnetic. The controller 19 may be configured to select a gripping force for gripping the object 10 in dependence on the material of the object 10. Advantageously, this can ensure that the gripping force used to grip the object 10 is appropriate for the object 10 located within the receptacle 12. This prevents the gripping force from being too high for the material of the object, which could lead to damaging the object 10, or too low for the material of the object, which could result in the fastener 14 being ineffective at gripping the object 10 securely within the receptacle 12. Accordingly, objects of any material may be held securely within the receptacle 12, which may result in fewer spillages and fewer distractions for the driver of the vehicle.

Figure 4 is a process flow chart outlining a method used in accordance with certain embodiments of the invention, to control operation of a receptacle 12 in dependence on the proximity of a vehicle occupant’s hand 17 to the receptacle 12, using the camera 7 of Figure 1 in operative communication with the controller 19 of Figure 2. The method is initiated by the camera 7 obtaining image data within the vehicle cabin 1, at step 301. In certain embodiments the camera 7 may be configured to continuously obtain the image data, or to periodically obtain image data at a predefined frequency. The obtained image data may be forwarded to the controller 19 for analysis where, at step 303, obtained image data is analysed to identify a vehicle occupant’s hand 17 within the obtained image data. As mentioned previously, this may comprise the use of image recognition software. Once a vehicle occupant’s hand 17 has been identified within the obtained image data, the position of the hand 17 is determined relative to the receptacle 12, at step 305. Where the vehicle cabin 1 comprises a plurality of different receptacles 12, step 305 may comprise determining the position of the hand 17 relative to the nearest receptacle 12. The position of the hand 17 relative to the receptacle 12 may be determined by the processor 25. At step 307 it is determined if at least a portion of the hand 17 lies at a distance that is less than or equal to a predefined threshold distance from the receptacle 12. If it is determined that no portion of the hand lies within the predefined threshold distance, then the processor 25 continues to analyse received obtained image data, and the method returns to step 303. If instead it is determined by the processor 25 that at least a portion of the identified hand lies within the predefined threshold distance of the receptacle 12, then the processor 25 generates a control signal for output to the receptacle 12, at step 308. Upon receipt of the control signal, at step 310, the operation of the receptacle 12 is controlled.

In certain embodiments, the predefined threshold distance may relate to a few centimetres, for example any distance within the range of 1cm to 10cm, including 1cm and 10cm. In certain embodiments the predefined threshold may delineate a control proximity boundary surrounding and offset from the receptacle 12 by the predetermined threshold distance, which control proximity boundary when intersected by at least a portion of the vehicle occupant’s hand 17 causes the controller 19 to generate the control signal for output to the receptacle 12.

The control proximity boundary may be geometrically shaped. For example, the control proximity boundary may be box-shaped, or spherically shaped. Effectively, the control proximity boundary relates to a volume of space offset from the receptacle 12 by the predefined threshold distance. In dependence on any portion of the control proximity boundary being intersected by at least a portion of the vehicle occupant’s hand 17, the controller 19 generates the control signal for operation of the receptacle 12. It is to be appreciated that not all of the portions of the control proximity boundary need to be offset from the receptacle 12 by the predefined threshold distance. For example, where the control proximity boundary is box-shaped (e.g. cube shaped), it is to be appreciated that some faces of the cube may not be offset from the receptacle 12 by the predefined threshold distance.

In certain embodiments, in order to enable the position of the vehicle occupant’s hand 17 to be determined relative to the receptacle 12, the camera 7 may relate to a 3D mapping controller arranged to generate a 3D model of the hand within the field of view 9. For example, in certain embodiments the camera 7 may relate to a Time-ofFlight (ToF) camera, in which each captured image pixel is associated with a distance on the basis of a time of return of a reflected illumination signal. To achieve this, the ToF camera may be configured with an illumination source arranged to illuminate the camera’s field of view. The incident illumination signal is subsequently reflected by objects present in the camera’s field of view, and the time of return of the reflected illumination signal is measured. In this way it is possible to associate a distance measurement to each imaged object. The illumination signal may relate to any electromagnetic signal, and need not be comprised in the visible spectrum. For example, in certain embodiments the illumination signal may operate in the infrared spectrum.

In those embodiments where the camera 7 comprises a ToF camera 27, the controller 19, and specifically the input 21 may be configured to receive both camera image data and image object distance information data from the ToF camera 27. This enables the controller 19, and more specifically the processor 25 to determine the position of the vehicle occupant’s hand 17 relative to a receptacle 12 from the received data.

Figure 5 is a schematic diagram illustrating the principle of operation of a ToF camera 27. A modulated illumination source 29 is used to illuminate a desired target 31. The incident illumination 33 is reflected by the target 31 and captured on a sensor 35 comprising an array of pixels. However, whilst simultaneously capturing the reflected modulated light 37, the pixels of the sensor 35 also capture visible light reflected from the target. Since the illumination signal is modulated 33, it may be distinguished from the visible light reflected from the target 31, which enables the time of flight of the modulated illumination signal to be measured. The time of flight taken for the modulated illumination signal to be incident on the target 31 and reflected back to the sensor 35 is measured when it is incident on the sensor 35. In this way, each captured image pixel may be associated with a distance of the corresponding image object on the basis of the measured time of flight required for the reflected modulated illumination signal 37 to be measured by the sensor 35. More specific details regarding operation of ToF cameras are widely available in the art, and for this reason a more detailed discussion is not necessary for present purposes.

Where the camera 7 of Figure 1 comprises a ToF camera 27, it is possible to generate a three-dimensional point cloud of the vehicle occupant’s hand located within the camera’s field of view 9. Figures 6a and 6b illustrate an example of a threedimensional point cloud 39 of the vehicle occupant’s hand 17, generated using the ToF camera 27. In certain embodiments the controller 19 may be configured to generate the three-dimensional point cloud using the image data and image object distance information received from the ToF camera 27. Figure 6a shows a point cloud 39 of the vehicle occupant’s hand 17 as it is approaching a rectangular-shaped control proximity boundary 41. In Figure 6b a portion of the point cloud 39 of the vehicle occupant’s hand 17 is intersecting a portion of the control proximity boundary 41. In this event, and as mentioned previously, the controller 19 is configured to generate a control signal for controlling operation of a receptacle 12 in dependence on the relative position of the hand 17 with respect to the receptacle 12.

In order to enable the position of the vehicle occupant’s hand 17 to be determined relative to a receptacle 12, the position of the receptacle 12 relative to the ToF camera may be determined. Again, this may be done using image recognition software. Since the position of the receptacle 12 relative to the ToF camera 27 is known, and the position of the vehicle occupant’s hand 17 relative to the ToF camera 27 is known, the position of the vehicle occupant’s hand 17 relative to the receptacle 12 may be determined using trigonometry. In certain embodiments, and in order to facilitate computation during use, the controller 19 may be provided with distance information of each receptacle 12 relative to the ToF camera 27 during an initial configuration of the ToF camera 27. This distance information may be stored and accessed for subsequent use when it is needed. This facilitates subsequent computation of the position of the hand relative to the receptacle 12, since only the distance of the vehicle occupant’s hand 17 with respect to the ToF camera 27, and the position relative to the known position of the receptacle 12 requires calculation.

In certain embodiments the controller 19 may be configured to control operation of the receptacle 12 in dependence on the distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being less than the predefined threshold, in a similar manner as previously described. In such embodiments it is envisaged that the vehicle occupant may cause the receptacle to transition between an open state and a closed state by moving their hand to a distance less than or equal to the predefined threshold distance.

In alternative embodiments the controller 19 may be configured to control operation of the receptacle 12 in dependence on the distance between at least a portion of the vehicle occupant’s hand 17 and the receptacle 12 being greater than or equal to the predefined threshold distance. For example, it is envisaged that as the receptacle is controlled by the vehicle occupant positioning their hand such that at least a portion of the hand lies at or within the predetermined threshold distance of the user operated control device 15, it is equally envisaged that, the control of the receptacle 12 may be ceased by the vehicle occupant retracting their hand to a position relative to the associated receptacle 12, that lies at a distance from the receptacle 12 greater than the predetermined threshold distance.

In a further embodiment, the controller 19 may be configured to determine from an analysis of the received camera image data whether the vehicle occupant’s hand 17 is the occupant’s left or right hand. The controller 19 may then be configured to output the control signal in dependence on whether the hand 17 is the occupant’s left or right hand. In this way it is possible to restrict the control of the receptacle 12 in dependence on whether a vehicle occupant’s left or right hand is determined to be at a distance from the receptacle 12 equal to or less than the predetermined threshold distance.

In certain embodiments, the controller 19 may be configured to determine if the vehicle occupant’s hand 17 is oriented palm upwards or downwards relative to the user operated control device 15 or relative to the camera 7, and determining if the hand 17 is the vehicle occupant’s left or right hand in dependence on the orientation of the hand. This may be determined on the basis of the reflectance signal from the hand 17, and/or by image object analysis. The skin texture of a palm of a hand is different to the skin texture of the back of a hand, and as a result the amount of incident light absorbed by the palm differs to the amount of incident light absorbed by the back of the hand. Accordingly, by configuring the controller to analyse the intensity of the reflected signal, which is indicative of the amount of incident illumination absorbed by the hand, it is possible for the controller to determine whether the hand is oriented palm upwards or downwards.

In certain embodiments, the controller 19 may be configured to determine which vehicle occupant the imaged hand belongs to, so, for example, whether the imaged hand 17 belongs to a driver of the vehicle or to a passenger. The operation of the receptacle 12 may then be controlled in dependence on which vehicle occupant the hand belongs to. For example, the operation of the receptacle 12 may be controlled in dependence on the hand 17 belonging to the driver 3 of the vehicle. This may help to reduce driver distraction by preventing the accidental operation of the receptacle 12 a passenger of the vehicle. One non-limiting way in which the controller 19 may determine which vehicle occupant the hand belongs to, is by monitoring and determining a direction of entry of the hand into the camera’s field of view 9 relative to the receptacle 12. This may be achieved from an analysis by the controller 19 of image data obtained by the camera 7. The direction of entry of the hand 17 into the camera’s field of view 9 may be indicative of where the vehicle occupant is seated in relation to the receptacle 12, and therefore provides a good assumption regarding which vehicle occupant the hand 17 belongs to.

Another non-limiting way in which the controller 19 may determine which vehicle occupant the hand belongs to, is by determining from an analysis of the received camera image data whether the vehicle occupant’s hand 17 is an occupant’s left or right hand, as described in a foregoing embodiment. This may be particularly useful for controlling operation of receptacles disposed between two vehicle occupants occupying a common seating row within the vehicle and facing the same direction, e.g. for receptacles arranged between the front seat passengers. By way of further explanation, in this example a first occupant is most likely to access the receptacles with a left hand whereas a second occupant is most likely to access the receptacles with a right hand (or vice versa). In this manner, the controller may discriminate between two vehicle occupants seated adjacent one another within the vehicle.

The non-limiting examples described above may be used independently or in combination to discriminate between multiple vehicle occupants, and to control operation of the receptacles in dependence on whom the hand belongs to.

In a further embodiment, the output 23 of the controller 19 may be arranged in use to output a control signal configured to vary the operation of the receptacle 12 in dependence on a variation in the relative position of the vehicle occupant’s hand 17 with respect to the receptacle 12. The input 21 may be arranged in use to receive a sequence of images obtained by the image capture device. The sequence of images may comprise two or more image frames. The processor 25 may be arranged in use to recognise the vehicle occupant’s hand 17 in the sequence of images, and determine if the position of the hand 17 with respect to the receptacle 12 varies in the sequence of images. The output 23 may then be arranged in use to output a control signal configured to vary the operation of the receptacle 12, in dependence on the determined variation of the position of the hand 17 with respect to the receptacle 12.

Whilst the preceding embodiments of the invention have been described within the context of a ToF camera, it is to be appreciated that alternative camera configurations may be used in accordance with the herein described embodiments. Any configuration of cameras may be used that enables image data of a hand relative to a receptacle to be obtained, and the position of the hand relative to the receptacle to be determined. For example, a configuration of two or more cameras each configured to enable a different perspective image of the hand relative to the receptacle to be captured may also be used. In such an arrangement the different perspective images of the hand relative to the receptacle would enable the controller to determine the position of the hand with respect to the receptacle by triangulation.

Similarly, in an alternative embodiment, the ToF camera of the preceding embodiments may be replaced by a conventional camera, in combination with an optical ruler, such as a LIDAR for example. In such an embodiment the LIDAR provides the image object distance information, whilst the camera provides image data. The controller may be configured in such embodiments to analyse the LIDAR data in combination with the obtained image data in order to determine the position of the vehicle occupant’s hand relative to the receptacle.

The preceding embodiments of the invention have been described within the context of the operation of a receptacle, such as a cup holder, in a vehicle cabin, wherein the cup holder’s volume is configured to receive a beverage container. Some non-limiting examples of receptacles that the invention may be configured to operate include a holder for other objects such as a mobile device including a mobile phone or a tablet, a pair of sunglasses, or stationery (e.g. a pen or a stylus). Accordingly, the holder’s volume may be arranged to receive the object.

It is to be appreciated that many modifications may be made to the above examples and embodiments without departing from the scope of the present invention as defined in the accompanying claims.

Claims

1. A method for controlling operation of a receptacle located within a vehicle cabin, the receptacle having a volume for receiving an object and being arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle, the method comprising:

detecting a hand of a vehicle occupant within a volume of space within the vehicle cabin;

determining the relative position of the hand with respect to the receptacle; and controlling operation of the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance.

2. The method of claim 1, comprising:

determining the relative position of the hand with respect to a control proximity boundary associated with the receptacle, the control proximity boundary defining a boundary offset from the receptacle by the predefined threshold distance; and controlling operation of the receptacle in dependence on the position of at least a portion of the hand intersecting the control proximity boundary.

3. The method of claim 1 or claim 2, comprising:

transitioning the receptacle, when it is in the open state, from the open state to the closed state in dependence on the distance between at least a portion of the hand and the receptacle being greater than the predefined threshold distance.

4. The method of any preceding claim, comprising:

transitioning the receptacle, when it is in the closed state, from the closed state to the open state in dependence on the distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

5. The method of any preceding claim, comprising:

determining if the hand is the vehicle occupant’s left or right hand; and controlling operation of the receptacle in dependence on whether the hand is the vehicle occupant’s left or right hand.

6. The method of claim 5, comprising:

determining if the hand is oriented palm upwards or downwards within the volume of space relative to the receptacle; and determining if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand.

7. The method of any preceding claim, comprising:

determining if the hand is a hand of a first vehicle occupant or of a second vehicle occupant; and controlling operation of the receptacle in dependence on which vehicle occupant the hand belongs to.

8. The method of claim 7, wherein the first vehicle occupant is a driver of the vehicle, and the method comprises:

controlling operation of the receptacle in dependence on the hand belonging to the driver of the vehicle.

9. The method of any one of claims 5 to 8, comprising:

determining a direction of entry of the hand into the volume of space relative to the receptacle.

10. The method of any preceding claim, comprising:

obtaining image data of the hand within the volume of space;

receiving a reflectance signal reflected from the hand;

determining a distance of the hand from a designated origin in dependence on the reflectance signal; and determining the relative position of the detected hand with respect to the receptacle in dependence on the distance of the hand relative to the designated origin, the obtained image data, and a known distance of the receptacle relative to the designated origin.

11. The method of claim 10, wherein the designated origin is coincident with the position of an image capture device.

12. The method of any preceding claim, comprising:

determining the presence of an object within the receptacle; and transitioning the receptacle from the open state to the closed state in dependence on the presence of the object within the receptacle.

13. The method of any preceding claim, wherein the receptacle comprises a fastener having a first state associated with the open state of the receptacle and to a second state associated with the closed state of the receptacle, in the second state the fastener being arranged to grip an object located within the receptacle, the method comprising:

varying the state of the fastener in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

14. The method of claim 13, wherein the fastener is configured to grip an object located within the receptacle with a gripping force when the fastener is in the second state, the method comprising:

identifying a material of the object; and selecting the value of the gripping force in dependence on the identified material.

15. The method of any preceding claim, wherein the receptacle comprises a movable surface arranged to lower an object into the volume of the receptacle, and the method comprises:

determining if the object is in contact with the surface; and lowering the object into the volume of the receptacle in dependence on the object being in contact with the surface.

16. The method of any preceding claim, comprising:

identifying a shape of an object located within the receptacle; determining if the shape is consistent with an expected shape; and varying the state of the receptacle from the open state to the closed state in dependence on the shape being consistent with the expected shape.

17. The method of any preceding claim, wherein the receptacle is a cup holder, the object is a beverage container, and the receptacle’s volume is arranged to receive the beverage container.

18. A controller for controlling operation of a receptacle located within a vehicle cabin, the receptacle having a volume for receiving an object and being arranged to transition between an open state in which an object located within the receptacle is removable therefrom, and a closed state in which the object is retained within the receptacle, the controller comprising:

an input configured to receive image data obtained by an image capture device;

a processor arranged in use to:

recognise a hand of a vehicle occupant from the image data, the hand being located within a volume of space within the vehicle cabin within which image data is obtained by the image capture device; and determine a position of the hand with respect to the receptacle; and an output arranged in use to output a control signal to the receptacle in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to a predefined threshold distance.

19. The controller of claim 18, wherein the processor is arranged in use to determine the position of the hand with respect to a control proximity boundary associated with the control device, the control proximity boundary defining a boundary offset from the control device by the predefined threshold distance; and the output is arranged in use to output the control signal in dependence on the position of at least a portion of the hand intersecting the control proximity boundary.

20. The controller of claim 18 or claim 19, wherein the output is configured in use to output a control signal comprising instructions for transitioning the receptacle, when it is in the open state, from the open state to the closed state, in dependence on the distance between at least a portion of the hand and the receptacle being greater than to the predefined threshold distance.

21. The controller of any one of claims 18 to 20, wherein the output is configured in use to output a control signal comprising instructions for transitioning the receptacle, when it is in the closed state, from the closed state to the open state in dependence on the distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

22. The controller of any one of claims 18 to 21, wherein the processor is arranged in use to determine if the hand is the vehicle occupant’s left or right hand; and the output is arranged in use to output the control signal in dependence on whether the hand is the vehicle occupant’s left or right hand.

23. The controller of claim 22, wherein the processor is arranged in use to determine if the hand is oriented palm upwards or downwards within the volume of space relative to the receptacle; and determine if the hand is the vehicle occupant’s left or right hand in dependence on the orientation of the hand.

24. The controller of any one of claims 18 to 23, wherein the processor is arranged in use to determine if the hand is a hand of a first vehicle occupant or of a second vehicle occupant; and the output is arranged in use to output the control signal in dependence on which vehicle occupant the hand belongs to.

25. The controller of claim 24, wherein the first vehicle occupant is a driver of the vehicle, and the output is arranged in use to output the control signal in dependence on the hand belonging to the driver of the vehicle.

26. The controller of any one of claims 22 to 25, wherein the processor is arranged in use to determine a direction of entry of the hand into the volume of space relative to the receptacle.

27. The controller of any one of claims 18 to 26, wherein:

the input is configured to receive data from a time-of-flight (ToF) image capture device comprising a sensor, the ToF image capture device being arranged in use to obtain image data of the hand, to illuminate the hand, and to measure a time of return of a reflected illumination signal, the time of return of the reflected illumination signal being proportional to a distance of the hand from the sensor, the data comprising the image data and the time of return of the reflected illumination signal; and wherein the processor is arranged in use to determine the relative position of the hand with respect to the receptacle in dependence on the determined distance of the hand from the sensor, the captured image, and a known distance of the receptacle relative to the sensor, wherein the distance of the hand from the sensor is determined in dependence on the time of return of the reflected illumination signal.

28. The controller of any one of claims 18 to 27, wherein the processor is arranged in use to determine the presence of any object within the receptacle; and the output is arranged in use to output a control signal comprising instructions for transitioning the receptacle from the open state to the closed state in dependence on the presence of the object within the receptacle.

29. The controller of any one of claims 18 to 28, wherein the receptacle comprises a fastener having a first state associated with the open state of the receptacle and a second state associated with the closed state of the receptacle, in the second state the fastener being arranged to grip an object located within the receptacle; and the output is arranged in use to output a control signal for varying the state of the fastener in dependence on a distance between at least a portion of the hand and the receptacle being less than or equal to the predefined threshold distance.

30. The controller of claim 29, wherein the fastener is configured to grip an object located within the receptacle with a gripping force when the fastener is in the second state, the processor is arranged in use to identify a material of the object located within the receptacle from the image data; and the output is arranged in use to output a control signal defining the value of the gripping force in dependence on the identified material.

31. The controller of any one of claims 18 to 30, wherein the receptacle comprises a movable surface arranged to lower an object into the volume of the receptacle, the processor is arranged in use to determine if the object is in contact with the surface; and the output is arranged in use to output a control signal for lowering the object into the volume of the receptacle in dependence on the object being in contact with the surface.

32. The controller of any one of claims 18 to 31, wherein the processor is arranged in use to:

identify a shape of any object located within the receptacle;

determine if the shape is consistent with an expected shape; and the output is arranged in use to:

output a control signal comprising instructions for varying the state of the receptacle from the open state to the closed state in dependence on the shape being consistent with the expected shape.

33. The controller of any one of claims 18 to 32, wherein the receptacle is a cup holder, and the receptacle’s volume is arranged to receive a beverage container.

34. A system comprising the controller of any one of claims 18 to 33 and a time-offlight (ToF) image capture device.

35. A vehicle comprising the controller of any one of claims 18 to 33 or the system of claim 34.

36. A vehicle arranged to carry out the method of any one of claims 1 to 17.

37. A computer program product comprising instructions for carrying out the method of any one of claims 1 to 17.

5

38. A computer readable data carrier having stored thereon instructions for carrying out the method of any one of claims 1 to 17.