WO2015017883A1 - Driving infringement flagging system - Google Patents

Driving infringement flagging system Download PDF

Info

Publication number
WO2015017883A1
WO2015017883A1 PCT/AU2014/000783 AU2014000783W WO2015017883A1 WO 2015017883 A1 WO2015017883 A1 WO 2015017883A1 AU 2014000783 W AU2014000783 W AU 2014000783W WO 2015017883 A1 WO2015017883 A1 WO 2015017883A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
image
determining whether
face
analysing
Prior art date
Application number
PCT/AU2014/000783
Other languages
French (fr)
Inventor
David Fletcher
Bernard Clark
Original Assignee
David Fletcher
Bernard Clark
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2013902946A external-priority patent/AU2013902946A0/en
Application filed by David Fletcher, Bernard Clark filed Critical David Fletcher
Priority to AU2014305641A priority Critical patent/AU2014305641A1/en
Priority to SG11201600924WA priority patent/SG11201600924WA/en
Publication of WO2015017883A1 publication Critical patent/WO2015017883A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness

Definitions

  • the present invention relates to a system and method for flagging that a driver may be infringing road rules by acting in a manner that is illegal while driving.
  • a potential driving infringement flagging system comprising:
  • an image source that captures or receives an image of a driver of a vehicle
  • a processor configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving
  • the processor is configured with image analysis rules to determining whether the driver is potentially acting in a manner that is illegal while driving.
  • the image analysis rules comprise rules for detecting use of a mobile telephone by the driver.
  • the image analysis rules comprise rules for detecting incorrect (or no) seatbelt use by the driver.
  • the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by analysing the image for a visual footprint of a mobile phone, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone.
  • the processor is configured to determine that there is a facial extension when the ratio of respective distances between each pupil and the ear on the
  • corresponding side of the driver's face is within a range of values.
  • the range of values is between 0.95 and 1 .05.
  • the processor is configured to determine that the ear should be in view when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values.
  • the range of values is between 1 .1 and 1 .8, and in another embodiment between 1 .6 and 1 .8.
  • the processor is configured to determine that a hand is proximal to the face when the hand is with a distance of four times the height of the driver's nose.
  • the processor is configured to determine that the ear is obscured the head is turned and the ear on the side of the face opposite to the direction in which the head is turned is visible. ln an embodiment the processor is configured to determine that the head is turned when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value. In an embodiment the value is 1 .1 and in an alternative embodiment the value is 1 .3.
  • the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that neither the eyes nor glasses are visible then the analysing outcome is that the driver may be using a mobile telephone.
  • the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone.
  • the value is 1 .8 and in an alternative embodiment the value is 2.
  • the processor is configured to determine whether glasses are being worn by the driver, and if so determined then the position of the pupil of each eye is approximated.
  • the processor is configured to analyse the image to determine whether the driver is correctly wearing a seatbelt, or not.
  • the processor is configured to determine whether the driver is correctly wearing a seatbelt by determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver.
  • the acute angle is within the range between 30 and 60 degrees.
  • the set value range is between 3 and 4 times the height of the nose.
  • a mobile telephone use while driving recognition system comprising: an image source that captures or receives an image of a driver of a vehicle;
  • a processor configured to analyse the image to determine whether the driver is using a mobile telephone, or not;
  • a seatbelt worn while driving recognition system comprising:
  • an image source that captures or receives an image of a driver of a vehicle
  • a processor configured to analyse the image to determine whether the driver is wearing a seatbelt, or not;
  • a method of detection of a potential driving infringement comprising:
  • the analysis comprising applying image analysis rules to the image to determining whether the driver is potentially acting in a manner that is illegal while driving.
  • applying the image analysis rules comprises detecting use of a mobile telephone by the driver.
  • applying the image analysis rules comprises detecting absence of correct seatbelt use by the driver.
  • the analysing the image comprises determining whether there is a visual footprint of a mobile phone in the image, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone. ln an embodiment the analysing the image comprises determining:
  • the step of determining whether there is facial extension comprises determining whether the ratio of respective distances between each pupil and the ear on the corresponding side of the face is within a range of values.
  • the range of values is between 0.95 and 1 .05.
  • the step of determining whether the face is forward facing comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values.
  • the range of values is between 1 .1 and 1 .8 and in an alternative embodiment between 1 .6 and 1 .8. When the face is within the range it is deemed to be forward facing.
  • the analysing the image comprises:
  • a hand is regarded as proximal to the face if it is with a distance of four times the height of the driver's nose.
  • the step of determining whether the ear is obscured when the face is not deemed to be forward facing comprises determining whether the head is turned and in which direction, and then determining whether the ear on the side of the face opposite to the direction in which the head is turned is visible.
  • determining whether the head is turned comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value. In an embodiment the value is 1 .1 and in alternative embodiment the value is 1 .3.
  • the analysing the image comprises: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that the eyes or glasses are not visible then the analysing outcome is that the driver may be using a mobile telephone.
  • the analysing the image comprises: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone.
  • the value is 1 .8 and in an alternative embodiment the value is 2.
  • the analysing the image comprises determining whether glasses are being worn by the driver the pupil, and if so determined then the position of the pupil of each eye is approximated.
  • the analysing the image comprises determining whether the driver is correctly wearing a seatbelt, or not.
  • the analysing the image comprises determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver.
  • the acute angle is within the range between 30 and 60 degrees.
  • the set value range is between 3 and 4 times the height of the nose.
  • one or more combinations of the analysis steps are used. According to the present invention there is provided method of detecting possible mobile telephone use while driving, said method comprising:
  • a method of detecting possible incorrect seatbelt use while within a vehicle comprising:
  • a computer program comprising instructions for controlling a processor to perform the method above.
  • Figure 1 is a schematic flow chart of an embodiment of a method of detecting a potential driving infringement according to an embodiment of the invention
  • Figure 2 is a schematic flow chart of an embodiment of a method of detecting possible incorrect seatbelt use while within a vehicle according to an embodiment of the invention
  • Figure 3 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
  • Figure 4 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
  • Figure 5 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
  • Figure 6 is a schematic simulation of an image processed in order to perform the method of Figure 2; and Figure 7 is a schematic block diagram of a system for detecting a potential driving infringement according to an embodiment of the invention.
  • a system 10 comprising a processor 12, a camera 14, a storage medium 16 and an output 18.
  • the processor may be a general purpose computer loaded with a computer program comprising instructions for causing the processor 12 to operate as the system 10 in combination with the other components of the system 10 and to perform an
  • the processor 12 may be a purpose configured processing device.
  • the processor 12 may comprise memory, or the storage 16 may comprise memory in which the instructions are stored and operated on.
  • the camera 14 may be an integral component of the system 10, or it may be external. In particular the camera 14 may be remotely located from the rest of the system. The camera 14 is generally portable and may be set up in a suitable location in order to observe the drivers.
  • the camera 14 may form part of another system, such as an automated number plate recognition system.
  • the system 10 may form part of another system, such as an automated number plate recognition system, or a traffic monitoring system.
  • the camera may be permanently installed to view a part of a roadway system, such as for example a roadside or overhead traffic monitoring camera.
  • the storage 16 may be an integral component of the system 10, or it may be external. In particular the storage 16 may be remotely located from the rest of the system. The storage 16 may form part of another system, such as an automated number plate recognition system.
  • the storage may be in the form of a hard disk drive or flash memory and in an embodiment stores a database of images of common mobile telephones. Also in an embodiment the storage 16 stored image from the camera 14 as they are taken.
  • the processor 12 may be configured as, or the computer program may be arranged to configure the processor 12 as, functional modules, each of which performs a step in the method.
  • the computer program may be stored in a non-volatile, non-transitory state, for example on a computer readable storage medium, such as on a hard disk drive, or flash memory, or permanent storage medium, such as a CD-ROM, DVD-ROM, Blueray-ROM etc.
  • the output 18 may be in the form of a display to an operator of the system 10 which shows the operator an indication that the driver may be acting in a manner that is not permitted.
  • the indication may be showing the image of the driver taken by the camera 14, that that a formal infringement process may be commenced by the operator.
  • the system 10 may issue infringement notices automatically, as these are regarded as the indication of infringement.
  • the indication may be an illumination of a signal light or an audible warning for the operator to manually observe the driver.
  • the system 10 comprises an image source, such as camera 14, that captures or receives an image of a diver of a vehicle; a processor 12 configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving; and an output 18 that indicates the result of the analysis.
  • image source such as camera 14
  • processor 12 configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving
  • output 18 that indicates the result of the analysis.
  • the processor is configured with image analysis rules to determine whether the driver is potentially acting in a manner that is illegal while driving.
  • the image analysis rules comprise rules for detecting use of a mobile telephone by the driver.
  • the image analysis rules comprise rules for detecting no or incorrect seatbelt use by the driver.
  • the image analysis rules may be for both detecting use of a mobile telephone by the driver and incorrect seatbelt use by the driver. They may also perform other image analysis. Generally they will be often be combined with or form part of an automatic number plate recognition system such that the infringement may be enforced by identifying the vehicle.
  • the method 100 commences with the processor 12 receiving an image of a diver of a vehicle from the image source. Often the image will be of the whole front of the vehicle and the system 10 will conduct number plate analysis according to known methods. In this embodiment of the present invention a portion of the image showing the driver is isolated from the rest of the image. Insofar as the present invention is concerned this isolated portion of the image is regarded as the image of the driver even if it is not separated from the rest of the image. Localisation to the driver can occur in a similar manner to number plate localisation, in that the position of the driver within the vehicle is generally in the same region in relation to the rest of the vehicle (or indeed the number plate). Variations to the position of the driver may be needed with different types of vehicle.
  • the method 100 may serially, or in parallel, perform a seatbelt worn correctly check at 200, which is described further in relation to Figure 2.
  • a preliminary step 104 may be to check whether a new face is in the image. There may also be a number of frames in a video image that are suitable, and there may be some processing to select the best image.
  • Step 106 is to analyse the image analysis to determine whether there is a visual footprint of a mobile phone in the image.
  • the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones stored in the storage 16.
  • the analysing outcome is that the driver may be using a mobile telephone and path 108 is taken.
  • the process proceeds to step 1 10.
  • Step 1 10 is a zero exclusion test. This comprises a number of precursor image analysis steps and then a number of sub-tests.
  • the precursor image analysis steps are performed on the image 300 with reference to Figure 3.
  • the permitter of the face 302 may be identified, such as by use of an edge detection or shape detection algorithm.
  • the eyes 304 may be identified and within those the pupils identified or estimated. In the case of glasses being detected and the pupils obscured, the positions of the pupils are estimated.
  • the distance A in the image of the separation of the pupils is calculated.
  • the position of the nose 306 is determined and the length D of the nose in the image is calculated.
  • the ratio of A to D is calculated as a fraction, that is A / D is calculated. This eliminates the scale of the image as a spurious factor, as well as size of the face.
  • the position of the ears 308L and 308R are detected. (R and L are right and left in the image, not from the point of view of the person).
  • the ears 308L and 308R extend horizontally from the perimeter of the face 302, roughly vertically half way up the face.
  • the most horizontally separated tips of the ears 308L and 308R are detected and the horizontal distance in the image from the respective tip to corresponding side of the face pupil is calculated E(L) and E(R).
  • this ratio of A to D is within a range in a high proportion of cases, it can use used to represent when the driver's head is or is not facing forward. When the head is turned from side to side A will be less than when facing forward.
  • the first subtest is to determine whether the driver's face is forward facing.
  • the step of determining whether the face is forward facing comprises determining whether the ratio of A to D (ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver) is within a range of values.
  • the range of values is between 1 .1 and 1 .8.
  • the range of values is 1 .6 to 1 .8.
  • a sub-test may include analysing the image to determine whether there is any "facial extension", that is when the face appears to the larger on one side.
  • the step of determining whether there is facial extension comprises determining whether the ratio of respective distances between each pupil and the ear on the corresponding side of the face, that is E(L) / E(R), is within a range of values. In an embodiment the range of values is between 0.95 and 1 .05. When this ration is outside of this range the face is regarded as extended.
  • a further sub-test may include analysing the image to determine whether a hand is visible in the image within a distance from the perimeter of the face 302. In an embodiment the distance is 4 x D. If as a result of these sub-tests, the face is determined to be forward facing, there is no facial extension and there is not hand detected within the required distance of the face, then the process takes path 1 12, indicating that the driver is deemed to not be using a mobile telephone. The process then proceeds to review a new face at 104. If however the subtests result in the face not being forward facing, or there may be facial extension, or a hand is detected within the required distance, or any of these are inconclusive, further analysis is required and the process proceeds to step 1 14.
  • step 1 14 when the face is determined to be forward facing a further comparison of the E(L) to E(R) ration is performed.
  • the ratio is outside of a range then there is the possibility of the face be "extended" by the driver's hand being on one ear.
  • the range is between 0.8 and 1 .2 and in an alternative embodiment the range is between 0.9 and 1 .1 .
  • the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 1 16.
  • the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain.
  • step 1 18 when the face is determined to be forward facing the analysis of this step is bypassed. Thus this analysis occurs when the face is not deemed to be forward facing.
  • this analysis it is determined whether an ear of the driver should be in view, but is obscured. This comprises determining whether the head is turned and in which direction, and then determining whether the ear on the side of the face opposite to the direction in which the head is turned is visible.
  • determining whether the head is turned comprises determining whether the ratio A to D (distance between pupils of the eyes of the driver divided by the height of the nose of the driver) is less than a value. In an embodiment the value is 1 .1 and in an alternative embodiment the value is 1 .3.
  • the determining whether the head is turned comprises determining whether the tip of the ear 328L is not visible or the tip of the ear 328R is inside the perimeter of the face 302 (when the face is turned to the right). Naturally if the face is turned to the left the testing is swapped. In the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 120. Again, the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 122.
  • step 122 an analysis is performed to determine whether there is a hand visible proximal to the face.
  • a hand is regarded as proximal to the face if it is within a distance of four times the height of the driver's nose, that is 4 x D.
  • the analysis outcome is that the driver may be using a mobile telephone and a flag is set at 124. Again, the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain.
  • the process then proceeds to step 126.
  • step 126 an analysis is performed to determine whether the diver's eyes are visible or for whether the driver's face is facing down. In an embodiment this comprises
  • the analysing outcome is that the driver may be using a mobile telephone.
  • the value is 1 .8 and in an alternative embodiment the value is 2.
  • the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 128.
  • the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. ln an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 130.
  • step 130 it is determined whether a flag has been set. In the event that a flag is not set 132 the driver is deemed to not be using a mobile telephone and the process proceeds to analyse a new face at 104. In the even that a flag is set the process proceeds at 134 to step 136.
  • Step 136 may perform further analysis, may issue an infringement 140, or may provide an indication via output 18 for a manual inspection by an operator.
  • the image may be stored in the storage 16 for later review on a display. If manually viewed and the driver is not acting in an illegal manner the process may proceed to delete the record at 138. In the event that manual inspection confirms that the driver is using a mobile telephone an infringement may be issued at 140 and the image is usually stored in case it is later needed in a legal process.
  • a process 200 of analysing the image comprises determining whether the driver is correctly wearing a seatbelt, or not. This process may be conducted in the same system 10 that conducts the method 100, or in an independent system 10.
  • the method 200 commences with the processor 12 receiving an image of a diver of a vehicle.
  • the image may the same as in method 100.
  • the analysis of the image to determine whether the driver is potentially acting in a manner that is illegal while driving, and in particular to determine whether the driver is correctly using a seatbelt using analysis conducted at step 204.
  • the outputting of an indication of the result of the analysis is performed at 216. Each of these may be performed by a corresponding module of the processor 12.
  • Step 204 is to analyse the image to determine whether the seatbelt is correctly used.
  • the precursor image analysis steps from method 100 may be used, or a precursor image analysis is performed on the image 330 with reference to Figure 6.
  • the permitter of the face 302 may be identified, such as by use of an edge detection algorithm.
  • the position of the nose 306 is determined and the length D of the nose in the image is calculated.
  • the seatbelt 340 is identified as a band of contrast below the face.
  • the angle ⁇ of the band is determined as is the vertical distance 344 from the bottom of the nose 336 to point of intersection 342 with the band.
  • the angle ⁇ is tested to fall within a range. In an embodiment the range is between 30 and 60°.
  • the distance 344 from the bottom of the nose 336 is also tested to fall within a range. In an embodiment the range is between 3 and 4 times the length of the nose 336, that is 3 x D to 4x D.
  • the seatbelt is determine to be correctly worn.
  • the process progresses down path 206.
  • the process may then consider a new face again at 204.
  • a flag is set. If either of the calculated values are outside of these ranges, then the outcome of the analysis is that the driver may not be correctly wearing the seatbelt, and a flag is set at 208. There may be a flag for each of the tests or a common flag may be used. In the case of a common flag, the flag may also be in common with the common flag used in method 100. In an embodiment the flag is not set if the outcome is uncertain. The process then proceeds to step 210. In step 210, is determined whether a flag has been set. In an embodiment the flags for angle and distance may be tested, with neither or only one being set, the result is that the driver is deemed to be using a seatbelt correctly, unless no seatbelt is detected.
  • the process proceeds down path 212 to analyse a new face at 204. In the even that no seatbelt is detected or both flags are set, or if the system 10 is configured such that only need needs to be set, then the process proceeds at 214 to step 216.
  • Step 216 may perform further analysis, may issue an infringement 220, or may provide an indication via output 18 for a manual inspection by an operator.
  • the image may be stored in the storage 16 for later review on a display. If manually view and the driver is not acting in an illegal manner the process may proceed to delete the record at 218. In the event that manual inspection confirms that the driver is not correctly using the seatbelt an infringement may be issued at 220 and the image is usually stored in case it is later needed in a legal process.
  • Step 216 may be combined with step 136 of method 100.
  • This process may also be performed on a passenger instead of a driver - the only difference being identifying the passenger in a larger image and processing the localised image of the passenger as if it were the image of the driver for purposes of determining whether the passenger is correctly using the seatbelt.
  • Advantages of the present invention are that enforcement of driver behaviour can be automated, or partially automated, with only likely candidates requiring visual inspection.
  • laws relating to mobile telephone use while driving and correct seatbelt use can be enforced with greater effectiveness and efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

A potential driving infringement flagging system comprises an image source that captures or receives an image of a driver of a vehicle or another person within the vehicle; a processor configured to analyse the image to determine whether the driver is using a mobile telephone while driving or whether the driver or the person is wearing a seatbelt; and an output that indicates the result of the analysis.

Description

Driving infringement flagging system
Field of the Invention The present invention relates to a system and method for flagging that a driver may be infringing road rules by acting in a manner that is illegal while driving.
Background It is common in many jurisdictions to provide cameras that monitor for drivers exceeding a speed limit in a given area. These cameras use RADAR or similar to detect the speed of the vehicle and if the speed is above a threshold a photograph of the vehicle is taken in order to process an infringement notice. It is also known to provide a camera with associated automatic number plate monitoring equipment for identifying the vehicle passing by in real time. This process uses optical character recognition to identify the number plate and look up the vehicle details associated with the number plate in a vehicle database. The real time nature of this system enables on the spot enforcement action, for instance if the vehicle has been reported as stolen.
Such "speeding" mentioned above is prohibited and is tested against the action of the vehicle, and not observed actions of the driver, notwithstanding that it is controlled by the driver. There are often other laws that relate to the actions of the driver. Many jurisdictions have laws that make using a mobile telephone while driving illegal. Also many jurisdictions have laws that require the driver (and other passengers) to be correctly restrained by a seatbelt.
There are currently no known systems for automatic detection of observed actions of a driver that infringe road rules related to prohibited or mandatory actions of the driver while driving. Such enforcement currently relies on police and traffic enforcement officers noticing an infringement taking place.
It is with this background that the present invention has been developed. Reference to prior art documents is not an admission that they form part of the common general knowledge of a skilled person in any jurisdiction.
Summary of the Present Invention
According to one aspect of the present invention there is provided a potential driving infringement flagging system, said system comprising:
an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving;
an output that indicates the result of the analysis.
In an embodiment the processor is configured with image analysis rules to determining whether the driver is potentially acting in a manner that is illegal while driving.
In one embodiment the image analysis rules comprise rules for detecting use of a mobile telephone by the driver.
In one embodiment the image analysis rules comprise rules for detecting incorrect (or no) seatbelt use by the driver.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by analysing the image for a visual footprint of a mobile phone, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether there is facial extension when the face is determined to be forward facing, and in the event that it is determined that there is facial extension then the analysing outcome is that the driver may be using a mobile telephone. ln an embodiment the processor is configured to determine that there is a facial extension when the ratio of respective distances between each pupil and the ear on the
corresponding side of the driver's face is within a range of values. In an embodiment the range of values is between 0.95 and 1 .05.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver should be in view, but is obscured when the face is determined not to be forward facing, and in the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the processor is configured to determine that the ear should be in view when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values. In an embodiment the range of values is between 1 .1 and 1 .8, and in another embodiment between 1 .6 and 1 .8.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver is obscured when the face is determined to be forward facing; and
3) determining whether there is a hand visible proximal to the face when an ear is not obscured, and in the event that it is determined that a hand is visible proximal to the face then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the processor is configured to determine that a hand is proximal to the face when the hand is with a distance of four times the height of the driver's nose.
In an embodiment the processor is configured to determine that the ear is obscured the head is turned and the ear on the side of the face opposite to the direction in which the head is turned is visible. ln an embodiment the processor is configured to determine that the head is turned when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value. In an embodiment the value is 1 .1 and in an alternative embodiment the value is 1 .3.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that neither the eyes nor glasses are visible then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone. In an embodiment the value is 1 .8 and in an alternative embodiment the value is 2.
In an embodiment the processor is configured to determine whether glasses are being worn by the driver, and if so determined then the position of the pupil of each eye is approximated.
In an embodiment combinations of the above configurations are used. In an embodiment the processor is configured to analyse the image to determine whether the driver is correctly wearing a seatbelt, or not.
In an embodiment the processor is configured to determine whether the driver is correctly wearing a seatbelt by determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver. In an embodiment the acute angle is within the range between 30 and 60 degrees. In an embodiment the set value range is between 3 and 4 times the height of the nose.
According to the present invention there is provided a mobile telephone use while driving recognition system, said system comprising: an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is using a mobile telephone, or not;
an output that indicates the result of the analysis.
According to the present invention there is provided a seatbelt worn while driving recognition system, said system comprising:
an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is wearing a seatbelt, or not;
an output that indicates the result of the analysis.
According to the present invention there is provided a method of detection of a potential driving infringement, said method comprising:
receiving an image of a driver of a vehicle;
analysing the image to determine whether the driver is potentially acting in a manner that is illegal while driving;
outputting an indication of the result of the analysis. In an embodiment the analysis comprising applying image analysis rules to the image to determining whether the driver is potentially acting in a manner that is illegal while driving.
In one embodiment applying the image analysis rules comprises detecting use of a mobile telephone by the driver.
In one embodiment applying the image analysis rules comprises detecting absence of correct seatbelt use by the driver.
In an embodiment the analysing the image comprises determining whether there is a visual footprint of a mobile phone in the image, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone. ln an embodiment the analysing the image comprises determining:
1 ) determining whether the driver's face is forward facing;
2) determining whether there is facial extension when the face is determined to be forward facing, and in the event that it is determined that there is facial extension then the analysing outcome is that the driver may be using a mobile telephone.
The step of determining whether there is facial extension comprises determining whether the ratio of respective distances between each pupil and the ear on the corresponding side of the face is within a range of values. In an embodiment the range of values is between 0.95 and 1 .05.
In an embodiment the analysing the image comprises:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver should be in view, but is obscured when the face is determined not to be forward facing, and in the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the step of determining whether the face is forward facing comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values. In an embodiment the range of values is between 1 .1 and 1 .8 and in an alternative embodiment between 1 .6 and 1 .8. When the face is within the range it is deemed to be forward facing. In an embodiment the analysing the image comprises:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver is obscured when the face is determined to be forward facing; and
3) determining whether there is a hand visible proximal to the face when an ear is not obscured, and in the event that it is determined that a hand is visible proximal to the face then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment a hand is regarded as proximal to the face if it is with a distance of four times the height of the driver's nose. The step of determining whether the ear is obscured when the face is not deemed to be forward facing comprises determining whether the head is turned and in which direction, and then determining whether the ear on the side of the face opposite to the direction in which the head is turned is visible.
In an embodiment determining whether the head is turned comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value. In an embodiment the value is 1 .1 and in alternative embodiment the value is 1 .3.
In an embodiment the analysing the image comprises: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that the eyes or glasses are not visible then the analysing outcome is that the driver may be using a mobile telephone.
In an embodiment the analysing the image comprises: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone. In an embodiment the value is 1 .8 and in an alternative embodiment the value is 2.
In an embodiment the analysing the image comprises determining whether glasses are being worn by the driver the pupil, and if so determined then the position of the pupil of each eye is approximated.
In an embodiment the analysing the image comprises determining whether the driver is correctly wearing a seatbelt, or not.
In an embodiment the analysing the image comprises determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver. In an embodiment the acute angle is within the range between 30 and 60 degrees. In an embodiment the set value range is between 3 and 4 times the height of the nose.
In an embodiment one or more combinations of the analysis steps are used. According to the present invention there is provided method of detecting possible mobile telephone use while driving, said method comprising:
receiving an image of a diver of a vehicle;
analysing the image to determine whether the driver is using a mobile telephone, or not; indicating the result of the analysis.
According to the present invention there is provided a method of detecting possible incorrect seatbelt use while within a vehicle, said method comprising:
receiving an image of a person within a vehicle;
analysing the image to determine whether the person is wearing a seatbelt, or not;
indicating the result of the analysis.
According to the present invention there is provided a computer program comprising instructions for controlling a processor to perform the method above.
In this specification the terms "having", "comprising" or "comprises" are used inclusively and not exclusively or exhaustively.
Description of Drawings
In order to provide a better understanding of the present invention preferred embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure 1 is a schematic flow chart of an embodiment of a method of detecting a potential driving infringement according to an embodiment of the invention;
Figure 2 is a schematic flow chart of an embodiment of a method of detecting possible incorrect seatbelt use while within a vehicle according to an embodiment of the invention; Figure 3 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
Figure 4 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
Figure 5 is a schematic simulation of an image processed in order to perform the method of Figure 1 ;
Figure 6 is a schematic simulation of an image processed in order to perform the method of Figure 2; and Figure 7 is a schematic block diagram of a system for detecting a potential driving infringement according to an embodiment of the invention.
Description of Embodiments of the Invention
Referring to the Figure 7 there is shown a system 10 according to an embodiment of the present invention comprising a processor 12, a camera 14, a storage medium 16 and an output 18. The processor may be a general purpose computer loaded with a computer program comprising instructions for causing the processor 12 to operate as the system 10 in combination with the other components of the system 10 and to perform an
embodiment of the method of the present invention, or the processor 12 may be a purpose configured processing device.
The processor 12 may comprise memory, or the storage 16 may comprise memory in which the instructions are stored and operated on.
The camera 14 may be an integral component of the system 10, or it may be external. In particular the camera 14 may be remotely located from the rest of the system. The camera 14 is generally portable and may be set up in a suitable location in order to observe the drivers. The camera 14 may form part of another system, such as an automated number plate recognition system. Indeed the system 10 may form part of another system, such as an automated number plate recognition system, or a traffic monitoring system. As such the camera may be permanently installed to view a part of a roadway system, such as for example a roadside or overhead traffic monitoring camera.
The storage 16 may be an integral component of the system 10, or it may be external. In particular the storage 16 may be remotely located from the rest of the system. The storage 16 may form part of another system, such as an automated number plate recognition system. The storage may be in the form of a hard disk drive or flash memory and in an embodiment stores a database of images of common mobile telephones. Also in an embodiment the storage 16 stored image from the camera 14 as they are taken.
The processor 12 may be configured as, or the computer program may be arranged to configure the processor 12 as, functional modules, each of which performs a step in the method. The computer program may be stored in a non-volatile, non-transitory state, for example on a computer readable storage medium, such as on a hard disk drive, or flash memory, or permanent storage medium, such as a CD-ROM, DVD-ROM, Blueray-ROM etc.
The output 18 may be in the form of a display to an operator of the system 10 which shows the operator an indication that the driver may be acting in a manner that is not permitted. The indication may be showing the image of the driver taken by the camera 14, that that a formal infringement process may be commenced by the operator.
Alternatively the system 10 may issue infringement notices automatically, as these are regarded as the indication of infringement. The indication may be an illumination of a signal light or an audible warning for the operator to manually observe the driver.
In the event of the system 10 analysing an image taken from the camera and no indication of an infringement is detected, this may be indicated to the operater, or there may be no indication given in this case.
Thus generally the system 10 comprises an image source, such as camera 14, that captures or receives an image of a diver of a vehicle; a processor 12 configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving; and an output 18 that indicates the result of the analysis.
More particularly in an embodiment the processor is configured with image analysis rules to determine whether the driver is potentially acting in a manner that is illegal while driving. In one example the image analysis rules comprise rules for detecting use of a mobile telephone by the driver. In another example the image analysis rules comprise rules for detecting no or incorrect seatbelt use by the driver. The image analysis rules may be for both detecting use of a mobile telephone by the driver and incorrect seatbelt use by the driver. They may also perform other image analysis. Generally they will be often be combined with or form part of an automatic number plate recognition system such that the infringement may be enforced by identifying the vehicle.
Referring to Figure 1 there is shown an embodiment of a method 100 of detection of a potential driving infringement. The method 100 commences with the processor 12 receiving an image of a diver of a vehicle from the image source. Often the image will be of the whole front of the vehicle and the system 10 will conduct number plate analysis according to known methods. In this embodiment of the present invention a portion of the image showing the driver is isolated from the rest of the image. Insofar as the present invention is concerned this isolated portion of the image is regarded as the image of the driver even if it is not separated from the rest of the image. Localisation to the driver can occur in a similar manner to number plate localisation, in that the position of the driver within the vehicle is generally in the same region in relation to the rest of the vehicle (or indeed the number plate). Variations to the position of the driver may be needed with different types of vehicle.
The method 100 may serially, or in parallel, perform a seatbelt worn correctly check at 200, which is described further in relation to Figure 2.
When the image is a frame of a video stream a preliminary step 104 may be to check whether a new face is in the image. There may also be a number of frames in a video image that are suitable, and there may be some processing to select the best image.
The analysis of the image to determine whether the driver is potentially acting in a manner that is illegal while driving, and in particular to determine whether the driver is using a mobile telephone, is broken down into a number of tests performed at 106, 1 10, 1 14, 1 16, 1 18, 122 and 126. The outputting of an indication of the result of the analysis is performed at 136. Each of these may be performed by a corresponding functional module of the processor 12. Step 106 is to analyse the image analysis to determine whether there is a visual footprint of a mobile phone in the image. The visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones stored in the storage 16. In the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone and path 108 is taken. In the even that there is no visual footprint of a common phone in the image or the analysis is uncertain, the process proceeds to step 1 10.
Step 1 10 is a zero exclusion test. This comprises a number of precursor image analysis steps and then a number of sub-tests. The precursor image analysis steps are performed on the image 300 with reference to Figure 3. The permitter of the face 302 may be identified, such as by use of an edge detection or shape detection algorithm. The eyes 304 may be identified and within those the pupils identified or estimated. In the case of glasses being detected and the pupils obscured, the positions of the pupils are estimated. The distance A in the image of the separation of the pupils is calculated. The position of the nose 306 is determined and the length D of the nose in the image is calculated. The ratio of A to D is calculated as a fraction, that is A / D is calculated. This eliminates the scale of the image as a spurious factor, as well as size of the face.
The position of the ears 308L and 308R are detected. (R and L are right and left in the image, not from the point of view of the person). The ears 308L and 308R extend horizontally from the perimeter of the face 302, roughly vertically half way up the face. The most horizontally separated tips of the ears 308L and 308R are detected and the horizontal distance in the image from the respective tip to corresponding side of the face pupil is calculated E(L) and E(R).
Because this ratio of A to D is within a range in a high proportion of cases, it can use used to represent when the driver's head is or is not facing forward. When the head is turned from side to side A will be less than when facing forward.
Generally a face is symmetrical and so one would expect E(L) to be about the same as E(R) when the face is facing forward. The first subtest is to determine whether the driver's face is forward facing. In an embodiment the step of determining whether the face is forward facing comprises determining whether the ratio of A to D (ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver) is within a range of values. In an embodiment the range of values is between 1 .1 and 1 .8. In an alternative embodiment the range of values is 1 .6 to 1 .8. When the face is within the range it is deemed to be forward facing.
A sub-test may include analysing the image to determine whether there is any "facial extension", that is when the face appears to the larger on one side. The step of determining whether there is facial extension comprises determining whether the ratio of respective distances between each pupil and the ear on the corresponding side of the face, that is E(L) / E(R), is within a range of values. In an embodiment the range of values is between 0.95 and 1 .05. When this ration is outside of this range the face is regarded as extended.
A further sub-test may include analysing the image to determine whether a hand is visible in the image within a distance from the perimeter of the face 302. In an embodiment the distance is 4 x D. If as a result of these sub-tests, the face is determined to be forward facing, there is no facial extension and there is not hand detected within the required distance of the face, then the process takes path 1 12, indicating that the driver is deemed to not be using a mobile telephone. The process then proceeds to review a new face at 104. If however the subtests result in the face not being forward facing, or there may be facial extension, or a hand is detected within the required distance, or any of these are inconclusive, further analysis is required and the process proceeds to step 1 14.
In step 1 14, when the face is determined to be forward facing a further comparison of the E(L) to E(R) ration is performed. When the ratio is outside of a range then there is the possibility of the face be "extended" by the driver's hand being on one ear. In an embodiment the range is between 0.8 and 1 .2 and in an alternative embodiment the range is between 0.9 and 1 .1 . In the event that it is determined that there is this degree of facial extension then the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 1 16. The flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 1 18. In step 1 18, when the face is determined to be forward facing the analysis of this step is bypassed. Thus this analysis occurs when the face is not deemed to be forward facing. In this analysis it is determined whether an ear of the driver should be in view, but is obscured. This comprises determining whether the head is turned and in which direction, and then determining whether the ear on the side of the face opposite to the direction in which the head is turned is visible. In an embodiment determining whether the head is turned comprises determining whether the ratio A to D (distance between pupils of the eyes of the driver divided by the height of the nose of the driver) is less than a value. In an embodiment the value is 1 .1 and in an alternative embodiment the value is 1 .3. In an embodiment the determining whether the head is turned comprises determining whether the tip of the ear 328L is not visible or the tip of the ear 328R is inside the perimeter of the face 302 (when the face is turned to the right). Naturally if the face is turned to the left the testing is swapped. In the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 120. Again, the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 122.
In step 122, an analysis is performed to determine whether there is a hand visible proximal to the face. In an embodiment a hand is regarded as proximal to the face if it is within a distance of four times the height of the driver's nose, that is 4 x D. In the event that it is determined that a hand is visible proximal to the face then the analysis outcome is that the driver may be using a mobile telephone and a flag is set at 124. Again, the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. In an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 126. In step 126, an analysis is performed to determine whether the diver's eyes are visible or for whether the driver's face is facing down. In an embodiment this comprises
determining whether the ratio of the distance between the pupils and the length of the nose (A / D) is more than a value and in the event that it is determined that the ratio is more than the value then the analysing outcome is that the driver may be using a mobile telephone. In an embodiment the value is 1 .8 and in an alternative embodiment the value is 2. In the event that it is determined that the eyes or glasses are not visible or that the driver is looking down then the analysing outcome is that the driver may be using a mobile telephone and a flag is set at 128. Yet again, the flag may be a single flag used in all the remaining tests, or it may be a specific flag indicating the reason for the flag being raised. ln an embodiment the flag is not set is the out is uncertain. The process then proceeds to step 130.
In step 130, it is determined whether a flag has been set. In the event that a flag is not set 132 the driver is deemed to not be using a mobile telephone and the process proceeds to analyse a new face at 104. In the even that a flag is set the process proceeds at 134 to step 136.
Step 136 may perform further analysis, may issue an infringement 140, or may provide an indication via output 18 for a manual inspection by an operator. In one embodiment the image may be stored in the storage 16 for later review on a display. If manually viewed and the driver is not acting in an illegal manner the process may proceed to delete the record at 138. In the event that manual inspection confirms that the driver is using a mobile telephone an infringement may be issued at 140 and the image is usually stored in case it is later needed in a legal process.
Referring to Figure 2 a process 200 of analysing the image comprises determining whether the driver is correctly wearing a seatbelt, or not. This process may be conducted in the same system 10 that conducts the method 100, or in an independent system 10.
The method 200 commences with the processor 12 receiving an image of a diver of a vehicle. The image may the same as in method 100. The analysis of the image to determine whether the driver is potentially acting in a manner that is illegal while driving, and in particular to determine whether the driver is correctly using a seatbelt using analysis conducted at step 204. The outputting of an indication of the result of the analysis is performed at 216. Each of these may be performed by a corresponding module of the processor 12.
Step 204 is to analyse the image to determine whether the seatbelt is correctly used.
The precursor image analysis steps from method 100 may be used, or a precursor image analysis is performed on the image 330 with reference to Figure 6. Again the permitter of the face 302 may be identified, such as by use of an edge detection algorithm. The position of the nose 306 is determined and the length D of the nose in the image is calculated. The seatbelt 340 is identified as a band of contrast below the face. The angle Θ of the band is determined as is the vertical distance 344 from the bottom of the nose 336 to point of intersection 342 with the band.
The angle Θ is tested to fall within a range. In an embodiment the range is between 30 and 60°. The distance 344 from the bottom of the nose 336 is also tested to fall within a range. In an embodiment the range is between 3 and 4 times the length of the nose 336, that is 3 x D to 4x D.
When these are within the ranges the seatbelt is determine to be correctly worn. The process progresses down path 206. The process may then consider a new face again at 204.
If no seatbelt is detected a flag is set. If either of the calculated values are outside of these ranges, then the outcome of the analysis is that the driver may not be correctly wearing the seatbelt, and a flag is set at 208. There may be a flag for each of the tests or a common flag may be used. In the case of a common flag, the flag may also be in common with the common flag used in method 100. In an embodiment the flag is not set if the outcome is uncertain. The process then proceeds to step 210. In step 210, is determined whether a flag has been set. In an embodiment the flags for angle and distance may be tested, with neither or only one being set, the result is that the driver is deemed to be using a seatbelt correctly, unless no seatbelt is detected. In this case, the process proceeds down path 212 to analyse a new face at 204. In the even that no seatbelt is detected or both flags are set, or if the system 10 is configured such that only need needs to be set, then the process proceeds at 214 to step 216.
Step 216 may perform further analysis, may issue an infringement 220, or may provide an indication via output 18 for a manual inspection by an operator. In one embodiment the image may be stored in the storage 16 for later review on a display. If manually view and the driver is not acting in an illegal manner the process may proceed to delete the record at 218. In the event that manual inspection confirms that the driver is not correctly using the seatbelt an infringement may be issued at 220 and the image is usually stored in case it is later needed in a legal process. Step 216 may be combined with step 136 of method 100. This process may also be performed on a passenger instead of a driver - the only difference being identifying the passenger in a larger image and processing the localised image of the passenger as if it were the image of the driver for purposes of determining whether the passenger is correctly using the seatbelt.
Advantages of the present invention are that enforcement of driver behaviour can be automated, or partially automated, with only likely candidates requiring visual inspection. In particular, laws relating to mobile telephone use while driving and correct seatbelt use can be enforced with greater effectiveness and efficiency.
Modifications may be made to the present invention with the context of that described and shown in the drawings. Such modifications are intended to form part of the invention described in this specification.

Claims

1 . A potential driving infringement flagging system, said system comprising:
an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is potentially acting in a manner that is illegal while driving;
an output that indicates the result of the analysis.
2. A system as claimed in claim 1 , wherein the processor is configured with image analysis rules to determining whether the driver is potentially acting in a manner that is illegal while driving.
3. A system as claimed in claim 2, wherein image analysis rules comprise rules for detecting use of a mobile telephone by the driver.
4. A system as claimed in claim 2, wherein the image analysis rules comprise rules for detecting incorrect seatbelt use by the driver.
5. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by analysing the image for a visual footprint of a mobile phone, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone.
6. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether there is facial extension when the face is determined to be forward facing, and in the event that it is determined that there is facial extension then the analysing outcome is that the driver may be using a mobile telephone.
7. A system as claimed in claim 6, wherein the processor is configured to determine that there is a facial extension when the ratio of respective distances between each pupil and the ear on the corresponding side of the driver's face is within a range of values. In an embodiment the range of values is between 0.95 and 1 .05.
8. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver should be in view, but is obscured when the face is determined not to be forward facing, and in the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone.
9. A system as claimed in claim 8, wherein the processor is configured to determine that the ear should be in view when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values.
10. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver is obscured when the face is determined to be forward facing; and
3) determining whether there is a hand visible proximal to the face when an ear is not obscured, and in the event that it is determined that a hand is visible proximal to the face then the analysing outcome is that the driver may be using a mobile telephone.
1 1 . A system as claimed in claim 10, wherein the processor is configured to determine that a hand is proximal to the face when the hand is with a distance of four times the height of the driver's nose.
12. A system as claimed in claim 10, wherein the processor is configured to determine that the ear is obscured when the head is turned and the ear on the side of the face opposite to the direction in which the head is turned is visible.
13. A system as claimed in claims 6, 8 or 10, wherein the processor is configured to determine that the head is turned when the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value.
14. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that neither the eyes nor glasses are visible then the analysing outcome is that the driver may be using a mobile telephone.
15. A system as claimed in claim 3, wherein the processor is configured to analyse the image to determine whether or not the driver is using a mobile telephone by: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone
16. A system as claimed in claim 13 or 15, wherein the processor is configured to determine whether glasses are being worn by the driver, and if so determined then the position of the pupil of each eye is approximated.
17. A system as claimed in claim 4, wherein the processor is configured to analyse the image to determine whether the driver is correctly wearing a seatbelt, or not.
18. A system as claimed in claim 17, wherein the processor is configured to determine whether the driver is correctly wearing a seatbelt by determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver.
19. A mobile telephone use while driving recognition system, said system comprising: an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is using a mobile telephone, or not;
an output that indicates the result of the analysis.
20. A seatbelt worn while driving recognition system, said system comprising:
an image source that captures or receives an image of a driver of a vehicle;
a processor configured to analyse the image to determine whether the driver is wearing a seatbelt, or not;
an output that indicates the result of the analysis.
21 . A method of detecting possible mobile telephone use while driving, said method comprising:
receiving an image of a driver of a vehicle;
analysing the image to determine whether the driver is using a mobile telephone, or not; indicating the result of the analysis.
22. A method of detecting possible incorrect seatbelt use while within a vehicle, said method comprising:
receiving an image of a person within a vehicle;
analysing the image to determine whether the person is wearing a seatbelt, or not;
indicating the result of the analysis.
23. A method of detection of a potential driving infringement, said method comprising: receiving an image of a driver of a vehicle;
analysing the image to determine whether the driver is potentially acting in a manner that is illegal while driving;
outputting an indication of the result of the analysis.
24. A method as claimed in claim 23, wherein the analysis comprising applying image analysis rules to the image to determining whether the driver is potentially acting in a manner that is illegal while driving.
25. A method as claimed in claim 24, wherein applying the image analysis rules comprises detecting use of a mobile telephone by the driver.
26. A method as claimed in claim 24, wherein applying the image analysis rules comprises detecting absence of correct seatbelt use by the driver.
27. A method as claimed in claim 25, wherein the analysing the image comprises determining whether there is a visual footprint of a mobile phone in the image, where the visual footprint is one of a plurality of footprints stored in a database of visual footprints of mobile phones, and in the event that the image includes the visual footprint then the analysing outcome is that the driver may be using a mobile telephone.
28. A method as claimed in claim 25, wherein the analysing the image comprises determining:
1 ) determining whether the driver's face is forward facing;
2) determining whether there is facial extension when the face is determined to be forward facing, and in the event that it is determined that there is facial extension then the analysing outcome is that the driver may be using a mobile telephone.
29. A method as claimed in claim 28, wherein whether there is facial extension comprises determining whether the ratio of respective distances between each pupil and the ear on the corresponding side of the face is within a range of values.
30. A method as claimed in claim 25, wherein the analysing the image comprises:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver should be in view, but is obscured when the face is determined not to be forward facing, and in the event that it is determined that the ear is obscured then the analysing outcome is that the driver may be using a mobile telephone.
31 . A method as claimed in claim 30, wherein the step of determining whether the face is forward facing comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is within a range of values.
32. A method as claimed in claim 25, wherein the analysing the image comprises:
1 ) determining whether the driver's face is forward facing;
2) determining whether an ear of the driver is obscured when the face is determined to be forward facing; and
3) determining whether there is a hand visible proximal to the face when an ear is not obscured, and in the event that it is determined that a hand is visible proximal to the face then the analysing outcome is that the driver may be using a mobile telephone.
33. A method as claimed in claim 32, wherein a hand is regarded as proximal to the face if it is with a distance of four times the height of the driver's nose.
34. A method as claimed in claim 32, wherein of determining whether the ear is obscured when the face is not deemed to be forward facing comprises determining whether the head is turned and in which direction, and then determining whether the ear on the side of the face opposite to the direction in which the head is turned is visible.
35. A method as claimed in claim 28, 30, or 32, wherein determining whether the head is turned comprises determining whether the ratio of distance between pupils of the eyes of the driver and the height of the nose of the driver is less than a value.
36. A method as claimed in claim 31 or 35, wherein the analysing the image comprises: determining whether the diver's eyes are visible or that glasses are visible and in the event that it is determined that the eyes or glasses are not visible then the analysing outcome is that the driver may be using a mobile telephone.
37. A method as claimed in claim 25, wherein the analysing the image comprises: determining whether the ratio of the distance between the pupils and the length of the nose is more than a value and in the event that it is determined that the ration is more than the value then the analysing outcome is that the driver may be using a mobile telephone.
38. A method as claimed in claim 31 , 35 or 36, wherein the analysing the image comprises determining whether glasses are being worn by the driver the pupil, and if so determined then the position of the pupil of each eye is approximated.
39. A method as claimed in claim 26, wherein the analysing the image comprises determining whether the driver is correctly wearing a seatbelt, or not.
40. A method as claimed in claim 39, wherein the analysing the image comprises determining whether there is a contrast line at an acute angle within a set value range from the bottom of the nose of the driver.
PCT/AU2014/000783 2013-08-06 2014-08-06 Driving infringement flagging system WO2015017883A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
AU2014305641A AU2014305641A1 (en) 2013-08-06 2014-08-06 Driving infringement flagging system
SG11201600924WA SG11201600924WA (en) 2013-08-06 2014-08-06 Driving infringement flagging system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2013902946 2013-08-06
AU2013902946A AU2013902946A0 (en) 2013-08-06 Driving infringement flagging system

Publications (1)

Publication Number Publication Date
WO2015017883A1 true WO2015017883A1 (en) 2015-02-12

Family

ID=52460420

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2014/000783 WO2015017883A1 (en) 2013-08-06 2014-08-06 Driving infringement flagging system

Country Status (3)

Country Link
AU (1) AU2014305641A1 (en)
SG (1) SG11201600924WA (en)
WO (1) WO2015017883A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109969195A (en) * 2019-03-25 2019-07-05 陕西科技大学 A kind of driver's face abnormal behavior alarm system and method and the vehicle based on it
CN110503831A (en) * 2019-08-28 2019-11-26 上海眼控科技股份有限公司 A kind of method and apparatus identifying driver's illegal activities
WO2020014731A1 (en) 2018-07-19 2020-01-23 Acusensus Pty Ltd Infringement detection method, device and system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108922175B (en) * 2018-06-22 2021-10-01 大连理工大学 Method and device for recording illegal behaviors of multiple motor vehicles crossing solid line

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005205943A (en) * 2004-01-20 2005-08-04 Omron Corp Vehicle control device and telephone usage detection method at telephone usage in driving
EP1744292A2 (en) * 2005-07-08 2007-01-17 Van de Weijdeven, Everhardus Franciscus Method for determining data of vehicles
CN103150556A (en) * 2013-02-20 2013-06-12 西安理工大学 Safety belt automatic detection method for monitoring road traffic

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005205943A (en) * 2004-01-20 2005-08-04 Omron Corp Vehicle control device and telephone usage detection method at telephone usage in driving
EP1744292A2 (en) * 2005-07-08 2007-01-17 Van de Weijdeven, Everhardus Franciscus Method for determining data of vehicles
CN103150556A (en) * 2013-02-20 2013-06-12 西安理工大学 Safety belt automatic detection method for monitoring road traffic

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020014731A1 (en) 2018-07-19 2020-01-23 Acusensus Pty Ltd Infringement detection method, device and system
US11610481B2 (en) 2018-07-19 2023-03-21 Acusensus Ip Pty Ltd Infringement detection method, device and system
US11990035B2 (en) 2018-07-19 2024-05-21 Acusensus Ip Pty Ltd Infringement detection method, device and system
CN109969195A (en) * 2019-03-25 2019-07-05 陕西科技大学 A kind of driver's face abnormal behavior alarm system and method and the vehicle based on it
CN109969195B (en) * 2019-03-25 2024-02-27 陕西科技大学 Driver facial behavior abnormality alarm system and method and vehicle based on system and method
CN110503831A (en) * 2019-08-28 2019-11-26 上海眼控科技股份有限公司 A kind of method and apparatus identifying driver's illegal activities

Also Published As

Publication number Publication date
AU2014305641A1 (en) 2016-03-03
SG11201600924WA (en) 2016-03-30

Similar Documents

Publication Publication Date Title
US9672440B2 (en) Damage recognition assist system
US9984568B2 (en) Wrong-way determination apparatus
CN109637158B (en) Intelligent vehicle confluence early warning method for expressway entrance ramp area
CN109758167A (en) Driver status detection device
EP2246831B1 (en) Dangerous drive preventive intention judgment system and dangerous drive preventive intention judgment method
JP6455866B2 (en) Monitoring device, monitoring method, and program
ES2689100T3 (en) Traffic behavior monitoring system
WO2015017883A1 (en) Driving infringement flagging system
KR101834838B1 (en) System and method for providing traffic information using image processing
CN109559519A (en) Monitoring device and its parking offense detection method, device, readable storage medium storing program for executing
JP7255582B2 (en) DANGEROUS ACTION ELIMINATION SYSTEM, DEVICE, METHOD AND PROGRAM
CN110866479A (en) Method, device and system for detecting that motorcycle driver does not wear helmet
KR102332517B1 (en) Image surveilance control apparatus
CN107004353B (en) Traffic violation management system and traffic violation management method
CN109146914B (en) Drunk driving behavior early warning method for expressway based on video analysis
CN103700220A (en) Fatigue driving monitoring device
CN111492416A (en) Violation monitoring system and violation monitoring method
JP6247869B2 (en) Driving evaluation device
CN110220721A (en) A kind of vehicle A column visual field appraisal procedure
CN111383248A (en) Method and device for judging red light running of pedestrian and electronic equipment
CN103956027A (en) Driver eye fatigue detection device and method
WO2018129653A1 (en) Method for collecting evidence of not yielding to school bus, apparatus, and school bus
CN115131960B (en) Intelligent safety processing method and device for road vehicle accidents
CN109949613A (en) Pedestrian's alarming device
CN212460040U (en) Traffic running height early warning device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14834506

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2014305641

Country of ref document: AU

Date of ref document: 20140806

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 14834506

Country of ref document: EP

Kind code of ref document: A1