WO2018154262A1

WO2018154262A1 - User detection

Info

Publication number: WO2018154262A1
Application number: PCT/GB2017/053306
Authority: WO
Inventors: Christopher Andrew DAVIES
Original assignee: Davies Christopher Andrew
Priority date: 2017-02-24
Filing date: 2017-11-02
Publication date: 2018-08-30
Also published as: GB201703043D0

Abstract

Method and system for detecting an interaction with a user comprising the steps of: presenting digital content to a user on a device. Determining if an interaction between the device and the user occurs while the digital content is being presented on the device. If it is determined that an interaction between the device and the user has occurred then taking an action.

Description

User Detection

Field of the Invention The present invention relates to a system and method for detecting the presence of a user operating a device, such as a computer or mobile device. Determining the presence of a user enables other functions, actions or processes to take place or to cease.

Background of the Invention

There can be several reasons that it is necessary to detect the presence of a user interacting with a device, such as a computer or smart phone. These may include determining if a person is viewing content on the device (for example, video content) or to determine if the content is being displayed on a screen without being viewed. This can be particularly important for mobile devices where resources such as computing and battery power can be limited. This may also be useful in reducing network bandwidth

requirements, as streamed or downloaded content may be paused when not being viewed or otherwise consumed. However, it is currently difficult to accurately determine that a user is viewing or interacting with a device. If the content is paused incorrectly (i.e. when a user is actually present and viewing the screen) then this will be decrease the usability of the device. Conversely, if the content is being displayed when the user is not viewing or otherwise interacting with the device, then subsequent actions, events or metrics based on that assumption that the content was viewed by a person may be inaccurate and resources both on the device, any attached network or server may be wasted.

In more specific examples, it may also be useful to substantiate that a smart phone user has interacted with digital content when that content is video. Confirmation that the video has played on the device may be used for various purposes, including accounting purposes (e.g. recording that a video advertisement has actually been viewed a particular number of times), regulatory purposes (e.g. confirming that training material has being viewed) or safety reasons (e.g. confirming that passengers have viewed a safety film before a journey commences). For advertising, in particular, there are several major issues facing the multi-million dollar mobile video advertising industry.

Determining when and if a mobile video advertisement is 'in view' is currently difficult to assess. The '100% viewability' metric is one of the current standards being adopted by the industry in order to invoice an advertiser and/or agency for the placement of a video advertisement (i.e. that the entire video unit is in view for the duration of the advertisement). There are a variety of methods adopted to invoice advertiser clients based on whether the video advertisement has played on the screen of the user and whether the entire advertisement has 'completed', i.e. played for the entire duration of a15, 30 second or 60 second advertisement, for example.

There is no current industry-wide method to verify that the user has in fact watched and viewed the advertisement that has played on their screen.

The second issue facing the industry relates to whether the advertisement has played in front of an actual human being. Such difficulties may be exploited in a fraudulent manner where advertisements are retrieved from a server in bulk in such a way that the server considers that they have been viewed and pays advertising fees accordingly but the process actually takes place without any real person viewing the material (e.g. busted/#ca9a9e24ca8c retrieved 3 February 2017).

A number of companies currently seek to offer 'non-human traffic detection' to detect and block page requests emanating from bots but this is not done within the advertisement unit.

The third and final important issue facing the advertising industry, in particular, relates to engagement and attention. A video may meet a viewability criteria and may be played in front of a genuine human but the user may actually be viewing other content on the website or app, unless the video advertisement is in full screen. Even when the video advertisement is played on the full screen, there is an assumption made that the user engages with the advertisement but this cannot be proven with current techniques.

Nevertheless, the advertiser and/or agency may be invoiced for the advertisement played on the device's screen.

There is also a need to be able to interact with a viewer of digital content without pausing the content.

Therefore, there is required a method and system that overcomes these problems. Summary of the Invention

A method and system for the detection of human interaction and/or engagement with content on a device (e.g. a mobile device such as a smartphone or tablet) is provided. An interaction with a user may be detected before or during playback, rendering or displaying of digital content such as video, audio, still images (or a series or slideshow of still images) or other material, for example. The interaction may be viewing the material (or simply looking at the screen when the material is being presented or played). This may be described as a passive interaction. The interaction may be more direct or dynamic, such as the user issuing commands or causing signals that are detected by the device. When still images (e.g. an interstitial ad) are skipped, this can take the form of exiting the content (e.g. removing the still image from the screen or presentation).

If the interaction is detected then the presentation of the digital content may commence or continue (if it has already started). If the interaction is not detected or ceases, perhaps for a particular length of time, then an action (or decision or result) may be taken by the device (or server providing the content to the device). This action may be for the presentation of the content to stop, to continue, to record that the interaction took place, to transmit data indicating that the interaction took place (or not if it didn't), to direct the user to a web site, to receive an answer from the user to a query, to skip the playback of the digital content and/or to provide new content, to provide a requested service to a user (e.g. to provide a news story or film) or any other action. If the presentation ceases then it may restart when the same or a different interaction with the user is detected or may continue with the playback of the digital content without skipping until the particular user interaction is detected. Where the digital content pauses or stops unless an interaction is detect, the presentation may start from where it left off (i.e. at the point that it was stopped or paused), close to where it left off (e.g. 0-10 seconds before or after) or may restart from the beginning, for example. The presentation or rendering may continue even if no interaction is detected and a different action may then result (e.g. different data are recorded). The action may comprise more than one component actions, for example.

The determination of the interaction may take place at the device, at a remote entity like a server or in different locations, for example. One or more actions may take place when the entirety of the digital content has played or been presented in full (e.g. with or without determining that the user is not interacting with the device or when this occurs and the content restarting one or more times resulting in the content playback completing).

The interaction may be detected following a challenge to the user to complete one or more tasks (e.g. press a button on the device or peripheral connected to the device, provide an answer to a query, and/or the user may perform a touch gesture on a screen of the device or simply face the screen or speak). The interaction may also be detected without requiring a response to a challenge by the user. For example, an imaging device may capture the face of the user to determine if they were watching the content as it was being presented or a microphone of the device may detect a sound from the user. This may be described as a passive interaction with the device.

Furthermore, the interaction may be measured. For example, measurements may be made to record how long through a presentation (e.g. of an advertisement or instructional video) a viewer or plurality of viewers started the interaction (e.g. pressing on a screen icon to initiate a timed interaction), whether and how many times they stopped and perhaps restarted the interaction (e.g. taking their finger off the screen and then back on to the icon), the average time for a view to be presented with the content or other measurements.

In accordance with a first aspect there is provided a method for detecting the presence of a user or for detecting an interaction with the user, the method comprising the steps of:

playing or presenting digital content to a user on a device;

determining if an interaction between the device and the user occurs while the digital content is being played or presented on the device;

if it is determined that an interaction between the device and the user has occurred then taking an action (e.g. a passive action such as continuing to playing the digital content or an active action such as recording, storing or transmitting data) and/or if it is determined that an interaction between the device and the user has not occurred then taking an alternative action (e.g. pausing or stopping the digital content). For example, the content may be stopped (e.g. no longer presented) when a person is not viewing the material or may continue but with data recording that the user did not necessarily view the content. This can prevent wasting resources (e.g. bandwidth, power, battery, etc.), especially with the content is paused or stopped and can also ensure that the recording of viewing events taking place is more accurate. Either or both the action and alternative action may be active or passive actions.

Optionally, the action may be continuing the presenting or playing of the digital content or preventing it from being stopped or paused.

Optionally, the digital content may be video content. Other forms of content may be used including still images, series of still images, audio and text (or any combination of these), for example.

Optionally, presenting the video content may be playing the video content.

Optionally, if it is determined that an interaction between the device and the user has not occurred then taking an alternative action. Preferably, the alternative action may be stopping or pausing (e.g. temporarily so that it may restart from the same position or playback time or location) the digital content from presenting or playing (or playing one or more additional items of digital content). In other words, the video may pause with the last frame frozen on screen or may be able to restart from the same point or time code (even if the last frame is not displayed).

Optionally, the method may further comprise the steps of:

after stopping the digital content from being presented then determining that an interaction between the device and the user occurs and in response then starting the presentation of the digital content to the user. In other words, upon failure to detect an interaction with the user then playback or presentation stops. If an interaction is subsequently detected then the playback may start again (either from where it left off or from the beginning) or the content may only be started again from the beginning or from a set point in the playback.

Optionally, the method may further comprise the step of determining a time from the stopping of the playing or presentation of the digital playing and the determining that the interaction between the device and the user has occurred; and

if the determined time is greater than or equal to a predetermined time then waiting for a command to replay the digital content or a different digital content; and

if the determined time is less than the predetermined time then continuing the playing or presentation of the digital content from where it was stopped. Therefore, if the interaction occurs within a short time (predetermined or selectable by a content provider or advertiser) after the content playback stops then it may restart from the same point within the content. Otherwise, the playback may need to restart from the beginning or may be prevented from restarting at all.

Preferably, the method may further comprise the step of initiating the interaction between the device and the user. This initiation may come from the device (e.g. as software controlling the playback of content) or from another (e.g. external) source.

Optionally, the interaction may be initiated at a random time, at a predetermined time, and/or at intervals.

Optionally, the digital content may be any one or more of: a still image, a static interstitial advertisement unit, an advertisement, an instructional video, a slideshow, and/or audio. Having an interactive skip in an interstitial may help ensure that a real human has skipped and not a bot.

Optionally, the method may further comprise the step of recording the digital content completing. This can be recorded by the device (e.g. in memory or another data store) and/or by a server or remote entity. This may also be recorded by a remote entity providing the content to the device (e.g. by monitoring signals or messages received from the device or devices).

Optionally, the digital content may be played or presented on a touch screen of the device and the interaction may further comprise receiving an indication that a touch event occurred on the touch screen. In other words, the interaction is the user touching the screen (e.g. capacitive or resistive display).

Optionally, the action may be ending the playing or presentation of the digital content. This may involve skipping to a new digital content (e.g. digital content requested by a user) or providing a new service.

Optionally, the action may further comprise providing a service to the user. This may be in the form of further digital content (e.g. the same or in a different format to the played digital content).

Optionally, the step of determining if an interaction between the device and the user occurs further comprises: measuring a duration of the interaction and if the duration is greater or equal to a predetermined time then determining that the interaction occurred and if the duration is less than the predetermined time then determining that the interaction has not occurred. In other words, there may be a requirement that the interaction takes place over a minimum amount of time for it to be registered as a valid interaction. During this time, it may be assumed that the user is viewing the playing digital content. This predetermined time may be varied or set to different values (e.g. it may be dependent on the digital content being played or the action that will follow it).

Optionally, the method may further comprise the step of displaying a countdown timer during the duration of the interaction.

Optionally, the countdown timer may be non-numerical (e.g. graphical).

Optionally, the countdown timer may be configured to indicate when the

predetermined time has expired.

Optionally, the method may further comprise the step of displaying at least one item on the touch screen, the touch event may correspond with a user touching the touch screen over the displayed at least item (e.g. an icon). For example, there may be one, two, three, four or more items for the user to touch or select. A different action or response data may be recorded for each different item. One or more item may be selected by the user in any combination either simultaneously or at different times.

Optionally, determining if the interaction between the device and the user occurs may comprise determining if the user touched the screen over the displayed at least one item within a period of time starting when the at least one item was displayed. This may be a predetermined time.

Preferably, the method may further comprise the step of displaying a countdown timer when the at least one item is displayed.

Optionally, the countdown timer may be non-numerical (e.g. graphical). This may be a graph or item decreasing or increasing in size, changing shape, and/or changing colour, for example.

Optionally, the displayed item may be part of the digital content. This may be a character or object displayed within the content (e.g. static or moving object).

Optionally, the displayed item may be displayed at intervals (e.g. regular or irregular) during the playing of the digital content. The object may need to be touched each time it is displayed to indicate an interaction with the user (i.e. checking if the user is paying attention to the content).

Optionally, the displayed item may be displayed at different locations on the touch screen between the intervals. This may be randomly chosen or at predetermined locations, for example.

Optionally, the step of determining if an interaction between the device and the user occurs may further comprise determining that the user touched the screen for a time greater than or equal to a time threshold. In other words, the user must touch the screen (at a particular location or region) for up to or including a set time (e.g. three seconds, 1 -5 seconds, etc.).

Preferably, the method may further comprise the step of providing an indication to the user that the time threshold had been reached. This allows the user to know how long to hold their finger on the screen to provide an accurate response.

Optionally, the displayed at least one item may correspond with a response to a query provided by the user. For example, choosing a particular product or service.

Optionally, the method may further comprise the step of transmitting data indicating the response over a network (e.g. the internet or a mobile network).

Optionally, the action may be to launch a web page on a browser or an application. Optionally, the action may be to stop the digital content and provide further digital content. In other words, to skip through content (e.g. an advertisement) to obtain further content (e.g. a selected video).

Optionally, the interaction between the device and the user may be an image captured by the device of the user. This may be an image of the particular user or any person. Optionally, the step of determining if the interaction between the device and the user occurs may further comprise detecting at least a part of the face (e.g. the whole face, certain features of the face, the eyes, etc.) of the user within the captured image. Face detection software may be used (e.g. within the device).

Preferably, the image captured by the device may be a video frame capture by a camera of the device.

Optionally, the at least a part of face of the user is detected continuously or at intervals. Only detecting the user at intervals may improve battery or computing resource use.

Preferably, the face of the user may be detected during a continuous video capture or stream.

Optionally, the method may further comprise the step of recording measurements of one or more user interactions. These measurements may be collated and/or viewed or analysed at a later time. For example, the success of a campaign may be measured based on the amount of time or number of times that the digital content (e.g. advertisements) is confirmed to have been viewed.

Optionally, the measurements may include any one or more of: a time after the digital content commenced playing when the user interaction occurred or completed; the number and/or duration of user interactions that started but did not complete during playing of the digital content; and a type or identifier of digital content played to a user when the user interaction occurred. Other metrics or measurements may be recorded.

According to a second aspect there is provided a device (e.g. a mobile device) for detecting the presence of or other interaction with a user comprising:

a display;

a processor; and

memory storing computer-executable instructions that, when executed by the processor, cause the device to:

present or play digital content to a user on the display;

determine if an interaction between the device and the user occurs while the digital content is being presented or played on the device;

if it is determined that an interaction between the device and the user has occurred then taking an action; and/or

if it is determined that an interaction between the device and the user has not occurred then taking an alternative action (e.g. stop digital content from playing). The device may have a suitable operating system (e.g. Android or iOS). Preferably, the display may be a touch screen and further wherein the interaction between the device and the user is a touch event on the touch screen.

Optionally, the device may further comprise:

a camera (or other imaging device), wherein the interaction between the device and the user is an image captured by the camera.

Preferably, the device may be a mobile device (e.g. smart phone or tablet computer).

Advantageously, the device may further comprise an interface (e.g. radio, telecommunications, WiFi (RTM), wireless, etc.) configured to remotely retrieve the digital content.

According to a third aspect there is provided one or more non-transitory computer- readable media storing computer readable instructions which, when executed, configure a computing device to perform a method for detecting the presence or other interaction with a user by:

presenting or playing digital content to a user on a device;

determining if an interaction between the device and the user occurs while the digital content is being presented or played on the device;

if it is determined that an interaction between the device and the user has not occurred then taking an alternative action (e.g. stopping digital content from playing or recording that the content did not complete or allowing the content to continue). This may the form of a mobile application, for example.

Optionally, the digital content may be played on a touch screen of the device and the interaction further comprises receiving an indication that a touch event occurred on the touch screen.

Optionally, the interaction between the device and the user may be an image captured by the device of the user.

According to a further aspect there may be provided a mobile application operating according to any of the described methods.

In accordance with a further aspect there is provided a method for detecting the presence of a user or for detecting an interaction with the user, the method comprising the steps of:

presenting or playing digital content to a user on a device; determining if an interaction between the device and the user occurs while the digital content is being presented or played on the device;

if it is determined that an interaction between the device and the user has not occurred then ceasing presentation, pausing or stopping the digital content.

In accordance with a further aspect, there is provided a system for implementing the method.

In accordance with a further aspect, there is provided a one or more non-transitory computer-readable media storing computer readable

instructions which, when executed, configure a computing device to perform the method.

The methods described above may be implemented as a computer program comprising program instructions to operate a computer. The computer program may be stored on a computer-readable medium.

The computer system may include a processor such as a central processing unit (CPU). The processor may execute logic in the form of a software program. The computer system may include a memory including volatile and non-volatile storage medium. A computer-readable medium may be included to store the logic or program instructions. The different parts of the system may be connected using a network (e.g. wireless networks and wired networks). The computer system may include one or more interfaces. The computer system may contain a suitable operating system such as UNIX, Windows (RTM) or Linux, for example.

It should be noted that any feature described above may be used with any particular aspect or embodiment of the invention. Brief description of the Figures

The present invention may be put into practice in a number of ways and

embodiments will now be described by way of example only and with reference to the accompanying drawings, in which:

Figure 1 shows a flow chart of a method for detecting the presence of a user;

Figures 2A to 2D show a series of screenshots from a device operating according to the method of Figure 1 ;

Figure 3 shows a screenshot from a device operating according to an alternative method; Figure 4 shows a schematic diagram of a system for detecting the presence of a user, given by way of example only; and

Figures 5 to 26 show a series of screenshots from a device operating according to alternative methods, given by way of examples only.

It should be noted that the figures are illustrated for simplicity and are not necessarily drawn to scale. Like features are provided with the same reference numerals.

Detailed description of the preferred embodiments

The following describes various embodiments of a system and method for detecting the presence of a user of a particular device. The examples are shown include a mobile device such as a smart phone or tablet computer but the procedure may be extended to other devices such as computers, desktop devices, notebook computers, kiosks, terminals, display screens, either fixed or moving (e.g. within a vehicle such as a bus, automobile or aircraft, for example) or other apparatus that may interact with human users.

Figure 1 shows a flow chart of a method 10 for detecting the presence of a user or determining that a user is interacting with the device that is playing or presenting digital content to a user. At step 20, the device receives an instruction to play the content. This instruction may originate with the user or may come from another source either internal or external to the device. Following the instruction to play the content, the device commences playback of the content at step 30. The digital content may take different forms including video, audio, combinations of video and audio, still images, a series of still images, music, advertising material, instructional content such as safety information, or any other content that may be presented to a person.

At step 40 the device makes a determination whether an interaction has or is occurring between the user and the device. The particular way that the device can make this determination is described in more detail later on but may include the use of sensors or user interface hardware within or connected to the device. This hardware may include but is not limited to cameras, touch screen, gyroscope, switches and buttons, finger print sensors, light sensors, one or more microphones, and/or capacitive or resistive sensors (or any combination of these). The interaction is either detected or not detected. When an interaction is detected then step 60 results in an action being taken. This action may be to ensure that playback continues and the method returns to determining whether or not the interaction between the user and device occurs (step 40). This may be carried out continuously or at intervals which may be irregular or regular intervals.

If no interaction is detected at step 50 then then an alternative action may be taken (step 70). Either action may be passive (i.e. allow operation to continue in its current form) or dynamic (i.e. a change occurs or data is recorded). The alternative action may be for the playback to be paused or to stop entirely, for example. Various actions or functions may take place following the pausing or stopping of the playback (or other alternative action) at step 70. For example, the method may return to the beginning and look for or receive a further instruction to play the content (step 20). This may start the content at the beginning or may restart the content where it last paused.

In some example implementations, only the action will ever take place (the dotted line around box 70 in figure 1 indicates that this step is not always present in the method).

Figures 2A to 2E show screenshots from a device (in this case a smart phone), which operates the method 10 described with reference to figure 1 . In this particular example implementation the interaction between the user and the device takes the form of a fingerprint icon 120 appearing on the display 100 of the device at a time during the playback of a video (e.g. an advertisement). As shown in figure 2A, surrounding the icon 120 is a graphical timing indicator 1 10 that takes the form of a circle that gradually loses its circumference to indicate a time to complete an action expiring. The timing indication 1 10 may also be numerical. In this example, the user must touch the display 100 of the device over or at the icon 120 when it appears. The screen 100 of the device is a touch screen and so can register this interaction.

Figure 2B shows the view of the screen 100 still showing the icon 120 before it has been pressed or selected by the user towards the end of the time limit. Therefore this figure shows only a very small part of the circle 1 10 remaining and the circumference line will disappear at the end of the time limit (e.g. 3 second, 5 seconds, 1 to 10 seconds, etc.). If the user fails to touch the icon 120 by the time that the timer expires then the alternative action may take place (e.g. the video displayed on the screen 1 10 may pause or stop).

Figure 2C shows a screenshot during the time that the user is touching the icon 120. The icon 120 has changed appearance in certain ways to indicate that it has registered the interaction (or at least started to register an extended interaction). For example, the fingerprint icon 120 has changed (in this example from white to grey). While in some implementations, a momentary touch of the icon 120 will be sufficient to register the interaction, in this particular example a prolonged interaction is required. In other words, the user must keep their finger pressed on the display 120 for a minimum amount of time, which may be predetermined. For example, the time may be 1 to 10 seconds or 3, 4 or 5 seconds, for example. As well as the change of the colour of the icon 120 or another indication that the interaction or touch has been recorded or is being recorded, a circle (or another circle 1 10') starts to grow in circumference (or it may also diminish as with the initial timer 1 10). The timer indicator circle 1 10' may also have a different colour to the countdown timer circle 1 10. Following this interaction, the action (step 60) may take place (e.g. the video continues). Figure 2C shows the countdown timer 1 10' and icon 120 partway through an extended press by the user. The circle 1 10' has grown to more than half its circumference in this figure. The size of the circle or graphical indicator may be proportional to the time left or the time passed for any of these events.

Figure 2D shows the display 100 after the icon 120 has been pressed by the user and reached the end of the countdown; therefore, the timer circle 1 10' has completed. This completes the interaction being detected allowing the video content also to complete (or at least continue to be shown or another action). For short videos or advertisements (e.g. 15, seconds, 30 seconds or up to 1 minute) only one interaction or icon 120 may appear during playback. However, for extended duration content, such as instructional videos, then the icon 120 may appear several times requiring the user to press the icon 120 for the required duration each time that it appears. More than one icon may appear at any one time.

The icon 120 (or icons) may appear in the same portion of the screen 100 each time or its location may change, either randomly or at a predetermined series of locations. This means that the user has to look out for the icon 120 and so they pay more attention to the content on screen. This may also prevent automated key presses from being instigated by malicious software simulating views of users that are not actually viewing the content.

Whilst this example shows an icon 120 in the form of a fingerprint, other icons may be used. Furthermore, the icon functionality may be replaced by the user having to press on features that are actually part of the content itself. In some examples this may be a particular character that appears or perhaps a brand logo. Again, the user may need to press on the particular object on the display 100 for a predetermined period of time that may be visually or audibly indicated. The icon 120 or other object that needs to be pressed may appear within the video itself on the screen 100 or as shown in figures 2A to 2D, in a portion of the display 100 that does not show the video.

Figure 4 shows a high level schematic diagram of a system 200 used to implement the method 10 described with reference to figure 1 . The system 200 shows a smartphone 210 as the device in this particular example. A server 230 having a connected database 240 supplies the device 210 with content through a network 250. The database may also store interactions with the user or store information recording how many playbacks of content fully and/or partially completed and any other viewing metrics. This network 250 may be the internet, a telecommunications network or a combination of these. The system 200 may be used with the example implementation involved using touch interactions described with a reference to figures 2A to 2D.

In a further example implementation, instead of (or as well as) the touch detection scheme, the interaction between the user and the device may be passive. Figure 4 shows schematically a user's face 260 viewing the display 100 of the device 210. The device 210 has an imaging sensor or camera 210 that has a view of the user's face 260 when viewing the display 100 (e.g. a front facing camera 210 is used).

The user can request content to be downloaded from the server 230 and be displayed on the screen 100 of the device 210. During playback of content, the camera 210 may capture the user's face 260 continuously or at intervals. Face detection software operating within the device 210 (or remotely) may determine whether or not the camera 210 is viewing a face 260 and if so then record the interaction with the user. When the user is not viewing the display 100 of the device 210 then the camera 210 does not view the user's face 260 and the face detection software or algorithm operating within the device 210 indicates this event as a lack of interaction between the user and the device resulting in the playback of the content pausing or stopping.

Figure 3 shows a (different) video on the display 100 after the system determines that the user's face 260 is not being captures by the camera 210. In this case, the video pauses or stops.

As well as improving the efficiency and performance of devices, this system and method may avoid data from being downloaded and displayed without being viewed by a user. The system may also be used to ensure that for video advertisements (in particular but not limited to this content):

(i) the viewability metrics more accurately reflect the number of interactions (e.g. touch based or face detection) and are intrinsically linked to users actually viewing the advertisement by eliciting and prompting an interaction;

(ii) the industry wide ad fraud and/or bot-threat may be reduced by ensuring a real human interacts with the device when the content is being displayed. This can be improved further by making the icon or item appear at a random time and/or random position on the device screen; and/or

(iii) the user/customer may be focused on the device screen as attention is required to ensure that the sensor prompt is touched and the circular interaction completes one circumnavigation (i.e. the visual timer indicator completes) or knowing that the content will pause if a face is not present in front of the screen or camera (face detection

implementation). This requirement of attention on the screen produces a significantly higher likelihood that the user not only views the advertisement or other material but more fully engages with it.

Whilst this particular example discusses mobile video advertisements within mobile web and apps that are played on a smartphone or tablet, this technology may be extended to other content and interaction types. For example, the system 200 and method 10 may be used to pose questions and answers during content play, for example during a video advertisement. Prompts may appear on screen with the user asked to touch and hold down on a particular favourite character, for example.

Further examples may include using the system 200 within any form of video content, not necessarily limited to video advertisements, to ensure validation that a human was present and engaged with the content. Examples may include online video tutorials that fail to complete when the user interaction is not detected. This may require that the video stops and moves back to its start if they did not interact during one or more key times or events during the tutorial.

The system 200 may also be extended to online consent mechanisms. For example the system may make it easier to ensure that users actually read and accept terms and conditions or privacy notices or to determine that a particular user is returning or is new to the system or to provide affirmative consent to the collection and/or use of personal data or for the purposes of withdrawing consent for these purposes. This may be especially useful where more stringent opt in processes are required.

In one example implementation, in the instance of video playback commencing (after optional buffering), a timer may be set for a random instant no less than five or ten seconds, and no more than 50% of the full duration of the video advertisement, to commence an interactive user verification check according to any interaction determination. When the timer elapses, a visual interaction prompt may be displayed on screen. The visual indicator appears in the form of a completed circle surrounding an image. The visual indicator may take different forms, including a fingerprint icon or logo, for example.

The completed circle within the visual indicator may be set as a semi-translucent white colour and immediately begins to countdown by reducing the circle in an

anticlockwise direction. In this example, there is a current three second countdown setting before the circle disappears but this can be set at any length of time. Preferably, this would not be longer than five to ten seconds when played within an advertisement context but could be set for a longer period if featured outside of an advertisement context. The semi- translucent colour of the countdown can also be changed to any colour or format. Once the visual indicator circle has fully counted down, the video content will terminate and the user will exit or return to a different screen.

If the user interacts with the visual interaction prompt before the indicator circle has fully counted down, by touching the surface of the smartphone or tablet on the circle or within any area inside the circle, interaction will be detected and determined. During the display of the visual interaction prompt and visual indicator circle, the main video content displayed on screen 100 may continue to play. Preferably, any touch interaction with the visual indicator circle will not result in the content being paused at any time. In one example implementation, the video advertisement may also be paused by the user at any time by pressing any part of the screen. Preferably, the user cannot pause the video when the visual interaction prompt commences and has fully completed.

Upon the touch interaction being detected, the indicator circle 1 10 may change to the colour red and reverse in a clockwise direction until it returns back to its initial starting point and the circle 1 10' is fully completed. Once the indicator circle 1 10' is complete the prompt will disappear from the screen 100 and the content will continue to play until completion. More than one visual interaction prompt may be deployed during any length of content, e.g. a three second visual indicator circle 1 10 may be deployed or presented after fifteen seconds of a sixty second advertisement being played and then again after forty five seconds.

The following describes in more detail an example implementation of the detection of a user using the camera 220 of the device 210. Apple iOS has a Core Image library including Face Recognition. The output may be a binary indication (i.e. face detected or face not detected).

The app may carry out the following procedure. There may be two views in a Main.storyboard. These are positioned at the top and bottom of the screen 100. The top view may contain the video (or other content being played) and the bottom view may show a video output of the front facing camera 220 of the device 210. The video content should only play while the user's face 260 is present and detected in the camera stream. In other examples, the captured view of the user's face 260 is not displayed on the screen 100.

When the app runs it sets up the video player in the viewDidl_oad() but does not start the video content from playing. It also starts a process for detecting the face of the user using setupAvCapture(). This then displays the camera footage on the view, setting up a callback captureOutput(). This captureOutput() function may be called every time a new video frame is shown. Inside this function is where the actual face detection process occurs. If the image is present the video will start to play (if not already playing) or will stop if the face is not present.

The captureOutput() may be called every time a new frame of video is present.

Whilst this provides a fast and accurate indication that the user is present, this may require a lot of processing power leading to reduced battery life. Therefore, throttling of this process may be achieved by reducing the number of times that the user is detected. In one example this may be reduced to four times a second rather than every frame of the content (e.g. 30 times a second). However, the Face Detection function may be unreliable at times. Therefore, a "sticky state" may be added to avoid flip/flops between starting and stopping the video.

Global variables may be used to tune the process and achieve a compromise between accuracy and performance. The time that the captureOutput() was last allowed to run may include a check at the top of this function to see if enough time had elapsed e.g. 250 milliseconds, if not then the function may immediately return. Alternatively, an update frequency may be scheduled on the video stream.

The next part is to handle occasions (frames) where the face is missed. To do this a buffer (e.g. two seconds) may only stop playing the video if the user's face had not been seen for a particular time (e.g. two or three seconds or more). The process for tracking the face is CPU intensive and uses a lot of power but advertising videos may be relatively short Therefore, longer videos may have more time between face detections, i.e. this time between detections (any type including touch, face detections and/or other detections) may be based on the length of the content.

In the App the FaceDetector functionality may use different accuracy levels. This depends on resolution settings, e.g. High Accuracy and Low Accuracy may correspond with camera resolutions of 1280x720 and 640x480, respectively. The accuracy setting may also be reduced to improve battery life and reduce processing requirements.

The app may continuously poll the camera to get the image and then run an algorithm to detect a face in the image. In effect there may be two loops of logic. A fast loop may continuously run a process to look for a face in the image. A second slower loop may check if it has seen at least one face in the last time frame (e.g. two seconds). The second loop is needed because occasionally the face detection algorithm may fail despite a face being present. In some situations (i.e. some hardware), the video may pause erroneously. This may be due to the device doing too much processing and it is unable to play the video and run the face detection code at the same time. To address this, the fast loop logic may be adjusted for particular hardware configurations. The number of times the second loop runs may be adjusted to optimise performance.

Other optimisations may be used. Reducing the resolution used to find the face may reduce the computational requirements. This may reduce latency issues. To illustrate with an example we may assume that it is only necessary to detect a face twice a second and every fourth attempt fails. This may detect that a face disappears/appears at a slower rate than is desirable. By reducing the resolution the face detection process may be run more often, e.g. four times a second with every fourth attempt failing. This may also improve latency.

Figures 5 to 23 show screen shots of further example interaction detections. These examples use the touch interaction detection method described above. However, one or more actions may take place depending on whether or not an icon is touched (and/or for the required length of time). These actions may include stopping the digital content from being played (e.g. video pausing) or may allow the video to continue whether or not the icon is touched.

Figures 5 and 9 shows the action as providing the user with a reward (e.g. some type of credit or a specific number of "coins" or other tokens). Figure 6 shows the icon of figure 5 during a touch event.

Figures 7 and 8 show screen shots of different icons, where the action is to change the digital content (e.g. to a new advertisement or to outtakes of the video).

Figure 10 shows a screen shot with an icon, where the action (after touching it) is to enter a competition. This may take the user to a web page, for example.

In figure 1 1 , the action is to donate to charity. Figure 12 shows an icon where a discount for a product or service is provided as the action. This screen shot also illustrates an alternative countdown timer (that may be used with any example).

Figures 13 to 19 show screen shots in which there are two displayed items or icons. A different action or data may be generated depending on which icon is selected. In the example of figure 13, the "dislike" icon is selected and part way through the touch timer. In the example of figure 14 the use indicates their particular preference regarding the video. Other data are captured from the user in figures 15 to 19.

Figures 20 to 22 show screen shots in which there are three displayed items or icons. A different action or data may be generated depending on which icon is selected. Figure 21 shows an example in which confirmation that the user is actually watching the content is provided by asking the user a question about the particular digital content. In this example, there may be a correct answer. If an incorrect answer is selected then data may be stored and/or transmitted indicating that the content was not watched or was not watched sufficiently carefully.

Figure 23 illustrates an example implementation enabling a user to skip videos or other content. In other words, digital content, for example, a video (e.g. an advertisement) may play for a period of time (e.g. 5, 10, 15 or up to 60 seconds) before the user can stop or exit this video (e.g. before it finishes). In some examples, the user may be presented with a further video (or other digital content) or a different outcome may occur.

In existing examples, a user may be presented with an "X" button at the top right of a video (or other location) that enables the user to exit the video or alternatively. As with YouTube (RTM), the user may be presented with a countdown notifying them of when they are able to skip the advertisement.

The touch screen interactive function, described above, may be used to implement similar skip options. For example, instead of an "X" button located at the top right hand side of the screen, there may be an icon or sign, (e.g. a finger print icon, a countdown circle or other timer indicator, either numerical or non-numerical) optionally adjacent or near words or other indicators such as "Skip Ad" or "Hold to Skip". When the user touches the area, item or icon and holds down to complete the circle (or otherwise provide an interaction with the device) a circle (which may be red in colour) may commence from a 12 o'clock position and begin to complete if the user maintains holding down the area. If the user removes their finger off the skip area during the completion of the circle around the fingerprint icon, the circle will then snap back or otherwise revert to an original position or indication and they will be required to hold down again to completion in order to exit the content.

If the circle is completed then an "X" or other indicator may appear enabling the user to exit the video. Alternatively, the skip procedure may commence when the user has held down the icon to the completion of the hold timer. If there is no interaction with the device, either via the displayed item or circle or otherwise before the circle or timer completes its countdown, then the circle, item or icon may disappear and there will be no X button presented to the user (or the option to skip is deactivated). In other words, the user may then be locked into the video advertisement until completion.

Whilst figure 23 shows the skipping icon or item associated with this functionality in the middle of the screen, it may be located at the top right hand side of the video (or anywhere else over or around the video) and may also be placed in a random position on screen.

According to a further example implementation (that may include any of the previously described features), the method may be implemented as a separate application or code running alongside or on top of an application or website that displays digital content. The following provides an example for use in advertising but may be extended for any display or rendering system of digital content.

A brand may produce an advertisement (i.e. digital content) that they wish to be displayed alongside or with content that a user has selected (e.g. a news or media website). An agency may place bookings across various media sites or media apps (e.g. mobile applications). A decision will be made to run the advertisement within a particular media app through an ad network (who may have an existing relationship with the media provider) to buy advertisement slots within the media sites and apps. Other intermediaries may provide the content at various stages. The brand would already have the video creative (digital content) developed in-house and this would be passed through to the ad network to run on the media website (or app).

Usually, the ad network will include certain tags and codes within the video creative that runs on the media site in order to measure whether the video played, the duration of the play and various other data points that they will use to measure the success or response of the advertisement campaign. This may result in payments based on these measurements. The present system may operate within or separately from the ad network or other intermediaries to offer an interactive measurement solution for in-app (or website) videos (digital content). Therefore, the new system and method does not require any form of SDK integration with the media app or website and may come in the form of a simple JavaScript ad tag that the ad network can overlay on top of the existing creative (digital content).

When the video content or advertisement appears on the media app or website, the interactive countdown function will appear within the ad creative (digital content). This can initiate a user interaction (that may also be recorded) and enable actions to occur. This may involve actions other than allowing the digital content to continue playing. In other words, the digital content may not necessarily pause or stop in the absence of a user interaction, as described previously (but this may still be the case for certain example implementations).

The system may further act as a third party measurement and verification service for in-app content. For example, this implementation may feature full screen video or rich media full screen interstitials (i.e. full-screen ads or content that cover the interface of a host app. They are typically displayed at natural transition points in the flow of an app, such as between activities or during the pause between levels in a game.) In this

implementation, the interactive countdown engagement circle may be within an

advertisement or digital content, in addition to the interactive skip feature. This example implementation may also measure whether the user interacted or engaged with certain parts of the digital content (e.g. video), ensuring that a real human is present and requiring a user to be looking at the screen when they exit the content.

A back end dashboard may also be provided to enable greater insights for agencies and advertisers about the effectiveness of an in-app video campaign. As an example, an agency could currently confirm that out of 1 million renders of the video across 1 million UK devices, 80% of users skipped and 20% had a completed view (i.e. the entire video played to completion). This may be implemented as a remote server, for example.

With the present system, for the same 1 million devices may be able to clarify (i) how many skipped, (ii) how many went to skip, stop skipping, then skipped later (iii) how many went to skip but then watched the entire content, (iv) how many touched the interactive circles for limited periods, (v) how many completed the interactive circles; and (vi) how many did nothing and just let the video play to completion, for example.

Figures 24 to 26 illustrate a further interactive skip feature. Such a feature may be incorporated into an in-app video, for example including the wording 'skip ad' may be present (or some other wording or icon). In this example, the fingerprint icon is also present but there is now no countdown circle unravelling from the outset (it may always be present or present at particular times).

As shown in figures 24 to 26, the icon stands alone and upon the user touching the icon, a red circle appears at the 12 o'clock position and starts to complete in a clockwise direction. Typically, a three second default complete time may be set for the circle to complete, which then triggers the skip (of the digital content) and enables the user to exit the content. The three second complete time is only an example can be altered to any time, such as 1 second, 5 seconds or 10 seconds. Other formats of the icon may be used.

Moreover, if the user takes their finger off the skip area whilst the circle is completing but before it is complete, it will immediately change direction in an anticlockwise direction and completely unravel back to the 12 o'clock position, leaving just the fingerprint icon as before. Other visual indications of this may be provided (e.g. numerical timers).

In this example implementation, the content on screen continues and only exits once the circle is fully complete. An example unravel time (when the user stops touching the fingerprint) may be quicker than the interaction confirmation time (e.g. one second to unravel). Therefore it may eventually take 5-6 seconds to skip the content, rather than three seconds, if the user does not continuously press down on the fingerprint icon.

In a further example implementation the skip function may be implemented in a slightly different manner. This may involve an On Demand service. For example, a content provider app on mobile device may enable the viewing of catch up non-live TV. When a 20 minute programme first starts the viewer may be required to watch three back to back ads before the content plays and this three (or other number) ad format appears once or twice again before the end of the programme.

The skip functionality may enable a user to hold down a fingerprint/circle for the entire duration of the first ad of the three (or a different number), in order to skip the next two ads and go straight to the programme (or other requested service). This could potentially require the user to hold down for an entire 15 seconds of a 15 second ad but the brand would be confident that the user had to look at the screen and therefore the advertisement and the content provider could potentially charge a higher premium for this guarantee or simply be able to record a full viewing of the ad. The user also gets to skip the next one, two (or more) ads by engaging with the first ad in this way.

Another similar scenario is rewarded in-app video ads. This is currently a huge developing market that requires users to watch unskippable video ads in order to get to the next level or acquire free coins or levels without having to make in-app purchases.

Advertisers will have the same issue that a content provider has in that they have no idea that the user watches the unskippable ad. Having a requirement to hold down on the circle would provide that confirmation of engagement that the user watched the reward ad before receiving the free tokens or whatever is on offer.

Figure 1 shows a flowchart of an implementation where the functionality is provided within a single app (or website code). However, with the scripted (JavaScript)

implementation the functionality may be split. For example:

1 . When the JavaScript ad tag is added to the creative (digital content) then the content (including either an interactive circle, interactive skip function or both) will be played when the content starts to run within an app or website.

2. Following the instruction to run an ad, the script functionality commences on the creative (within the digital content).

3. The script functionality then still determines interaction between a user and a device if the user touches the circle or the skip icon. 4. An action is still taken if the user completes the circle or skip, e.g. launching a website, app store or exiting the video.

Advantageously, the functionality may operate without requiring the owner or host of the app that the content is running on to provide functionality with the app. The interaction measurement(s) may be transmitted back to external servers from within the advertisement. This enables the creation of a dashboard (i.e. recording metrics) for a campaign to measure averages across all users that viewed the ad but also on a specific device level. For example the system will be able to determine if particular users skipped the ad, touched the interactive circle, completed the circle, etc.. In this example, the script does not pause the playing of the ad in the absence of a user interaction.

These examples are described with reference to advertisements but other digital content may be used. These may include instructional or safety videos or content, or teaching materials, for example.

As will be appreciated by the skilled person, details of the above embodiment may be varied without departing from the scope of the present invention, as defined by the appended claims.

For example, other interactions may be detected that may be passive or active. These may include audible interactions (e.g. listening for the voice of the user). The user may be rewarded for completely watching the digital content by receiving a service, unlocking a function, receiving a credit or receiving more digital content, for example.

Whilst a particular iOS face detection library has been described, other software

components using different operating systems or software may be used with similar functionality. Any of the features or combinations of features described with reference to the touch detection examples may be used with any one or more features described with reference to the image capture examples. In the face detection example, the same types of actions and alternative actions may be take place as described with reference to the touch interaction examples (e.g. the content may continue even if a face is not detected but the occurrence of this event may be recorded).

Many combinations, modifications, or alterations to the features of the above embodiments will be readily apparent to the skilled person and are intended to form part of the invention. Any of the features described specifically relating to one embodiment or example may be used in any other embodiment by making the appropriate changes.

Claims

CLAIMS:

1 . A method for detecting an interaction with a user comprising the steps of:

presenting digital content to a user on a device;

determining if an interaction between the device and the user occurs while the digital content is being presented on the device; and

if it is determined that an interaction between the device and the user has occurred then taking an action.

2. The method of claim 1 , wherein the action is continuing the presentation of the digital content.

3. The method of claim 1 or claim 2, wherein if it is determined that an interaction between the device and the user has not occurred then taking an alternative action.

4. The method of claim 3, wherein the alternative action is stopping the digital content from being presented, or presenting one or more additional items of digital content.

5. The method of claim 4, wherein stopping the digital content is pausing the digital content or pausing the presentation of the digital content.

6. The method of claim 4 or claim 5 further comprising the steps of:

after stopping the digital content from being presented then determining that an interaction between the device and the user occurs and in response then starting the presentation of the digital content to the user.

7. The method of claim 6 further comprising the step of determining a time from the stopping of the presentation of the digital playing and the determining that the interaction between the device and the user has occurred; and

if the determined time is greater than or equal to a predetermined time then waiting for a command to replay or present again the digital content or a different digital content; and

if the determined time is less than the predetermined time then continuing the presentation of the digital content from where it was stopped.

8. The method of claim 1 , wherein the action is ending the presentation of the digital content.

9. The method of claim 8, wherein the action further comprises providing a service to the user.

10. The method of claim 8 or claim 9, wherein the step of determining if an interaction between the device and the user occurs further comprises: measuring a duration of the interaction and if the duration is greater or equal to a predetermined time then determining that the interaction occurred and if the duration is less than the predetermined time then determining that the interaction has not occurred.

1 1 . The method of claim 10 further comprising the step of displaying a countdown timer during the duration of the interaction.

12. The method of claim 1 1 , wherein the countdown timer is non-numerical.

13. The method of claim 1 1 or claim 12, wherein the countdown timer is configured to indicate when the predetermined time has expired.

14. The method according to any previous claim, wherein the digital content is presented on a touch screen of the device and the interaction further comprises receiving an indication that a touch event occurred on the touch screen.

15. The method of claim 14 further comprising the step of displaying at least one item on the touch screen, wherein the touch event corresponds with a user touching the touch screen over the displayed at least one item.

16. The method of claim 15, wherein determining if the interaction between the device and the user occurs further comprises determining if the user touched the screen over the displayed at least one item within a period of time starting when the item was displayed.

17. The method of claim 16 further comprising the step of displaying a countdown timer when the at least one item is displayed.

18. The method of claim 17, wherein the countdown timer is non-numerical.

19. The method according to any of claims 15 to 18, wherein the displayed at least one item is part of the digital content.

20. The method according to any of claims 15 to 19, wherein the displayed item is displayed at intervals during the playing of the digital content.

21 . The method of claim 20, wherein the displayed at least one item is displayed at different locations on the touch screen between the intervals.

22. The method according to any of claims 15 to 21 wherein the step of determining if an interaction between the device and the user occurs further comprises determining that the user touched the screen for a time greater than or equal to a time threshold.

23. The method of claim 22, further comprising the step of providing an indication to the user that the time threshold had been reached.

24. The method according to any of claims 15 to 23, wherein the displayed at least one item corresponds with a response to a query provided by the user.

25. The method of claim 24 further comprising the step of transmitting data indicating the response over a network.

26. The method according to any previous claims, wherein the action is to launch a web page on a browser.

27. The method according to any of claims 1 to 25, wherein the action is to stop, pause or skip the digital content and provide further digital content.

28. The method according to any of claims 1 to 13, wherein the interaction between the device and the user is an image captured by the device of the user.

29. The method of claim 28, wherein the step of determining if the interaction between the device and the user occurs further comprises detecting at least a part of the face of the user within the captured image.

30. The method of claim 28 or claim 28, wherein the image captured by the device is a video frame capture by a camera of the device.

31 . The method of claim 28 or claim 30, wherein the at least part of the face of the user is detected continuously or at intervals.

32. The method according to any of claims 28 to 31 , wherein the at least part of the face of the user is detected during a continuous video capture or stream.

33. The method according to any previous claim, wherein the digital content is video content.

34. The method of claim 33, wherein presenting the video content is playing the video content.

35. The method according to any previous claim further comprising the step of initiating the interaction between the device and the user.

36. The method of claim 35, wherein the interaction is initiated at a random time, at a predetermined time, and/or at intervals.

37. The method according to any previous claim, wherein the digital content is any one or more of: a still image, a static interstitial advertisement unit, an advertisement, an instructional video, a slideshow, and/or audio.

38. The method according to any previous claim further comprising the step of recording the digital content completing.

39. The method according to any previous claim further comprising the step of recording measurements of one or more user interactions.

40. The method of claim 39, wherein the measurements include any one or more of: a time after the digital content commenced presentation when the user interaction occurred or completed;

the number and/or duration of user interactions that started but did not complete during the presentation of the digital content; and

a type or identifier of digital content played to a user when the user interaction occurred.

41 . A device for detecting the presence of a user comprising:

a display;

a processor; and

present digital content to a user on the display;

determine if an interaction between the device and the user occurs while the digital content is being presented on the device; and

42. The device of claim 41 , wherein the display is a touch screen and further wherein the interaction between the device and the user is a touch event on the touch screen.

43. The device of claim 41 or claim 42 further comprising:

a camera, wherein the interaction between the device and the user is an image captured by the camera.

44. The device according to any of claims 41 to 43, wherein the device is a mobile device.

45. The device according to any of claims 41 to 44 further comprising an interface configured to remotely retrieve the digital content.

46. One or more non-transitory computer-readable media storing computer readable instructions which, when executed, configure a computing device to perform a method for detecting the presence of a user by: presenting digital content to a user on a device;

47. The one or more non-transitory computer-readable media of claim 46, wherein the digital content is played on a touch screen of the device and the interaction further comprises receiving an indication that a touch event occurred on the touch screen.

48. The one or more non-transitory computer-readable media of claim 47, wherein the interaction between the device and the user is an image captured by the device of the user.