WO2021257881A1 - Image frames with unregistered users obfuscated - Google Patents

Image frames with unregistered users obfuscated Download PDF

Info

Publication number
WO2021257881A1
WO2021257881A1 PCT/US2021/037904 US2021037904W WO2021257881A1 WO 2021257881 A1 WO2021257881 A1 WO 2021257881A1 US 2021037904 W US2021037904 W US 2021037904W WO 2021257881 A1 WO2021257881 A1 WO 2021257881A1
Authority
WO
WIPO (PCT)
Prior art keywords
image frame
computing device
face
registered
user
Prior art date
Application number
PCT/US2021/037904
Other languages
French (fr)
Inventor
Vishal Sharma
Rafael ZOTTO
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Publication of WO2021257881A1 publication Critical patent/WO2021257881A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31User authentication
    • G06F21/32User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245Protecting personal data, e.g. for financial or medical purposes

Definitions

  • Computing devices such as desktops, laptops, mobile phones, tablets, and handheld personal computers (PC) comprise cameras to capture images and videos.
  • a built-in camera of a computing device such as a webcam of a laptop, enables users to capture pictures or videos.
  • an application may use the images or videos captured through the camera.
  • an application that provides video conferencing functionality when initiated, may enable the user to interact with other users through video conferencing applications installed at the respective computing devices.
  • Fig. 1 illustrates a computing device to cause display of image frames with faces of unregistered users being obfuscated, according to an example implementation of the present subject matter.
  • FIG. 2 illustrates a computing device to provide image frames with unregistered users obfuscated to applications executing on the computing device, in accordance with an example implementation of the present subject matter.
  • Fig. 3 illustrates a method for providing image frames with obfuscated faces of users not registered with a computing device to an application executing on the computing device, in accordance with another example implementation of the present subject matter.
  • Fig. 4 illustrates a method of editing an image frame to obfuscate a user not registered with a computing device, according to an example of the present subject matter.
  • Fig. 5 illustrates a method of editing an image frame to obfuscate portions of the image frame other than a portion corresponding to a registered user, according to an example of the present subject matter.
  • FIG. 6 illustrates a computing environment for providing image frames with obfuscated unregistered users to applications to cause display of the image frames with the obfuscated unregistered users, according to an example implementation of the present subject matter.
  • Applications running on a computing device may be interfaced with a camera of the computing device or another camera coupled to the computing device to provide various functionalities.
  • a social media application may access the camera to enable a user to capture and upload his image
  • a video calling application may access the camera to enable the user to participate in a video call.
  • a video conferencing application may interface with the camera to receive a video captured by the camera and transmit the video stream so captured to other remote computing devices over a communication network.
  • a corresponding video conferencing application executing on such a remote computing device may receive the transmitted video stream and render the video on a display device coupled to the remote computing device.
  • a user of the computing device initiates an application to capture an image or video through the camera
  • individuals, other than the user, present in the vicinity of the user’s computing device may be included in the captured image or video.
  • the video stream captured and transmitted to other remote computing devices may contain images of such individuals.
  • the video stream is likely to include images of several other individuals.
  • an image or video captured by a user may include such other individuals.
  • images of such other individuals are often undesirable in a video conference session since they may cause distractions.
  • the present subject matter describes examples of techniques of providing image frames captured by a camera coupled to a computing device, to applications executing on the computing device with unregistered users obfuscated in the image frame, i.e., a user not registered with the computing device.
  • An image frame may be understood as a photo captured by a camera and a sequence of image frames may constitute a video.
  • image frame used herein may refer to an image or a frame of a video, as the case may be.
  • a face present in an image frame captured by the camera is identified. If the face in the image frame is recognized to be of an unregistered user, an application executing on the computing device is caused to display the image frame with the unregistered user obfuscated. Since the application displays unregistered users such that such unregistered users are unidentifiable, instances of violation of the privacy of individuals who are captured in the image frame may be avoided.
  • a face of a user registered with the computing device is transmitted by a video conferencing application.
  • the present subject matter also enables video conference sessions to be conducted without potential distraction from individuals who are not intended to be participants in the sessions.
  • Fig. 1 shows a computing device 100 to cause display of image frames with unregistered users obfuscated, according to an example implementation of the present subject matter.
  • Examples of the computing device 100 may include but are not limited to, an electronic device, such as a desktop computer, a personal computer, a laptop, a smartphone, a personal digital assistant (PDAs), and a tablet.
  • PDAs personal digital assistant
  • a face recognition engine 102 of the computing device 100 may control the computing device 100, such that image frames are caused to be displayed without unregistered users being identifiable.
  • the face recognition engine 102 may be implemented as a combination of hardware and programming (for example, programmable instructions) to implement certain functionalities in the computing device 100.
  • such combinations of hardware and programming may be implemented in several different ways.
  • the face recognition engine 102 is configured to fetch and execute computer-readable instructions stored in a memory (not shown in Fig. 1) of the computing device 100.
  • the face recognition engine 102 may be interfaced with a camera (not shown in Fig. 1) that is operable in conjunction with the computing device 100 to capture image frames.
  • the face recognition engine 102 is operable to identify a face present in an image frame captured by the camera. If the face in the image frame is recognized to be of an unregistered user, the face recognition engine 102 causes the image frame to be displayed with the unregistered user obfuscated.
  • the image frame without the unregistered user being identifiable may be displayed on a display device (not shown in Fig. 1) coupled to the computing device 100.
  • the image frame without the unregistered user may be provided to applications executing on the computing device 100.
  • the image frame without the unregistered user may be provided to a video conferencing application executing on the computing device 100.
  • the video conferencing application may thus enable registered users of the computing device 100 to engage in video conferencing sessions with other remote users without video streams displayed in the sessions including the unregistered user.
  • the techniques to identify unregistered users and edit image frames to obfuscate the unregistered users from the image frames are discussed in detail subsequently.
  • the present subject matter provides for privacy protection since the image frames that are displayed contain registered users alone.
  • a registered user using the computing device 100 for example, to engage in a voice over internet protocol (VOIP)-based video call with a remote user is assured that the image frames transmitted to the remote user are devoid of any unregistered users that may be present in the vicinity.
  • VOIP voice over internet protocol
  • FIG. 2 illustrates a computing device 200 to provide image frames devoid of unregistered users to applications executing on the computing device 200, in accordance with an example implementation of the present subject matter.
  • Examples of the computing device 200 may include a variety of electronic devices that may incorporate or be interfaced with a camera 202.
  • the camera 202 may be inbuilt or integrated into the computing device 200, such as a webcam.
  • the webcam may be a complementary metal-oxide-semiconductor (CMOS) camera in an example.
  • CMOS complementary metal-oxide-semiconductor
  • the camera 202 may be an external camera, such as an external webcam, coupled to the computing device 200 through a universal serial bus (USB) interface.
  • USB universal serial bus
  • the computing device 200 also comprises a display device 204.
  • the display device 204 may be a visual display unit (VDU), such as a screen or a monitor that may be integral to the computing device 200 or may be coupled to the computing device 200.
  • VDU visual display unit
  • an interface(s) 206 may serve to communicatively couple the external camera or display device to the computing device 200.
  • the interface(s) 206 may include a variety of software and hardware interfaces that allow interaction of the computing device 200 with other communication and computing devices, such as network entities, web servers, and external repositories, and peripheral devices.
  • the interface(s) 206 may also enable the coupling of internal components of the computing device 200 with each other.
  • the interface(s) 206 may couple the display device 204 and the camera 202 to a processor 208 of the computing device 200 that may control the operation of the various components of the computing device 200.
  • the processor(s) 208 may be implemented as microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) 208 is configured to fetch and execute computer-readable instructions stored in memory 210 of the computing device 200.
  • the memory 210 may include any computer-readable medium including, for example, volatile memory (e.g., RAM), and/or non-volatile memory (e.g., EPROM, flash memory, etc.).
  • the processor 208 executes an operating system (OS) 212 of the computing device 200 residing in the memory 210.
  • OS operating system
  • the OS 212 is a set of instructions that manages the hardware and software of the computing device 200 to enable the computing device 200 to function.
  • the processor 208 may also execute firmware 214 of the computing device 200 residing in the memory 210.
  • Firmware 214 may comprise sets of instructions, for example, proprietary firmware that may, among other things, enable the OS 212 to manage the operation of hardware devices, such as the display device 204 and the camera 202.
  • the OS 212 executes application(s) 216 to provide various services.
  • An application may be understood as a set of instructions to enable a functionality in the computing device 200.
  • the application(s) 216 may be either native to the OS 212 or maybe a third-party application installed on the OS 212. Examples of the application(s) 216 include but are not limited to, applications that may access the camera 202 to provide functionalities relating to capturing, storing, and/or sharing of images or videos herein referred to as image frames.
  • the OS 212 executes a video conferencing application 218 to provide functionalities, such as video calling and video conferencing.
  • the application(s) 216 may cause the camera 202 to capture image frames of individuals who may be present in the field of view of the camera 202.
  • a face recognition engine 220 of the computing device 200 may be implemented as an interface between the application(s) 216 and the camera 202 to provide the image frames captured by the camera 202 to the application(s) 216 without identifiable unregistered users who may have been present in the field of view of the camera 202 and captured in the image frames.
  • the face recognition engine 220 may recognize a user who may have previously registered himself with the computing device 200 as a registered user. Users other than the registered user may be considered as unregistered users.
  • the face recognition engine 220 may obtain an image frame captured by the camera 202 and modify the image frame to obfuscate an unregistered user from the image frame if such a user were present in the image frame.
  • the modified image frame is made available to the application(s) 216.
  • the application(s) 216 may in turn cause display of the image frame without the unregistered user being identifiable therein.
  • the face recognition engine 220 may be similar to the above-described face recognition engine 102 and may be implemented as a combination of hardware and programming to implement functionalities, such as identifying and removing unregistered users from image frames so as to cause display of the image frames without the unregistered users being identifiable.
  • engine(s) comprising combinations of hardware and programming may be implemented in several different ways.
  • the programming for the engine(s) may be processor-executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the engine(s) may include a processing resource (for example, implemented as either a single processor or a combination of multiple processors), to execute such instructions.
  • the machine-readable storage medium may store instructions that, when executed by the processing resource, implement engine(s).
  • the computing device 200 may include the machine-readable storage medium storing the instructions and the processing resource to execute the instructions or the machine-readable storage medium may be separate but accessible to the computing device 200 and the processing resource.
  • engine(s) may be implemented by electronic circuitry.
  • the engine(s) may also comprise other engine(s) (not shown) that supplement functions of the computing device 200.
  • the face recognition engine 220 may control access of the camera 202 by the application(s) 216.
  • the face recognition engine 220 may be implemented in the firmware 214 of the computing device 200.
  • the face recognition engine 220 may be coupled to the OS 212 or may reside in the OS 212 as a device driver for the camera 202.
  • the face recognition engine 220 implemented as the device driver may either be a default driver for the camera 202 or may be implemented in the computing device 200 in addition to other drivers of the camera 202.
  • the application(s) 216 may be interfaced with the camera 202 via the face recognition engine 220 acting as the driver for the camera 202 either by default or by way of user selection in cases where drivers other than the face recognition engine 220 may exist in the computing device 200.
  • the face recognition engine 220 provides for image frames devoid of the unregistered user to be made available to the applications 216 for their consumption, thereby avoiding modifications in the applications 216.
  • the face recognition engine 220 comprises a face registration module 222 to register users with the computing device 200.
  • the face registration module 222 may provide an interface for registering the users. Users, such as an owner of the computing device 200 or users otherwise authorized to use the computing device 200 may perform a registration process to register themselves with the computing device 200. The registration process may be performed once by each user to enable users to register for subsequent interactions with the applications that use the face recognition engine 220.
  • the face registration module 222 may capture information relating to the face of a user, such as face descriptors corresponding to the user during a registration process.
  • the face registration module 222 may cause a prompt to be displayed on the display device 204 prompting the user to submit images of his face.
  • the user may provide previously captured images of his face, for example, from his social media account in response to the prompt.
  • an authorized person such as an administrator of an organization may also carry out the registration process to register a user, such as an employee of the organization to the computing device 200.
  • the authorized person in one example, may use an image of the user, such as a profile picture of the user included in his profile registered with the organization to register the user.
  • the face registration module 222 may request a user performing a registration process for an identification code, such as an employee id or a preregistered password.
  • the user may opt for images of his face to be captured by the camera 202 for submitting the same to the face registration module 222. Accordingly, the face registration module 222 may further cause the camera 202 to capture images of the face of the user.
  • Each of the users registering with the computing device 200 may, at completion of the registration process, be allocated an account password. Thus, multiple users registered with the computing device 200 may each have a distinct account password that may be usable with their respective interactions with the face recognition engine 220 subsequently.
  • face descriptors corresponding to the face may be extracted.
  • the face descriptors may include, but are not limited to, geometrical features of the face of the user or features of a retina or iris of the user.
  • the face descriptors corresponding to each of the users that perform the registration process successfully, i.e., registered users, may be stored in a face descriptor database 224.
  • the face descriptor database 224 may be accessed for recognition of the registered users who may be present in the image frames captured by the camera 202 at future instances.
  • the face descriptor database 224 has been shown to reside internal to the computing device 200. However, implementations where the face descriptor database 224 may reside external to the computing device 200, for example, an external server implemented for registering the users with the computing device 200 are also possible.
  • the face registration module 222 may not perform the registration process for registration and recognition of users. Rather, the face recognition engine 220 may leverage registration processes that may have been performed by the OS 212 of the computing device 200 to register users to authorized them to use the computing device 200. Accordingly, in one example, the face registration module 222 may obtain information relating to faces of registered users of the computing device 200 from the OS 212 of the computing device 200. The face registration module 222 may identify the users registered with the computing device 200 based on the information obtained from the OS 212.
  • the face recognition engine 220 comprises an image frame modifier 226 to detect faces, if present in an image frame captured by the camera 202.
  • the image frame modifier 226 may implement a variety of face detection techniques to detect the presence of a face in the image frame.
  • the image frame modifier 226 may use OpenCV-based face detection techniques, such as Haar Cascades method to detect the presence of the face in an image frame.
  • the image frame modifier 226 detects a face to be present in the image frame, it initiates a process to recognize the face.
  • the image frame modifier 226 extracts face descriptors of the detected face and compares the same with face descriptors of users registered with the computing device 200.
  • the face descriptors may be stored in the face descriptor database 224 or in any other repository comprising information relating to faces of users registered with the computing device 200, may be retrieved for the purpose. If the face descriptors of the detected face match with the face descriptors of any of the users registered with the computing device 200, the detected face is identified to be of a registered user.
  • the match may be based on a predefined threshold of accuracy.
  • predefined settings such as user-configurable settings may define whether, say, a 70% match or a 100% match of the face descriptors of the detected face with the face descriptors of any of the registered users is to be considered by the image frame modifier 226 for identifying the registered users.
  • the image frame modifier 226 modifies the image frame to remove the unregistered user from the image frame.
  • the image frame modifier 226 may employ a variety of techniques to modify the image frame to obfuscate the unregistered user.
  • Obfuscating the unregistered user from the image frame may be understood as any modification of the image frame that may render the unregistered user in the image frame unidentifiable.
  • modification of the image frame may comprise blurring or pixelating a portion of the image frame corresponding to the unregistered user.
  • the image frame modifier 226 may modify the image frame to remove the unregistered user by substituting a portion of the image frame containing the unregistered user with pixels from other areas of the image frame.
  • each of the detected faces is identified to be of a registered user or an unregistered user and each of the unregistered users is removed from the image frame.
  • the unregistered face alone may be obfuscated from the image frame, for instance, by removing or blurring the unregistered face.
  • the modified image frame may contain, at least partially, parts of the body corresponding to the unregistered face, had the same been captured by the camera 202, along with the unregistered face that is obfuscated.
  • the image frame modifier 226 may modify the image frame such that a portion of the image frame containing a registered user is unaltered while other portions are obfuscated.
  • the image frame modifier 226 may modify the portions of the image frame not containing the registered user by pixelating such portions or by superimposing pixels of a predetermined color over such portions, in one example.
  • the image frame modifier 226 may also provide for a background of the registered user in the image frame to be modified.
  • the image frame modifier 226 may modify the image frame such that a portion of the image frame corresponding to such a registered user is also obfuscated.
  • the registered user may use his account password to interact with the face recognition engine 220. The user may define settings at the face recognition engine 220, such that upon his recognition, his image is caused to be displayed after a verification of the account password.
  • FIG. 3 illustrates a method for providing image frames with faces of users not registered with a computing device being obfuscated, to an application executing on the computing device, in accordance with another example implementation of the present subject matter.
  • the method 300 may be implemented in a variety of electronic devices, for the ease of explanation, the present description of the example method 300 to provide image frames without identifiable faces of unregistered users is provided in reference to the above-described computing device 100 and computing device 200.
  • the order in which the method 300 is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method 300, or an alternative method.
  • the method 300 may be implemented by a processor(s) or computing device(s) through any suitable hardware, non-transitory machine- readable instructions, or combination thereof.
  • blocks of the method 300 may be performed by programmed computing devices.
  • the blocks of the method 300 may be executed based on instructions stored in a non-transitory computer- readable medium, as will be readily understood.
  • the non-transitory computer- readable medium may include, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media.
  • an image frame captured by a camera such as a camera 202 of the computing device 200 is received.
  • a plurality of faces in the image frame is detected.
  • the face recognition engine 220 of the computing device 200 may implement face detection techniques to determine presence of the plurality of faces. For instance, the face recognition engine 220 may use Haar face detection method to detect the presence of the plurality of faces in the image frame. Further, upon detection of the presence of the plurality of faces in the image frame, each of the plurality of faces may be recognized as a face of a user either registered or not registered with the computing device 200.
  • face recognition may comprise identifying faces of users who may have previously registered with the computing device 200 as registered faces and identifying faces other than the registered faces amongst the plurality of faces in the image frame as unregistered faces.
  • the image frame is edited to obfuscate faces of users not registered with the computing device 200 from amongst the plurality of faces in the image frame.
  • the edited image frame is provided to an application executing on the computing device 200.
  • the application may be a video conferencing application, such as the video conferencing application 218.
  • the video conferencing application 218 is disabled from displaying, in video conferencing sessions conducted using the video conferencing application 218, the unregistered faces such that they are identifiable.
  • Fig. 4 illustrates a method 400 of editing an image frame to obfuscate a user not registered with a computing device, according to an example of the present subject matter
  • Fig. 5 illustrates a method 500 of editing the image frame to obfuscate portions of the image frame other than a portion corresponding to a registered user, according to an example of the present subject matter.
  • the methods 400 and 500 may also be implemented in a variety of electronic devices, however, for the ease of explanation, methods 400 and 500 are described in reference to the computing device 200.
  • the order in which the methods 400 and 500 are described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the methods 400 and 500, or an alternative method.
  • the methods 400 and 500 may be implemented by processor(s) or computing device(s) through any suitable hardware, non-transitory machine-readable instructions, or combination thereof.
  • blocks of the methods 400 and 500 may be performed by programmed computing devices.
  • the blocks of the methods 400 and 500 may be executed based on instructions stored in a non-transitory computer-readable medium, as will be readily understood.
  • the non-transitory computer-readable medium may include, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media.
  • an image frame captured by the camera 202 of the computing device 200 is received, and, at block 404, a plurality of faces in the image frame is detected.
  • face descriptors of each of the plurality of faces detected in the image frame are compared to face descriptors of users registered with the computing device 200. This enables a user, corresponding to each of the plurality of faces detected in the image frame, to be recognized as a registered user or an unregistered user.
  • at block 408 based on the comparison at least one face from amongst the plurality of faces detected in the image frame may be identified to be of a user not registered with the computing device 200.
  • a portion of the image frame corresponding to the user not registered with the computing device 200 is modified to obfuscate the user.
  • the modification may, in one example, comprise blurring or pixelating the portion of the image frame corresponding to the unregistered user, such that the user is not identifiable.
  • the modification may comprise other techniques, such as replacing pixels in the portion of the image frame depicting the unregistered user with pixels from other areas of the image frame.
  • the portion of the image frame corresponding to the unregistered user may be modified, while, in another example, the portion of the image frame corresponding to the face of the unregistered user alone may be modified.
  • the modified image frame is provided to the application(s) 216 executing on the computing device 200.
  • the modified image frame may be provided to a video recording application so as to cause the video recording application to display the image frame without the unregistered user.
  • the image frame is modified to obfuscate portions of the image frame other than a portion corresponding to the registered user.
  • the modified image frame may contain the face of the registered user with other portions of the image frame being modified, for example, by blurring or pixelating.
  • the modified image frame may contain, at least partially, other body parts along with the face of the registered user.
  • Fig. 6 illustrates an example computing environment 600 for providing image frames without identifiable unregistered faces to applications executing in the computing environment 600, according to an example implementation of the present subject matter.
  • the computing environment 600 includes a processing resource 604 communicatively coupled to a non-transitory computer-readable medium 602 through a communication link 606.
  • the processing resource 604 fetches and executes computer-readable instructions 612 from the non-transitory computer-readable medium 602.
  • the processing resource 604 can be a processor of a computing device, such as the computing device 100 or the computing device 200.
  • the non-transitory computer-readable medium 602 can be, for example, an internal memory device or an external memory device.
  • the communication link 606 may be a direct communication link, such as one formed through a memory read/write interface.
  • the communication link 606 may be an indirect communication link, such as one formed through a network interface.
  • the processing resource 604 can access the non-transitory computer-readable medium 602 through a network 608.
  • the network 608 may be a single network or a combination of multiple networks and may use a variety of different communication protocols.
  • the processing resource 604 and the non-transitory computer- readable medium 602 may also be communicatively coupled to a data source(s) 610.
  • the data source(s) 610 may be used to store data, such as image frames captured by a camera and information relating to faces of users registered with the computing device, in an example.
  • the non-transitory computer- readable medium 602 includes a set of computer-readable instructions 612 that may, in one example, be executable by the processing resource 604 for providing image frames captured by a camera of a computing device without identifiable unregistered users to an application executing on the computing device.
  • the set of computer-readable instructions 612 can be accessed by the processing resource 604 through the communication link 606 and subsequently executed to provide such image frames.
  • the non-transitory computer-readable medium 602 may include a set of instructions executable by the processing resource 604 to detect a face in an image frame. Further, the non-transitory computer-readable medium 602 may also include instructions that may, in one example, be executable to recognize a face if the same is detected to be present in the image frame. The instructions may be executable by the processing resource 604 to recognize the face to be of an unregistered user. In an example, to detect and recognize the face in the image frame, face detection and face recognition techniques may be used. Accordingly, instructions executable to cause face detection and recognition may be executed by the processing resource 604, in one example.
  • instructions that may be executable by the processing resource 604 to receive information relating to faces of users registered with the computing device and store the information in a face descriptor database, may be included in the non-transitory computer-readable medium 602.
  • the information may comprise face descriptors of users registered with the computing device to enable recognition of such users.
  • a face detected to be present in the image frame may be recognized to be of a registered user or unregistered user by comparing face descriptors of the detected face to the face descriptors of registered users that may be retrieved from the face descriptor database.
  • the non-transitory computer-readable medium 602 may further include a set of instructions that may be executable by the processing resource 604 to obfuscate an unregistered user from the image frame when the unregistered user is identified based on face recognition. As explained previously, to obfuscate the unregistered user from the image frame, the image frame may be modified in a variety of ways.
  • the non-transitory computer- readable medium 602 may also include instructions that may be executable to provide the image frame to an application to cause display of the image frame with the obfuscated unregistered user by the application.
  • the methods and devices of the present subject matter provide for the display of image frames without identifiable unregistered users even if such users were captured in an image frame.
  • Applications are caused to display image frames without identifiable unregistered users to prevent instances of violation of privacy of unregistered users captured in the image frames.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Security & Cryptography (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Bioethics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)

Abstract

Techniques for providing image frames with unregistered faces being obfuscated are described. According to an example of the present subject matter, a face present in an image frame captured by a camera is identified. If the face in the image frame is recognized to be of an unregistered user, a display of the image frame with the unregistered user being obfuscated is caused.

Description

IMAGE FRAMES WITH UNREGISTERED USERS OBFUSCATED
BACKGROUND
[001] Computing devices, such as desktops, laptops, mobile phones, tablets, and handheld personal computers (PC) comprise cameras to capture images and videos. For example, a built-in camera of a computing device, such as a webcam of a laptop, enables users to capture pictures or videos.
[002] There may also be applications running on the computing device that may use the images or videos captured through the camera. For example, an application that provides video conferencing functionality, when initiated, may enable the user to interact with other users through video conferencing applications installed at the respective computing devices.
BRIEF DESCRIPTION OF DRAWINGS
[003] The detailed description is described with reference to the accompanying figures. It should be noted that the description and figures are merely examples of the present subject matter and are not meant to represent the subject matter itself.
[004] Fig. 1 illustrates a computing device to cause display of image frames with faces of unregistered users being obfuscated, according to an example implementation of the present subject matter.
[005] Fig. 2 illustrates a computing device to provide image frames with unregistered users obfuscated to applications executing on the computing device, in accordance with an example implementation of the present subject matter.
[006] Fig. 3 illustrates a method for providing image frames with obfuscated faces of users not registered with a computing device to an application executing on the computing device, in accordance with another example implementation of the present subject matter. [007] Fig. 4 illustrates a method of editing an image frame to obfuscate a user not registered with a computing device, according to an example of the present subject matter.
[008] Fig. 5 illustrates a method of editing an image frame to obfuscate portions of the image frame other than a portion corresponding to a registered user, according to an example of the present subject matter.
[009] Fig. 6 illustrates a computing environment for providing image frames with obfuscated unregistered users to applications to cause display of the image frames with the obfuscated unregistered users, according to an example implementation of the present subject matter.
[0010] Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description, however, the description is not limited to the examples and/or implementations provided in the drawings.
DETAILED DESCRIPTION
[0011] Applications running on a computing device may be interfaced with a camera of the computing device or another camera coupled to the computing device to provide various functionalities. For instance, a social media application may access the camera to enable a user to capture and upload his image, while a video calling application may access the camera to enable the user to participate in a video call. In another example, a video conferencing application may interface with the camera to receive a video captured by the camera and transmit the video stream so captured to other remote computing devices over a communication network. A corresponding video conferencing application executing on such a remote computing device may receive the transmitted video stream and render the video on a display device coupled to the remote computing device.
[0012] When a user of the computing device initiates an application to capture an image or video through the camera, individuals, other than the user, present in the vicinity of the user’s computing device may be included in the captured image or video. For instance, if individuals other than a user who is interacting with remote users over a video conference session are present within a field of view of the camera, the video stream captured and transmitted to other remote computing devices may contain images of such individuals. For example, if any of the users interacting with each other over the video conference session is located in a public place, such as a train, flight, or cafe, the video stream is likely to include images of several other individuals.
[0013] Sometimes, it may be undesirable for an image or video captured by a user to include such other individuals. For example, images of such other individuals are often undesirable in a video conference session since they may cause distractions. Also, there may be concerns relating to violation of privacy of such other individuals owing them being included in the captured image or video.
[0014] The present subject matter describes examples of techniques of providing image frames captured by a camera coupled to a computing device, to applications executing on the computing device with unregistered users obfuscated in the image frame, i.e., a user not registered with the computing device. An image frame may be understood as a photo captured by a camera and a sequence of image frames may constitute a video. Thus, the term ‘image frame’ used herein may refer to an image or a frame of a video, as the case may be.
[0015] In an example implementation of the present subject matter, a face present in an image frame captured by the camera is identified. If the face in the image frame is recognized to be of an unregistered user, an application executing on the computing device is caused to display the image frame with the unregistered user obfuscated. Since the application displays unregistered users such that such unregistered users are unidentifiable, instances of violation of the privacy of individuals who are captured in the image frame may be avoided.
[0016] Accordingly, in one example, even in cases where a video stream captured by the camera contains a set of faces, a face of a user registered with the computing device alone is transmitted by a video conferencing application. Thus, the present subject matter also enables video conference sessions to be conducted without potential distraction from individuals who are not intended to be participants in the sessions.
[0017] The above techniques are further described with reference to Fig. 1 to Fig. 6. It should be noted that the description and the figures merely illustrate the principles of the present subject matter along with examples described herein and should not be construed as limiting the present subject matter. It is thus understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present subject matter. Moreover, all statements herein reciting principles, aspects, and implementations of the present subject matter, as well as specific examples thereof, are intended to encompass equivalents thereof.
[0018] Fig. 1 shows a computing device 100 to cause display of image frames with unregistered users obfuscated, according to an example implementation of the present subject matter. Examples of the computing device 100 may include but are not limited to, an electronic device, such as a desktop computer, a personal computer, a laptop, a smartphone, a personal digital assistant (PDAs), and a tablet.
[0019] A face recognition engine 102 of the computing device 100 may control the computing device 100, such that image frames are caused to be displayed without unregistered users being identifiable. In an example, the face recognition engine 102 may be implemented as a combination of hardware and programming (for example, programmable instructions) to implement certain functionalities in the computing device 100. In examples described herein, such combinations of hardware and programming may be implemented in several different ways. Among other capabilities, the face recognition engine 102 is configured to fetch and execute computer-readable instructions stored in a memory (not shown in Fig. 1) of the computing device 100.
[0020] To implement techniques to cause display of image frames with unregistered users obfuscated, in one implementation, the face recognition engine 102 may be interfaced with a camera (not shown in Fig. 1) that is operable in conjunction with the computing device 100 to capture image frames. The face recognition engine 102 is operable to identify a face present in an image frame captured by the camera. If the face in the image frame is recognized to be of an unregistered user, the face recognition engine 102 causes the image frame to be displayed with the unregistered user obfuscated.
[0021] In an example, the image frame without the unregistered user being identifiable may be displayed on a display device (not shown in Fig. 1) coupled to the computing device 100. In an example, the image frame without the unregistered user may be provided to applications executing on the computing device 100. For example, the image frame without the unregistered user may be provided to a video conferencing application executing on the computing device 100. The video conferencing application may thus enable registered users of the computing device 100 to engage in video conferencing sessions with other remote users without video streams displayed in the sessions including the unregistered user. The techniques to identify unregistered users and edit image frames to obfuscate the unregistered users from the image frames are discussed in detail subsequently.
[0022] Thus, the present subject matter provides for privacy protection since the image frames that are displayed contain registered users alone. A registered user using the computing device 100, for example, to engage in a voice over internet protocol (VOIP)-based video call with a remote user is assured that the image frames transmitted to the remote user are devoid of any unregistered users that may be present in the vicinity.
[0023] Reference is now made to Fig. 2 that illustrates a computing device 200 to provide image frames devoid of unregistered users to applications executing on the computing device 200, in accordance with an example implementation of the present subject matter. Examples of the computing device 200 may include a variety of electronic devices that may incorporate or be interfaced with a camera 202.
[0024] In an example, the camera 202 may be inbuilt or integrated into the computing device 200, such as a webcam. The webcam may be a complementary metal-oxide-semiconductor (CMOS) camera in an example. Although not shown, in an example, the camera 202 may be an external camera, such as an external webcam, coupled to the computing device 200 through a universal serial bus (USB) interface.
[0025] In an example, the computing device 200 also comprises a display device 204. The display device 204 may be a visual display unit (VDU), such as a screen or a monitor that may be integral to the computing device 200 or may be coupled to the computing device 200.
[0026] It would be understood that while in the example implementation illustrated in Fig. 2, the display device 204 and the camera 202 are integral to the computing device 200, other implementations, where either or both of these components are external to the computing device 200, are also feasible. In such cases, an interface(s) 206 may serve to communicatively couple the external camera or display device to the computing device 200.
[0027] The interface(s) 206 may include a variety of software and hardware interfaces that allow interaction of the computing device 200 with other communication and computing devices, such as network entities, web servers, and external repositories, and peripheral devices. The interface(s) 206 may also enable the coupling of internal components of the computing device 200 with each other. For example, the interface(s) 206 may couple the display device 204 and the camera 202 to a processor 208 of the computing device 200 that may control the operation of the various components of the computing device 200.
[0028] The processor(s) 208 may be implemented as microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor(s) 208 is configured to fetch and execute computer-readable instructions stored in memory 210 of the computing device 200. The memory 210 may include any computer-readable medium including, for example, volatile memory (e.g., RAM), and/or non-volatile memory (e.g., EPROM, flash memory, etc.). [0029] The processor 208 executes an operating system (OS) 212 of the computing device 200 residing in the memory 210. The OS 212 is a set of instructions that manages the hardware and software of the computing device 200 to enable the computing device 200 to function. The processor 208 may also execute firmware 214 of the computing device 200 residing in the memory 210. Firmware 214 may comprise sets of instructions, for example, proprietary firmware that may, among other things, enable the OS 212 to manage the operation of hardware devices, such as the display device 204 and the camera 202.
[0030] In an example, the OS 212 executes application(s) 216 to provide various services. An application may be understood as a set of instructions to enable a functionality in the computing device 200. The application(s) 216 may be either native to the OS 212 or maybe a third-party application installed on the OS 212. Examples of the application(s) 216 include but are not limited to, applications that may access the camera 202 to provide functionalities relating to capturing, storing, and/or sharing of images or videos herein referred to as image frames. In an example implementation, the OS 212 executes a video conferencing application 218 to provide functionalities, such as video calling and video conferencing.
[0031] In operation, when any of the application(s) 216, that may access content captured by the camera 202 to render the content on the display device 204 or transmit the content to another device, is initiated, the application(s) 216 may cause the camera 202 to capture image frames of individuals who may be present in the field of view of the camera 202. In an example implementation of the present subject matter, a face recognition engine 220 of the computing device 200 may be implemented as an interface between the application(s) 216 and the camera 202 to provide the image frames captured by the camera 202 to the application(s) 216 without identifiable unregistered users who may have been present in the field of view of the camera 202 and captured in the image frames. For the purpose, the face recognition engine 220 may recognize a user who may have previously registered himself with the computing device 200 as a registered user. Users other than the registered user may be considered as unregistered users. [0032] In an example, the face recognition engine 220 may obtain an image frame captured by the camera 202 and modify the image frame to obfuscate an unregistered user from the image frame if such a user were present in the image frame. The modified image frame is made available to the application(s) 216. The application(s) 216 may in turn cause display of the image frame without the unregistered user being identifiable therein.
[0033] In an example, the face recognition engine 220 may be similar to the above-described face recognition engine 102 and may be implemented as a combination of hardware and programming to implement functionalities, such as identifying and removing unregistered users from image frames so as to cause display of the image frames without the unregistered users being identifiable.
[0034] In examples described herein, engine(s) comprising combinations of hardware and programming may be implemented in several different ways. For example, the programming for the engine(s) may be processor-executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the engine(s) may include a processing resource (for example, implemented as either a single processor or a combination of multiple processors), to execute such instructions. In the present examples, the machine-readable storage medium may store instructions that, when executed by the processing resource, implement engine(s). In such examples, the computing device 200 may include the machine-readable storage medium storing the instructions and the processing resource to execute the instructions or the machine-readable storage medium may be separate but accessible to the computing device 200 and the processing resource. In other examples, engine(s) may be implemented by electronic circuitry. In an example, in addition to the aforementioned face recognition engine 220, the engine(s) may also comprise other engine(s) (not shown) that supplement functions of the computing device 200.
[0035] In accordance with an example implementation of the present subject matter, the face recognition engine 220 may control access of the camera 202 by the application(s) 216. According to an example, the face recognition engine 220 may be implemented in the firmware 214 of the computing device 200. In another example, the face recognition engine 220 may be coupled to the OS 212 or may reside in the OS 212 as a device driver for the camera 202. The face recognition engine 220 implemented as the device driver, may either be a default driver for the camera 202 or may be implemented in the computing device 200 in addition to other drivers of the camera 202. Accordingly, the application(s) 216, for example, the video conferencing application 218 executing on the OS 212, may be interfaced with the camera 202 via the face recognition engine 220 acting as the driver for the camera 202 either by default or by way of user selection in cases where drivers other than the face recognition engine 220 may exist in the computing device 200. Thus, the face recognition engine 220 provides for image frames devoid of the unregistered user to be made available to the applications 216 for their consumption, thereby avoiding modifications in the applications 216.
[0036] In an example implementation of the present subject matter, the face recognition engine 220 comprises a face registration module 222 to register users with the computing device 200. For example, the face registration module 222 may provide an interface for registering the users. Users, such as an owner of the computing device 200 or users otherwise authorized to use the computing device 200 may perform a registration process to register themselves with the computing device 200. The registration process may be performed once by each user to enable users to register for subsequent interactions with the applications that use the face recognition engine 220.
[0037] In one example, the face registration module 222 may capture information relating to the face of a user, such as face descriptors corresponding to the user during a registration process. For the purpose, the face registration module 222 may cause a prompt to be displayed on the display device 204 prompting the user to submit images of his face. In an example, the user may provide previously captured images of his face, for example, from his social media account in response to the prompt.
[0038] In some situations, an authorized person, such as an administrator of an organization may also carry out the registration process to register a user, such as an employee of the organization to the computing device 200. The authorized person, in one example, may use an image of the user, such as a profile picture of the user included in his profile registered with the organization to register the user. In an example, to ensure that unauthorized users are disabled from registering themselves or others with the computing device 200, the face registration module 222 may request a user performing a registration process for an identification code, such as an employee id or a preregistered password.
[0039] In an example, the user may opt for images of his face to be captured by the camera 202 for submitting the same to the face registration module 222. Accordingly, the face registration module 222 may further cause the camera 202 to capture images of the face of the user. Each of the users registering with the computing device 200 may, at completion of the registration process, be allocated an account password. Thus, multiple users registered with the computing device 200 may each have a distinct account password that may be usable with their respective interactions with the face recognition engine 220 subsequently.
[0040] Based on the images of the face of the user captured during the registration process, face descriptors corresponding to the face may be extracted. In an example, the face descriptors may include, but are not limited to, geometrical features of the face of the user or features of a retina or iris of the user. The face descriptors corresponding to each of the users that perform the registration process successfully, i.e., registered users, may be stored in a face descriptor database 224. The face descriptor database 224 may be accessed for recognition of the registered users who may be present in the image frames captured by the camera 202 at future instances.
[0041] In the example implementation depicted in Fig. 2, the face descriptor database 224 has been shown to reside internal to the computing device 200. However, implementations where the face descriptor database 224 may reside external to the computing device 200, for example, an external server implemented for registering the users with the computing device 200 are also possible.
[0042] Similarly, implementations where registration of the users with the computing device 200 is performed independent of the face recognition engine 220 are also possible. For instance, in some example implementations, the face registration module 222 may not perform the registration process for registration and recognition of users. Rather, the face recognition engine 220 may leverage registration processes that may have been performed by the OS 212 of the computing device 200 to register users to authorized them to use the computing device 200. Accordingly, in one example, the face registration module 222 may obtain information relating to faces of registered users of the computing device 200 from the OS 212 of the computing device 200. The face registration module 222 may identify the users registered with the computing device 200 based on the information obtained from the OS 212.
[0043] In an example implementation, the face recognition engine 220 comprises an image frame modifier 226 to detect faces, if present in an image frame captured by the camera 202. The image frame modifier 226 may implement a variety of face detection techniques to detect the presence of a face in the image frame. In one example, the image frame modifier 226 may use OpenCV-based face detection techniques, such as Haar Cascades method to detect the presence of the face in an image frame.
[0044] In operation, when the image frame modifier 226 detects a face to be present in the image frame, it initiates a process to recognize the face. In an example, to recognize the face, the image frame modifier 226 extracts face descriptors of the detected face and compares the same with face descriptors of users registered with the computing device 200. The face descriptors that may be stored in the face descriptor database 224 or in any other repository comprising information relating to faces of users registered with the computing device 200, may be retrieved for the purpose. If the face descriptors of the detected face match with the face descriptors of any of the users registered with the computing device 200, the detected face is identified to be of a registered user. On the other hand, if the face descriptors of the detected face do not match, the detected face is identified to be of an unregistered user. In an example, the match may be based on a predefined threshold of accuracy. For example, predefined settings, such as user-configurable settings may define whether, say, a 70% match or a 100% match of the face descriptors of the detected face with the face descriptors of any of the registered users is to be considered by the image frame modifier 226 for identifying the registered users. Upon identifying the face to be of an unregistered user based on the comparison, the image frame modifier 226 modifies the image frame to remove the unregistered user from the image frame.
[0045] The image frame modifier 226 may employ a variety of techniques to modify the image frame to obfuscate the unregistered user. Obfuscating the unregistered user from the image frame may be understood as any modification of the image frame that may render the unregistered user in the image frame unidentifiable. In one example, modification of the image frame may comprise blurring or pixelating a portion of the image frame corresponding to the unregistered user. In one example, the image frame modifier 226 may modify the image frame to remove the unregistered user by substituting a portion of the image frame containing the unregistered user with pixels from other areas of the image frame.
[0046] In one example, if several faces are detected to be present in the image frame captured by the camera 202, each of the detected faces is identified to be of a registered user or an unregistered user and each of the unregistered users is removed from the image frame. In an example, when a detected face is identified to belong to an unregistered user, the unregistered face alone may be obfuscated from the image frame, for instance, by removing or blurring the unregistered face. Thus, in some examples, the modified image frame may contain, at least partially, parts of the body corresponding to the unregistered face, had the same been captured by the camera 202, along with the unregistered face that is obfuscated.
[0047] In an example, the image frame modifier 226 may modify the image frame such that a portion of the image frame containing a registered user is unaltered while other portions are obfuscated. The image frame modifier 226 may modify the portions of the image frame not containing the registered user by pixelating such portions or by superimposing pixels of a predetermined color over such portions, in one example. Thus, the image frame modifier 226 may also provide for a background of the registered user in the image frame to be modified.
[0048] In an example, from amongst the several faces detected to be present in the image frame captured by the camera 202, multiple faces may be of registered users. If any of the registered users do not wish to be included in the image frame, the image frame modifier 226 may modify the image frame such that a portion of the image frame corresponding to such a registered user is also obfuscated. In an example, to instruct the image frame modifier 226 to affect his exclusion, the registered user may use his account password to interact with the face recognition engine 220. The user may define settings at the face recognition engine 220, such that upon his recognition, his image is caused to be displayed after a verification of the account password.
[0049] Fig. 3 illustrates a method for providing image frames with faces of users not registered with a computing device being obfuscated, to an application executing on the computing device, in accordance with another example implementation of the present subject matter.
[0050] Although the method 300 may be implemented in a variety of electronic devices, for the ease of explanation, the present description of the example method 300 to provide image frames without identifiable faces of unregistered users is provided in reference to the above-described computing device 100 and computing device 200.
[0051] The order in which the method 300 is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method 300, or an alternative method. Furthermore, the method 300 may be implemented by a processor(s) or computing device(s) through any suitable hardware, non-transitory machine- readable instructions, or combination thereof.
[0052] It may be understood that blocks of the method 300 may be performed by programmed computing devices. The blocks of the method 300 may be executed based on instructions stored in a non-transitory computer- readable medium, as will be readily understood. The non-transitory computer- readable medium may include, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media.
[0053] Referring to Fig. 3, at block 302, an image frame captured by a camera, such as a camera 202 of the computing device 200 is received. At block 304, a plurality of faces in the image frame is detected. In an example implementation, the face recognition engine 220 of the computing device 200 may implement face detection techniques to determine presence of the plurality of faces. For instance, the face recognition engine 220 may use Haar face detection method to detect the presence of the plurality of faces in the image frame. Further, upon detection of the presence of the plurality of faces in the image frame, each of the plurality of faces may be recognized as a face of a user either registered or not registered with the computing device 200. In an example, face recognition may comprise identifying faces of users who may have previously registered with the computing device 200 as registered faces and identifying faces other than the registered faces amongst the plurality of faces in the image frame as unregistered faces.
[0054] Thereupon, at block 306, the image frame is edited to obfuscate faces of users not registered with the computing device 200 from amongst the plurality of faces in the image frame. At block 308, the edited image frame is provided to an application executing on the computing device 200. For example, the application may be a video conferencing application, such as the video conferencing application 218.
[0055] By providing the edited image frame having the faces of users not registered with the computing device 200 obfuscated therefrom to the video conferencing application 218, the video conferencing application 218 is disabled from displaying, in video conferencing sessions conducted using the video conferencing application 218, the unregistered faces such that they are identifiable.
[0056] Fig. 4 illustrates a method 400 of editing an image frame to obfuscate a user not registered with a computing device, according to an example of the present subject matter and Fig. 5 illustrates a method 500 of editing the image frame to obfuscate portions of the image frame other than a portion corresponding to a registered user, according to an example of the present subject matter.
[0057] Alike method 300, the methods 400 and 500 may also be implemented in a variety of electronic devices, however, for the ease of explanation, methods 400 and 500 are described in reference to the computing device 200. [0058] The order in which the methods 400 and 500 are described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the methods 400 and 500, or an alternative method. Furthermore, the methods 400 and 500 may be implemented by processor(s) or computing device(s) through any suitable hardware, non-transitory machine-readable instructions, or combination thereof.
[0059] It may be understood that blocks of the methods 400 and 500 may be performed by programmed computing devices. The blocks of the methods 400 and 500 may be executed based on instructions stored in a non-transitory computer-readable medium, as will be readily understood. The non-transitory computer-readable medium may include, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media.
[0060] Referring to Fig. 4, at block 402, an image frame captured by the camera 202 of the computing device 200 is received, and, at block 404, a plurality of faces in the image frame is detected. At block 406, face descriptors of each of the plurality of faces detected in the image frame are compared to face descriptors of users registered with the computing device 200. This enables a user, corresponding to each of the plurality of faces detected in the image frame, to be recognized as a registered user or an unregistered user. Accordingly, at block 408, based on the comparison, at least one face from amongst the plurality of faces detected in the image frame may be identified to be of a user not registered with the computing device 200.
[0061] When the image frame is determined to include a user not registered with the computing device 200 based on recognition of the face of the user, at block 410, a portion of the image frame corresponding to the user not registered with the computing device 200 is modified to obfuscate the user. The modification may, in one example, comprise blurring or pixelating the portion of the image frame corresponding to the unregistered user, such that the user is not identifiable. In other examples, the modification may comprise other techniques, such as replacing pixels in the portion of the image frame depicting the unregistered user with pixels from other areas of the image frame. In an example, the portion of the image frame corresponding to the unregistered user may be modified, while, in another example, the portion of the image frame corresponding to the face of the unregistered user alone may be modified.
[0062] At block 412, the modified image frame is provided to the application(s) 216 executing on the computing device 200. For instance, the modified image frame may be provided to a video recording application so as to cause the video recording application to display the image frame without the unregistered user.
[0063] Reference is now made to the method 500 that also provides a modified image frame to the applications(s) 216 executing on the computing device 200 to cause display of the image frame with the unregistered user obfuscated. Blocks 502, 504, and 506 of method 500 being similar to the blocks 402, 404, and 406 of method 500 have not been elaborated to maintain brevity of the present description.
[0064] At block 508, based on the comparison carried out at block 506, at least one face from amongst the plurality of faces detected in the image frame may be identified to be of a user registered with the computing device 200. Having recognized a registered user, at block 510, the image frame is modified to obfuscate portions of the image frame other than a portion corresponding to the registered user. In one example, the modified image frame may contain the face of the registered user with other portions of the image frame being modified, for example, by blurring or pixelating. Similarly, in another example, the modified image frame may contain, at least partially, other body parts along with the face of the registered user.
[0065] Such techniques to modify the image frame to obfuscate portions of the image frame other than those corresponding to the registered user, enable a background of the registered user to be obscured. Thus, inanimate objects in the field of view of the camera 202 may also be obfuscated in addition to the unregistered users. At block 512, the modified image is made available to the application(s) 216 executing on the computing device 200. The application(s) 216 may in turn use the modified image and may cause, for example, storing, sharing, or displaying of the modified image. [0066] Fig. 6 illustrates an example computing environment 600 for providing image frames without identifiable unregistered faces to applications executing in the computing environment 600, according to an example implementation of the present subject matter. In one exemplary implementation, the computing environment 600 includes a processing resource 604 communicatively coupled to a non-transitory computer-readable medium 602 through a communication link 606. In an example, the processing resource 604 fetches and executes computer-readable instructions 612 from the non-transitory computer-readable medium 602.
[0067] For example, the processing resource 604 can be a processor of a computing device, such as the computing device 100 or the computing device 200. The non-transitory computer-readable medium 602 can be, for example, an internal memory device or an external memory device. In one implementation, the communication link 606 may be a direct communication link, such as one formed through a memory read/write interface. In another implementation, the communication link 606 may be an indirect communication link, such as one formed through a network interface. In such a case, the processing resource 604 can access the non-transitory computer-readable medium 602 through a network 608. The network 608 may be a single network or a combination of multiple networks and may use a variety of different communication protocols.
[0068] The processing resource 604 and the non-transitory computer- readable medium 602 may also be communicatively coupled to a data source(s) 610. The data source(s) 610 may be used to store data, such as image frames captured by a camera and information relating to faces of users registered with the computing device, in an example.
[0069] In an example implementation, the non-transitory computer- readable medium 602 includes a set of computer-readable instructions 612 that may, in one example, be executable by the processing resource 604 for providing image frames captured by a camera of a computing device without identifiable unregistered users to an application executing on the computing device. The set of computer-readable instructions 612 can be accessed by the processing resource 604 through the communication link 606 and subsequently executed to provide such image frames.
[0070] In an example, the non-transitory computer-readable medium 602 may include a set of instructions executable by the processing resource 604 to detect a face in an image frame. Further, the non-transitory computer-readable medium 602 may also include instructions that may, in one example, be executable to recognize a face if the same is detected to be present in the image frame. The instructions may be executable by the processing resource 604 to recognize the face to be of an unregistered user. In an example, to detect and recognize the face in the image frame, face detection and face recognition techniques may be used. Accordingly, instructions executable to cause face detection and recognition may be executed by the processing resource 604, in one example.
[0071] For example, to enable recognition of faces of users registered with the computing device, in one example, instructions, that may be executable by the processing resource 604 to receive information relating to faces of users registered with the computing device and store the information in a face descriptor database, may be included in the non-transitory computer-readable medium 602. The information may comprise face descriptors of users registered with the computing device to enable recognition of such users. A face detected to be present in the image frame may be recognized to be of a registered user or unregistered user by comparing face descriptors of the detected face to the face descriptors of registered users that may be retrieved from the face descriptor database.
[0072] The non-transitory computer-readable medium 602 may further include a set of instructions that may be executable by the processing resource 604 to obfuscate an unregistered user from the image frame when the unregistered user is identified based on face recognition. As explained previously, to obfuscate the unregistered user from the image frame, the image frame may be modified in a variety of ways. The non-transitory computer- readable medium 602 may also include instructions that may be executable to provide the image frame to an application to cause display of the image frame with the obfuscated unregistered user by the application. [0073] Thus, the methods and devices of the present subject matter provide for the display of image frames without identifiable unregistered users even if such users were captured in an image frame. Applications are caused to display image frames without identifiable unregistered users to prevent instances of violation of privacy of unregistered users captured in the image frames.
[0074] Although implementations of providing image frames with obfuscated unregistered users have been described in a language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations for providing image frames without identifiable unregistered users.

Claims

Claims:
1. A computing device comprising: a camera to capture image frames; a face recognition engine, coupled to the camera, to: identify a face present in an image frame captured by the camera; recognize the face to be of an unregistered user; and cause to display the image frame with the unregistered user being obfuscated.
2. The computing device as claimed in claim 1 , wherein the face recognition engine is to: recognize the face to be of a registered user; and cause to display the image frame with the registered user.
3. The computing device as claimed in claim 1 , wherein the face recognition engine is implemented in a firmware of the computing device.
4. The computing device as claimed in claim 1 , wherein the computing device comprises: an operating system (OS), wherein the face recognition engine is coupled to the OS as a device driver for the camera; and a video conferencing application executing on the OS, wherein the face recognition engine is to provide the image frame with the unregistered user obfuscated to the video conferencing application to cause display of the image frame with the unregistered user obfuscated.
5. The computing device as claimed in claim 1 , wherein the face recognition engine comprises a face registration module to provide an interface for registering users with the computing device.
6. The computing device as claimed in claim 5, wherein the face registration module is to obtain information relating to faces of registered users of the computing device from the OS of the computing device.
7. The computing device as claimed in claim 1 , wherein the face recognition engine is to access a face descriptor database comprising information relating to faces of registered users of the computing device.
8. The computing device as claimed in claim 1 , wherein the face recognition engine comprises an image frame modifier to: detect presence of the face present in the image frame captured by the camera; compare face descriptors of the detected face with face descriptors of users registered with the computing device; identify, based on the comparison, the face to be of the unregistered user; and modify the image frame to obfuscate the unregistered user in the image frame.
9. The computing device as claimed in claim 8, wherein the image frame modifier is to modify the image frame to remove the unregistered user by substituting a portion of the image frame containing the unregistered user with pixels from other areas of the image frame.
10. A method comprising: receiving an image frame captured by a camera of a computing device; detecting a plurality of faces in the image frame; editing the image frame to obfuscate faces of users not registered with the computing device from amongst the plurality of faces in the image frame; and providing the edited image frame to an application executing on the computing device.
11. The method as claimed in claim 10, wherein the application is a video- conferencing application.
12. The method as claimed in claim 10, wherein the method further comprises: recognizing a registered face from amongst the plurality of faces in the image frame, wherein the editing comprises removing portions of the image frame other than a portion of the image frame containing the registered face.
13. The method as claimed in claim 10, wherein the method further comprises creating a face descriptor database comprising information of faces of users registered with the computing device.
14. A non-transitory computer-readable medium comprising computer- readable instructions executable by a processing resource to: detect a face to be present in an image frame; recognize the face to be of an unregistered user; obfuscate the unregistered user in the image frame; and provide the image frame to an application to cause display of the image frame with the unregistered user obfuscated.
15. The non-transitory computer-readable medium as claimed in claim 14 further comprising computer-readable instructions executable to: receive information relating to faces of users registered with the computing device; and store the information in a face descriptor database, wherein the information is retrieved to recognize the face to be of the unregistered user.
PCT/US2021/037904 2020-06-17 2021-06-17 Image frames with unregistered users obfuscated WO2021257881A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN202021025415 2020-06-17
IN202021025415 2020-06-17

Publications (1)

Publication Number Publication Date
WO2021257881A1 true WO2021257881A1 (en) 2021-12-23

Family

ID=79268458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2021/037904 WO2021257881A1 (en) 2020-06-17 2021-06-17 Image frames with unregistered users obfuscated

Country Status (1)

Country Link
WO (1) WO2021257881A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230135312A1 (en) * 2021-11-04 2023-05-04 International Business Machines Corporation Obfuscation during video conferencing
US20230153963A1 (en) * 2021-11-18 2023-05-18 Citrix Systems, Inc. Online meeting non-participant detection and remediation

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190138795A1 (en) * 2017-11-07 2019-05-09 Ooma, Inc. Automatic Object Detection and Recognition via a Camera System
US20190149543A1 (en) * 2016-10-27 2019-05-16 Tencent Technology (Shenzhen) Company Limited Face verification method and device
US20190278976A1 (en) * 2018-03-11 2019-09-12 Krishna Khadloya Security system with face recognition

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190149543A1 (en) * 2016-10-27 2019-05-16 Tencent Technology (Shenzhen) Company Limited Face verification method and device
US20190138795A1 (en) * 2017-11-07 2019-05-09 Ooma, Inc. Automatic Object Detection and Recognition via a Camera System
US20190278976A1 (en) * 2018-03-11 2019-09-12 Krishna Khadloya Security system with face recognition

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230135312A1 (en) * 2021-11-04 2023-05-04 International Business Machines Corporation Obfuscation during video conferencing
US11665316B2 (en) * 2021-11-04 2023-05-30 International Business Machines Corporation Obfuscation during video conferencing
US20230153963A1 (en) * 2021-11-18 2023-05-18 Citrix Systems, Inc. Online meeting non-participant detection and remediation

Similar Documents

Publication Publication Date Title
US11551482B2 (en) Facial recognition-based authentication
KR102350507B1 (en) Access control method, access control device, system and storage medium
WO2020211388A1 (en) Behavior prediction method and device employing prediction model, apparatus, and storage medium
US10699116B2 (en) Eye gaze authentication
IL256885A (en) Apparatus and methods for facial recognition and video analytics to identify individuals in contextual video streams
US20170061258A1 (en) Method, apparatus, and computer program product for precluding image capture of an image presented on a display
WO2013100699A1 (en) Method, apparatus, and computer-readable recording medium for authenticating a user
WO2021257881A1 (en) Image frames with unregistered users obfuscated
GB2513218A (en) Object detection metadata
WO2021169616A1 (en) Method and apparatus for detecting face of non-living body, and computer device and storage medium
US9183430B2 (en) Portable electronic apparatus and interactive human face login method
US20160125240A1 (en) Systems and methods for secure biometric processing
US20220171889A1 (en) Controlling display devices based on viewing angles
US10970953B2 (en) Face authentication based smart access control system
JP2019118021A (en) Shooting control system, shooting control method, program, and recording medium
US20210374385A1 (en) System and method for face recognition
WO2023024473A1 (en) Living body detection method and apparatus, and electronic device, computer-readable storage medium and computer program product
WO2017157435A1 (en) A method and system for visual privacy protection for mobile and wearable devices
CN115035530A (en) Image processing method, image text obtaining method, device and electronic equipment
CN116361761A (en) Information shielding method, information shielding device and electronic equipment
WO2022231702A1 (en) Integrating and detecting visual data security token in data via graphics processing circuitry using a frame buffer
US20190057271A1 (en) Image processing method, photographing device and storage medium
US20230095955A1 (en) Object alteration in image
CN112685588B (en) Resource recommendation method, device, equipment and storage medium
US20240179371A1 (en) Video management device, video management system, and video management method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21825873

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21825873

Country of ref document: EP

Kind code of ref document: A1