WO2023061262A1

WO2023061262A1 - Image processing method and apparatus, and device and storage medium

Info

Publication number: WO2023061262A1
Application number: PCT/CN2022/123845
Authority: WO
Inventors: 吴义孝
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-10-15
Filing date: 2022-10-08
Publication date: 2023-04-20
Also published as: CN115982708A

Abstract

The embodiments of the present application belong to the technical field of computers. Disclosed are an image processing method and apparatus, and a device and a storage medium. The method is applied to an electronic device, which supports a trusted execution environment, wherein there is an independently running micro-kernel in the trusted execution environment. The method comprises: in response to a call request of a first program for an image sensor during an image information collection process, acquiring permission information of the first program by means of a micro-kernel (310); authenticating the permission information to obtain an authentication result (320); in response to the authentication result indicating that the first program has information collection permission of the image sensor, calling the image sensor by means of the micro-kernel, so as to collect image information (330); and processing the image information to obtain an image processing result (340). By means of the method, the complexity of image processing in a trusted execution environment can be reduced while the security of image processing can be guaranteed.

Description

图像处理方法、装置、设备及存储介质Image processing method, device, equipment and storage medium

本申请要求于2021年10月15日提交的申请号为202111205107.6、发明名称为“图像处理方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202111205107.6 and the title of the invention "Image processing method, device, equipment and storage medium" filed on October 15, 2021, the entire contents of which are incorporated by reference in this application .

技术领域technical field

本申请实施例涉及计算机技术领域，特别涉及一种图像处理方法、装置、设备及存储介质。The embodiments of the present application relate to the field of computer technology, and in particular, to an image processing method, device, device, and storage medium.

背景技术Background technique

安卓(Android)***将安全设计贯穿***架构的各个层面，覆盖***内核、虚拟机、应用框架层以及应用层的各个环节，力求在开放的同时，保护用户的数据、应用程序和电子设备的安全。The Android (Android) system runs through all levels of the system architecture with security design, covering all aspects of the system kernel, virtual machine, application framework layer, and application layer, and strives to protect the security of users' data, applications, and electronic devices while being open. .

传统的Android通路上，采用通用执行环境(Rich Execution Environment，REE)。为了保障数据处理的安全性，还可以在Android通路上直接添加可信任执行环境(Trusted Execution Environment，TEE)，这种情况下Android通路与TEE之间存在耦合关系，因此，在TEE中数据处理的过程中，需要在***内核、虚拟机、应用框架层以及应用层的各个环节依次授权。On the traditional Android path, the general execution environment (Rich Execution Environment, REE) is adopted. In order to ensure the security of data processing, a Trusted Execution Environment (TEE) can also be directly added to the Android channel. In this case, there is a coupling relationship between the Android channel and the TEE. Therefore, the data processing in the TEE During the process, it needs to be authorized in sequence at each link of the system kernel, virtual machine, application framework layer, and application layer.

发明内容Contents of the invention

本申请实施例提供了一种图像处理方法、装置、设备及存储介质。所述技术方案如下：Embodiments of the present application provide an image processing method, device, device, and storage medium. Described technical scheme is as follows:

根据本申请的一方面内容，提供了一种图像处理方法，应用于电子设备中，所述电子设备支持可信执行环境，所述可信执行环境中存在独立运行的微内核，所述方法包括：According to one aspect of the present application, an image processing method is provided, which is applied to an electronic device, the electronic device supports a trusted execution environment, and there is an independently running microkernel in the trusted execution environment, the method includes :

响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息；Responding to the call request of the first program to the image sensor during the image information collection process, acquiring the license information of the first program through the microkernel;

通过所述微内核对所述许可信息进行鉴权，得到鉴定结果，所述鉴定结果包括所述第一程序具备或者不具备所述图像传感器的信息采集权限；Authenticating the license information through the microkernel to obtain an authentication result, the authentication result including whether the first program has or does not have the information collection authority of the image sensor;

响应于所述第一程序具备所述图像传感器的信息采集权限，通过所述微内核调用所述图像传感器进行图像信息的采集；In response to the first program possessing the information collection authority of the image sensor, calling the image sensor through the microkernel to collect image information;

通过所述微内核对所述图像信息进行处理，得到图像处理结果，所述图像处理结果用于对所述第一程序的运行进程进行指示。The image information is processed by the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

根据本申请的另一方面内容，提供了一种图像处理装置，所述装置支持可信执行环境，所述可信执行环境中存在独立运行的微内核，所述装置包括：According to another aspect of the present application, an image processing device is provided, the device supports a trusted execution environment, and there is a microkernel running independently in the trusted execution environment, and the device includes:

获取模块，用于响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息；An acquisition module, configured to acquire the license information of the first program through the microkernel in response to the call request of the first program to the image sensor during the image information acquisition process;

鉴权模块，用于通过所述微内核对所述许可信息进行鉴权，得到鉴定结果，所述鉴定结果包括所述第一程序具备或者不具备所述图像传感器的信息采集权限；An authentication module, configured to authenticate the license information through the microkernel to obtain an authentication result, the authentication result including whether the first program has or does not have the information collection authority of the image sensor;

采集模块，用于响应于所述第一程序具备所述图像传感器的信息采集权限，通过所述微内核调用所述图像传感器进行图像信息的采集；An acquisition module, configured to call the image sensor through the microkernel to acquire image information in response to the first program possessing the information acquisition authority of the image sensor;

处理模块，用于通过所述微内核对所述图像信息进行处理，得到图像处理结果，所述图像处理结果用于对所述第一程序的运行进程进行指示。The processing module is configured to process the image information through the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

根据本申请的另一方面内容，提供了一种电子设备，所述电子设备包括处理器、和与所述处理器相连的存储器，以及存储在所述存储器上的程序指令，所述处理器执行所述程序指令时实现如本申请各个方面提供的图像处理方法。According to another aspect of the present application, an electronic device is provided, the electronic device includes a processor, a memory connected to the processor, and program instructions stored in the memory, and the processor executes The program instructions implement the image processing method provided in various aspects of the present application.

根据本申请的另一方面内容，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有程序指令，所述程序指令被处理器执行时实现如本申请各个方面提供的图像处理方法。According to another aspect of the present application, a computer-readable storage medium is provided, wherein program instructions are stored in the computer-readable storage medium, and when the program instructions are executed by a processor, the image provided by various aspects of the present application is realized. Approach.

根据本申请的另一个方面内容，提供了一种计算机程序产品(或计算机程序)，所述计算机程序产品(或计算机程序)包括计算机指令，所述计算机指令存储在计算机可读存储介质中。计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令，所述处理器执行所述计算机指令，使得所述计算机设备执行上述图像处理方法的各种可选实现方式中提供的方法。According to another aspect of the present application, a computer program product (or computer program) is provided, the computer program product (or computer program) includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned image processing methods provided in various optional implementation manners. method.

附图说明Description of drawings

图1示出了本申请一个示例性实施例提供的电子设备的示意图；FIG. 1 shows a schematic diagram of an electronic device provided by an exemplary embodiment of the present application;

图2示出了本申请一个示例性实施例提供的许可信息传递的示意图；Fig. 2 shows a schematic diagram of license information transfer provided by an exemplary embodiment of the present application;

图3示出了本申请一个示例性实施例提供的图像处理方法的流程图；FIG. 3 shows a flowchart of an image processing method provided by an exemplary embodiment of the present application;

图4示出了本申请另一个示例性实施例提供的图像处理方法的流程图；FIG. 4 shows a flowchart of an image processing method provided by another exemplary embodiment of the present application;

图5示出了本申请另一个示例性实施例提供的图像处理方法的流程图；FIG. 5 shows a flowchart of an image processing method provided by another exemplary embodiment of the present application;

图6示出了本申请另一个示例性实施例提供的图像处理方法的流程图；FIG. 6 shows a flowchart of an image processing method provided by another exemplary embodiment of the present application;

图7示出了本申请一个示例性实施例提供的图像信息传递的安全通路的示意图；Fig. 7 shows a schematic diagram of a secure path for image information transmission provided by an exemplary embodiment of the present application;

图8示出了本申请一个示例性实施例提供的图像处理装置的框图；Fig. 8 shows a block diagram of an image processing device provided by an exemplary embodiment of the present application;

图9示出了本申请一个示例性实施例提供的计算机设备的结构示意图。Fig. 9 shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present application.

具体实施方式Detailed ways

对本申请中涉及的名词解释如下。The terms involved in this application are explained as follows.

通用执行环境，是电子设备上通用的执行环境，运行通用的操作***，比如Android***、苹果***(iOS)、以及鸿蒙***(HarmonyOS)。The universal execution environment is a common execution environment on electronic devices, running common operating systems, such as Android system, Apple system (iOS), and Hongmeng system (HarmonyOS).

可信执行环境，是与REE并存的执行环境。它是一个独立的执行区域，提供了介于REE和安全元件(Secure Element，SE)之间的安全性的框架，能够确保加载到可信执行环境中的代码和数据的安全性。其中，运行在可信执行环境中的应用程序，即为受信任的应用程序(Trusted Application，TA)。Trusted execution environment is an execution environment that coexists with REE. It is an independent execution area that provides a security framework between REE and Secure Element (SE), which can ensure the security of code and data loaded into the trusted execution environment. Among them, the application program running in the trusted execution environment is a trusted application program (Trusted Application, TA).

微内核，是一种能够提供必要服务的操作***内核；其中，这些必要服务包括任务、线程、交互进程通信以及内存管理等。所有服务(包括设备驱动)在用户模式下运行，而处理这些服务同处理其他的任何一个程序一样。因为每个服务只是在自己的地址空间运行，所以这些服务之间彼此之间都受到了保护。A microkernel is an operating system kernel that can provide necessary services; these necessary services include tasks, threads, interactive process communication, and memory management. All services (including device drivers) run in user mode, and handling these services is the same as handling any other program. Because each service just runs in its own address space, these services are protected from each other.

处于长期打开状态(Always ON，AON)的摄像头，也即AON摄像头，示例性的，AON摄像头一般是电子设备上的前置摄像头。AON摄像头的应用场景(简称AON场景)包括但不限于如下应用场景：A camera that is always on (Always ON, AON), that is, an AON camera. Exemplarily, the AON camera is generally a front camera on an electronic device. The application scenarios of AON cameras (referred to as AON scenarios) include but are not limited to the following application scenarios:

·解锁场景· Unlock scene

在解锁场景下通过AON摄像头采集图像信息，电子设备对上述图像信息进行信息提取，得到用户信息，比如用户的人脸信息、或者手势信息、或者虹膜信息等；电子设备基于用户信息进行验证，最终确定是否进行解锁。In the unlocking scene, the image information is collected by the AON camera, and the electronic device extracts the above image information to obtain user information, such as the user's face information, or gesture information, or iris information, etc.; the electronic device verifies based on the user information, and finally Determine whether to unlock.

·用户信息采集场景· User information collection scene

比如，在身份信息、解锁信息等用户信息的采集场景下，通过AON摄像头采集图像信息，电子设备对上述图像信息进行信息提取，得到用户信息，比如人脸信息、或者手势信息、或者虹膜信息等；电子设备对上述用户信息进行存储，用于身份验证或者解锁验证。For example, in the collection scene of user information such as identity information and unlocking information, the image information is collected by the AON camera, and the electronic device extracts the above image information to obtain user information, such as face information, or gesture information, or iris information, etc. ; The electronic device stores the above user information for identity verification or unlocking verification.

·注视不息屏场景·Focus on the non-stop screen scene

在注视不息屏场景下通过AON摄像头采集图像信息，电子设备对上述图像信息进行信息提取，得到用户行为信息；在预设时间段内不存在外部输入的情况下，电子设备基于用户行为信息确定用户正在注视屏幕，则不执行息屏操作。上述外部输入是指通过电子设备上的输入设备进行输入操作。In the scene of non-stop watching the screen, the image information is collected by the AON camera, and the electronic device extracts the information from the above image information to obtain user behavior information; when there is no external input within the preset time period, the electronic device determines based on the user behavior information. If the user is looking at the screen, the off-screen operation will not be performed. The aforementioned external input refers to an input operation performed through an input device on an electronic device.

上述人脸信息、手势信息、以及虹膜信息等用户信息属于隐私信息，电子设备在采集或者处理上述用户信息时，需要在可信执行环境中执行，以保证上述用户信息的安全。The above-mentioned user information such as face information, gesture information, and iris information is private information. When electronic devices collect or process the above-mentioned user information, they need to be executed in a trusted execution environment to ensure the security of the above-mentioned user information.

传统的可信执行环境直接添加在***通路上，这使得可信执行环境与***通路之间存在耦合关系，也因此，在可信执行环境中数据处理的过程中，需要在***内核、虚拟机、应用框架层以及应用层的各个环节依次授权，以保证数据、以及数据处理过程的安全性。针对上述技术问题，本申请提供了一种图像处理方法，该方法应用于电子设备中，该电子设备支持可信执行环境，该可信执行环境中存在独立运行的微内核，该方法包括：The traditional trusted execution environment is directly added to the system path, which causes a coupling relationship between the trusted execution environment and the system path. Therefore, in the process of data processing in the trusted execution environment, it is necessary to , the application framework layer and each link of the application layer are authorized in sequence to ensure the security of the data and the data processing process. In view of the above technical problems, the present application provides an image processing method, which is applied to an electronic device, the electronic device supports a trusted execution environment, and there is an independently running microkernel in the trusted execution environment, the method includes:

响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核获取第一程序的许可信息；In response to the call request of the first program to the image sensor during the image information collection process, the license information of the first program is obtained through the microkernel;

通过微内核对许可信息进行鉴权，得到鉴定结果，鉴定结果包括第一程序具备或者不具备图像传感器的信息采集权限；Authenticating the license information through the microkernel to obtain an appraisal result, the appraisal result including whether the first program has or does not have the information collection authority of the image sensor;

响应于第一程序具备图像传感器的信息采集权限，通过微内核调用图像传感器进行图像信息的采集；In response to the fact that the first program has the information collection authority of the image sensor, the microkernel calls the image sensor to collect image information;

通过微内核对图像信息进行处理，得到图像处理结果，图像处理结果用于对第一程序的运行进程进行指示。The image information is processed by the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

在一些实施例中，电子设备上与微内核并行运行有主操作***；In some embodiments, a main operating system runs on the electronic device in parallel with the microkernel;

通过微内核获取第一程序的许可信息，包括：Obtain the license information of the first program through the microkernel, including:

通过主操作***获取许可信息；Obtain license information through the main operating system;

通过微内核从主操作***中获取许可信息。Licensing information is obtained from the main operating system through the microkernel.

在一些实施例中，通过微内核从主操作***中获取许可信息，包括：In some embodiments, license information is obtained from the host operating system through the microkernel, including:

通过微内核的第一硬件抽象层接收主操作***的第二硬件抽象层传递的许可信息。The license information delivered by the second hardware abstraction layer of the main operating system is received by the first hardware abstraction layer of the microkernel.

在一些实施例中，图像传感器包括处于长期打开状态的前置摄像头，前置摄像头用于采集环境图像以对程序运行的应用场景进行实时获取；In some embodiments, the image sensor includes a front-facing camera that is in a long-term open state, and the front-facing camera is used to collect environmental images to obtain real-time application scenarios of program running;

响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核获取第一程序的许可信息，包括：In response to the call request of the first program to the image sensor during the image information collection process, the license information of the first program is obtained through the microkernel, including:

通过微内核接收前置摄像头采集到的环境图像；Receive the environmental image collected by the front camera through the microkernel;

通过微内核基于环境图像进行程序运行的应用场景分析；Application scenario analysis of program running based on the environment image through the microkernel;

响应于应用场景为第一应用场景，通过微内核获取第一程序的许可信息；第一应用场景包括第一程序调用前置摄像头进行图像信息采集的场景。In response to the fact that the application scenario is the first application scenario, the license information of the first program is obtained through the microkernel; the first application scenario includes a scenario where the first program calls the front camera to collect image information.

在一些实施例中，可信执行环境中预存有验证信息；In some embodiments, verification information is pre-stored in the trusted execution environment;

通过微内核对许可信息进行鉴权，得到鉴定结果，包括：The license information is authenticated through the microkernel, and the authentication results are obtained, including:

通过微内核对许可信息与验证信息进行匹配；Match the license information and verification information through the microkernel;

响应于许可信息与验证信息匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果。In response to the matching of the permission information and the verification information, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.

在一些实施例中，许可信息包括公钥，验证信息包括私钥；In some embodiments, the license information includes a public key and the verification information includes a private key;

通过微内核对许可信息与验证信息进行匹配，包括：Match the license information and verification information through the microkernel, including:

通过微内核对公钥与私钥进行配对；Pair the public key with the private key through the microkernel;

响应于许可信息与验证信息匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果，包括：In response to the match between the license information and the verification information, the microkernel obtains an authentication result that the first program has the information collection authority of the image sensor, including:

响应于公钥与私钥匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果。In response to the matching of the public key and the private key, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.

在一些实施例中，许可信息包括第一指纹信息，验证信息包括可信执行环境中预存的第二指纹信息；In some embodiments, the license information includes first fingerprint information, and the verification information includes second fingerprint information pre-stored in the trusted execution environment;

响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核调用指纹传感器进行第一指纹信息的采集；In response to the call request of the first program to the image sensor during the image information collection process, the microkernel calls the fingerprint sensor to collect the first fingerprint information;

通过微内核计算第一指纹信息与第二指纹信息的相似度；Calculating the similarity between the first fingerprint information and the second fingerprint information through the microkernel;

响应于相似度大于预设相似度，通过微内核确定第一指纹信息与第二指纹信息匹配。In response to the similarity being greater than the preset similarity, the microkernel determines that the first fingerprint information matches the second fingerprint information.

通过微内核调用图像传感器进行图像信息的采集之后，包括：After the microkernel calls the image sensor to collect image information, it includes:

通过主操作***将图像信息存储至可信执行环境中的图像存储器中；storing the image information in the image memory in the trusted execution environment through the main operating system;

通过微内核从图像存储器中读取图像信息。The image information is read from the image memory through the microkernel.

在一些实施例中，第一程序包括***程序和第三方应用程序中的至少一种。In some embodiments, the first program includes at least one of a system program and a third-party application program.

上述图像处理方法中步骤实现的详细内容请参考如下实施例。Please refer to the following embodiments for detailed content of implementation of the steps in the above image processing method.

图1示出了本申请一个示例性实施例提供的电子设备100的框图。该电子设备100包括处理器120和存储器140，存储器140中存储有至少一条指令，指令由处理器120加载并执行以实现如本申请各个方法实施例所述的图像处理方法。Fig. 1 shows a block diagram of an electronic device 100 provided by an exemplary embodiment of the present application. The electronic device 100 includes a processor 120 and a memory 140, at least one instruction is stored in the memory 140, and the instruction is loaded and executed by the processor 120 to implement the image processing method described in each method embodiment of the present application.

处理器120可以包括一个或者多个处理核心。处理器120利用各种接口和线路连接整个电子设备100内的各个部分，通过运行或执行存储在存储器140内的指令、程序、代码集或指令集，以及调用存储在存储器140内的数据，执行电子设备100的各种功能和处理数据。可选的，处理器120可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器120可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)、调制解调器和前处理单元等中的一种或几种的组合。其中，CPU主要处理操作***、用户界面和应用程序等；GPU用于负责显示屏所需要显示的内容的渲染和绘制；调制解调器用于处理无线通信；前处理单元用于提供前处理功能，示例性的，对原始数据处理之后得到执行某项功能所需的输入数据，那么对原始数据的处理即是前处理。可以理解的是，上述调制解调器也可以不集成到处理器120中，单独通过一块芯片进行实现。 Processor 120 may include one or more processing cores. The processor 120 uses various interfaces and circuits to connect various parts of the entire electronic device 100, and executes or executes instructions, programs, code sets or instruction sets stored in the memory 140, and calls data stored in the memory 140, to execute Various functions of the electronic device 100 and processing data. Optionally, the processor 120 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware. The processor 120 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, and a pre-processing unit. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the content that needs to be displayed on the display screen; the modem is used to handle wireless communication; the pre-processing unit is used to provide pre-processing functions, exemplary Yes, after the raw data is processed to obtain the input data required to perform a certain function, then the processing of the raw data is pre-processing. It can be understood that, the above-mentioned modem may not be integrated into the processor 120, but may be implemented by a single chip.

示例性的，前处理单元上运行有可认证微内核，该微内核独立于主操作***之外运行。在本申请提供的实施例中，微内核运行于可信执行环境中，主操作***运行于通用执行环境，微内核与主操作***协作实现图像处理方法的步骤。Exemplarily, a certifiable microkernel runs on the pre-processing unit, and the microkernel runs independently of the main operating system. In the embodiment provided by the present application, the microkernel runs in a trusted execution environment, the main operating system runs in a general execution environment, and the microkernel and the main operating system cooperate to implement the steps of the image processing method.

比如，在第一程序请求调用图像传感器进行信息采集的情况下，主操作***获取第一程序的许可信息，将许可信息从自身的硬件抽象层(Hardware Abstraction Layer，HAL)传递至微内核的硬件抽象层；微内核对许可信息进行鉴权，验证第一程序是否具备图像传感器的信息采集权限；在第一程序具备图像传感器的信息采集权限的情况下，由微内核执行图像信息的处理步骤。For example, when the first program requests to call the image sensor for information collection, the main operating system obtains the license information of the first program, and transfers the license information from its own hardware abstraction layer (Hardware Abstraction Layer, HAL) to the hardware of the microkernel Abstraction layer: the microkernel authenticates the license information, and verifies whether the first program has the information acquisition authority of the image sensor; if the first program has the information acquisition authority of the image sensor, the microkernel executes the image information processing steps.

若应用程序在微内核中运行，微内核基于图像处理结果执行第一程序接下来的进程；若应用程序仅在微内核中进行图像信息的前处理，微内核对图像传感器采集的图像信息处理完成后，将图像处理结果反馈给主操作***，由主操作***基于图像处理结果执行第一程序接下来的进程。If the application program runs in the microkernel, the microkernel executes the next process of the first program based on the image processing result; if the application program only performs pre-processing of image information in the microkernel, the microkernel completes the processing of the image information collected by the image sensor Afterwards, the image processing result is fed back to the main operating system, and the main operating system executes the next process of the first program based on the image processing result.

在一些实施例中，在处理器120上单独划分部分硬件资源，将这一部分硬件资源划分给前处理单元。In some embodiments, part of the hardware resources are separately allocated on the processor 120, and this part of hardware resources is allocated to the pre-processing unit.

在另一些实施例中，前处理单元不集成到处理器120中，单独通过一块芯片(即前处理芯片)进行实现。如图2，电子设备可以包括前处理芯片220和应用芯片(Application Processor，AP)240。示例性的，应用芯片240可以是图1中的处理器120，或者，可以是图1中处理器120的集成部分。In some other embodiments, the pre-processing unit is not integrated into the processor 120, but implemented by a single chip (ie, the pre-processing chip). As shown in FIG. 2 , the electronic device may include a pre-processing chip 220 and an application chip (Application Processor, AP) 240. Exemplarily, the application chip 240 may be the processor 120 in FIG. 1 , or may be an integrated part of the processor 120 in FIG. 1 .

前处理芯片220上部署有可信任执行环境，可信任执行环境中存在独立使用的硬件资源，比如，小型CPU 222和DSP 224。CPU 222上运行有微内核，微内核的***架构(FrameWork，FW)21包括硬件抽象层22。应用芯片240上部署有通用执行环境，应用芯片240上运行有主操作***，主操作***的***架构包括硬件抽象层23。A trusted execution environment is deployed on the pre-processing chip 220, and there are independently used hardware resources in the trusted execution environment, such as a small CPU 222 and a DSP 224. A microkernel runs on the CPU 222, and the system architecture (FrameWork, FW) 21 of the microkernel includes a hardware abstraction layer 22. A general execution environment is deployed on the application chip 240 , and a main operating system runs on the application chip 240 . The system architecture of the main operating system includes a hardware abstraction layer 23 .

在本申请提供的实施例中，微内核与主操作***之间许可信息的传递，通过硬件抽象层22和硬件抽象层23来实现。示例性的，应用芯片240上还部署有马里亚纳多媒体(硬件)服务接入(Mariana Multimedia Service，MMS)框架24；主操作***还可以通过MMS框架24来获取多媒体应用程序中的许可信息。In the embodiment provided by this application, the transfer of license information between the microkernel and the main operating system is realized through the hardware abstraction layer 22 and the hardware abstraction layer 23 . Exemplarily, a Mariana Multimedia (hardware) service access (Mariana Multimedia Service, MMS) framework 24 is also deployed on the application chip 240; the main operating system can also obtain the license information in the multimedia application through the MMS framework 24.

存储器140可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory，ROM)。可选的，该存储器140包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器140可用于存储指令、程序、代码、代码集或指令集。存储器140可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作***的指令、用于至少一个功能的指令(比如触控功能、声音播放功能、图像处理功能等)、用于实现下述各个方法实施例的指令等；存储数据区可存储下面各个方法实施例中涉及到的数据等。The memory 140 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory, ROM). Optionally, the memory 140 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium). The memory 140 may be used to store instructions, programs, codes, sets of codes or sets of instructions. The memory 140 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image processing function, etc.), Instructions and the like for implementing the following method embodiments; the storage data area can store data and the like involved in the following method embodiments.

示例性的，上述主操作***可以是Android***、或iOS、或HarmonyOS。Exemplarily, the above-mentioned main operating system may be an Android system, or iOS, or HarmonyOS.

示例性的，电子设备100可以包括智能手机、平板电脑、电子书阅读器、MP3(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)播放器、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、膝上型便携计算机和台式计算机中、笔记本电脑的至少一种。本申请实施例对电子设备100的设备类型不加以限定。Exemplarily, the electronic device 100 may include a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, moving picture expert compression standard audio level 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, Motion Image Expert Compression Standard Audio Level 4) At least one of player, laptop computer and desktop computer, notebook computer. The embodiment of the present application does not limit the device type of the electronic device 100 .

图3示出了本申请一个示例性实施例提供的图像处理方法的流程图。该图像处理方法可以应用于电子设备中。在图3中，图像处理方法包括：Fig. 3 shows a flowchart of an image processing method provided by an exemplary embodiment of the present application. The image processing method can be applied to electronic equipment. In Figure 3, the image processing method includes:

步骤310：响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核获取第一程序的许可信息。Step 310: Obtain license information of the first program through the microkernel in response to the call request of the first program to the image sensor during image information collection.

电子设备支持可信执行环境，可信执行环境中存在独立运行的微内核。电子设备通过微内核响应第一程序在图像信息采集过程中对图像传感器的调用请求，在可信执行环境中通过微内核获取第一程序的许可信息。The electronic device supports a trusted execution environment, and there is a microkernel running independently in the trusted execution environment. The electronic device responds to the call request of the first program to the image sensor during the image information collection process through the microkernel, and acquires the license information of the first program through the microkernel in the trusted execution environment.

在一个实施例中，电子设备在微内核上运行第一程序；响应于第一程序请求在图像信息采集过程中对图像传感器的调用请求，通过微内核获取第一程序的许可信息。In one embodiment, the electronic device runs the first program on the microkernel; in response to the first program's request to call the image sensor during image information collection, the license information of the first program is obtained through the microkernel.

电子设备还支持通用执行环境，通用执行环境中存在运行的主操作***，主操作***与微内核并行运行在电子设备上；其中，通用执行环境与可信执行环境被分配使用不同的硬件资源，也即通用执行环境与可信执行环境之间存在硬件隔离。在另一个实施例中，电子设备在主操作***上运行第一程序；响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过主操作***获取第一程序的许可信息；通过微内核从主操作***中获取第一程序的许可信息。The electronic device also supports a general-purpose execution environment, in which there is a running main operating system, and the main operating system and the microkernel run in parallel on the electronic device; wherein, the general-purpose execution environment and the trusted execution environment are allocated to use different hardware resources, That is, there is hardware isolation between the general execution environment and the trusted execution environment. In another embodiment, the electronic device runs the first program on the main operating system; in response to the call request of the first program to the image sensor during the image information collection process, obtains the license information of the first program through the main operating system; The microkernel obtains the license information of the first program from the main operating system.

可选地，微内核包括第一硬件抽象层，主操作***包括第二硬件抽象层；微内核与主操作***之间信息交互，通过第一硬件抽象层与第二硬件抽象层来实现；电子设备通过微内核的第一硬件抽象层接收主操作***的第二硬件抽象层传递的许可信息，也即电子设备将许可信息从第一硬件抽象层传递至第二硬件抽象层。Optionally, the microkernel includes a first hardware abstraction layer, and the main operating system includes a second hardware abstraction layer; the information interaction between the microkernel and the main operating system is realized through the first hardware abstraction layer and the second hardware abstraction layer; The device receives the license information transferred by the second hardware abstraction layer of the main operating system through the first hardware abstraction layer of the microkernel, that is, the electronic device transfers the license information from the first hardware abstraction layer to the second hardware abstraction layer.

可选地，微内核与主操作***之间通过网络连接；通过微内核接收主操作***通过网络连接发送的许可信息。Optionally, the microkernel is connected to the main operating system through a network; the microkernel receives the license information sent by the main operating system through the network connection.

示例性的，第一程序中预先设置有许可信息，该许可信息用于鉴定第一程序是否具备调用图像传感器进行图像信息采集的权限。示例性的，该许可信息可以是程序开发时写入第一程序中的，或者，还可以是基于用户授权写入第一程序中的。电子设备可以通过主操作***或者微内核运行MMS架构，通过MMS架构从第一程序中获取许可信息。Exemplarily, permission information is preset in the first program, and the permission information is used to identify whether the first program has the right to call the image sensor to collect image information. Exemplarily, the license information may be written into the first program during program development, or may also be written into the first program based on user authorization. The electronic device can run the MMS architecture through the main operating system or the microkernel, and obtain the license information from the first program through the MMS architecture.

示例性的，上述第一程序是电子设备中正在运行的程序。可选地，第一程序包括***程序、应用程序、第三方应用程序中的至少一种。其中，***程序是指控制和协调计算机(即电子设备)及外部设备，支持应用软件开发和运行的程序；其是无需用户干预的各种程序的集合，主要功能是调度、监控和维护计算机***，负责管理计算机***中各种独立的硬件，使得它们可以协调工作。应用程序是指***本身自带的应用程序，也即***生厂商提供的应用程序。第三方应用程序是指非***本身自带的、也非用户自己制作的应用程序，也即***生产商以外的应用公司提供的应用程序。Exemplarily, the above-mentioned first program is a program running in the electronic device. Optionally, the first program includes at least one of a system program, an application program, and a third-party application program. Among them, the system program refers to the program that controls and coordinates the computer (that is, electronic equipment) and peripheral equipment, and supports the development and operation of application software; it is a collection of various programs that do not require user intervention, and its main function is to schedule, monitor and maintain computer systems , responsible for managing various independent hardware in the computer system so that they can work in harmony. The application program refers to the application program provided by the system itself, that is, the application program provided by the system manufacturer. Third-party applications refer to applications that are not built into the system itself or produced by users themselves, that is, applications provided by application companies other than the system manufacturer.

步骤320：通过微内核对许可信息进行鉴权，得到鉴定结果，鉴定结果包括第一程序具备或者不具备图像传感器的信息采集权限的鉴定结果。Step 320: Authenticating the license information through the microkernel to obtain an authentication result, the authentication result including the authentication result that the first program has or does not have the information collection authority of the image sensor.

电子设备在可信执行环境中通过微内核对许可信息进行鉴权，若得到第一程序具备图像传感器的信息采集权限的鉴定结果，则执行步骤330；若得到第一程序不具备图像传感器的信息采集权限的鉴定结果，则终止图像处理方法的执行。The electronic device authenticates the license information through the microkernel in the trusted execution environment, and if it obtains the authentication result that the first program has the information collection authority of the image sensor, execute step 330; if it obtains the information that the first program does not have the image sensor The execution of the image processing method is terminated if the identification result of the acquisition authority is obtained.

电子设备中可信执行环境被分配使用单独的存储区域，该存储区域用于可信执行环境中程序运行的相关数据的存储。该存储区域中存储有验证信息，也即可信执行环境中预存有验证信息，该验证信息用于对许可信息进行验证，以确定第一程序是否具备图像传感器的信息采集权限。电子设备通过微内核从存储区域读取验证信息，通过微内核对许可信息与验证信息进行匹配；响应于许可信息与验证信匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果；响应于许可信息与验证信息不匹配，通过微内核得到第一程序不具备图像传感器的信息采集权限的鉴定结果。The trusted execution environment in the electronic device is assigned to use a separate storage area, and the storage area is used for storing data related to program operation in the trusted execution environment. Verification information is stored in the storage area, that is, verification information is pre-stored in the trusted execution environment, and the verification information is used to verify the license information to determine whether the first program has the information collection authority of the image sensor. The electronic device reads the verification information from the storage area through the microkernel, and matches the license information with the verification information through the microkernel; in response to the match between the license information and the verification letter, obtains the identification that the first program has the information collection authority of the image sensor through the microkernel Result; in response to the mismatch between the permission information and the verification information, an identification result that the first program does not have the information collection authority of the image sensor is obtained through the microkernel.

示例性的，电子设备通过微内核确定许可信息与验证信息是否相同；响应于许可信息与验证信息相同，确定第一程序具备图像传感器的信息采集权限。比如，验证信息为第一许可证，许可信息为第二许可证，其中，第一许可证是向第一程序开放图像传感器的信息采集权限所发放的证明文件；电子设备通过微内核确定第二许可证与第一许可证是否相同，响应于第二许可证与第一许可证相同，确定第一程序具备图像传感器的信息采集权限。Exemplarily, the electronic device determines whether the license information is the same as the verification information through the microkernel; in response to the license information being the same as the verification information, it is determined that the first program has the information collection authority of the image sensor. For example, the verification information is the first license, and the license information is the second license, wherein the first license is a certificate issued by opening the image sensor's information collection authority to the first program; the electronic device determines the second license through the microkernel. Whether the license is the same as the first license, in response to whether the second license is the same as the first license, it is determined that the first program has the information collection authority of the image sensor.

示例性的，电子设备通过微内核确定许可信息与验证信息是否存在一一对应关系；响应于许可信息与验证信息之间存在一一对应关系，确定第一程序具备图像传感器的信息采集权限。比如，验证信息包括验证函数与验证数值，许可信息为许可数值；响应于验证数值与许可数值符合验证函数，确定第一程序具备图像传感器的信息采集权限；其中，验证数值与许可数值符合验证函数，是指将许可数值输入验证函数，得到的解为验证数值。示例性的，验证函数包括哈希函数，相应的，验证数值为哈希值。Exemplarily, the electronic device determines whether there is a one-to-one correspondence between the license information and the verification information through the microkernel; in response to the one-to-one correspondence between the license information and the verification information, determine that the first program has the information collection authority of the image sensor. For example, the verification information includes a verification function and a verification value, and the permission information is a permission value; in response to the verification value and the permission value conforming to the verification function, it is determined that the first program has the information collection authority of the image sensor; wherein, the verification value and the permit value conform to the verification function , means that the permission value is input into the verification function, and the obtained solution is the verification value. Exemplarily, the verification function includes a hash function, and correspondingly, the verification value is a hash value.

步骤330：响应于第一程序具备图像传感器的信息采集权限，通过微内核调用图像传感器进行图像信息的采集。Step 330: In response to the fact that the first program has the information collection authority of the image sensor, the microkernel calls the image sensor to collect image information.

电子设备在第一程序具备图像传感器的信息采集权限的情况下，在可信执行环境中通过微内核调用图像传感器进行图像信息的采集。Under the condition that the first program has the information collection authority of the image sensor, the electronic device calls the image sensor through the microkernel in the trusted execution environment to collect image information.

示例性的，电子设备上设置有图像传感器；电子设备响应于第一程序具备图像传感器的信息采集权限，通过微内核调用图像传感器进行图像信息的采集。Exemplarily, the electronic device is provided with an image sensor; in response to the first program having the information collection authority of the image sensor, the electronic device calls the image sensor through the microkernel to collect image information.

示例性的，图像采集设备上设置有图像传感器、通信模块和信息处理模块；电子设备与图像采集设备之间存在通信连接；在第一程序具备图像传感器的信息采集权限的情况下，电子设备上微内核通过通信连接调用图像采集设备上的图像传感器，以通过图像传感器进行图像信息的采集。示例性的，上述通信连接包括有线连接或者无线连接；示例性的，无线连接可以包括蓝牙连接、近场通信(Near Field Communication，NFC)、以及无线保真(WIreless FIdelity，WIFI)连接等。Exemplarily, an image sensor, a communication module, and an information processing module are set on the image acquisition device; there is a communication connection between the electronic device and the image acquisition device; when the first program has the information acquisition authority of the image sensor, the electronic device The microkernel calls the image sensor on the image acquisition device through the communication connection, so as to collect image information through the image sensor. Exemplarily, the above-mentioned communication connection includes a wired connection or a wireless connection; exemplary, the wireless connection may include a Bluetooth connection, a near field communication (Near Field Communication, NFC), and a wireless fidelity (WIreless FIdelity, WIFI) connection, etc.

图像传感器可以应用于拍照场景，还可以应用于生物特征信息的信息采集场景。其中，信息采集场景是指通过图像传感器采集涉及隐私的用户信息的应用场景，比如，采集人脸信息、虹膜信息、以及手势信息等用户信息的应用场景。在上述信息采集场景下，电子设备通过微内核调用图像传感器进行图像信息的采集。The image sensor can be applied to the scene of taking pictures, and can also be applied to the scene of information collection of biometric information. The information collection scenario refers to an application scenario in which user information related to privacy is collected by an image sensor, for example, an application scenario in which user information such as face information, iris information, and gesture information is collected. In the above information collection scenario, the electronic device calls the image sensor through the microkernel to collect image information.

步骤340：通过微内核对图像信息进行处理，得到图像处理结果，图像处理结果用于对第一程序的运行进程进行指示。Step 340: Process the image information through the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

电子设备在可信执行环境中通过微内核对图像信息进行处理，得到图像处理结果，图像处理结果用于指示第一程序的运行进程。The electronic device processes the image information through the microkernel in the trusted execution environment to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

示例性的，图像处理结果可以是第一程序运行过程中的中间数据，也即是第一程序继续运行所需的中间数据。此时，若是第一程序运行在可信执行环境中的微内核上，电子设备通过微内核将图像处理结果作为第一程序的中间输入数据，继续运行第一程序；若是第一程序运行在通用执行环境中的主操作***上，电子设备通过微内核将图像处理结果传递至主操作***；通过主操作***将图像处理结果作为第一程序的中间输入数据，继续运行第一程序。Exemplarily, the image processing result may be intermediate data during the running of the first program, that is, intermediate data required by the first program to continue running. At this time, if the first program runs on the microkernel in the trusted execution environment, the electronic device uses the image processing result as the intermediate input data of the first program through the microkernel, and continues to run the first program; On the main operating system in the execution environment, the electronic device transmits the image processing result to the main operating system through the microkernel; the main operating system uses the image processing result as the intermediate input data of the first program, and continues to run the first program.

比如，在注视不息屏场景下，电子设备通过微内核对人脸图像进行信息提取，得到用户行为信息(也即图像处理结果)；电子设备通过微内核用户行为信息传递至主操作***；在用户行为信息指示用户正在注视屏幕的情况下，电子设备通过主操作***控制电子设备不息屏。For example, in the scene of watching the screen continuously, the electronic device extracts information from the face image through the microkernel to obtain user behavior information (that is, the image processing result); the electronic device transmits the user behavior information to the main operating system through the microkernel; When the user behavior information indicates that the user is staring at the screen, the electronic device controls the electronic device to keep the screen off through the main operating system.

示例性的，图像处理结果可以是第一程序运行的最终结果。此时，若是第一程序运行在可信执行环境中的微内核上，电子设备可以通过微内核将图像处理结果反馈在第一程序的用户界面上；若是第一程序运行在通用执行环境中的主操作***上，电子设备通过微内核将图像处理结果传递至主操作***；通过主操作***将图像处理结果反馈在第一程序的用户界面上。Exemplarily, the image processing result may be the final result of running the first program. At this time, if the first program is running on the microkernel in the trusted execution environment, the electronic device can feed back the image processing result on the user interface of the first program through the microkernel; if the first program is running in the general execution environment On the main operating system, the electronic device transmits the image processing result to the main operating system through the microkernel; the image processing result is fed back to the user interface of the first program through the main operating system.

比如，在用户信息采集场景下，电子设备通过微内核对人脸图像进行信息提取，得到人脸信息；电子设备通过微内核将人脸信息存储至存储区域；通过微内核将人脸信息已存储的图像处理结果传递至主操作***；通过主操作***在第一程序的用户界面上显示人脸信息已存储的图像处理结果。For example, in the user information collection scenario, the electronic device extracts information from the face image through the microkernel to obtain face information; the electronic device stores the face information in the storage area through the microkernel; The image processing result is transmitted to the main operating system; the image processing result of the stored face information is displayed on the user interface of the first program through the main operating system.

比如，在电子设备的解锁场景下，电子设备通过微内核对人脸图像进行信息提取，得到第一人脸信息；通过微内核将第一人脸信息与可信执行环境中预存的第二人脸信息进行匹配；响应于第一人脸信息与第二人脸信息匹配，通过微内核将解锁成功的信息传递至主操作***；通过主操作***基于解锁成功的信息，显示***程序或者应用程序的用户界面。For example, in the unlocking scenario of an electronic device, the electronic device extracts information from the face image through the microkernel to obtain the first face information; Face information is matched; in response to the matching of the first face information and the second face information, the unlocking success information is transmitted to the main operating system through the microkernel; the system program or application program is displayed based on the unlocking success information through the main operating system user interface.

综上所述，本实施例提供的图像处理方法，为可信执行环境下的数据处理分配使用一个微内核，这个微内核是在电子设备中单独运行的；对于图像处理，首先获取许可信息，通过微内核基于许可信息进行鉴权，以验证第一程序是否具备图像传感器的信息采集权限，在确定第一程序具备上述信息采集权限之后，通过微内核对图像传感器采集的图像信息进行处理，得到图像处理结果。也就是说，上述图像信息处理时在可信执行环境中执行的步骤，均是由微内核来单独实现，这使得可信执行环境与电子设备中的***通路之间处于解耦状态，因此，在图像处理过程中无需再在***通路上的***内核、虚拟机、应用框架以及应用层的各个环节依次授权，来保证图像处理处于可信执行环境，大大降低了可信执行环境中图像处理的繁琐程度，提高了可信执行环境中图像处理的效率。To sum up, the image processing method provided in this embodiment allocates a microkernel for data processing in a trusted execution environment, and this microkernel runs independently in the electronic device; for image processing, firstly obtain the license information, The microkernel is used to perform authentication based on the license information to verify whether the first program has the information collection authority of the image sensor. After determining that the first program has the above information collection authority, the image information collected by the image sensor is processed through the microkernel to obtain Image processing results. That is to say, the steps performed in the trusted execution environment during the above image information processing are all independently implemented by the microkernel, which makes the trusted execution environment and the system path in the electronic device in a state of decoupling. Therefore, In the image processing process, there is no need to authorize the system kernel, virtual machine, application framework, and application layer in sequence on the system path to ensure that the image processing is in a trusted execution environment, which greatly reduces the burden of image processing in the trusted execution environment. cumbersomeness, improving the efficiency of image processing in trusted execution environments.

上述图像传感器包括AON摄像头；在电子设备处于开机状态的情况下，AON摄像头具备处于长期打开状态的特性。其中，AON摄像头可以用于采集环境图像以对程序运行的应用场景进行实时获取。上述环境图像是通过AON摄像头对电子设备所处环境采集得到的图像。在一些实施例中，电子设备可以通过AON摄像头来采集环境图像，基于环境图像来分析AON摄像头的应用场景，进而确定是否执行图3中所示的图像处理方法。示例性的，如图4，上述步骤310可以通过如下步骤312至步骤316来实现：The aforementioned image sensor includes an AON camera; when the electronic device is turned on, the AON camera has the characteristic of being in a long-term on state. Among them, the AON camera can be used to collect environmental images to obtain real-time application scenarios of program running. The foregoing environment image is an image collected by the AON camera from the environment where the electronic device is located. In some embodiments, the electronic device may collect an environment image through the AON camera, analyze an application scenario of the AON camera based on the environment image, and then determine whether to execute the image processing method shown in FIG. 3 . Exemplarily, as shown in FIG. 4, the above step 310 may be implemented through the following steps 312 to 316:

步骤312：通过微内核接收AON摄像头采集到的环境图像。Step 312: Receive the environmental image collected by the AON camera through the microkernel.

电子设备上的AON摄像头在采集到环境图像之后，将环境图像发送至微内核，以基于环境图像进行AON摄像头的应用场景分析。可选地，AON摄像头为前置摄像头。After the AON camera on the electronic device collects the environment image, it sends the environment image to the microkernel, so as to analyze the application scene of the AON camera based on the environment image. Optionally, the AON camera is a front camera.

示例性的，电子设备通过微内核控制前置摄像头按照预设周期采集环境图像，以进行前置摄像头的应用场景分析。示例性的，上述预设周期可以根据电子设备的工作状态切换。比如，在电子设备处于息屏状态下，电子设备通过微内核控制前置摄像头按照第一预设周期采集环境图像；在电子设备处于亮屏状态下，电子设备通过微内核控制前置摄像头按照第二预设周期采集环境图像；其中，第一预设周期与第二预设周期的周期长度不同。又比如，在电子设备处于亮屏状态下，电子设备在存在用户操作的情况下，通过微内核控制前置摄像头按照第三预设周期采集环境图像；在指定时间段内不存在用户操作的情况下，通过微内核控制前置摄像头按照第四预设周期采集环境图像；其中，第三预设周期与第四预设周期的周期长度不同。Exemplarily, the electronic device controls the front camera through the microkernel to collect environmental images according to a preset period, so as to analyze the application scene of the front camera. Exemplarily, the aforementioned preset period may be switched according to the working state of the electronic device. For example, when the electronic device is in the off-screen state, the electronic device controls the front camera through the microkernel to collect environmental images according to the first preset cycle; The environment image is collected in two preset periods; wherein, the period lengths of the first preset period and the second preset period are different. For another example, when the electronic device is in the bright screen state, the electronic device controls the front camera through the microkernel to collect environmental images according to the third preset cycle when there is user operation; there is no user operation within the specified time period Next, the microkernel is used to control the front camera to collect environmental images according to a fourth preset period; wherein, the third preset period and the fourth preset period have different period lengths.

示例性的，在电子设备上前置摄像头的相邻位置设置有接近传感器；电子设备通过微内核控制接近传感器对接近物体的检测；响应于接近传感器检测到用户与电子设备之间的距离小于距离阈值，通过微内核调用前置摄像头对环境图像进行采集。Exemplarily, a proximity sensor is disposed adjacent to the front camera on the electronic device; the electronic device controls the proximity sensor to detect an approaching object through a microkernel; in response to the proximity sensor detecting that the distance between the user and the electronic device is less than the distance Threshold, through the microkernel to call the front camera to collect the environment image.

步骤314：通过微内核基于环境图像进行程序运行的应用场景分析。Step 314: Analyzing the application scenario of the program running based on the environment image through the microkernel.

在本申请实施例中，对程序运行的应用场景分析也即是对AON摄像头的应用场景分析。电子设备通过微内核对环境图像进行分析，分析AON摄像头的应用场景。示例性的，电子设备通过微内核提取环境图像中的特征信息，该特征信息用于指示AON摄像头的应用场景。比如，该特征信息为手势信息，则该特征信息指示AON摄像头用于采集手势信息以触发特定功能的应用场景。又比如，该特征信息为人脸信息，则该特征信息指示AON摄像头用于采集人脸信息以进行设备解锁或者功能解锁的应用场景。In the embodiment of the present application, the analysis of the application scene of the program running is also the analysis of the application scene of the AON camera. The electronic device analyzes the environmental image through the microkernel, and analyzes the application scenarios of the AON camera. Exemplarily, the electronic device extracts feature information in the environment image through the microkernel, and the feature information is used to indicate an application scenario of the AON camera. For example, if the feature information is gesture information, the feature information indicates an application scenario where the AON camera is used to collect gesture information to trigger a specific function. For another example, if the feature information is face information, the feature information indicates an application scenario where the AON camera is used to collect face information for device unlocking or function unlocking.

示例性的，电子设备还可以通过微内核调用人工智能(Artificial Intelligence，AI)分析模型对环境图像进行应用场景分析。Exemplarily, the electronic device may also invoke an artificial intelligence (Artificial Intelligence, AI) analysis model to perform application scene analysis on the environment image through the microkernel.

步骤316：响应于应用场景为第一应用场景，通过微内核获取第一程序的许可信息；第一应用场景包括第一程序调用AON摄像头进行信息采集的场景。Step 316: In response to the fact that the application scenario is the first application scenario, obtain the license information of the first program through the microkernel; the first application scenario includes a scenario where the first program invokes the AON camera to collect information.

电子设备确定AON摄像头的应用场景为第一应用场景，则执行通过微内核获取第一程序的许可信息的步骤。The electronic device determines that the application scenario of the AON camera is the first application scenario, and then executes the step of obtaining the license information of the first program through the microkernel.

示例性的，在应用场景为第一应用场景的情况下，电子设备通过微内核基于许可信息对应的关键字从第一程序中获取许可信息。Exemplarily, when the application scenario is the first application scenario, the electronic device obtains the license information from the first program based on the keyword corresponding to the license information through the microkernel.

在另一些实施例中，上述步骤312至步骤316可以由主操作***来实现。示例性的，电子设备通过主操作***接收AON摄像头采集到的环境图像；通过主操作***基于环境图像进行AON摄像头的应用场景分析；响应于应用场景为第一应用场景，通过主操作***获取第一程序的许可信息。In some other embodiments, the above step 312 to step 316 may be implemented by the main operating system. Exemplarily, the electronic device receives the environmental image collected by the AON camera through the main operating system; analyzes the application scene of the AON camera based on the environmental image through the main operating system; in response to the application scene being the first application scene, obtains the second application scene through the main operating system. License information for a program.

在另一些实施例中，在电子设备上第一应用场景的触发，还可以是由用户操作来实现的，示例性的，电子设备通过主操作***(或者微内核)响应触控屏上对信息采集功能(第一程序中的部分程序功能)的触发操作，执行信息采集程序，通过主操作***(或者微内核)获取第一程序的许可信息，以验证第一程序是否具备AON摄像头的信息采集权限。In some other embodiments, the triggering of the first application scene on the electronic device can also be realized by user operation. Exemplarily, the electronic device responds to the information on the touch screen through the main operating system (or microkernel) The trigger operation of the collection function (part of the program function in the first program), execute the information collection program, obtain the license information of the first program through the main operating system (or microkernel), to verify whether the first program has the information collection of the AON camera authority.

需要说明的是，若第一程序的许可信息由主操作***获取得到，则之后主操作***将许可信息传递至微内核。It should be noted that, if the license information of the first program is obtained by the main operating system, then the main operating system transfers the license information to the microkernel.

综上所述，本申请实施例提供的图像处理方法，是基于环境图像进行应用场景分析，在第一程序调用AON摄像头进行信息采集的场景下，对第一程序是否具备AON摄像头的信息采集权限进行验证，进而实现图像处理，这一系列过程可以在微内核中完成，保证了图像处理的安全性。To sum up, the image processing method provided by the embodiment of the present application is based on the analysis of the application scene based on the environmental image. In the scene where the first program calls the AON camera for information collection, whether the first program has the information collection authority of the AON camera To verify and then realize image processing, this series of processes can be completed in the microkernel, which ensures the security of image processing.

示例性的，验证第一程序具备图像传感器设备的信息采集权限的方式可以包括但不限于以下两种：Exemplarily, the manner of verifying that the first program has the information collection authority of the image sensor device may include but not limited to the following two:

一、采用数字签名的鉴权方式1. Authentication method using digital signature

如图5，采用数字签名的鉴权方式时，图3中步骤320可以通过步骤321至步骤322实现，如下所示：As shown in Figure 5, when the authentication method of digital signature is adopted, step 320 in Figure 3 can be realized through steps 321 to 322, as shown below:

步骤321：通过微内核对公钥与私钥进行配对。Step 321: pair the public key and the private key through the microkernel.

许可信息包括公钥，对应的，验证信息包括私钥。在可信执行环境中存储区域上公钥，公钥与私钥之间存在一一对应关系；在主操作***上运行第一程序的情况下，电子设备通过主操作***将公钥发送至微内核，通过微内核对公钥和私钥进行配对。The license information includes a public key, and correspondingly, the verification information includes a private key. There is a one-to-one correspondence between the public key and the private key on the storage area in the trusted execution environment; in the case of running the first program on the main operating system, the electronic device sends the public key to the microcomputer through the main operating system. Kernel, through the microkernel to pair the public key and private key.

在一些实施例中，公钥和私钥的配对方式可以是解密的方式。示例性的，上述许可信息可以是公钥；电子设备通过微内核基于私钥对信息加密，生成数字签名(signature)；通过微内核采用公钥对数字签名进行解密，得到解密信息；通过微内核对解密信息与信息进行对比，响应于解密信息与信息相同，确定公钥与私钥匹配。In some embodiments, the pairing method of the public key and the private key may be a decryption method. Exemplarily, the above license information may be a public key; the electronic device encrypts the information based on the private key through the microkernel to generate a digital signature (signature); uses the public key to decrypt the digital signature through the microkernel to obtain decrypted information; The decrypted information is compared to the information, and in response to the decrypted information being identical to the information, it is determined that the public key matches the private key.

示例性的，上述许可信息中包括公钥；电子设备通过微内核采用公钥对信息进行加密，得到公钥的加密信息，然后采用私钥对公钥的加密信息进行解密；通过微内核采用私钥对信息进行加密，得到私钥的加密信息，然后采用公钥对私钥的加密信息进行解密；在私钥成功解密公钥的加密文件、且公钥成功解密私钥的加密文件的情况下，通过微内核确定公钥与私钥匹配。需要说明的是，本实施例中对上述私钥和公钥的验证步骤的执行顺序不加以限定。Exemplarily, the above license information includes a public key; the electronic device encrypts the information with the public key through the microkernel to obtain the encrypted information of the public key, and then uses the private key to decrypt the encrypted information of the public key; Encrypt the information with the key to obtain the encrypted information of the private key, and then use the public key to decrypt the encrypted information of the private key; when the private key successfully decrypts the encrypted file of the public key, and the public key successfully decrypts the encrypted file of the private key , the microkernel determines that the public key matches the private key. It should be noted that, in this embodiment, there is no limitation on the execution order of the verification steps of the above-mentioned private key and public key.

在另一些实施例中，上述许可信息中还可以包括公钥的加密信息；电子设备通过主操作***将公钥的加密信息发送至微内核；通过微内核采用私钥对公钥的加密信息进行解密；响应于私钥对公钥的加密信息成功解密，通过微内核确定公钥与私钥匹配。In some other embodiments, the above license information may also include encrypted information of the public key; the electronic device sends the encrypted information of the public key to the microkernel through the main operating system; Decryption; in response to the private key successfully decrypting the encrypted information of the public key, determining, by the microkernel, that the public key matches the private key.

步骤322：响应于公钥与私钥匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果。Step 322: In response to the matching of the public key and the private key, obtain an authentication result that the first program has the information collection authority of the image sensor through the microkernel.

在另一些实施例中，私钥和公钥的匹配还可以通过主操作***与微内核之间的交互实现，示例性的，电子设备通过主操作***基于公钥对信息进行加密，得到公钥的加密信息，将公钥的加密信息传递至微内核；通过微内核基于私钥对公钥的加密信息进行解密，响应于对公钥的加密信息成功解密，生成第一反馈信息，基于私钥对第一反馈信息加密，采用数字签名的方式生成私钥的加密信息，将私钥的加密信息发送至主操作***；通过主操作***基于公钥对私钥的加密信息进行解密，在对私钥的加密信息成功解密的情况下，得到第一反馈信息。示例性的，上述第一反馈信息用于指示第一程序具备AON摄像头的图像采集权限。对于第一反馈信息采用私钥进行加密，使得主操作***在具备公钥的情况下，才能够获得鉴权成功的反馈信息，可以对私钥与公钥进行再次匹配验证，保证第一程序真实具备AON摄像头的信息采集权限。In other embodiments, the matching of the private key and the public key can also be realized through the interaction between the main operating system and the microkernel. Exemplarily, the electronic device encrypts the information based on the public key through the main operating system to obtain the public key The encrypted information of the public key is passed to the microkernel; the encrypted information of the public key is decrypted based on the private key through the microkernel, and the first feedback information is generated in response to the successful decryption of the encrypted information of the public key, based on the private key Encrypt the first feedback information, generate encrypted information of the private key by digital signature, and send the encrypted information of the private key to the main operating system; decrypt the encrypted information of the private key based on the public key through the main operating system, and If the encrypted information of the key is successfully decrypted, the first feedback information is obtained. Exemplarily, the above-mentioned first feedback information is used to indicate that the first program has the image acquisition permission of the AON camera. The first feedback information is encrypted with the private key, so that the main operating system can obtain the feedback information of successful authentication only when it has the public key, and can verify the matching of the private key and the public key again to ensure the authenticity of the first program. Possess the information collection authority of the AON camera.

示例性的，上述公钥的加密信息还可以是对许可证的加密，电子设备还可以在通过微内核对公钥与私钥匹配的同时，对许可证进行验证，从而实现对第一程序具备图像传感器的信息采集权限的双重验证。Exemplarily, the encryption information of the above public key may also be the encryption of the license, and the electronic device may also verify the license while matching the public key and the private key through the microkernel, so that the first program has Double verification of the information collection authority of the image sensor.

二、采用指纹匹配的鉴权方式Second, the authentication method using fingerprint matching

如图6，采用指纹匹配的鉴权方式时，图3中步骤310可以通过步骤318实现，步骤320可以通过步骤323至步骤325实现，如下所示：As shown in Figure 6, when the authentication mode of fingerprint matching is adopted, step 310 in Figure 3 can be realized through step 318, and step 320 can be realized through steps 323 to 325, as shown below:

步骤318：响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核调用指纹传感器进行第一指纹信息的采集。Step 318: In response to the call request of the first program to the image sensor during image information collection, the microkernel calls the fingerprint sensor to collect the first fingerprint information.

在第一程序请求调用图像传感器进行图像信息采集的情况下，电子设备通过微内核确定第一程序具备指纹传感器的调用权限，则调用指纹传感器进行第一指纹信息的采集。When the first program requests to call the image sensor to collect image information, the electronic device determines through the microkernel that the first program has the calling authority of the fingerprint sensor, and then calls the fingerprint sensor to collect the first fingerprint information.

示例性的，在第一程序运行于主操作***上的情况下，电子设备通过主操作***将第一程序的对指纹传感器的调用许可信息传递至微内核；通过微内核基于调用许可信息进行鉴权，得到第一程序具备指纹传感器的调用权限。Exemplarily, in the case that the first program runs on the main operating system, the electronic device transmits the calling permission information of the first program to the fingerprint sensor to the microkernel through the main operating system; the microkernel performs authentication based on the calling permission information The first program has the calling authority of the fingerprint sensor.

步骤323：通过微内核计算第一指纹信息与第二指纹信息的相似度。Step 323: Calculate the similarity between the first fingerprint information and the second fingerprint information through the microkernel.

可信执行环境中存储有电子设备的拥有者的第一指纹信息；电子设备通过微内核计算第一指纹信息与第二指纹信息的相似度。The first fingerprint information of the owner of the electronic device is stored in the trusted execution environment; the electronic device calculates the similarity between the first fingerprint information and the second fingerprint information through the microkernel.

步骤324：响应于相似度大于或者等于预设相似度，通过微内核确定第一指纹信息与第二指纹信息匹配。Step 324: In response to the similarity being greater than or equal to the preset similarity, the microkernel determines that the first fingerprint information matches the second fingerprint information.

电子设备通过微内核确定第一指纹信息与第二指纹信息匹配，即是确定许可信息与验证信息匹配。The electronic device determines that the first fingerprint information matches the second fingerprint information through the microkernel, that is, determines that the license information matches the verification information.

步骤325：响应于第一指纹信息与第二指纹信息匹配，通过微内核得到第一程序具备图像传感器的信息采集权限的鉴定结果。Step 325: In response to the match between the first fingerprint information and the second fingerprint information, obtain an authentication result that the first program has the information collection authority of the image sensor through the microkernel.

综上所述，本实施例提供的图像处理方法，可以通过数字签名或者指纹识别的方式来实现对第一程序具备图像传感器的信息采集权限的鉴定，还能够保证鉴权的准确性。To sum up, the image processing method provided by this embodiment can realize the identification that the first program has the information collection authority of the image sensor through digital signature or fingerprint identification, and can also ensure the accuracy of the authentication.

在可信执行环境中设置有图像传感器对应使用的图像存储器，电子设备上运行的主操作***上包括图像传感器的子***，子***具备向图像存储器写入图像信息的权限；电子设备通过图像传感器采集得到图像信息之后，通过子***将图像信息存储至可信执行环境中的图像存储器中，通过微内核从图像存储器中读取图像信息。示例性的，上述子***与图像存储器之间通过摄像头接口连接；如图7，在本申请的实施例中，电子设备为图像处理提供了一条安全通路，电子设备上运行有图像传感器410的子***420，电子设备调用图像传感器410采集得到图像信息之后，由子***420通过摄像头接口将图像信息存储至图像存储器430中，以方便图像处理时微内核对图像信息的获取。这一安全通路保证了图像信息的安全性。The image memory corresponding to the image sensor is set in the trusted execution environment. The main operating system running on the electronic device includes a subsystem of the image sensor, and the subsystem has the authority to write image information to the image memory; the electronic device passes the image sensor After the image information is collected, the image information is stored in the image memory in the trusted execution environment through the subsystem, and the image information is read from the image memory through the microkernel. Exemplarily, the above-mentioned subsystem is connected to the image memory through a camera interface; as shown in FIG. In the system 420, after the electronic device invokes the image sensor 410 to collect the image information, the subsystem 420 stores the image information in the image memory 430 through the camera interface, so as to facilitate the acquisition of the image information by the microkernel during image processing. This secure channel ensures the security of the image information.

请参考图8，其示出了本申请一个示例性实施例提供的图像处理装置的结构框图。该装置支持可信执行环境，可信执行环境中存在独立运行的微内核。该图像处理装置可以通过软件、硬件或者两者的结合实现成为电子设备的全部或一部分。该装置包括：Please refer to FIG. 8 , which shows a structural block diagram of an image processing apparatus provided by an exemplary embodiment of the present application. The device supports a trusted execution environment, and there is an independently running microkernel in the trusted execution environment. The image processing apparatus can be implemented as all or a part of electronic equipment through software, hardware or a combination of the two. The unit includes:

获取模块510，用于响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核获取第一程序的许可信息；An acquisition module 510, configured to acquire the license information of the first program through the microkernel in response to the first program's call request to the image sensor during the image information collection process;

鉴权模块520，用于通过微内核对许可信息进行鉴权，得到鉴定结果，鉴定结果包括第一程序具备或者不具备图像传感器的信息采集权限；An authentication module 520, configured to authenticate the license information through the microkernel to obtain an authentication result, the authentication result including whether the first program has or does not have the information collection authority of the image sensor;

采集模块530，用于响应于第一程序具备图像传感器的信息采集权限，通过微内核调用图像传感器进行图像信息的采集；The collection module 530 is configured to call the image sensor through the microkernel to collect image information in response to the first program possessing the information collection authority of the image sensor;

处理模块540，用于通过微内核对图像信息进行处理，得到图像处理结果，图像处理结果用于对第一程序的运行进程进行指示。The processing module 540 is configured to process the image information through the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.

在一些实施例中，电子设备上与微内核并行运行有主操作***；获取模块510，用于：In some embodiments, there is a main operating system running in parallel with the microkernel on the electronic device; the acquiring module 510 is configured to:

在一些实施例中，获取模块510，用于：In some embodiments, the acquisition module 510 is configured to:

在一些实施例中，图像传感器包括处于长期打开状态的前置摄像头，前置摄像头用于采集环境图像以对程序运行的应用场景进行实时获取；获取模块510，用于：In some embodiments, the image sensor includes a front-facing camera in a long-term open state, and the front-facing camera is used to collect environmental images to obtain real-time application scenarios of program running; the acquisition module 510 is configured to:

响应于应用场景为第一应用场景，通过微内核获取第一程序的许可信息；第一应用场景为第一程序调用前置摄像头进行图像信息采集的场景。In response to the fact that the application scenario is the first application scenario, the license information of the first program is obtained through the microkernel; the first application scenario is a scenario in which the first program calls the front camera to collect image information.

在一些实施例中，可信执行环境中预存有验证信息；鉴权模块520，用于：In some embodiments, verification information is pre-stored in the trusted execution environment; the authentication module 520 is configured to:

在一些实施例中，许可信息包括公钥，验证信息包括私钥；鉴权模块520，用于：In some embodiments, the license information includes a public key, and the verification information includes a private key; the authentication module 520 is configured to:

在一些实施例中，许可信息包括第一指纹信息，验证信息包括可信执行环境中预存的第二指纹信息；In some embodiments, the permission information includes first fingerprint information, and the verification information includes second fingerprint information pre-stored in the trusted execution environment;

获取模块510，用于响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过微内核调用指纹传感器进行第一指纹信息的采集；The acquiring module 510 is configured to call the fingerprint sensor through the microkernel to collect the first fingerprint information in response to the call request of the first program to the image sensor during the image information collection process;

鉴权模块520，用于通过微内核计算第一指纹信息与第二指纹信息的相似度；响应于相似度大于预设相似度，通过微内核确定第一指纹信息与第二指纹信息匹配。The authentication module 520 is configured to calculate the similarity between the first fingerprint information and the second fingerprint information through the microkernel; in response to the similarity being greater than a preset similarity, determine that the first fingerprint information matches the second fingerprint information through the microkernel.

在一些实施例中，电子设备上与微内核并行运行有主操作***；该装置还包括存储模块550；In some embodiments, a main operating system runs in parallel with the microkernel on the electronic device; the device also includes a storage module 550;

存储模块550，用于通过主操作***将图像信息存储至可信执行环境中的图像存储器中；The storage module 550 is used to store the image information in the image memory in the trusted execution environment through the main operating system;

获取模块510，用于通过微内核从图像存储器中读取图像信息。An acquisition module 510, configured to read image information from the image memory through the microkernel.

图9示出了本申请一个示例性实施例提供的计算机设备的结构示意图。该计算机设备可以是执行如本申请提供的图像处理方法的设备，该计算机设备可以是电子设备或者终端。具体来讲：Fig. 9 shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present application. The computer device may be a device for executing the image processing method provided in this application, and the computer device may be an electronic device or a terminal. Specifically:

计算机设备600包括中央处理单元(CPU，Central Processing Unit)601、包括随机存取存储器(RAM，Random Access Memory)602和只读存储器(ROM，Read Only Memory)603的***存储器604，以及连接***存储器604和中央处理单元601的***总线605。计算机设备600还包括帮助计算机内的各个器件之间传输信息的基本输入/输出***(I/O***，Input Output System)606，和用于存储操作***613、应用程序614和其他程序模块615的大容量存储设备607。The computer device 600 includes a central processing unit (CPU, Central Processing Unit) 601, a system memory 604 including a random access memory (RAM, Random Access Memory) 602 and a read-only memory (ROM, Read Only Memory) 603, and a connection system memory 604 and the system bus 605 of the central processing unit 601. The computer device 600 also includes a basic input/output system (I/O system, Input Output System) 606 that helps to transmit information between various devices in the computer, and is used to store an operating system 613, application programs 614 and other program modules 615 mass storage device 607 .

基本输入/输出***606包括有用于显示信息的显示器608和用于用户输入信息的诸如鼠标、键盘之类的输入设备609。其中显示器608和输入设备609都通过连接到***总线605的输入输出控制器610连接到中央处理单元601。基本输入/输出***606还可以包括输入输出控制器610以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入输出控制器610还提供输出到显示屏、打印机或其他类型的输出设备。The basic input/output system 606 includes a display 608 for displaying information and input devices 609 such as a mouse and a keyboard for user input of information. Both the display 608 and the input device 609 are connected to the central processing unit 601 through the input and output controller 610 connected to the system bus 605 . The basic input/output system 606 may also include an input output controller 610 for receiving and processing input from a number of other devices such as a keyboard, mouse, or electronic stylus. Similarly, input output controller 610 also provides output to a display screen, printer, or other type of output device.

大容量存储设备607通过连接到***总线605的大容量存储控制器(未示出)连接到中央处理单元601。大容量存储设备607及其相关联的计算机可读介质为计算机设备600提供非易失性存储。也就是说，大容量存储设备607可以包括诸如硬盘或者紧凑型光盘只读存储器(CD-ROM，Compact Disc Read Only Memory)驱动器之类的计算机可读介质(未示出)。 Mass storage device 607 is connected to central processing unit 601 through a mass storage controller (not shown) connected to system bus 605 . Mass storage device 607 and its associated computer-readable media provide non-volatile storage for computer device 600 . That is, the mass storage device 607 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read Only Memory (CD-ROM, Compact Disc Read Only Memory) drive.

计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、可擦除可编程只读存储器(EPROM，Erasable Programmable Read Only Memory)、带电可擦可编程只读存储器(EEPROM,Electrically Erasable Programmable Read Only Memory)、闪存或其他固态存储其技术，CD-ROM、数字通用光盘(DVD，Digital Versatile Disc)或固态硬盘(SSD，Solid State Drives)、其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。其中，随机存取记忆体可以包括电阻式随机存取记忆体(ReRAM，Resistance Random Access Memory)和动态随机存取存储器(DRAM，Dynamic Random Access Memory)。当然，本领域技术人员可知计算机存储介质不局限于上述几种。上述的***存储器604和大容量存储设备607可以统称为存储器。Computer readable media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage Its technology, CD-ROM, Digital Versatile Disc (DVD, Digital Versatile Disc) or Solid State Drives (SSD, Solid State Drives), other optical storage, tape cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices. Wherein, random access memory may include resistive random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory). Certainly, those skilled in the art know that the computer storage medium is not limited to the above-mentioned ones. The aforementioned system memory 604 and mass storage device 607 may be collectively referred to as memory.

根据本申请的各种实施例，计算机设备600还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即计算机设备600可以通过连接在***总线605上的网络接口单元611连接到网络612，或者说，也可以使用网络接口单元611来连接到其他类型的网络或远程计算机***(未示出)。According to various embodiments of the present application, computer device 600 may also operate on a remote computer connected to a network through a network such as the Internet. That is, the computer device 600 can be connected to the network 612 through the network interface unit 611 connected to the system bus 605, or in other words, the network interface unit 611 can also be used to connect to other types of networks or remote computer systems (not shown).

上述存储器还包括一个或者一个以上的程序，一个或者一个以上程序存储于存储器中，被配置由CPU执行，以实现如上所述的图像处理方法。The above-mentioned memory also includes one or more programs, one or more programs are stored in the memory and configured to be executed by the CPU to implement the above-mentioned image processing method.

本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质存储有至少一条指令，所述至少一条指令由处理器加载并执行以实现如上各个实施例所述的图像处理方法。The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the image processing method described in each of the above embodiments .

可选地，该计算机可读存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、固态硬盘(SSD，Solid State Drives)或光盘等。其中，随机存取记忆体可以包括电阻式随机存取记忆体(ReRAM，Resistance Random Access Memory)和动态随机存取存储器(DRAM，Dynamic Random Access Memory)。Optionally, the computer-readable storage medium may include: a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a solid-state hard drive (SSD, Solid State Drives) or an optical disc, etc. Wherein, random access memory may include resistive random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).

本申请实施例还提供了一种计算机程序产品(或计算机程序)，该计算机程序产品(或计算机程序)包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取计算机指令，该处理器执行计算机指令，使得计算机设备执行上述图像处理方法的各种可选实现方式中提供的方法。The embodiment of the present application also provides a computer program product (or computer program), where the computer program product (or computer program) includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementation manners of the above-mentioned image processing method.

Claims

一种图像处理方法，应用于电子设备中，所述电子设备支持可信执行环境，所述可信执行环境中存在独立运行的微内核，所述方法包括：An image processing method, applied to an electronic device, the electronic device supports a trusted execution environment, and there is an independently running microkernel in the trusted execution environment, the method comprising:

响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息；Responding to the call request of the first program to the image sensor during the image information collection process, acquiring the license information of the first program through the microkernel;

通过所述微内核对所述许可信息进行鉴权，得到鉴定结果，所述鉴定结果包括所述第一程序具备或者不具备所述图像传感器的信息采集权限；Authenticating the license information through the microkernel to obtain an authentication result, the authentication result including whether the first program has or does not have the information collection authority of the image sensor;

响应于所述第一程序具备所述图像传感器的信息采集权限，通过所述微内核调用所述图像传感器进行图像信息的采集；In response to the first program possessing the information collection authority of the image sensor, calling the image sensor through the microkernel to collect image information;

通过所述微内核对所述图像信息进行处理，得到图像处理结果，所述图像处理结果用于对所述第一程序的运行进程进行指示。The image information is processed by the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.
根据权利要求1所述的方法，所述电子设备上与所述微内核并行运行有主操作***；According to the method according to claim 1, a main operating system runs in parallel with the microkernel on the electronic device;

所述通过所述微内核获取所述第一程序的许可信息，包括：The acquiring the license information of the first program through the microkernel includes:

通过所述主操作***获取所述许可信息；Obtain the license information through the main operating system;

通过所述微内核从所述主操作***中获取所述许可信息。The license information is obtained from the main operating system through the microkernel.
根据权利要求2所述的方法，所述通过所述微内核从所述主操作***中获取所述许可信息，包括：The method according to claim 2, said obtaining said license information from said main operating system through said microkernel, comprising:

通过所述微内核的第一硬件抽象层接收所述主操作***的第二硬件抽象层传递的所述许可信息。The license information delivered by the second hardware abstraction layer of the main operating system is received by the first hardware abstraction layer of the microkernel.
根据权利要求1至3任一所述的方法，所述图像传感器包括处于长期打开状态的前置摄像头，所述前置摄像头用于采集环境图像以对程序运行的应用场景进行实时获取；According to the method according to any one of claims 1 to 3, the image sensor includes a front-facing camera in a long-term open state, and the front-facing camera is used to collect environmental images to obtain real-time application scenarios of program running;

所述响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息，包括：The obtaining the license information of the first program through the microkernel in response to the call request of the first program to the image sensor during the image information collection process includes:

通过所述微内核接收所述前置摄像头采集到的环境图像；receiving the environmental image collected by the front camera through the microkernel;

通过所述微内核基于所述环境图像进行程序运行的应用场景分析；Analyzing application scenarios of program execution based on the environment image through the microkernel;

响应于所述应用场景为第一应用场景，通过所述微内核获取所述第一程序的许可信息；所述第一应用场景包括所述第一程序调用所述前置摄像头进行图像信息采集的场景。Responding to the fact that the application scenario is a first application scenario, obtaining license information of the first program through the microkernel; the first application scenario includes that the first program invokes the front camera to collect image information Scenes.
根据权利要求1至3任一所述的方法，所述可信执行环境中预存有验证信息；According to the method according to any one of claims 1 to 3, verification information is pre-stored in the trusted execution environment;

所述通过所述微内核对所述许可信息进行鉴权，得到鉴定结果，包括：The authentication of the license information through the microkernel to obtain an authentication result includes:

通过所述微内核对所述许可信息与所述验证信息进行匹配；matching the license information with the verification information through the microkernel;

响应于所述许可信息与所述验证信息匹配，通过所述微内核得到所述第一程序具备所述图像传感器的信息采集权限的鉴定结果。In response to the match between the permission information and the verification information, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.
根据权利要求5所述的方法，所述许可信息包括公钥，所述验证信息包括私钥；The method according to claim 5, the license information includes a public key, and the verification information includes a private key;

所述通过所述微内核对所述许可信息与所述验证信息进行匹配，包括：The matching of the license information and the verification information through the microkernel includes:

通过所述微内核对所述公钥与所述私钥进行配对；pairing the public key with the private key through the microkernel;

所述响应于所述许可信息与所述验证信息匹配，通过所述微内核得到所述第一程序具备所述图像传感器的信息采集权限的鉴定结果，包括：In response to the match between the permission information and the verification information, obtaining the identification result that the first program has the information collection authority of the image sensor through the microkernel includes:

响应于所述公钥与所述私钥匹配，通过所述微内核得到所述第一程序具备所述图像传感器的信息采集权限的鉴定结果。In response to the match between the public key and the private key, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.
根据权利要求5所述的方法，所述许可信息包括第一指纹信息，所述验证信息包括所述可信执行环境中预存的第二指纹信息；The method according to claim 5, wherein the permission information includes first fingerprint information, and the verification information includes second fingerprint information pre-stored in the trusted execution environment;

所述响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息，包括：The obtaining the license information of the first program through the microkernel in response to the call request of the first program to the image sensor during the image information collection process includes:

响应于所述第一程序在图像信息采集过程中对所述图像传感器的调用请求，通过所述微内核调用指纹传感器进行所述第一指纹信息的采集；In response to the call request of the first program to the image sensor during the image information collection process, the microkernel calls the fingerprint sensor to collect the first fingerprint information;

所述通过所述微内核对所述许可信息与所述验证信息进行匹配，包括：The matching of the license information and the verification information through the microkernel includes:

通过所述微内核计算所述第一指纹信息与所述第二指纹信息的相似度；calculating the similarity between the first fingerprint information and the second fingerprint information by the microkernel;

响应于所述相似度大于预设相似度，通过所述微内核确定所述第一指纹信息与所述第二指纹信息匹配。In response to the similarity being greater than a preset similarity, the microkernel determines that the first fingerprint information matches the second fingerprint information.
根据权利要求1至3任一所述的方法，所述电子设备上与所述微内核并行运行有主操作***；According to the method described in any one of claims 1 to 3, a main operating system runs in parallel with the microkernel on the electronic device;

所述通过所述微内核调用所述图像传感器进行图像信息的采集之后，包括：After the microkernel is used to call the image sensor to collect image information, it includes:

通过所述主操作***将所述图像信息存储至所述可信执行环境中的图像存储器中；storing the image information in an image memory in the trusted execution environment through the host operating system;

通过所述微内核从所述图像存储器中读取所述图像信息。The image information is read from the image memory through the microkernel.
根据权利要求1至3任一所述的方法，所述第一程序包括***程序和第三方应用程序中的至少一种。According to the method according to any one of claims 1 to 3, the first program includes at least one of a system program and a third-party application program.
一种图像处理装置，所述装置支持可信执行环境，所述可信执行环境中存在独立运行的微内核，所述装置包括：An image processing device, the device supports a trusted execution environment, and there is an independently running microkernel in the trusted execution environment, and the device includes:

获取模块，用于响应于第一程序在图像信息采集过程中对图像传感器的调用请求，通过所述微内核获取所述第一程序的许可信息；An acquisition module, configured to acquire the license information of the first program through the microkernel in response to the call request of the first program to the image sensor during the image information acquisition process;

鉴权模块，用于通过所述微内核对所述许可信息进行鉴权，得到鉴定结果，所述鉴定结果包括所述第一程序具备或者不具备所述图像传感器的信息采集权限；An authentication module, configured to authenticate the license information through the microkernel to obtain an authentication result, the authentication result including whether the first program has or does not have the information collection authority of the image sensor;

采集模块，用于响应于所述第一程序具备所述图像传感器的信息采集权限，通过所述微内核调用所述图像传感器进行图像信息的采集；An acquisition module, configured to call the image sensor through the microkernel to acquire image information in response to the first program possessing the information acquisition authority of the image sensor;

处理模块，用于通过所述微内核对所述图像信息进行处理，得到图像处理结果，所述图像处理结果用于对所述第一程序的运行进程进行指示。The processing module is configured to process the image information through the microkernel to obtain an image processing result, and the image processing result is used to indicate the running process of the first program.
根据权利要求10所述的装置，所述电子设备上与所述微内核并行运行有主操作***；The device according to claim 10, a main operating system runs in parallel with the microkernel on the electronic device;

所述获取模块，用于：The acquisition module is used for:

通过所述主操作***获取所述许可信息；Obtain the license information through the main operating system;

通过所述微内核从所述主操作***中获取所述许可信息。The license information is obtained from the main operating system through the microkernel.
根据权利要求11所述的装置，所述获取模块，用于：The device according to claim 11, the acquisition module is configured to:

通过所述微内核的第一硬件抽象层接收所述主操作***的第二硬件抽象层传递的所述许可信息。The license information delivered by the second hardware abstraction layer of the main operating system is received by the first hardware abstraction layer of the microkernel.
根据权利要求10至12任一所述的装置，所述图像传感器包括处于长期打开状态的前置摄像头，所述前置摄像头用于采集环境图像以对程序运行的应用场景进行实时获取；According to the device according to any one of claims 10 to 12, the image sensor includes a front-facing camera in a long-term open state, and the front-facing camera is used to collect environmental images to obtain real-time application scenarios of program running;

所述获取模块，用于：The acquisition module is used for:

通过所述微内核接收所述前置摄像头采集到的环境图像；receiving the environmental image collected by the front camera through the microkernel;

通过所述微内核基于所述环境图像进行程序运行的应用场景分析；Analyzing application scenarios of program execution based on the environment image through the microkernel;

响应于所述应用场景为第一应用场景，通过所述微内核获取所述第一程序的许可信息；所述第一应用场景包括所述第一程序调用所述前置摄像头进行图像信息采集的场景。Responding to the fact that the application scenario is a first application scenario, obtaining license information of the first program through the microkernel; the first application scenario includes that the first program invokes the front camera to collect image information Scenes.
根据权利要求10至12任一所述的装置，所述可信执行环境中预存有验证信息；According to the device according to any one of claims 10 to 12, verification information is pre-stored in the trusted execution environment;

所述鉴权模块，用于：The authentication module is used for:

通过所述微内核对所述许可信息与所述验证信息进行匹配；matching the license information with the verification information through the microkernel;

响应于所述许可信息与所述验证信息匹配，通过所述微内核得到所述第一程序具备所述图像传感器的信息采集权限的鉴定结果。In response to the match between the permission information and the verification information, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.
根据权利要求14所述的装置，所述许可信息包括公钥，所述验证信息包括私钥；The apparatus of claim 14, the license information includes a public key, and the verification information includes a private key;

所述鉴权模块，用于：The authentication module is used for:

通过所述微内核对所述公钥与所述私钥进行配对；pairing the public key with the private key through the microkernel;

响应于所述公钥与所述私钥匹配，通过所述微内核得到所述第一程序具备所述图像传感器的信息采集权限的鉴定结果。In response to the match between the public key and the private key, an authentication result that the first program has the information collection authority of the image sensor is obtained through the microkernel.
根据权利要求14所述的装置，所述许可信息包括第一指纹信息，所述验证信息包括所述可信执行环境中预存的第二指纹信息；The apparatus according to claim 14, wherein the license information includes first fingerprint information, and the verification information includes second fingerprint information pre-stored in the trusted execution environment;

所述获取模块，用于响应于所述第一程序在图像信息采集过程中对所述图像传感器的调用请求，通过所述微内核调用指纹传感器进行所述第一指纹信息的采集；The acquisition module is configured to, in response to the call request of the first program to the image sensor during the image information collection process, call the fingerprint sensor through the microkernel to collect the first fingerprint information;

所述鉴权模块，用于通过所述微内核计算所述第一指纹信息与所述第二指纹信息的相似度；响应于所述相似度大于预设相似度，通过所述微内核确定所述第一指纹信息与所述第二指纹信息匹配。The authentication module is configured to calculate the similarity between the first fingerprint information and the second fingerprint information through the microkernel; in response to the similarity being greater than a preset similarity, determine the The first fingerprint information matches the second fingerprint information.
根据权利要求10至12任一所述的装置，所述电子设备上与所述微内核并行运行有主操作***；The device according to any one of claims 10 to 12, wherein a main operating system runs in parallel with the microkernel on the electronic device;

所述装置还包括存储模块；The device also includes a storage module;

所述存储模块，用于通过所述微内核调用所述图像传感器进行图像信息的采集之后，通过所述主操作***将所述图像信息存储至所述可信执行环境中的图像存储器中；The storage module is configured to store the image information in the image memory in the trusted execution environment through the main operating system after calling the image sensor through the microkernel to collect image information;

所述获取模块，用于通过所述微内核从所述图像存储器中读取所述图像信息。The acquiring module is configured to read the image information from the image memory through the microkernel.
一种电子设备，所述电子设备包括处理器、和与所述处理器相连的存储器，以及存储在所述存储器上的程序指令，所述处理器执行所述程序指令时实现如权利要求1至9任一所述的图像处理方法。An electronic device, the electronic device includes a processor, a memory connected to the processor, and program instructions stored on the memory, when the processor executes the program instructions, the following claims 1 to 1 are implemented. 9 any one of the image processing methods.
一种计算机可读存储介质，所述计算机可读存储介质中存储有程序指令，所述程序指令被处理器执行时实现如权利要求1至9任一所述的图像处理方法。A computer-readable storage medium, wherein program instructions are stored in the computer-readable storage medium, and when the program instructions are executed by a processor, the image processing method according to any one of claims 1 to 9 is realized.
一种计算机程序产品，所述计算机程序产品包括计算机指令，所述计算机指令存储在计算机可读存储介质中；计算机设备的处理器从所述计算机可读存储介质读取所述计算机指令，所述处理器执行所述计算机指令时实现如权利要求1至9中任一所述的图像处理方法。A computer program product comprising computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, the When the processor executes the computer instructions, the image processing method according to any one of claims 1 to 9 is implemented.