WO2023236115A1

WO2023236115A1 - Image processing method and apparatus, and storage medium

Info

Publication number: WO2023236115A1
Application number: PCT/CN2022/097695
Authority: WO
Inventors: 刘晓林; 宋伟
Original assignee: 北京小米移动软件有限公司
Priority date: 2022-06-08
Filing date: 2022-06-08
Publication date: 2023-12-14
Also published as: CN117546479A

Abstract

The present disclosure relates to an image processing method and apparatus, and a storage medium. The image processing method comprises: acquiring an original image which is collected by an image collection sensor in a second terminal, and determining an image display instruction; obtaining a display image of the original image on the basis of the image display instruction and the original image; and displaying a display image of the original image. By means of the image processing method disclosed in the present disclosure, image processing is performed, such that the implementation is fast and smooth.

Description

一种图像处理方法、装置及存储介质An image processing method, device and storage medium

技术领域Technical field

本公开涉及终端技术领域，尤其涉及一种图像处理方法、装置及存储介质。The present disclosure relates to the field of terminal technology, and in particular, to an image processing method, device and storage medium.

背景技术Background technique

相关技术中，在小像素的图像采集传感器中，包含有图像采集传感器捕获的原始图像的处理机制，而大像素的图像采集传感器对于图像处理的技术方法，依赖于终端设备的图像前端引擎以及图像处理引擎高耦合的图像处理方式，无法做到直接由图像处理引擎处理捕获的原始图像，导致图像采集传感器必须要与图片处理器在同一设备上才可使用。In related technologies, small-pixel image acquisition sensors include a processing mechanism for the original image captured by the image acquisition sensor, while large-pixel image acquisition sensors rely on the image front-end engine and image processing technology of the terminal device. The highly coupled image processing method of the processing engine cannot directly process the captured original image by the image processing engine, so the image acquisition sensor must be used on the same device as the image processor.

发明内容Contents of the invention

为克服相关技术中存在的问题，本公开提供一种图像处理方法、装置及存储介质。In order to overcome problems existing in related technologies, the present disclosure provides an image processing method, device and storage medium.

根据本公开实施例的第一方面，提供一种图像处理方法，应用于第一终端，包括：According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, applied to a first terminal, including:

获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令；Obtain the original image collected by the image acquisition sensor in the second terminal, and determine the image display instruction;

基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像；Based on the image display instruction and the original image, obtain a display image of the original image;

显示所述原始图像的显示图像。Display the display image of the original image.

在一种实施方式中，所述基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像，包括：In one implementation, obtaining a display image of the original image based on the image display instruction and the original image includes:

获取源缓存数据，所述源缓存数据为缓存在内存中的所述原始图像的源数据；Obtain source cache data, which is the source data of the original image cached in the memory;

基于所述源缓存数据，得到匹配所述图像显示指令的显示图像缓存队列；Based on the source cache data, obtain a display image cache queue matching the image display instruction;

将所述显示图像缓存队列中的显示图像，作为所述原始图像的显示图像。The display image in the display image cache queue is used as the display image of the original image.

在一种实施方式中，所述图像显示指令包括预览图像显示指令，或者预览图像显示指令和预览图像的拍照图像显示指令；In one embodiment, the image display instructions include a preview image display instruction, or a preview image display instruction and a photographed image display instruction of the preview image;

所述基于所述源缓存数据，得到匹配所述图像显示指令的显示图像缓存队列，包括：Obtaining a display image cache queue matching the image display instruction based on the source cache data includes:

基于所述源缓存数据中包括的会话捕获创建请求，获取所述源缓存数据的图像尺寸以及图像格式；Obtain the image size and image format of the source cache data based on the session capture creation request included in the source cache data;

基于所述源缓存数据的图像尺寸以及图像格式，构建基础图像处理目标；Construct a basic image processing target based on the image size and image format of the source cache data;

配置所述基础图像处理目标的目标输出流，所述目标输出流包括显示所述原始图像的预览图像的输出流，或者显示所述原始图像的预览图像的输出流以及显示所述预览图像的拍照图像的输出流；Configuring a target output stream of the basic image processing target, the target output stream including an output stream displaying a preview image of the original image, or an output stream displaying a preview image of the original image and a photograph displaying the preview image Image output stream;

基于所述目标输出流对应的显示图像，生成所述原始图像的显示图像缓存队列。Based on the display image corresponding to the target output stream, a display image cache queue of the original image is generated.

在一种实施方式中，所述源缓存数据中包括的会话捕获创建请求采用如下方式预先创建：In one implementation, the session capture creation request included in the source cache data is pre-created in the following manner:

创建捕获请求函数，所述捕获请求函数的参数中包括对所述原始图像的重处理指令；Create a capture request function, the parameters of the capture request function include reprocessing instructions for the original image;

根据所述原始图像的显示图像格式，创建输入配置对象，所述输入配置对象包括所述原始图像尺寸以及图像格式；Create an input configuration object according to the display image format of the original image, the input configuration object including the original image size and image format;

基于已创建的捕获请求函数，将所述输入配置对象传输至***底层，由***底层创建捕获输入配置对象的会话捕获创建请求。Based on the created capture request function, the input configuration object is transmitted to the bottom layer of the system, and the bottom layer of the system creates a session capture creation request that captures the input configuration object.

在一种实施方式中，所述基于所述源缓存数据中包括的会话捕获创建请求，获取所述源缓存数据的图像尺寸以及图像格式，包括：In one implementation, obtaining the image size and image format of the source cache data based on the session capture creation request included in the source cache data includes:

基于多次会话捕获创建请求对应的捕获操作，获取到所述原始图像的源缓存数据；Based on the capture operations corresponding to multiple session capture creation requests, obtain the source cache data of the original image;

缓存所述原始图像的源缓存数据。Cache the source cache data of the original image.

在一种实施方式中，所述方法还包括：In one embodiment, the method further includes:

响应于创建输入配置对象完成，设置输入配置对象的监听机制；In response to completion of creating the input configuration object, set the listening mechanism of the input configuration object;

基于所述监听机制，监听传输至***底层的输入配置对象。Based on the monitoring mechanism, the input configuration object transmitted to the bottom layer of the system is monitored.

若未监听到传输至***底层的输入配置对象，显示提示信息；If the input configuration object transmitted to the bottom layer of the system is not monitored, a prompt message is displayed;

所述提示信息用于提示不能获取到所述原始图像对应的输入配置对象。The prompt information is used to prompt that the input configuration object corresponding to the original image cannot be obtained.

根据本公开实施例的第二方面，提供一种图像处理方法，应用于第二终端，所述图像处理方法包括：According to a second aspect of an embodiment of the present disclosure, an image processing method is provided, applied to a second terminal, and the image processing method includes:

基于图像采集传感器采集原始图像；Collect original images based on image acquisition sensors;

将所述原始图像发送至第一终端，基于所述第一终端的所述图像显示指令和所述原始图像得到所述原始图像的显示图像。The original image is sent to the first terminal, and a display image of the original image is obtained based on the image display instruction of the first terminal and the original image.

根据本公开实施例的第三方面，提供一种图像处理装置，应用于第一终端，所述图像处理装置包括：According to a third aspect of an embodiment of the present disclosure, an image processing device is provided, applied to a first terminal, and the image processing device includes:

确定单元，用于获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令；a determination unit, used to obtain the original image collected by the image acquisition sensor in the second terminal, and determine the image display instruction;

处理单元，用于基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像；A processing unit configured to obtain a display image of the original image based on the image display instruction and the original image;

显示单元，显示所述原始图像的显示图像。A display unit displays the display image of the original image.

在一种实施方式中，所述处理单元采用如下方式基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像：In one implementation, the processing unit obtains the display image of the original image based on the image display instruction and the original image in the following manner:

在一种实施方式中，所述处理单元中图像显示指令包括预览图像显示指令，或者预览图像显示指令和预览图像的拍照图像显示指令；In one embodiment, the image display instructions in the processing unit include a preview image display instruction, or a preview image display instruction and a photographed image display instruction of the preview image;

在一种实施方式中，所述处理单元源缓存数据中包括的会话捕获创建请求采用如下方式预先创建：In one implementation, the session capture creation request included in the processing unit source cache data is pre-created in the following manner:

在一种实施方式中，所述处理单元采用如下方式基于所述源缓存数据中包括的会话捕获创建请求，获取所述源缓存数据的图像尺寸以及图像格式：In one implementation, the processing unit obtains the image size and image format of the source cache data based on the session capture creation request included in the source cache data in the following manner:

在一种实施方式中，所述处理单元还用于：In one implementation, the processing unit is also used to:

根据本公开实施例的第四方面，提供一种图像处理装置，应用于第二终端，所述图像处理装置包括：According to a fourth aspect of an embodiment of the present disclosure, an image processing device is provided, applied to a second terminal, and the image processing device includes:

采集单元，用于基于图像采集传感器采集原始图像，并由所述第一终端确定图像显示指令；An acquisition unit, configured to acquire original images based on the image acquisition sensor, and determine image display instructions by the first terminal;

发送单元，用于将所述原始图像发送至第一终端，基于所述第一终端的所述图像显示指令和所述原始图像得到所述原始图像的显示图像。A sending unit, configured to send the original image to a first terminal, and obtain a display image of the original image based on the image display instruction of the first terminal and the original image.

根据本公开实施例第五方面，提供一种图像处理装置，包括：According to a fifth aspect of the embodiment of the present disclosure, an image processing device is provided, including:

处理器；用于存储处理器可执行指令的存储器；Processor; memory used to store instructions executable by the processor;

其中，所述处理器被配置为：执行第一方面或第一方面任意一种实施方式中所述的图像处理方法。Wherein, the processor is configured to: execute the image processing method described in the first aspect or any implementation manner of the first aspect.

根据本公开实施例第六方面，提供一种图像处理装置，包括：According to a sixth aspect of the present disclosure, an image processing device is provided, including:

其中，所述处理器被配置为：执行第二方面或第二方面任意一种实施方式中所述的图像处理方法。Wherein, the processor is configured to: execute the image processing method described in the second aspect or any implementation manner of the second aspect.

根据本公开实施例第七方面，提供一种存储介质，所述存储介质中存储有指令，当所述存储介质中的指令由终端的处理器执行时，使得终端能够执行第一方面或第一方面任意一种实施方式中所述的图像处理方法。According to a seventh aspect of the embodiments of the present disclosure, a storage medium is provided. Instructions are stored in the storage medium. When the instructions in the storage medium are executed by a processor of the terminal, the terminal can execute the first aspect or the first aspect. The image processing method described in any one of the embodiments.

根据本公开实施例第八方面，提供一种存储介质，所述存储介质中存储有指令，当所述存储介质中的指令由终端的处理器执行时，使得终端能够执行第二方面或第二方面任意一种实施方式中所述的图像处理方法。According to an eighth aspect of the embodiments of the present disclosure, a storage medium is provided. Instructions are stored in the storage medium. When the instructions in the storage medium are executed by a processor of the terminal, the terminal can execute the second aspect or the second aspect. The image processing method described in any one of the embodiments.

本公开的实施例提供的技术方案可以包括以下有益效果：获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令，基于图像显示指令和原始图像，得到原始图像的显示图像，显示原始图像的显示图像。本公开实施例提供的图片处理方法实现了可以由图像处理引擎处理第二终端中图像采集传感器采集的原始图像，实现第一终端和第二终端处于不同终端，在第一终端上可以实时预览到第二终端采集到的图像的目的，实现方式快捷流畅。The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: obtain the original image collected by the image acquisition sensor in the second terminal, determine the image display instruction, obtain the display image of the original image based on the image display instruction and the original image, and display Display image of the original image. The image processing method provided by the embodiment of the present disclosure enables the image processing engine to process the original image collected by the image acquisition sensor in the second terminal, and realizes that the first terminal and the second terminal are in different terminals, and can be previewed in real time on the first terminal. The purpose of the image collected by the second terminal is to achieve the purpose in a quick and smooth way.

应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

附图说明Description of the drawings

此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

图1是根据一示例性实施例示出的一种应用于第一终端图像处理方法的流程图。FIG. 1 is a flow chart illustrating an image processing method applied to a first terminal according to an exemplary embodiment.

图2是根据一示例性实施例示出的一种得到原始图像的显示图像的流程图。FIG. 2 is a flowchart illustrating a method of obtaining a display image of an original image according to an exemplary embodiment.

图3是根据一示例性实施例示出的一种得到匹配图像显示指令的显示图像缓存队列的流程图。FIG. 3 is a flowchart illustrating a method of obtaining a display image cache queue that matches an image display instruction according to an exemplary embodiment.

图4是根据一示例性实施例示出的一种预先创建会话捕获请求的流程图。Figure 4 is a flowchart illustrating a pre-created session capture request according to an exemplary embodiment.

图5是根据一示例性实施例示出的一种***底层处理过程的示意图。Figure 5 is a schematic diagram of a system underlying processing process according to an exemplary embodiment.

图6是根据一示例性实施例示出的一种获取源缓存数据的图像尺寸以及图像格式的流程图。FIG. 6 is a flow chart for obtaining the image size and image format of source cache data according to an exemplary embodiment.

图7是根据一示例性实施例示出的一种应用于第一终端图像处理方法中监听机制的流程图。FIG. 7 is a flow chart illustrating a monitoring mechanism applied in the first terminal image processing method according to an exemplary embodiment.

图8是根据一示例性实施例示出的一种应用于第一终端图像处理方法中监听机制的流程图。Figure 8 is a flowchart illustrating a monitoring mechanism applied in the first terminal image processing method according to an exemplary embodiment.

图9是根据一示例性实施例示出的一种应用于第二终端的图像处理方法的流程图。FIG. 9 is a flowchart of an image processing method applied to a second terminal according to an exemplary embodiment.

图10是根据一示例性实施例示出的***数据流图的示意图。Figure 10 is a schematic diagram of a system data flow diagram according to an exemplary embodiment.

图11是根据一示例性实施例示出的一种应用于第一终端的图像处理装置框图。FIG. 11 is a block diagram of an image processing device applied to a first terminal according to an exemplary embodiment.

图12是根据一示例性实施例示出的一种应用于第二终端的图像处理装置框图。FIG. 12 is a block diagram of an image processing apparatus applied to a second terminal according to an exemplary embodiment.

图13是根据一示例性实施例示出的一种用于图像处理的装置的框图。FIG. 13 is a block diagram of an apparatus for image processing according to an exemplary embodiment.

具体实施方式Detailed ways

这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

本公开实施例提供的图像处理方法、装置及存储介质可以应用于第一终端和第二终端处于不同位置，在第一终端上可以实时预览到第二终端采集到的图像的场景。例如，本公开实施例提供的图像处理方法可以应用于通过图像采集传感器等终端采集到的图像，经过网络等方式传输给手机等终端进行图像处理并预览图像显示的场景。The image processing method, device and storage medium provided by the embodiments of the present disclosure can be applied to a scene where the first terminal and the second terminal are in different positions, and the first terminal can preview the image collected by the second terminal in real time. For example, the image processing method provided by the embodiments of the present disclosure can be applied to a scenario where images collected by terminals such as image acquisition sensors are transmitted to terminals such as mobile phones through the network for image processing and preview of image display.

相关技术中，存在终端只能处理来自于本终端的图像，然而无法实时处理来自于其他终端的图像的技术。In related technologies, there is a technology in which a terminal can only process images from its own terminal, but cannot process images from other terminals in real time.

本公开中涉及到的图像采集是指图像采集是相机与终端通过图像采集卡相互链接，图像采集卡接收相机的模拟信号或数字信号，并将信号处理转换为适用于终端的信息。The image acquisition involved in this disclosure means that the camera and the terminal are linked to each other through an image acquisition card. The image acquisition card receives the analog signal or digital signal of the camera and converts the signal processing into information suitable for the terminal.

图像处理器(ISP)主要包括了诸如IFE/BPS/IPE/JPEG等硬件模块，它们分别承担了不同的图像处理任务。其中，IFE(Image Front End)会针对预览以及视频做颜色校正、下采样、去马赛克统计3A数据的处理；BPS(Bayer processing segment)主要用于拍照图像数据的坏点去除、相位对焦、去马赛克、下采样、HDR处理以及Bayer的混合降噪处理；IPE(Image processing engine)主要承担例如硬件降噪、图像的裁剪、降噪、颜色处理、细节增强等图像处理工作；JPEG用于拍照数据的存储通过该硬件模块进行jpeg编码工作。The image processor (ISP) mainly includes hardware modules such as IFE/BPS/IPE/JPEG, which respectively undertake different image processing tasks. Among them, IFE (Image Front End) will perform color correction, downsampling, and demosaicing statistical 3A data processing for previews and videos; BPS (Bayer processing segment) is mainly used for dead pixel removal, phase focusing, and demosaicing of photographed image data. , downsampling, HDR processing and Bayer's hybrid noise reduction processing; IPE (Image processing engine) is mainly responsible for image processing such as hardware noise reduction, image cropping, noise reduction, color processing, detail enhancement, etc.; JPEG is used for photographing data Storage uses this hardware module to perform jpeg encoding work.

本公开中涉及到的图像处理器主要用了BPS(Bayer processing segment)、IPE(Image processing engine)以及对图像编码的模块。The image processor involved in this disclosure mainly uses BPS (Bayer processing segment), IPE (Image processing engine) and image encoding modules.

目前，图像处理技术在存在着研究的进步空间，例如终端以手机为例，让用户操作的手机相机可以通过内存处理传输而来的原始图片，并且进行显示预览。实现第一终端和第二终端处于不同终端，在第一终端上可以实时预览到第二终端采集到的图像的目的，进而实现第二终端预览图像显示的过程快捷流畅的效果。At present, there is room for research progress in image processing technology. For example, the terminal takes a mobile phone as an example, allowing the user-operated mobile phone camera to process the original image transmitted through the memory and display and preview it. The first terminal and the second terminal are in different terminals, and the image collected by the second terminal can be previewed in real time on the first terminal, thereby achieving a fast and smooth process of previewing the image display on the second terminal.

本公开实施例提供一种图像处理方法，在该图像处理方法中，获取第二终端采集的原始图像，对原始图像进行重处理得到原始图像的显示图像显示原始图像的显示图像。本公开实施例提供的图片处理方法实现了可以由图像处理引擎处理第二终端采集的原始图像，实现第一终端和第二终端处于不同终端，在第一终端上可以实时预览到第二终端采集到的图像的目的，实现方式快捷流畅。Embodiments of the present disclosure provide an image processing method. In the image processing method, the original image collected by the second terminal is obtained, and the original image is reprocessed to obtain a display image of the original image and display the display image of the original image. The image processing method provided by the embodiment of the present disclosure enables the image processing engine to process the original image collected by the second terminal, realizes that the first terminal and the second terminal are in different terminals, and the first terminal can preview in real time what is collected by the second terminal. The purpose of the image is achieved quickly and smoothly.

图1是根据一示例性实施例示出的一种应用于第一终端图像处理方法的流程图。如图1所示，包括以下步骤。FIG. 1 is a flow chart illustrating an image processing method applied to a first terminal according to an exemplary embodiment. As shown in Figure 1, it includes the following steps.

在步骤S11中，获得第二终端中图像采集传感器采集的原始图像，并确定图像显示指令。In step S11, the original image collected by the image collection sensor in the second terminal is obtained, and the image display instruction is determined.

在步骤S12中，基于图像显示指令和原始图像，得到原始图像的显示图像。In step S12, based on the image display instruction and the original image, a display image of the original image is obtained.

在步骤S13中，显示原始图像的显示图像。In step S13, the display image of the original image is displayed.

本公开实施例中，第二终端中图像采集传感器采集到原始图像之后，可以通过网络将原始图像传输至第一终端，第一终端对接收到的原始图像并确定图像显示指令，根据图像显示指令和原始图像，使得第一终端能够预览原始图像显示。In the embodiment of the present disclosure, after the image acquisition sensor in the second terminal collects the original image, the original image can be transmitted to the first terminal through the network. The first terminal determines the image display instruction for the received original image, and according to the image display instruction and the original image, so that the first terminal can preview the original image display.

例如，第一终端可以是手机，第二终端可以是图像采集传感器。由图像采集传感器的摄像头采集到的原始图像，可以通过网络将原始图像传输至手机上，利用手机的图像处理器中图像引擎模块对原始图像确定图像显示指令，根据图像显示指令和原始图像，更够原始图像能够在手机屏幕上进行预览显示图像，可以理解的是，手机屏幕上显示的是图像采集传感器采集到的图像。For example, the first terminal may be a mobile phone, and the second terminal may be an image acquisition sensor. The original image collected by the camera of the image acquisition sensor can be transmitted to the mobile phone through the network. The image engine module in the image processor of the mobile phone is used to determine the image display instruction for the original image. According to the image display instruction and the original image, the original image is updated. The original image can be previewed and displayed on the mobile phone screen. It is understandable that what is displayed on the mobile phone screen is the image collected by the image acquisition sensor.

本公开实施例中，提供的图像处理方法中第一终端和第二终端可以为不同的终端，从而实现能够查看处于不同位置的图像。In the embodiment of the present disclosure, in the image processing method provided, the first terminal and the second terminal may be different terminals, thereby enabling viewing of images at different locations.

图2是根据一示例性实施例示出的一种得到原始图像的显示图像的流程图。如图2所示，基于图像显示指令和原始图像，得到原始图像的显示图像，包括以下步骤。FIG. 2 is a flowchart illustrating a method of obtaining a display image of an original image according to an exemplary embodiment. As shown in Figure 2, based on the image display instruction and the original image, obtaining the display image of the original image includes the following steps.

在步骤S21中，获取源缓存数据，源缓存数据为缓存在内存中的所述原始图像的源数据。In step S21, source cache data is obtained, and the source cache data is the source data of the original image cached in the memory.

在步骤S22中，基于源缓存数据，得到匹配图像显示指令的显示图像缓存队列。In step S22, based on the source cache data, a display image cache queue matching the image display instruction is obtained.

在步骤S23中，将显示图像缓存队列中的显示图像，作为原始图像的显示图像。In step S23, the display image in the display image cache queue is used as the display image of the original image.

本公开实施例中，以第一终端是手机为例，在手机获取到原始图像后，将原始图像缓存在手机的内存之中，手机***底层根据内存中缓存的原始图像获取源缓存数据，基于图像显示指令，对原始图像的源缓存数据进行匹配图像显示指令，得到原始图像的显示图像缓存队列，将显示图像缓存队列中的显示图像，作为原始图像的显示图像，可以理解的是，手机屏幕可以预览显示图像，也可以对显示图像进行拍照。In this disclosed embodiment, taking the first terminal as a mobile phone as an example, after the mobile phone obtains the original image, the original image is cached in the memory of the mobile phone. The bottom layer of the mobile phone system obtains the source cache data based on the original image cached in the memory. The image display instruction matches the source cache data of the original image with the image display instruction to obtain the display image cache queue of the original image. The display image in the image cache queue will be displayed as the display image of the original image. It can be understood that the mobile phone screen You can preview the displayed image or take a photo of the displayed image.

本公开实施例中，提供的图像显示方法，能够将原始图像输入至***底层进行图像处理。In the embodiment of the present disclosure, the image display method provided can input the original image to the bottom layer of the system for image processing.

图3是根据一示例性实施例示出的一种得到匹配图像显示指令的显示图像缓存队列的流程图。如图3所示，图像显示指令包括预览图像显示指令，或预览图像显示指令和预览图像的拍照图像显示指令，基于源缓存数据，得到匹配图像显示指令的显示图像缓存队列，包括以下步骤。FIG. 3 is a flowchart illustrating a method of obtaining a display image cache queue that matches an image display instruction according to an exemplary embodiment. As shown in Figure 3, the image display instruction includes a preview image display instruction, or a preview image display instruction and a photographed image display instruction of the preview image. Based on the source cache data, obtaining a display image cache queue that matches the image display instruction includes the following steps.

在步骤S31中，基于源缓存数据中包括的会话捕获创建请求，获取源缓存数据的图像尺寸以及图像格式。In step S31, based on the session capture creation request included in the source cache data, the image size and image format of the source cache data are obtained.

本公开实施例中，根据源缓存数据的会话捕获请求发送创建指令，将源缓存数据的图像尺寸以及图像格式输入至源缓存数据的会话捕获请求中。例如，源缓存数据的图像尺寸为1280*720，图像格式为RAW格式，那么会话获得的图像尺寸为1280*720，图像格式为RAW格式。In the embodiment of the present disclosure, a creation instruction is sent according to the session capture request of the source cache data, and the image size and image format of the source cache data are input into the session capture request of the source cache data. For example, if the image size of the source cache data is 1280*720 and the image format is RAW format, then the image size obtained by the session is 1280*720 and the image format is RAW format.

在步骤S32中，基于源缓存数据的图像尺寸以及图像格式，构建基础图像处理目标。In step S32, a basic image processing target is constructed based on the image size and image format of the source cache data.

本公开实施例中，根据源缓存数据的图像尺寸以及图像格式，构建基础图像处理目标。其中，基础图像处理目标是指对于源缓存数据的图像进行基础的图像处理。In the embodiment of the present disclosure, a basic image processing target is constructed according to the image size and image format of the source cache data. Among them, the basic image processing target refers to performing basic image processing on the image of the source cache data.

在步骤S33中，配置基础图像处理目标的目标输出流，目标输出流包括显示原始图像的预览图像的输出流，或者显示原始图像的预览图像的输出流以及显示预览图像的拍照图像的输出流。In step S33, a target output stream of the basic image processing target is configured. The target output stream includes an output stream that displays a preview image of the original image, or an output stream that displays a preview image of the original image and an output stream of a photographed image that displays the preview image.

本公开实施例中，图像显示指令的过程需要配置基础图像处理目标的目标输出流，终端显示的图像根据目标输出流中的数据确定。其中，目标输出流包括显示原始图像的预览图像的输出流，或者显示原始图像的预览图像的输出流以及显示预览图像的拍照图像的输出流。In the embodiment of the present disclosure, the process of image display instruction requires configuring the target output stream of the basic image processing target, and the image displayed by the terminal is determined based on the data in the target output stream. The target output stream includes an output stream that displays a preview image of the original image, or an output stream that displays a preview image of the original image and an output stream that displays a photographed image of the preview image.

可以理解的是，目标输出流输出的图像尺寸、格式可以由预设至第一终端的数据确定。It can be understood that the image size and format output by the target output stream can be determined by the data preset to the first terminal.

在步骤S34中，基于目标输出流对应的显示图像，生成原始图像的显示图像缓存队列。In step S34, a display image cache queue of the original image is generated based on the display image corresponding to the target output stream.

本公开实施例中，根据目标输出流对应的显示图像，可以生成原始图像的显示图像缓存队列，原始图像的显示图像缓存队列是输出显示图像的途径，原始图像的显示图像缓存队列中可以有多个待显示图像，待显示图像在显示图像的缓存队列中的顺序，决定了显示图像的显示顺序。In the embodiment of the present disclosure, according to the display image corresponding to the target output stream, a display image cache queue of the original image can be generated. The display image cache queue of the original image is a way to output the display image. There can be multiple display image cache queues of the original image. There are images to be displayed, and the order of the images to be displayed in the cache queue of the displayed images determines the display order of the displayed images.

例如，显示图像的缓存队列中待显示图像的顺序为图a、图b……图n，那么在第一终端显示图像的顺序为图a、图b……图n。For example, if the order of images to be displayed in the cache queue of display images is picture a, picture b...picture n, then the order of images displayed on the first terminal is picture a, picture b...picture n.

图4是根据一示例性实施例示出的一种预先创建会话捕获请求的流程图。如图4所示，源缓存数据中包括会话捕获创建请求的预先创建，包括以下步骤。Figure 4 is a flowchart illustrating a pre-created session capture request according to an exemplary embodiment. As shown in Figure 4, the source cache data includes the pre-creation of the session capture creation request, including the following steps.

在步骤S41中，创建捕获请求函数，捕获请求函数的参数中包括对原始图像的重处理指令。In step S41, a capture request function is created, and the parameters of the capture request function include instructions for reprocessing the original image.

在步骤S42中，根据原始图像的显示图像格式，创建输入配置对象，输入配置对象包括原始图像尺寸以及图像格式。In step S42, an input configuration object is created according to the display image format of the original image, and the input configuration object includes the original image size and image format.

在步骤S43中，基于已创建的捕获请求函数，将输入配置对象传输至***底层，由***底层创建捕获输入配置对象的会话捕获创建请求。In step S43, based on the created capture request function, the input configuration object is transmitted to the system bottom layer, and the system bottom layer creates a session capture creation request that captures the input configuration object.

本公开实施例中，需要建立源缓存数据的会话捕获请求，通过会话捕获请求建立第一终端接收到的原始图像信息与***底层之间的数据传输。In the embodiment of the present disclosure, a session capture request for the source cache data needs to be established, and data transmission between the original image information received by the first terminal and the bottom layer of the system is established through the session capture request.

源缓存数据的会话捕获请求包括创建捕获请求函数，其中，捕获请求函数的参数中包括对原始图像的图像显示指令。根据原始图像的显示图像格式，创建输入配置对象，其中，输入配置对象包括图像尺寸及图像格式。根据已创建的捕获请求函数，将输入配置对象传输至***底层，由***底层创建捕获输入配置对象的会话捕获创建请求。The session capture request of the source cache data includes creating a capture request function, wherein the parameters of the capture request function include image display instructions for the original image. Create an input configuration object according to the display image format of the original image, where the input configuration object includes image size and image format. According to the created capture request function, the input configuration object is transferred to the bottom layer of the system, and the bottom layer of the system creates a session capture creation request to capture the input configuration object.

图5是根据一示例性实施例示出的一种***底层处理过程的示意图。参阅图5所示，以第一终端是手机为例进行说明。当手机接收到重处理指令后，将获取到的原始图像缓存在内存中，创建会话捕获请求，根据原始图像的显示图像格式建立输入配置对象，将输入配置对象通过会话捕获请求传输至手机***底层，由手机***底层对原始图像进行处理。Figure 5 is a schematic diagram of a system underlying processing process according to an exemplary embodiment. Referring to FIG. 5 , the first terminal is a mobile phone for example. When the mobile phone receives the reprocessing instruction, it caches the obtained original image in the memory, creates a session capture request, establishes an input configuration object according to the display image format of the original image, and transmits the input configuration object to the bottom layer of the mobile phone system through the session capture request. , the original image is processed by the bottom layer of the mobile phone system.

其中，手机***底层的媒体流中包括了输入配置对象以及目标输出流，手机***底层根据输入配置对象和目标输出流对原始图像进行基础图像处理。基础图像处理根据拜耳处理部分进行例如对图像的语法分析、图像的统计、自动调整白平衡、经过图像处理器对图像数据的坏点去除、去马赛克以及拜耳的混合降噪处理等处理。Among them, the media stream at the bottom of the mobile phone system includes the input configuration object and the target output stream. The bottom layer of the mobile phone system performs basic image processing on the original image based on the input configuration object and the target output stream. Basic image processing is based on the Bayer processing part, such as image syntax analysis, image statistics, automatic adjustment of white balance, removal of bad pixels from image data through the image processor, demosaicing, and Bayer hybrid noise reduction processing.

接述上例，手机接到原始图像之后，根据图像尺寸以及图像格式形成输入配置对象，手机***底层通过媒体流接收到输入配置对象之后，对于原始图像进行可以是对图像的语法分析、图像的统计、自动调整白平衡和经过图像处理器的处理过程，形成目标输出流，将***底层处理好的图像通过目标输出流输出至会话中。Continuing the above example, after the mobile phone receives the original image, it forms an input configuration object according to the image size and image format. After the bottom layer of the mobile phone system receives the input configuration object through the media stream, the original image can be syntax analysis of the image, image processing, etc. Statistics, automatic adjustment of white balance and processing by the image processor form a target output stream, and the images processed by the bottom layer of the system are output to the session through the target output stream.

本公开实施例中，为了实现显示图像连贯，会话可以多次发送请求。In the embodiment of the present disclosure, in order to achieve consistent display of images, the session can send requests multiple times.

图6是根据一示例性实施例示出的一种获取源缓存数据的图像尺寸以及图像格式的流程图。如图6所示，包括以下步骤。FIG. 6 is a flow chart for obtaining the image size and image format of source cache data according to an exemplary embodiment. As shown in Figure 6, it includes the following steps.

在步骤S51中，基于多次会话捕获创建请求对应的捕获操作，获取到原始图像的源缓存数据。In step S51, the source cache data of the original image is obtained based on the capture operations corresponding to the multiple session capture creation requests.

在步骤S52中，缓存原始图像的源缓存数据。In step S52, the source cache data of the original image is cached.

本公开实施例中，可以根据会话捕获请求进行多次捕获操作，将获取的原始图像缓存在源缓存数据中。以手机为例，会话可以每秒发送12次请求，那么源缓存数据中每秒增加12组原始图像的数据，通过手机***底层对原始图像数据进行处理，将处理结果输出至手机屏幕并进行显示。可以理解的是，手机能够多次发送会话请求并进行捕获操作，使原始图像的显示能够连贯，用户在手机屏幕上观看到的可以是视频的形式。In the embodiment of the present disclosure, multiple capture operations can be performed according to the session capture request, and the acquired original images are cached in the source cache data. Taking a mobile phone as an example, a session can send 12 requests per second, then 12 sets of original image data are added to the source cache data every second, the original image data is processed through the bottom layer of the mobile phone system, and the processing results are output to the mobile phone screen and displayed. . It is understandable that the mobile phone can send session requests and perform capture operations multiple times, so that the original image can be displayed coherently, and what the user watches on the mobile phone screen can be in the form of a video.

本公开实施例中，源缓存数据能够成功传输至***底层，第一终端可以对传输过程进行监听。In this disclosed embodiment, the source cache data can be successfully transmitted to the bottom layer of the system, and the first terminal can monitor the transmission process.

图7是根据一示例性实施例示出的一种应用于第一终端图像处理方法中监听机制的流程图。如图7所示，包括以下步骤。FIG. 7 is a flow chart illustrating a monitoring mechanism applied in the first terminal image processing method according to an exemplary embodiment. As shown in Figure 7, it includes the following steps.

在步骤S61中，响应于创建输入配置对象完成，设置输入配置对象的监听机制。In step S61, in response to completion of creating the input configuration object, a monitoring mechanism of the input configuration object is set.

在步骤S62中，基于监听机制，监听传输至***底层的输入配置对象。In step S62, based on the listening mechanism, the input configuration object transmitted to the bottom layer of the system is monitored.

本公开实施例中，第一终端图像处理方法中设置输入配置对象的监听机制，监听对象是创建输入配置对象，对传输至***底层的输入配置对象执行监听，In the embodiment of the present disclosure, a monitoring mechanism for input configuration objects is set in the first terminal image processing method. The monitoring object is to create an input configuration object and perform monitoring on the input configuration object transmitted to the bottom layer of the system.

图8是根据一示例性实施例示出的一种应用于第一终端图像处理方法中监听机制的流程图。如图8所示，包括以下步骤。Figure 8 is a flowchart illustrating a monitoring mechanism applied in the first terminal image processing method according to an exemplary embodiment. As shown in Figure 8, it includes the following steps.

在步骤S71中，若未监听到传输至***底层的输入配置对象，显示提示信息。In step S71, if the input configuration object transmitted to the bottom layer of the system is not monitored, a prompt message is displayed.

在步骤S72中，提示信息用于提示不能获取到原始图像对应的输入配置对象。In step S72, the prompt information is used to prompt that the input configuration object corresponding to the original image cannot be obtained.

本公开实施例中，监听机制若未监听到输入配置对象传输至***底层，显示提示信息，提示信息的内容为不能获取到原始图像对应的输入配置对象。In this disclosed embodiment, if the monitoring mechanism does not detect that the input configuration object is transmitted to the bottom layer of the system, a prompt message is displayed. The content of the prompt message is that the input configuration object corresponding to the original image cannot be obtained.

例如，存在一个输入配置对象未成功传输至***底层，监听机制监听该输入配置对象，在第一终端显示提示信息，提示信息的内容为不能获取该输入配置对象。For example, if there is an input configuration object that is not successfully transmitted to the bottom layer of the system, the monitoring mechanism monitors the input configuration object and displays a prompt message on the first terminal. The content of the prompt message is that the input configuration object cannot be obtained.

本公开实施例中，提供的图像处理方法中监听机制能够实时监听输入配置对象传输至 ***底层的结果，若并未传输成功，则进行提示。In the embodiment of the present disclosure, the monitoring mechanism in the image processing method provided can monitor the result of the input configuration object being transmitted to the bottom layer of the system in real time. If the transmission is not successful, a prompt will be issued.

图9是根据一示例性实施例示出的一种应用于第二终端的图像处理方法的流程图。如图9所示，图像处理方法，包括以下步骤。FIG. 9 is a flowchart of an image processing method applied to a second terminal according to an exemplary embodiment. As shown in Figure 9, the image processing method includes the following steps.

在步骤S81中，基于图像采集传感器采集原始图像。In step S81, original images are collected based on the image collection sensor.

在步骤S82中，将原始图像发送至第一终端，基于第一终端的所述图像显示指令和原始图像得到原始图像的显示图像。In step S82, the original image is sent to the first terminal, and a display image of the original image is obtained based on the image display instruction of the first terminal and the original image.

本公开实施例中，第二终端可以是图像采集传感器，第一终端可以是手机，图像采集传感器采集原始图像，将原始图像发送至手机，由手机对原始图像进行重处理，最终在手机上得到原始图像的显示图像。In the embodiment of the present disclosure, the second terminal can be an image acquisition sensor, and the first terminal can be a mobile phone. The image acquisition sensor collects original images, sends the original images to the mobile phone, and the mobile phone reprocesses the original images, and finally obtains the original image on the mobile phone. Display image of the original image.

接述上例，原始图像可以是天空，图像采集传感器采集到天空的原始图像之后，将天空的原始图像发送至手机，由手机对原始图像进行重处理，最终在手机屏幕上显示的是天空。Following the above example, the original image can be the sky. After the image acquisition sensor collects the original image of the sky, it sends the original image of the sky to the mobile phone. The mobile phone reprocesses the original image, and finally displays the sky on the screen of the mobile phone.

图10是根据一示例性实施例示出的***数据流图的示意图。参阅图10所示，本公开实施例中***数据流图分为两种数据流向，一种是数据流，一种是命令流。Figure 10 is a schematic diagram of a system data flow diagram according to an exemplary embodiment. Referring to Figure 10, the system data flow diagram in the embodiment of the present disclosure is divided into two data flow directions, one is the data flow and the other is the command flow.

本公开实施例的一种实现方式是通过Java的方法实现图像处理。以Java的方法、第一终端为手机、第二终端为图像采集器为例，图像处理方法数据流分为两部分，其中一部分为应用发送会话指令以及相应图像数据到相机服务，相机服务收到会话指令之后将相应图像数据传递到Hal层，Hal层再将会话指令以及相应图像数据发送至图像处理器中，以上数据流传递的目的是利用手机处理原始图像并显示。One implementation manner of the embodiments of the present disclosure is to implement image processing through Java methods. Taking the Java method as an example, the first terminal is a mobile phone and the second terminal is an image collector. The image processing method data flow is divided into two parts. One part is for the application to send session instructions and corresponding image data to the camera service. The camera service receives After the session command, the corresponding image data is passed to the Hal layer, and the Hal layer then sends the session command and the corresponding image data to the image processor. The purpose of the above data stream transfer is to use the mobile phone to process the original image and display it.

图像处理方法数据流向的另一部分是硬件网络接收到从图像采集器传递而来的原始图像数据，将原始图像数据传输至相机的移动信息网络，再传输至Java本地接口，最后传输至应用，达到控制图像采集器进行图像采集，并在手机上进行预览的过程。Another part of the data flow of the image processing method is that the hardware network receives the original image data passed from the image collector, transmits the original image data to the camera's mobile information network, then transmits it to the Java local interface, and finally transmits it to the application to achieve The process of controlling the image collector to collect images and preview them on the mobile phone.

***数据流图中的命令流，命令传输的流向，应用与Java本地接口之间可以互相传输命令，Java本地接口与相机的移动信息网之间可以相互传输命令，相机的移动信息网与硬件网络之间可以相互传输命令。The command flow in the system data flow diagram, the direction of command transmission, the application and the Java local interface can transmit commands to each other, the Java local interface and the camera's mobile information network can transmit commands to each other, the camera's mobile information network and the hardware network Commands can be transmitted to each other.

其中，将Java本地接口、相机的移动信息网络、相机服务、Hal层归到为用户层，即不需要通过***底层进行图像处理过程，将硬件网络和图像处理器归为内核层，即通过***底层进行图像处理过程。Among them, the Java local interface, the camera's mobile information network, the camera service, and the Hal layer are classified as the user layer, that is, there is no need to perform image processing through the bottom layer of the system, and the hardware network and image processor are classified as the kernel layer, that is, through the system The bottom layer performs image processing.

综上，本公开实施例提供的图像处理方法，是一种能够满足图像采集设备与原始图像显示设备分别位于不同终端时能够进行图像的显示，且实现方式快捷流畅的图像处理方案。In summary, the image processing method provided by the embodiments of the present disclosure is an image processing solution that can display images when the image acquisition device and the original image display device are located at different terminals, and the implementation method is fast and smooth.

需要说明的是，本领域内技术人员可以理解，本公开实施例上述涉及的各种实施方式/实施例中可以配合前述的实施例使用，也可以是独立使用。无论是单独使用还是配合前述的实施例一起使用，其实现原理类似。本公开实施中，部分实施例中是以一起使用的实施方式进行说明的。当然，本领域内技术人员可以理解，这样的举例说明并非对本公开实施例的限定。It should be noted that those skilled in the art can understand that the various implementations/embodiments mentioned above in the embodiments of the present disclosure can be used in conjunction with the foregoing embodiments or can be used independently. Whether used alone or in conjunction with the foregoing embodiments, the implementation principles are similar. In the implementation of the present disclosure, some embodiments are described in terms of implementations used together. Of course, those skilled in the art can understand that such illustrations do not limit the embodiments of the present disclosure.

基于相同的构思，本公开实施例还提供一种应用于第一终端的图像处理装置。Based on the same concept, embodiments of the present disclosure also provide an image processing device applied to a first terminal.

可以理解的是，本公开实施例提供的应用于第一终端的图像处理装置为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。结合本公开实施例中所公开的各示例的单元及算法步骤，本公开实施例能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同的方法来实现所描述的功能，但是这种实现不应认为超出本公开实施例的技术方案的范围。It can be understood that, in order to implement the above functions, the image processing device applied to the first terminal provided by the embodiment of the present disclosure includes a corresponding hardware structure and/or software module to perform each function. Combined with the units and algorithm steps of each example disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to go beyond the scope of the technical solutions of the embodiments of the present disclosure.

图11是根据一示例性实施例示出的一种应用于第一终端的图像处理装置框图。参照图11，该应用于第一终端的图像处理装置100包括确定单元101，处理单元102，显示单元103。FIG. 11 is a block diagram of an image processing device applied to a first terminal according to an exemplary embodiment. Referring to FIG. 11 , the image processing device 100 applied to the first terminal includes a determination unit 101 , a processing unit 102 , and a display unit 103 .

确定单元101，用于获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令。The determination unit 101 is used to obtain the original image collected by the image collection sensor in the second terminal, and determine the image display instruction.

处理单元102，用于基于图像显示指令和原始图像，得到原始图像的显示图像。The processing unit 102 is configured to obtain a display image of the original image based on the image display instruction and the original image.

显示单元103，用于显示原始图像的显示图像。The display unit 103 is used to display the display image of the original image.

在一种实施方式中，处理单元102用于：获取源缓存数据，源缓存数据为缓存在内存中的所述原始图像的源数据；基于源缓存数据，得到匹配图像显示指令的显示图像缓存队列；将显示图像缓存队列中的显示图像，作为原始图像的显示图像。In one implementation, the processing unit 102 is configured to: obtain source cache data, which is the source data of the original image cached in the memory; and obtain a display image cache queue matching the image display instruction based on the source cache data. ;The display image in the display image cache queue will be used as the display image of the original image.

在一种实施方式中，处理单元102中图像显示指令包括预览图像显示指令，或者预览图像显示指令和预览图像的拍照图像显示指令；基于所述源缓存数据，得到匹配所述图像显示指令的显示图像缓存队列，包括：基于源缓存数据中包括的会话捕获创建请求，获取源缓存数据的图像尺寸以及图像格式；基于源缓存数据的图像尺寸以及图像格式，构建基础图像处理目标；配置所述基础图像处理目标的目标输出流，目标输出流包括显示原始图像的预览图像的输出流，或者显示原始图像的预览图像的输出流以及显示预览图像的拍照图像的输出流；基于目标输出流对应的显示图像，生成原始图像的显示图像缓存队列。In one implementation, the image display instructions in the processing unit 102 include a preview image display instruction, or a preview image display instruction and a photographed image display instruction of the preview image; based on the source cache data, a display matching the image display instruction is obtained The image cache queue includes: obtaining the image size and image format of the source cache data based on the session capture creation request included in the source cache data; building a basic image processing target based on the image size and image format of the source cache data; configuring the base The target output stream of the image processing target, the target output stream includes an output stream that displays a preview image of the original image, or an output stream that displays a preview image of the original image, and an output stream that displays a photographed image of the preview image; a display based on the target output stream Image, generates a display image cache queue of the original image.

在一种实施方式中，处理单元102源缓存数据中包括的会话捕获创建请求采用如下方式预先创建：创建捕获请求函数，捕获请求函数的参数中包括对原始图像的重处理指令；根据原始图像的显示图像格式，创建输入配置对象，输入配置对象包括原始图像尺寸以及图像格式；基于已创建的捕获请求函数，将输入配置对象传输至***底层，由***底层创建捕获输入配置对象的会话捕获创建请求。In one implementation, the session capture creation request included in the source cache data of the processing unit 102 is pre-created in the following manner: a capture request function is created, and the parameters of the capture request function include instructions for reprocessing the original image; Display the image format, create an input configuration object, the input configuration object includes the original image size and image format; based on the created capture request function, transfer the input configuration object to the bottom layer of the system, and the bottom layer of the system creates a session capture creation request to capture the input configuration object .

在一种实施方式中，处理单元102采用如下方式基于所述源缓存数据中包括的会话捕获创建请求，获取源缓存数据的图像尺寸以及图像格式：基于多次会话捕获创建请求对应的捕获操作，获取到原始图像的源缓存数据；缓存原始图像的源缓存数据。In one implementation, the processing unit 102 obtains the image size and image format of the source cache data based on the session capture creation request included in the source cache data in the following manner: based on the capture operations corresponding to multiple session capture creation requests, Obtain the source cache data of the original image; cache the source cache data of the original image.

在一种实施方式中，处理单元102还用于：响应于创建输入配置对象完成，设置输入配置对象的监听机制；基于监听机制，监听传输至***底层的输入配置对象。In one implementation, the processing unit 102 is further configured to: in response to completion of creating the input configuration object, set a monitoring mechanism for the input configuration object; based on the monitoring mechanism, monitor the input configuration object transmitted to the bottom layer of the system.

在一种实施方式中，处理单元102还用于：若未监听到传输至***底层的输入配置对象，显示提示信息；提示信息用于提示不能获取到原始图像对应的输入配置对象。In one implementation, the processing unit 102 is also configured to: display prompt information if the input configuration object transmitted to the bottom layer of the system is not monitored; the prompt information is used to prompt that the input configuration object corresponding to the original image cannot be obtained.

基于相同的构思，本公开实施例还提供一种应用于第二终端的图像处理装置。Based on the same concept, embodiments of the present disclosure also provide an image processing device applied to a second terminal.

图12是根据一示例性实施例示出的一种应用于第二终端的图像处理装置框图。参照图12，该应用于第一终端的图像处理装置200包括采集单元201，发送单元202。FIG. 12 is a block diagram of an image processing apparatus applied to a second terminal according to an exemplary embodiment. Referring to FIG. 12 , the image processing device 200 applied to the first terminal includes a collection unit 201 and a sending unit 202 .

采集单元201，用于基于图像采集传感器采集原始图像，并由第一终端确定图像显示指令；The acquisition unit 201 is used to collect original images based on the image acquisition sensor, and determine the image display instructions by the first terminal;

发送单元202，用于将原始图像发送至第一终端，基于第一终端的所述图像显示指令和原始图像得到原始图像的显示图像。The sending unit 202 is configured to send the original image to the first terminal, and obtain a display image of the original image based on the image display instruction of the first terminal and the original image.

关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the devices in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

图13是根据一示例性实施例示出的一种用于图像处理的装置300的框图。例如，装置300可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等。FIG. 13 is a block diagram of an apparatus 300 for image processing according to an exemplary embodiment. For example, the device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

参照图13，装置300可以包括以下一个或多个组件：处理组件302，存储器304，电力组件306，多媒体组件308，音频组件310，输入/输出(I/O)接口312，传感器组件314，以及通信组件316。Referring to Figure 13, device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input/output (I/O) interface 312, sensor component 314, and Communication component 316.

处理组件302通常控制装置300的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件302可以包括一个或多个处理器320来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件302可以包括一个或多个模块，便于处理组件302和其他组件之间的交互。例如，处理组件302可以包括多媒体模块，以方便多媒体组件308和处理组件302之间的交互。 Processing component 302 generally controls the overall operations of device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 302 may include one or more modules that facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

存储器304被配置为存储各种类型的数据以支持在装置300的操作。这些数据的示例包括用于在装置300上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器304可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。 Memory 304 is configured to store various types of data to support operations at device 300 . Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

电力组件306为装置300的各种组件提供电力。电力组件306可以包括电源管理***，一个或多个电源，及其他与为装置300生成、管理和分配电力相关联的组件。 Power component 306 provides power to the various components of device 300 . Power components 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 300 .

多媒体组件308包括在所述装置300和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件308包括一个前置摄像头和/或后置摄像头。当装置300处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜***或具有焦距和光学变焦能力。 Multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, multimedia component 308 includes a front-facing camera and/or a rear-facing camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

音频组件310被配置为输出和/或输入音频信号。例如，音频组件310包括一个麦克风(MIC)，当装置300处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器304或经由通信组件316发送。在一些实施例中，音频组件310还包括一个扬声器，用于输出音频信号。 Audio component 310 is configured to output and/or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or sent via communication component 316 . In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

I/O接口312为处理组件302和***接口模块之间提供接口，上述***接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 312 provides an interface between the processing component 302 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.

传感器组件314包括一个或多个传感器，用于为装置300提供各个方面的状态评估。例如，传感器组件314可以检测到装置300的打开/关闭状态，组件的相对定位，例如所述组件为装置300的显示器和小键盘，传感器组件314还可以检测装置300或装置300一个组件的位置改变，用户与装置300接触的存在或不存在，装置300方位或加速/减速和装置300的温度变化。传感器组件314可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件314还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件314还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。 Sensor component 314 includes one or more sensors for providing various aspects of status assessment for device 300 . For example, the sensor component 314 can detect the open/closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, and the sensor component 314 can also detect a change in position of the device 300 or a component of the device 300. , the presence or absence of user contact with device 300 , device 300 orientation or acceleration/deceleration and temperature changes of device 300 . Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

通信组件316被配置为便于装置300和其他设备之间有线或无线方式的通信。装置300 可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件316经由广播信道接收来自外部广播管理***的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件316还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。 Communication component 316 is configured to facilitate wired or wireless communication between apparatus 300 and other devices. Device 300 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 316 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

在示例性实施例中，装置300可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。In an exemplary embodiment, apparatus 300 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the above method.

在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器304，上述指令可由装置300的处理器320执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 304 including instructions, which can be executed by the processor 320 of the device 300 to complete the above method is also provided. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

可以理解的是，本公开中“多个”是指两个或两个以上，其它量词与之类似。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。It can be understood that "plurality" in this disclosure refers to two or more, and other quantifiers are similar. "And/or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the related objects are in an "or" relationship. The singular forms "a", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

进一步可以理解的是，术语“第一”、“第二”等用于描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开，并不表示特定的顺序或者重要程度。实际上，“第一”、“第二”等表述完全可以互换使用。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other and do not imply a specific order or importance. In fact, expressions such as "first" and "second" can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information.

进一步可以理解的是，除非有特殊说明，“连接”包括两者之间不存在其他构件的直接连接，也包括两者之间存在其他元件的间接连接。It should be further understood that, unless otherwise specified, "connection" includes a direct connection without other components between the two, and also includes an indirect connection with other elements between the two.

进一步可以理解的是，本公开实施例中尽管在附图中以特定的顺序描述操作，但是不应将其理解为要求按照所示的特定顺序或是串行顺序来执行这些操作，或是要求执行全部所示的操作以得到期望的结果。在特定环境中，多任务和并行处理可能是有利的。It will be further understood that although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, this should not be understood as requiring that these operations be performed in the specific order shown or in a serial order, or that it is required that Perform all operations shown to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. .

应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利范围来限制。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosure is limited only by the appended rights.

Claims

一种图像处理方法，其特征在于，应用于第一终端，所述图像处理方法包括：An image processing method, characterized in that it is applied to a first terminal, and the image processing method includes:

获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令；Obtain the original image collected by the image acquisition sensor in the second terminal, and determine the image display instruction;

基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像；Based on the image display instruction and the original image, obtain a display image of the original image;

显示所述原始图像的显示图像。Display the display image of the original image.
根据权利要求1所述的图像处理方法，其特征在于，所述基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像，包括：The image processing method according to claim 1, characterized in that, obtaining a display image of the original image based on the image display instruction and the original image includes:

获取源缓存数据，所述源缓存数据为缓存在内存中的所述原始图像的源数据；Obtain source cache data, which is the source data of the original image cached in the memory;

基于所述源缓存数据，得到匹配所述图像显示指令的显示图像缓存队列；Based on the source cache data, obtain a display image cache queue matching the image display instruction;

将所述显示图像缓存队列中的显示图像，作为所述原始图像的显示图像。The display image in the display image cache queue is used as the display image of the original image.
根据权利要求2所述的图像处理方法，其特征在于，所述图像显示指令包括预览图像显示指令，或者预览图像显示指令和预览图像的拍照图像显示指令；The image processing method according to claim 2, characterized in that the image display instruction includes a preview image display instruction, or a preview image display instruction and a photographed image display instruction of the preview image;

所述基于所述源缓存数据，得到匹配所述图像显示指令的显示图像缓存队列，包括：Obtaining a display image cache queue matching the image display instruction based on the source cache data includes:

基于所述源缓存数据中包括的会话捕获创建请求，获取所述源缓存数据的图像尺寸以及图像格式；Obtain the image size and image format of the source cache data based on the session capture creation request included in the source cache data;

基于所述源缓存数据的图像尺寸以及图像格式，构建基础图像处理目标；Construct a basic image processing target based on the image size and image format of the source cache data;

配置所述基础图像处理目标的目标输出流，所述目标输出流包括显示所述原始图像的预览图像的输出流，或者显示所述原始图像的预览图像的输出流以及显示所述预览图像的拍照图像的输出流；Configuring a target output stream of the basic image processing target, the target output stream including an output stream displaying a preview image of the original image, or an output stream displaying a preview image of the original image and a photograph displaying the preview image Image output stream;

基于所述目标输出流对应的显示图像，生成所述原始图像的显示图像缓存队列。Based on the display image corresponding to the target output stream, a display image cache queue of the original image is generated.
根据权利要求3所述的图像处理方法，其特征在于，所述源缓存数据中包括的会话捕获创建请求采用如下方式预先创建：The image processing method according to claim 3, characterized in that the session capture creation request included in the source cache data is pre-created in the following manner:

创建捕获请求函数，所述捕获请求函数的参数中包括对所述原始图像的重处理指令；Create a capture request function, the parameters of the capture request function include reprocessing instructions for the original image;

根据所述原始图像的显示图像格式，创建输入配置对象，所述输入配置对象包括所述原始图像尺寸以及图像格式；Create an input configuration object according to the display image format of the original image, the input configuration object including the original image size and image format;

基于已创建的捕获请求函数，将所述输入配置对象传输至***底层，由***底层创建捕获输入配置对象的会话捕获创建请求。Based on the created capture request function, the input configuration object is transmitted to the bottom layer of the system, and the bottom layer of the system creates a session capture creation request that captures the input configuration object.
根据权利要求3或4所述的图像处理方法，其特征在于，所述基于所述源缓存数据中包括的会话捕获创建请求，获取所述源缓存数据的图像尺寸以及图像格式，包括：The image processing method according to claim 3 or 4, characterized in that, based on the session capture creation request included in the source cache data, obtaining the image size and image format of the source cache data includes:

基于多次会话捕获创建请求对应的捕获操作，获取到所述原始图像的源缓存数据；Based on the capture operations corresponding to multiple session capture creation requests, obtain the source cache data of the original image;

缓存所述原始图像的源缓存数据。Cache the source cache data of the original image.
根据权利要求4所述的图像处理方法，其特征在于，所述方法还包括：The image processing method according to claim 4, characterized in that the method further includes:

响应于创建输入配置对象完成，设置输入配置对象的监听机制；In response to completion of creating the input configuration object, set the listening mechanism of the input configuration object;

基于所述监听机制，监听传输至***底层的输入配置对象。Based on the monitoring mechanism, the input configuration object transmitted to the bottom layer of the system is monitored.
根据权利要求6所述的图像处理方法，其特征在于，所述方法还包括：The image processing method according to claim 6, characterized in that the method further includes:

若未监听到传输至***底层的输入配置对象，显示提示信息；If the input configuration object transmitted to the bottom layer of the system is not monitored, a prompt message is displayed;

所述提示信息用于提示不能获取到所述原始图像对应的输入配置对象。The prompt information is used to prompt that the input configuration object corresponding to the original image cannot be obtained.
一种图像处理方法，其特征在于，应用于第二终端，所述图像处理方法包括：An image processing method, characterized in that it is applied to a second terminal, and the image processing method includes:

基于图像采集传感器采集原始图像；Collect original images based on image acquisition sensors;

将所述原始图像发送至第一终端，基于所述第一终端的图像显示指令和所述原始图像得到所述原始图像的显示图像。The original image is sent to the first terminal, and a display image of the original image is obtained based on the image display instruction of the first terminal and the original image.
一种图像处理装置，其特征在于，应用于第一终端，所述图像处理装置包括：An image processing device, characterized in that it is applied to a first terminal, and the image processing device includes:

确定单元，用于获取第二终端中图像采集传感器采集的原始图像，并确定图像显示指令；a determination unit, used to obtain the original image collected by the image acquisition sensor in the second terminal, and determine the image display instruction;

处理单元，用于基于所述图像显示指令和所述原始图像，得到所述原始图像的显示图像；A processing unit configured to obtain a display image of the original image based on the image display instruction and the original image;

显示单元，用于显示所述原始图像的显示图像。A display unit configured to display a display image of the original image.
一种图像处理装置，其特征在于，应用于第二终端，所述图像处理装置包括：An image processing device, characterized in that it is applied to a second terminal, and the image processing device includes:

采集单元，用于基于图像采集传感器采集原始图像，并由第一终端确定图像显示指令；An acquisition unit, used to acquire original images based on the image acquisition sensor, and determine the image display instructions by the first terminal;

发送单元，用于将所述原始图像发送至第一终端，基于所述第一终端的所述图像显示指令和所述原始图像得到所述原始图像的显示图像。A sending unit, configured to send the original image to a first terminal, and obtain a display image of the original image based on the image display instruction of the first terminal and the original image.
一种图像处理装置，其特征在于，包括：An image processing device, characterized in that it includes:

处理器；processor;

用于存储处理器可执行指令的存储器；Memory used to store instructions executable by the processor;

其中，所述处理器被配置为：执行权利要求1至7中任意一项所述的图像处理方法，或执行权利要求8中的图像处理方法。Wherein, the processor is configured to: execute the image processing method according to any one of claims 1 to 7, or execute the image processing method according to claim 8.
一种存储介质，其特征在于，所述存储介质中存储有指令，当所述存储介质中的指令由终端的处理器执行时，使得终端能够执行权利要求1至7中任意一项所述的图像处理方法，或执行权利要求8中的图像处理方法。A storage medium, characterized in that instructions are stored in the storage medium. When the instructions in the storage medium are executed by a processor of a terminal, the terminal can execute the method described in any one of claims 1 to 7. Image processing method, or perform the image processing method in claim 8.