WO2023065523A1

WO2023065523A1 - Isp-based image processing method and apparatus, storage medium, and camera device

Info

Publication number: WO2023065523A1
Application number: PCT/CN2021/140342
Authority: WO
Inventors: 段亚勇
Original assignee: 上海闻泰电子科技有限公司
Priority date: 2021-10-20
Filing date: 2021-12-22
Publication date: 2023-04-27
Also published as: CN113992850B; CN113992850A

Abstract

Embodiments of the present disclosure provide an ISP-based image processing method and apparatus, a storage medium, and a camera device. The method comprises: determining the current photographing scene mode; obtaining a corresponding calibration image file; processing the calibration image file on the basis of an image processing algorithm corresponding to the current photographing scene mode to obtain a simulation image; and optimizing a parameter configuration file of the image processing algorithm according to the simulation image, and processing the calibration image file according to the optimized image processing algorithm to obtain a target image. According to the present invention, the calibration image file is processed by means of the image processing algorithm, the image processing algorithm is debugged and optimized according to a processing result, and since the calibration image file is pre-stored, continuous photographing is not needed by processing the pre-stored calibration image file, and algorithm debugging obstacles caused by uncontrollable factors such as the change of a photographing environment and the change of a photographing angle can be avoided, so that the algorithm debugging accuracy is improved while the algorithm debugging efficiency is improved.

Description

基于ISP的图像处理方法、装置、存储介质和摄像设备ISP-based image processing method, device, storage medium and camera equipment

本公开要求于2021年10月20日提交中国专利局、申请号为202111221020.8、发明名称为“基于ISP的图像处理方法及装置、存储介质和摄像设备”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。This disclosure claims the priority of the Chinese patent application with the application number 202111221020.8 submitted to the China Patent Office on October 20, 2021, and the title of the invention is "ISP-based image processing method and device, storage medium and camera equipment", the entire content of which Incorporated by reference in this disclosure.

技术领域technical field

本公开涉及一种基于ISP的图像处理方法及装置、存储介质和摄像设备。The disclosure relates to an ISP-based image processing method and device, a storage medium, and an imaging device.

背景技术Background technique

目前，相机传感器硬件上的缺陷，导致从相机传感器输出的原始数字图像存在很多缺陷，因此ISP(Image Signal Processing，图像信号处理)技术应运而生，由于不断的优化和扩充，ISP模块的中的图像处理算法种类也随之增加，例如黑电平矫正、坏点矫正、白平衡矫正、镜头阴影矫正等。然而随着相机的发展，以及拍摄需求的不断提升，在不同的场景下，可能需要新的算法去处理图像来适应不同的场景需求，例如适应夜景模式、人像美颜、背景虚化模式等。At present, the defects in the hardware of the camera sensor lead to many defects in the original digital image output from the camera sensor, so the ISP (Image Signal Processing, Image Signal Processing) technology came into being. Due to continuous optimization and expansion, the ISP module The types of image processing algorithms have also increased, such as black level correction, bad point correction, white balance correction, lens shadow correction, etc. However, with the development of cameras and the continuous improvement of shooting requirements, in different scenarios, new algorithms may be required to process images to meet the needs of different scenarios, such as adapting to night scene mode, portrait beauty, background blur mode, etc.

以夜景模式为例，普通的拍照模式在夜晚拍照的图像质量并不客观，无论是从高光处的清晰度，还是暗光的亮度，或者动态物体的对齐方向上都存在较大缺陷。针对这一问题，一般是在ISP流程的raw域植入第三方的夜景模式算法来对图像进行处理。但是，要解决这个问题避免不了对算法参数的调试，通常这种大型算法的参数多达数千个，若想调试出一个完美的参数搭配，是非常困难的。通常需要在不同的环境中，使用集成算法的设备和对比设备对同一物体进行拍照，在对得到的图像进行对比，根据对比结果对算法进行调试。Taking the night scene mode as an example, the image quality of ordinary photo mode at night is not objective, whether it is from the clarity of high light, the brightness of dark light, or the alignment direction of dynamic objects, there are big defects. To solve this problem, a third-party night scene mode algorithm is generally implanted in the raw domain of the ISP process to process the image. However, to solve this problem, the debugging of algorithm parameters cannot be avoided. Usually, there are thousands of parameters in such a large-scale algorithm. It is very difficult to debug a perfect parameter matching. Usually, it is necessary to use the integrated algorithm equipment and comparison equipment to take pictures of the same object in different environments, compare the obtained images, and debug the algorithm according to the comparison results.

然而，采用上述方式需要不断拍摄图像，从而浪费大量时间，同时，由于拍摄环境、拍摄角度及抖动等因素的影响，会使图像对比的难度增大，既浪费时间，也使得算法调试难度较大，调试效率及准确性不高。However, using the above method requires continuous shooting of images, which wastes a lot of time. At the same time, due to the influence of factors such as shooting environment, shooting angle, and shaking, it will make image comparison more difficult, which not only wastes time, but also makes algorithm debugging difficult. , The debugging efficiency and accuracy are not high.

发明内容Contents of the invention

(一)要解决的技术问题(1) Technical problems to be solved

由于拍摄环境、拍摄角度及抖动等因素的影响，会使图像对比的难度增大，既浪费时间，也使得算法调试难度较大，调试效率及准确性不高。Due to the influence of factors such as shooting environment, shooting angle and shaking, it will make image comparison more difficult, which not only wastes time, but also makes algorithm debugging difficult, and the debugging efficiency and accuracy are not high.

(二)技术方案(2) Technical solutions

根据本公开公开的各种实施例，提供一种基于ISP的图像处理方法、及装置、存储介质和摄像设备。According to various embodiments of the present disclosure, an ISP-based image processing method, device, storage medium, and imaging device are provided.

一种基于ISP的图像处理方法，所述方法包括：An ISP-based image processing method, the method comprising:

确定当前拍摄场景模式；Determine the current shooting scene mode;

获取对应于所述当前拍摄场景模式的标定图像文件；Acquiring a calibration image file corresponding to the current shooting scene mode;

基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；Processing the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image.

作为本公开实施例一种可选的实施方式,As an optional implementation manner of the embodiment of the present disclosure,

获取对应于所述当前拍摄场景模式的标定图像文件，包括：获取当前拍摄场景下拍摄的初始图像；将所述初始图像对应的数字图像文件替换为预设数字图像文件，将所述初始图像对应的数据信息替换为预设数据信息，以得到所述标定图像文件。Obtaining a calibration image file corresponding to the current shooting scene mode includes: obtaining an initial image taken under the current shooting scene; replacing the digital image file corresponding to the initial image with a preset digital image file, and converting the initial image corresponding to The data information is replaced with preset data information to obtain the calibration image file.

作为本公开实施例一种可选的实施方式,根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，包括：将所述仿真图像与预设图像进行对比，根据对比结果对所述图像处理算法的参数配置文件进行优化。As an optional implementation manner of the embodiment of the present disclosure, optimizing the parameter configuration file of the image processing algorithm according to the simulated image includes: comparing the simulated image with a preset image, and comparing the The parameter configuration file of the image processing algorithm described above is optimized.

作为本公开实施例一种可选的实施方式,根据对比结果对所述图像处理算法的参数配置文件进行优化，包括：在所述仿真图像相对于所述预设图像存在缺陷的条件下，则对所述图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于所述预存图像不存在缺陷。As an optional implementation manner of the embodiment of the present disclosure, optimizing the parameter configuration file of the image processing algorithm according to the comparison result includes: under the condition that the simulated image has defects relative to the preset image, then The parameter configuration file of the image processing algorithm is debugged until the simulated image processed according to the debugged image processing algorithm has no defects relative to the pre-stored image.

作为本公开实施例一种可选的实施方式,所述当前拍摄场景模式包括夜景模式、人像模式、背景虚化模式及广角镜头模式中的任意一种。As an optional implementation manner of the embodiment of the present disclosure, the current shooting scene mode includes any one of night scene mode, portrait mode, background blur mode and wide-angle lens mode.

作为本公开实施例一种可选的实施方式，所述标定图像文件包括：数字图像缓冲寄存器和元数据；其中，所述数字图像缓冲寄存器为摄像头传感器采集的原始数字图像文件，所述元数据为拍照时图像的属性信息，所述属性信息至少包括：感光度、曝光时间、白平衡增益及颜色校正矩阵。As an optional implementation of the embodiment of the present disclosure, the calibration image file includes: a digital image buffer register and metadata; wherein, the digital image buffer register is an original digital image file collected by a camera sensor, and the metadata It is the attribute information of the image when taking pictures, and the attribute information at least includes: sensitivity, exposure time, white balance gain and color correction matrix.

作为本公开实施例一种可选的实施方式，所述基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像，包括：调取ISP模块中所述当前拍摄场景模式相应所述图像处理算法；基于所述当前拍摄场景模式相应的所述图像处理算法，对所述标定图像文件进行处理，以得到所述标定图像文件对应的所述仿真图像。As an optional implementation manner of the embodiment of the present disclosure, the processing of the calibration image file based on the image processing algorithm in the ISP module corresponding to the current shooting scene mode to obtain a simulation image includes: calling the ISP module The current shooting scene mode corresponds to the image processing algorithm; based on the image processing algorithm corresponding to the current shooting scene mode, the calibration image file is processed to obtain the simulation corresponding to the calibration image file image.

作为本公开实施例一种可选的实施方式，针对不同的所述仿真图像，分别对应不同的所述参数配置文件，且对应相同的预设因素，其中，所述预设因素至少包括：拍摄图像角度以及拍摄图像抖动。As an optional implementation manner of the embodiment of the present disclosure, different simulation images correspond to different parameter configuration files and correspond to the same preset factors, wherein the preset factors at least include: shooting Image angles and camera shake.

作为本公开实施例一种可选的实施方式，所述获取对应于所述当前拍摄场景模式的标定图像文件；基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像，包括：接收用户输入的图像调试指令；响应所述图像调试指令，获取对应于所述当前拍摄场景模式的标定图像文件；基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。As an optional implementation of the embodiment of the present disclosure, the acquisition of the calibration image file corresponding to the current shooting scene mode; based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, the calibration image The file is processed to obtain a simulation image; the parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image, including: receiving the user input image debugging instruction; in response to the image debugging instruction, obtain the calibration image file corresponding to the current shooting scene mode; perform the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module process to obtain a simulation image; optimize the parameter configuration file of the image processing algorithm according to the simulation image, and process the calibration image file according to the optimized image processing algorithm to obtain a target image.

一种基于ISP的图像处理装置，该装置包括：An ISP-based image processing device, the device comprising:

确定模块，配置成确定当前拍摄场景模式；A determination module configured to determine the current shooting scene mode;

获取模块，配置成获取对应于所述当前拍摄场景模式的标定图像文件；An acquisition module configured to acquire a calibration image file corresponding to the current shooting scene mode;

第一处理模块，配置成基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；The first processing module is configured to process the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

第二处理模块，配置成根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The second processing module is configured to optimize the parameter configuration file of the image processing algorithm according to the simulated image, and process the calibration image file according to the optimized image processing algorithm to obtain a target image.

作为本公开实施例一种可选的实施方式，所述获取模块，具体配置成：获取当前拍摄场景下拍摄的初始图像；将所述初始图像对应的数字图像文件替换为预设数字图像文件，将所述初始图像对应的数据信息替换为预设数据信息，以得到所述标定图像文件。As an optional implementation manner of the embodiment of the present disclosure, the acquiring module is specifically configured to: acquire an initial image captured in the current shooting scene; replace the digital image file corresponding to the initial image with a preset digital image file, The data information corresponding to the initial image is replaced with preset data information to obtain the calibration image file.

作为本公开实施例一种可选的实施方式，所述第二处理模块，具体配置成：将所述仿真图像与预设图像进行对比，根据对比结果对所述图像处理算法的参数配置文件进行优化。As an optional implementation manner of the embodiment of the present disclosure, the second processing module is specifically configured to: compare the simulated image with a preset image, and perform a parameter configuration file of the image processing algorithm according to the comparison result optimization.

作为本公开实施例一种可选的实施方式，所述第二处理模块，具体配置成：在所述仿真图像相对于所述预设图像存在缺陷的条件下，对所述图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于所述预存图像不存在缺陷。As an optional implementation manner of the embodiment of the present disclosure, the second processing module is specifically configured to: under the condition that the simulated image has defects relative to the preset image, modify the parameters of the image processing algorithm The configuration file is debugged until the simulated image processed according to the debugged image processing algorithm has no defects relative to the pre-stored image.

作为本公开实施例一种可选的实施方式，所述第一处理模块，具体配置成：调取ISP模块中所述当前拍摄场景模式相应图像处理算法；基于所述当前拍摄场景模式相应的图像处理算法，对所述标定图像文件进行处理，以得到所述标定图像文件对应的所述仿真图像。As an optional implementation of the embodiment of the present disclosure, the first processing module is specifically configured to: call the image processing algorithm corresponding to the current shooting scene mode in the ISP module; based on the image corresponding to the current shooting scene mode A processing algorithm is used to process the calibration image file to obtain the simulation image corresponding to the calibration image file.

作为本公开实施例一种可选的实施方式，图像调试指令接收模块，配置成接收用户输入的图像调试指令；响应模块，配置成响应所述图像调试指令，获取对应于所述当前拍摄场景模式的标定图像文件；基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。As an optional implementation manner of the embodiment of the present disclosure, the image debugging instruction receiving module is configured to receive the image debugging instruction input by the user; the response module is configured to respond to the image debugging instruction and obtain the image corresponding to the current shooting scene mode The calibration image file; based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, the calibration image file is processed to obtain a simulation image; the parameter configuration file of the image processing algorithm is performed according to the simulation image. Optimizing, processing the calibration image file according to the optimized image processing algorithm to obtain the target image.

一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，所述计算机可读存储介质上存储有基于ISP的图像处理指令，所述基于ISP的图像处理指令被一个或多个处理器执行时实现如上述实施例所述的基于ISP的图像处理方法。One or more non-volatile computer-readable storage media storing computer-readable instructions, the computer-readable storage medium storing ISP-based image processing instructions, and the ISP-based image processing instructions being controlled by one or more When each processor is executed, the ISP-based image processing method as described in the above-mentioned embodiments is realized.

一种摄像设备，该摄像设备包括：上述实施例所述的基于ISP的图像处理装置；或者，该摄像设备包括：一个或多个处理器、存储器和存储在所述存储器上并可在所述一个或多个处理器上运行的基于ISP的图像处理指令，所述基于ISP的图像处理指令被所述一个或多个处理器执行时实现如上述实施例所述的基于ISP的图像处理方法。A kind of photographing equipment, this photographing equipment comprises: the image processing device based on ISP described in above-mentioned embodiment; Or, this photographing equipment comprises: one or more processors, memory and storage on described memory and can be in described memory An ISP-based image processing instruction running on one or more processors, when the ISP-based image processing instruction is executed by the one or more processors, implements the ISP-based image processing method as described in the foregoing embodiments.

本公开的其他特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本公开而了解。本公开的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得，本公开的一个或多个实施例的细节在下面的附图和描述中提出。Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the disclosure will be realized and attained by the structure particularly pointed out in the written description, claims hereof as well as the accompanying drawings, the details of one or more embodiments of the disclosure being set forth in the accompanying drawings and the description below.

为使本公开的上述目的、特征和优点能更明显易懂，下文特举可选实施例，并配合所附附图，作详细说明如下。In order to make the above objects, features and advantages of the present disclosure more comprehensible, optional embodiments are given below and described in detail in conjunction with the accompanying drawings.

附图说明Description of drawings

为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can also be obtained from these drawings without paying creative labor.

图1是根据本发明一个或多个实施例的基于ISP的图像处理方法的流程图；1 is a flowchart of an ISP-based image processing method according to one or more embodiments of the present invention;

图2是根据本发明一个或多个实施例的基于ISP的图像处理方法的流程图；2 is a flowchart of an ISP-based image processing method according to one or more embodiments of the present invention;

图3是根据本发明一个或多个实施例的基于ISP的图像处理装置的框图；3 is a block diagram of an ISP-based image processing device according to one or more embodiments of the present invention;

图4是根据本发明一个或多个实施例的摄像设备的框图。FIG. 4 is a block diagram of an imaging apparatus according to one or more embodiments of the present invention.

具体实施方式Detailed ways

为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways than described here; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, and Not all examples.

在本公开实施例中，“示例性的”或者“例如”等词来表示作例子、例证或说明。本公开实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念，此外，在本公开实施例的描述中，除非另有说明，“多个”的含义是指两个或两个以上。In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be construed as being preferred or advantageous over other embodiments or designs. To be precise, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "plurality" refers to two one or more.

下面参考图1和图2描述根据本发明实施例的基于ISP的图像处理方法，如图1所示，本发明实施例的基于ISP的图像处理方法至少包括步骤S1-步骤S4。The following describes an ISP-based image processing method according to an embodiment of the present invention with reference to FIG. 1 and FIG. 2. As shown in FIG.

步骤S1，确定当前拍摄场景模式。Step S1, determining the current shooting scene mode.

在实施例中，当前拍摄场景模式包括夜景模式、人像美颜模式、背景虚化模式及广角镜头模式等，根据拍摄需求不同，用户可选择不同的拍摄场景模式，相应的，不同拍摄场景模式对应的图像处理算法也不同，对拍摄图像的处理方式也不同。通过确定当前拍摄场景模式，便于在当前拍摄场景模式下进行图像的拍摄，以便调取相应的图像处理算法来对拍摄图像进行处理，最终输出用户需求的图像。In the embodiment, the current shooting scene mode includes night scene mode, portrait beauty mode, background blur mode and wide-angle lens mode, etc., according to different shooting requirements, the user can select different shooting scene modes, correspondingly, different shooting scene modes correspond to The image processing algorithms are also different, and the processing methods for the captured images are also different. By determining the current shooting scene mode, it is convenient to shoot images in the current shooting scene mode, so as to call the corresponding image processing algorithm to process the captured images, and finally output the image required by the user.

步骤S2，获取对应于当前拍摄场景模式的标定图像文件。Step S2, acquiring a calibration image file corresponding to the current shooting scene mode.

在实施例中，标定图像文件是提前存储至文件***指定目录下的图像文件，通过对某一物体进行多次拍摄，得到多组离线图像文件，将图像质量相对较好的离线图像文件作为标定图像文件，并存储至摄像设备例如手机的文件***指定目录下，以便后续的调取使用。In the embodiment, the calibration image file is an image file stored in the specified directory of the file system in advance. Multiple sets of offline image files are obtained by taking multiple shots of an object, and the offline image file with relatively good image quality is used as the calibration The image file is stored in the specified directory of the file system of the camera device, such as a mobile phone, for subsequent retrieval and use.

可以理解的是，标定图像文件包括数字图像buffer(缓冲寄存器)和metadata(元数据)，数字图像buffer指的是从摄像头传感器采集的原始数字图像文件，例如通过图像传感器采集的原始图像。Metadata数据指的是拍照时图像的各种属性，例如ISO(International Standards Organization，感光度)、曝光时间、白平衡增益及颜色校正矩阵等属性数据。通过获取对应于当前拍摄场景模式的标定图像文件，便于使用标定图像文件替代实时拍摄的图像文件进行图像处理，由于标定图像文件是预存的，因此，使用标定图像文件，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等的不可控因素导致的调试障碍，从而，在提高调试效率的同时，提高了调试的准确性。其中，不同拍摄场景模式下，由于图像处理需求不同，预存的标定图像文件对应不同。It can be understood that the calibration image file includes a digital image buffer (buffer register) and metadata (metadata), and the digital image buffer refers to an original digital image file collected from a camera sensor, such as an original image collected by an image sensor. Metadata data refers to various attributes of the image when taking pictures, such as ISO (International Standards Organization, sensitivity), exposure time, white balance gain, and color correction matrix and other attribute data. By obtaining the calibration image file corresponding to the current shooting scene mode, it is convenient to use the calibration image file instead of the real-time shooting image file for image processing. Since the calibration image file is pre-stored, using the calibration image file does not need to continuously shoot, and can avoid shooting Debugging obstacles caused by uncontrollable factors such as changes in the environment and shooting angles, thereby improving debugging accuracy while improving debugging efficiency. Among them, in different shooting scene modes, due to different image processing requirements, the pre-stored calibration image files correspond to different ones.

步骤S3，基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像。Step S3, based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, the calibration image file is processed to obtain a simulation image.

在实施例中，获取当前拍摄场景模式的标定图像文件后，根据当前拍摄场景模式调用ISP模块中对应于该当前拍摄场景模式的图像处理算法，例如当前拍摄场景模式为夜景模式，则调用对应于夜间模式的图像处理算法，又例如当前拍摄场景模式为人像美颜模式，则调用对应于人像美颜模式的图像处理算法。进而，基于ISP模块中的调取的相应图像处理算法对标定图像文件进行处理，即可得到文件***中预先存放的标定图像文件经处理后的仿真图像。In an embodiment, after obtaining the calibration image file of the current shooting scene mode, the image processing algorithm corresponding to the current shooting scene mode in the ISP module is called according to the current shooting scene mode, for example, the current shooting scene mode is night scene mode, then the call corresponding to The image processing algorithm of the night mode, and for example, the current shooting scene mode is the portrait beautification mode, and the image processing algorithm corresponding to the portrait beautification mode is invoked. Furthermore, by processing the calibration image file based on the corresponding image processing algorithm called in the ISP module, the processed simulation image of the calibration image file stored in the file system in advance can be obtained.

可以理解的是，通过将不同拍摄场景模式的图像处理算法写入ISP模块中的特定节点，以在不同场景模式下对标定图像文件的处理，从而达到在不同场景下对图像处理的不同需求。It can be understood that by writing the image processing algorithms of different shooting scene modes into specific nodes in the ISP module, the calibration image files can be processed in different scene modes, so as to meet different requirements for image processing in different scenes.

步骤S4，根据仿真图像对图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对标定图像文件进行处理，得到目标图像。Step S4, optimize the parameter configuration file of the image processing algorithm according to the simulation image, and process the calibration image file according to the optimized image processing algorithm to obtain the target image.

在实施例中，目标图像是当前拍摄场景模式下的质量最优的图像或者可以满足用户需求的图像。得到仿真图像后，检测仿真图像是否存在缺陷，若仿真图像存在缺陷，则根据仿真图像对图像处理算法的参数配置文件进行优化，例如通过修改图像处理算法中的参数配置文件，在修改完成后，再次触发拍照流程，对标定图像文件再次进行仿真，再次输出仿真图像，直至得到目标图像。In an embodiment, the target image is an image with the best quality in the current shooting scene mode or an image that can meet user requirements. After obtaining the simulated image, check whether there is a defect in the simulated image. If there is a defect in the simulated image, optimize the parameter configuration file of the image processing algorithm according to the simulated image. For example, by modifying the parameter configuration file in the image processing algorithm, after the modification is completed, Trigger the photographing process again, simulate the calibration image file again, and output the simulation image again until the target image is obtained.

可以理解的是，在不同参数配置文件下输出的仿真图像，该仿真图像除了参数配置文件不同，其余不可控因素均相同，例如拍摄角度及抖动情况等不可控因素均相同，因此，仅通过对图像处理算法中的参数配置文件进行优化，即可得到质量最优的目标图像。It is understandable that the simulated images output under different parameter configuration files have the same uncontrollable factors except for the different parameter configuration files, such as the uncontrollable factors such as shooting angle and jitter. Therefore, only through the By optimizing the parameter configuration file in the image processing algorithm, the target image with the best quality can be obtained.

根据本发明实施例的基于ISP的图像处理方法，通过获取对应于当前拍摄场景模式的标定图像文件，基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像，根据仿真图像对图像处理算法的参数配置文件进行优化，并根据优化后的图像处理算法对标定图像文件进行处理，得到目标图像，由此，通过图像处理算法对标定图像文件进行处理，根据处理结果对图像处理算法进行调试优化，由于标定图像文件是预存的，通过对预存的标定图像文件进行处理，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等不可控因素导致的算法调试障碍，从而，在提高算法调试效率的同时，提高了算法调试的准确性。According to the ISP-based image processing method of the embodiment of the present invention, by obtaining the calibration image file corresponding to the current shooting scene mode, the calibration image file is processed based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image , optimize the parameter configuration file of the image processing algorithm according to the simulation image, and process the calibration image file according to the optimized image processing algorithm to obtain the target image, thus, process the calibration image file through the image processing algorithm, according to the processing As a result, the image processing algorithm was debugged and optimized. Since the calibration image files are pre-stored, by processing the pre-stored calibration image files, there is no need to continuously shoot, and the algorithm debugging obstacles caused by uncontrollable factors such as shooting environment and shooting angle changes can be avoided. , while improving the efficiency of algorithm debugging, the accuracy of algorithm debugging is improved.

在一些实施例中，获取对应于当前拍摄场景模式的标定图像文件，包括：获取当前拍摄场景下拍摄的初始图像；将初始图像对应的数字图像文件替换为预设数字图像文件，将初始图像对应的数据信息替换为预设数据信息，以得到标定图像文件。In some embodiments, obtaining a calibration image file corresponding to the current shooting scene mode includes: obtaining an initial image taken under the current shooting scene; replacing the digital image file corresponding to the initial image with a preset digital image file, and converting the initial image corresponding to The data information is replaced with the preset data information to obtain the calibration image file.

在实施例中，获取当前拍摄场景下拍摄的初始图像的过程中，例如触发拍照流程时，使用预设的数字图像文件代替初始图像对应的数字图像文件，使用预设数据信息代替初始图像对应的数据信息，因此，在触发拍照流程时，得到的图像文件为标定图像文件。通过使用当前拍摄场景模式的标定图像文件代替初始图像，并在使用图像处理算法进行图像处理时，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等的不可控因素导致的调试障碍，从而，在提高调试效率的同时，提高了调试的准确性。In the embodiment, in the process of acquiring the initial image captured in the current shooting scene, for example, when the photographing process is triggered, the preset digital image file is used to replace the digital image file corresponding to the initial image, and the preset data information is used to replace the corresponding digital image file of the initial image. Data information, therefore, when the photo-taking process is triggered, the obtained image file is a calibration image file. By using the calibration image file of the current shooting scene mode to replace the initial image, and when using the image processing algorithm for image processing, there is no need to continuously shoot, which can avoid debugging obstacles caused by uncontrollable factors such as shooting environment and shooting angle changes. While improving the debugging efficiency, the debugging accuracy is improved.

在一些实施例中，根据仿真图像对图像处理算法的参数配置文件进行优化，包括：将仿真图像与预设图像进行对比，根据对比结果对图像处理算法的参数配置文件进行优化。In some embodiments, optimizing the parameter configuration file of the image processing algorithm according to the simulation image includes: comparing the simulation image with a preset image, and optimizing the parameter configuration file of the image processing algorithm according to the comparison result.

在实施例中，预设图像可以理解为至少需要达到的图像质量，得到仿真图像后，通过将仿真图像与预设图像进行对比，例如可以进行图像的各项参数对比，根据比较结果对图像处理算法的参数配置文件进行调整优化，以对图像处理算法进行调试优化，在对图像处理算法进行调试优化后，根据优化后的图像处理算法再次对标定图像进行处理，再根据处理结果优化图像处理算法，从而实现对图像处理算法的不断调整。In the embodiment, the preset image can be understood as at least the image quality that needs to be achieved. After the simulated image is obtained, by comparing the simulated image with the preset image, for example, various parameters of the image can be compared, and the image can be processed according to the comparison result. Adjust and optimize the parameter configuration file of the algorithm to debug and optimize the image processing algorithm. After debugging and optimizing the image processing algorithm, process the calibration image again according to the optimized image processing algorithm, and then optimize the image processing algorithm according to the processing results , so as to realize the continuous adjustment of the image processing algorithm.

在一些实施例中，根据对比结果对图像处理算法的参数配置文件进行优化，包括：若仿真图像相对于预设图像存在缺陷，则对图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于预存图像不存在缺陷。In some embodiments, optimizing the parameter configuration file of the image processing algorithm according to the comparison results includes: if the simulation image has defects relative to the preset image, then debugging the parameter configuration file of the image processing algorithm until the parameter configuration file is adjusted according to the debugged image. Compared with the pre-stored image, the simulated image processed by the image processing algorithm has no defects.

在实施例中，若仿真图像与预设图像相比，仿真图像存在缺陷，例如仿真图像的清晰度较低，此时，通过对图像处理算法的参数配置文件进行调试，并在调试完成后继续对标定图像文件进行处理，得到仿真图像，通过不断调整图像处理算法的配置文件参数，并在参数调整完成后，再次触发拍照流程，对标定图像文件再次进行处理，直至得到在当前拍摄场景下的仿真图像相对于预设图像不存在缺陷，如清晰度高于或等于预设图像时，停止对参数配置文件进行调试，输出此时对应的图像，即目标图像。In an embodiment, if the simulated image has defects compared with the preset image, for example, the definition of the simulated image is low, at this time, the parameter configuration file of the image processing algorithm is debugged, and after the debugging is completed, continue Process the calibration image file to obtain a simulation image. By continuously adjusting the configuration file parameters of the image processing algorithm, and after the parameter adjustment is completed, trigger the photographing process again, and process the calibration image file again until the current shooting scene is obtained. Compared with the preset image, the simulated image has no defects. For example, when the resolution is higher than or equal to the preset image, stop debugging the parameter configuration file and output the corresponding image at this time, that is, the target image.

在另一些实施例中，在对标定图像文件进行处理时，可将获取标定图像文件、调试算法配置文件、拍摄动作及仿真过程进行整合，实现一键调试，从而提升图像处理算法的调试效率。In some other embodiments, when processing the calibration image file, the acquisition of the calibration image file, debugging algorithm configuration file, shooting action and simulation process can be integrated to realize one-key debugging, thereby improving the debugging efficiency of the image processing algorithm.

下面结合图2对本发明实施例的基于ISP的图像处理方法进行说明，如图2所示，为本发明一个实施例的基于ISP的图像处理方法的流程图。The ISP-based image processing method of the embodiment of the present invention will be described below with reference to FIG. 2 , as shown in FIG. 2 , which is a flow chart of the ISP-based image processing method of an embodiment of the present invention.

步骤S11，在夜景场景模式下拍照。Step S11, taking pictures in the night scene scene mode.

步骤S12，获取当前拍摄场景下拍摄的初始图像。Step S12, acquiring an initial image captured in the current shooting scene.

步骤S13，将初始图像对应的数字图像文件替换为预设数字图像文件，将初始图像对应的数据信息替换为预设数据信息，以得到标定图像文件。Step S13, replacing the digital image file corresponding to the initial image with a preset digital image file, and replacing the data information corresponding to the initial image with preset data information, so as to obtain a calibration image file.

步骤S14，基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像。Step S14, based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, the calibration image file is processed to obtain a simulation image.

步骤S15，将仿真图像与预设图像进行对比。Step S15, comparing the simulated image with the preset image.

步骤S16，判断仿真图像相对于预设图像是否存在缺陷，若是，执行步骤S17；若否，执行步骤S18。Step S16, judging whether there is a defect in the simulated image relative to the preset image, if yes, execute step S17; if not, execute step S18.

步骤S17，对图像处理算法的参数配置文件进行调试。Step S17, debugging the parameter configuration file of the image processing algorithm.

步骤S18，结束。Step S18, end.

应该理解的是，虽然图1-2的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图1-2中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the various steps in the flow charts of FIGS. 1-2 are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 1-2 may include a plurality of sub-steps or stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, these sub-steps or stages The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

基于同一发明构思，作为对上述方法的实现，本公开实施例还提供了一种基于ISP的图像处理装置，该装置实施例与前述方法实施例对应，为便于阅读，本装置实施例不再对前述方法实施例中的细节内容进行逐一赘述，但应当明确，本实施例中的装置能够对应实现前述方法实施例中的全部内容。Based on the same inventive concept, as an implementation of the above method, the embodiment of the present disclosure also provides an ISP-based image processing device. The details in the foregoing method embodiments are described one by one, but it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiments.

下面描述本发明实施例的基于ISP的图像处理装置。The ISP-based image processing device of the embodiment of the present invention is described below.

如图3所示，本发明实施例的基于ISP的图像处理装置2，该装置包括：确定模块20、获取模块21、第一处理模块22和第二处理模块23，确定模块20配置成确定当前拍摄场景模式；获取模块21配置成获取对应于当前拍摄场景模式的标定图像文件；第一处理模块22配置成基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像；第二处理模块23配置成根据仿真图像对图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。As shown in Figure 3, the ISP-based image processing device 2 of the embodiment of the present invention includes: a determination module 20, an acquisition module 21, a first processing module 22 and a second processing module 23, and the determination module 20 is configured to determine the current Shooting scene mode; the acquisition module 21 is configured to obtain a calibration image file corresponding to the current shooting scene mode; the first processing module 22 is configured to process the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, and obtain Simulation image: the second processing module 23 is configured to optimize the parameter configuration file of the image processing algorithm according to the simulation image, and process the calibration image file according to the optimized image processing algorithm to obtain the target image.

根据本发明实施例的基于ISP的图像处理装置2，通过获取对应于当前拍摄场景模式的标定图像文件，基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像，根据仿真图像对图像处理算法的参数配置文件进行优化，并根据优化后的图像处理算法对标定图像文件进行处理，得到目标图像，由此，通过图像处理算法对标定图像文件进行处理，根据处理结果对图像处理算法进行调试优化，由于标定图像文件是预存的，通过对预存的标定图像文件进行处理，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等不可控因素导致的算法调试障碍，从而，在提高算法调试效率的同时，提高了算法调试的准确性。According to the ISP-based image processing device 2 of the embodiment of the present invention, by acquiring the calibration image file corresponding to the current shooting scene mode, the calibration image file is processed based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation Image, optimize the parameter configuration file of the image processing algorithm according to the simulation image, and process the calibration image file according to the optimized image processing algorithm to obtain the target image, thus, process the calibration image file through the image processing algorithm, according to The processing results debug and optimize the image processing algorithm. Since the calibration image files are pre-stored, by processing the pre-stored calibration image files, there is no need for continuous shooting, which can avoid algorithm debugging obstacles caused by uncontrollable factors such as shooting environment and shooting angle changes. Therefore, while improving the efficiency of algorithm debugging, the accuracy of algorithm debugging is improved.

作为本公开实施例一种可选的实施方式，获取模块21，具体配置成：获取当前拍摄场景下拍摄的初始图像；将初始图像对应的数字图像文件替换为预设数字图像文件，将初始图像对应的数据信息替换为预设数据信息，以得到标定图像文件。As an optional implementation of the embodiment of the present disclosure, the acquiring module 21 is specifically configured to: acquire the initial image captured in the current shooting scene; replace the digital image file corresponding to the initial image with a preset digital image file, and convert the initial image The corresponding data information is replaced with the preset data information to obtain the calibration image file.

作为本公开实施例一种可选的实施方式，所述第二处理模块23，具体配置成：将仿真图像与预设图像进行对比，根据对比结果对图像处理算法的参数配置文件进行优化。As an optional implementation of the embodiment of the present disclosure, the second processing module 23 is specifically configured to: compare the simulated image with the preset image, and optimize the parameter configuration file of the image processing algorithm according to the comparison result.

作为本公开实施例一种可选的实施方式，所述第二处理模块23，具体配置成：当仿真图像相对于预设图像存在缺陷时，对图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于预存图像不存在缺陷。As an optional implementation of the embodiment of the present disclosure, the second processing module 23 is specifically configured to: when the simulated image has defects relative to the preset image, debug the parameter configuration file of the image processing algorithm until the Compared with the pre-stored image, the simulated image processed by the debugged image processing algorithm has no defects.

本实施例提供的基于ISP的图像处理装置可以执行上述方法实施例提供的基于ISP的图像处理方法，其实现原理与技术效果类似，此处不再赘述。上述基于ISP的图像处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行上述各个模块对应的操作。The ISP-based image processing apparatus provided in this embodiment can execute the ISP-based image processing method provided in the above method embodiment, and its implementation principle and technical effect are similar, and will not be repeated here. Each module in the above-mentioned ISP-based image processing device may be fully or partially realized by software, hardware or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

在一个实施例中，提出了一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，所述计算机可读存储介质上存储有基于ISP的图像处理指令，所述基于ISP的图像处理指令被一个或多个处理器执行时实现如上述实施例所述的基于ISP的图像处理方法。In one embodiment, one or more non-transitory computer-readable storage media storing computer-readable instructions are proposed, the computer-readable storage medium is stored with ISP-based image processing instructions, and the ISP-based When the image processing instructions are executed by one or more processors, the ISP-based image processing method as described in the foregoing embodiments is implemented.

下面描述本发明实施例的摄像设备。The imaging apparatus of the embodiment of the present invention is described below.

在具体实施例中，该摄像设备例如为但不限于具有拍照功能的相机、手机、平板电脑、可穿戴设备等电子设备。In a specific embodiment, the imaging device is, for example, but not limited to electronic devices such as a camera with a camera function, a mobile phone, a tablet computer, and a wearable device.

在一些实施例中，摄像设备包括：上述实施例的基于ISP的图像处理装置2，即该电子设备可包括：配置成确定当前拍摄场景模式的确定模块20；以及配置成获取对应于当前拍摄场景模式的标定图像文件的获取模块21；以及配置成基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像的第一处理模块22；以及配置成根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像的第二处理模块23。由此，在一些实施例中，当该摄像设备在进行图像处理时，其具体实现方式与本发明上述任意一个实施例所描述的基于ISP的图像处理装置2类似，具体可参见前述关于该基于ISP的图像处理装置2部分的描述，为减少冗余，此处不再赘述。In some embodiments, the imaging device includes: the ISP-based image processing device 2 of the above-mentioned embodiments, that is, the electronic device may include: a determining module 20 configured to determine the current shooting scene mode; The acquisition module 21 of the calibration image file of the mode; and the first processing module 22 configured to process the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image; and configuration The parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain the second processing module 23 of the target image. Therefore, in some embodiments, when the imaging device is performing image processing, its specific implementation method is similar to the ISP-based image processing device 2 described in any of the above-mentioned embodiments of the present invention. The description of part 2 of the image processing device of the ISP is omitted here to reduce redundancy.

在另一些实施例中，如图4所示，该摄像设备包括一个或多个处理器(processor)1000，图4中以一个或多个处理器1000为例；和存储器(memory)2000，还可以包括通信接口(Communication Interface)3000和总线4000。其中，一个或多个处理器1000、存储器2000、通信接口3000可以通过总线4000完成相互间的通信。通信接口3000可以用于信息传输。一个或多个处理器1000可以调用存储器2000中的逻辑指令，以执行上述实施例的基于ISP的图像处理方法。In some other embodiments, as shown in FIG. 4, the imaging device includes one or more processors (processor) 1000, and one or more processors 1000 are taken as an example in FIG. 4; and memory (memory) 2000, also A communication interface (Communication Interface) 3000 and a bus 4000 may be included. Wherein, one or more processors 1000 , memory 2000 , and communication interface 3000 may communicate with each other through the bus 4000 . The communication interface 3000 can be used for information transmission. One or more processors 1000 may invoke logic instructions in the memory 2000 to execute the ISP-based image processing method of the above-mentioned embodiments.

此外，上述的存储器2000中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。In addition, the logic instructions in the above-mentioned memory 2000 may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as an independent product.

存储器2000作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序，如本发明实施例中的方法对应的程序指令/模块。一个或多个处理器1000通过运行存储在存储器2000中的软件程序、指令以及模块，从而执行功能应用以及数据处理，即实现上述方法实施例中的基于ISP的图像处理方法。As a computer-readable storage medium, the memory 2000 can be used to store software programs and computer-executable programs, such as program instructions/modules corresponding to the methods in the embodiments of the present invention. One or more processors 1000 execute functional applications and data processing by running software programs, instructions and modules stored in the memory 2000, that is, implement the ISP-based image processing method in the above method embodiments.

存储器2000可包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序；存储数据区可存储根据终端设备的使用所创建的数据等。此外，存储器2000可以包括高速随机存取存储器，还可以包括非易失性存储器。The memory 2000 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal device, and the like. In addition, the memory 2000 may include a high-speed random access memory, and may also include a non-volatile memory.

本发明实施例的技术方案可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括一个或多个指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明实施例所述方法的全部或部分步骤。而前述的存储介质可以是非暂态存储介质，包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等多种可以存储程序代码的介质，也可以是暂态存储介质。The technical solutions of the embodiments of the present invention can be embodied in the form of software products, which are stored in a storage medium and include one or more instructions to make a computer device (which can be a personal computer, a server, or a network equipment, etc.) to perform all or part of the steps of the method described in the embodiment of the present invention. The aforementioned storage medium can be a non-transitory storage medium, including: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc. A medium that can store program code, or a transitory storage medium.

需要说明的是，本发明实施例的摄像设备在进行拍照时，其具体实现方式与本发明实施例的基于ISP的图像处理方法的具体实现方式类似，具体请参见方法部分的描述，为了减少冗余，此处不再赘述。It should be noted that when the camera device in the embodiment of the present invention takes pictures, its specific implementation is similar to the specific implementation of the ISP-based image processing method in the embodiment of the present invention. For details, please refer to the description of the method part. In order to reduce redundant I will not repeat them here.

根据本发明实施例的摄像设备，通过获取对应于当前拍摄场景模式的标定图像文件，基于ISP模块中对应于当前拍摄场景模式的图像处理算法对标定图像文件进行处理，得到仿真图像，根据仿真图像对图像处理算法的参数配置文件进行优化，并根据优化后的图像处理算法对标定图像文件进行处理，得到目标图像，由此，通过图像处理算法对标定图像文件进行处理，根据处理结果对图像处理算法进行调试优化，由于标定图像文件是预存的，通过对预存的标定图像文件进行处理，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等不可控因素导致的算法调试障碍，从而，在提高算法调试效率的同时，提高了算法调试的准确性。According to the imaging device of the embodiment of the present invention, by obtaining the calibration image file corresponding to the current shooting scene mode, the calibration image file is processed based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, and the simulation image is obtained. According to the simulation image Optimize the parameter configuration file of the image processing algorithm, and process the calibration image file according to the optimized image processing algorithm to obtain the target image, thus, process the calibration image file through the image processing algorithm, and process the image according to the processing result The algorithm is debugged and optimized. Since the calibration image files are pre-stored, by processing the pre-stored calibration image files, there is no need to continuously shoot, which can avoid algorithm debugging obstacles caused by uncontrollable factors such as shooting environment and shooting angle changes, thus improving the algorithm. While improving debugging efficiency, the accuracy of algorithm debugging is improved.

当用于本发明中时，虽然术语“第一”、“第二”等可能会在本发明中使用以描述各元件，但这些元件不应受到这些术语的限制。这些术语仅用于将一个元件与另一个元件区别开。比如，在不改变描述的含义的情况下，第一元件可以叫做第二元件，并且同样第，第二元件可以叫做第一元件，只要所有出现的“第一元件”一致重命名并且所有出现的“第二元件”一致重命名即可。第一元件和第二元件都是元件，但可以不是相同的元件。When used in the present invention, although the terms 'first', 'second', etc. may be used in the present invention to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without changing the meaning of the description, a first element could be called a second element, and likewise, a second element could be called a first element, as long as all occurrences of "first element" are renamed consistently and all occurrences of "Second component" can be renamed consistently. The first element and the second element are both elements, but may not be the same element.

本发明中使用的用词仅用于描述实施例并且不用于限制权利要求。如在实施例以及权利要求的描述中使用的，除非上下文清楚地表明，否则单数形式的“一个”(a)、“一个”(an)和“所述”(the)旨在同样包括复数形式。类似地，如在本发明中所使用的术语“和/或”是指包含一个或一个以上相关联的列出的任何以及所有可能的组合。另外，当用于本发明中时，术语“包括”(comprise)及其变型“包括”(comprises)和/或包括(comprising)等指陈述的特征、整体、步骤、操作、元素，和/或组件的存在，但不排除一个或一个以上其它特征、整体、步骤、操作、元素、组件和/或这些的分组的存在或添加。The terms used in the present invention are used to describe the embodiments only and are not used to limit the claims. As used in the examples and description of the claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well unless the context clearly indicates otherwise . Similarly, the term "and/or" as used in the present invention is meant to include any and all possible combinations of one or more of the associated listed ones. In addition, when used in the present invention, the term "comprise" and its variants "comprises" and/or comprising (comprising) etc. refer to stated features, integers, steps, operations, elements, and/or The presence of a component does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groupings of these.

所描述的实施例中的各方面、实施方式、实现或特征能够单独使用或以任意组合的方式使用。所描述的实施例中的各方面可由软件、硬件或软硬件的结合实现。所描述的实施例也可以由存储有计算机可读代码的计算机可读介质体现，该计算机可读代码包括可由至少一个计算装置执行的指令。所述计算机可读介质可与任何能够存储数据的数据存储装置相关联，该数据可由计算机***读取。用于举例的计算机可读介质可以包括只读存储器、随机存取存储器、CD-ROM、HDD、DVD、磁带以及光数据存储装置等。所述计算机可读介质还可以分布于通过网络联接的计算机***中，这样计算机可读代码就可以分布式存储并执行。The aspects, implementations, implementations or features of the described embodiments can be used alone or in any combination. Aspects of the described embodiments can be implemented by software, hardware or a combination of hardware and software. The described embodiments may also be embodied by a computer-readable medium storing computer-readable code comprising instructions executable by at least one computing device. The computer readable medium can be associated with any data storage device that can store data that can be read by a computer system. By way of example, computer readable media may include read only memory, random access memory, CD-ROM, HDD, DVD, magnetic tape, optical data storage devices, and the like. The computer readable medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

上述技术描述可参照附图，这些附图形成了本发明的一部分，并且通过描述在附图中示出了依照所描述的实施例的实施方式。虽然这些实施例描述的足够详细以使本领域技术人员能够实现这些实施例，但这些实施例是非限制性的；这样就可以使用其它的实施例，并且在不脱离所描述的实施例的范围的情况下还可以做出变化。比如，流程图中所描述的操作顺序是非限制性的，因此在流程图中阐释并且根据流程图描述的两个或两个以上操作的顺序可以根据若干实施例进行改变。作为另一个例子，在若干实施例中，在流程图中阐释并且根据流程图描述的一个或一个以上操作是可选的，或是可删除的。另外，某些步骤或功能可以添加到所公开的实施例中，或两个以上的步骤顺序被置换。所有这些变化被认为包含在所公开的实施例以及权利要求中。The above technical description may refer to the accompanying drawings, which form a part hereof, and in which are shown by way of description implementations in accordance with the described embodiments. Although these embodiments are described in sufficient detail to enable those skilled in the art to practice these embodiments, these embodiments are not limiting; as such, other embodiments can be used, and without departing from the scope of the described embodiments Circumstances can also be changed. For example, the order of operations described in the flow diagrams is non-limiting, and thus the order of two or more operations illustrated in and described with respect to the flow diagrams may be changed according to several embodiments. As another example, in several embodiments, one or more operations illustrated in and described with respect to the flowcharts are optional, or may be deleted. Additionally, certain steps or functionality may be added to the disclosed embodiments, or the order of two or more steps permuted. All such variations are considered to be encompassed by the disclosed embodiments as well as the claims.

另外，上述技术描述中使用术语以提供所描述的实施例的透彻理解。然而，并不需要过于详细的细节以实现所描述的实施例。因此，实施例的上述描述是为了阐释和描述而呈现的。上述描述中所呈现的实施例以及根据这些实施例所公开的例子是单独提供的，以添加上下文并有助于理解所描述的实施例。上述说明书不用于做到无遗漏或将所描述的实施例限制到本发明的精确形式。根据上述教导，若干修改、选择适用以及变化是可行的。在某些情况下，没有详细描述为人所熟知的处理步骤以避免不必要地影响所描述的实施例。Additionally, terminology was used in the above technical description to provide a thorough understanding of the described embodiments. However, undue detail is not required to implement the described embodiments. Thus, the foregoing description of the embodiments has been presented for purposes of illustration and description. The embodiments presented in the foregoing description, and examples disclosed according to these embodiments, are provided individually to add context and to facilitate understanding of the described embodiments. The above description is not intended to be exhaustive or to limit the described embodiments to the precise forms of the invention. Several modifications, alternatives, and variations are possible in light of the above teachings. In some instances, well known process steps have not been described in detail in order not to unnecessarily obscure the described embodiments.

工业实用性Industrial Applicability

本公开提供的基于ISP的图像处理方法，通过图像处理算法对标定图像文件进行处理，根据处理结果对图像处理算法进行调试优化，由于标定图像文件是预存的，通过对预存的标定图像文件进行处理，无需不断拍摄，可以避免拍摄环境及拍摄角度变化等不可控因素导致的算法调试障碍，从而，在提高算法调试效率的同时，提高了算法调试的准确性，具有很强的工业实用性。The ISP-based image processing method provided by this disclosure processes the calibration image file through the image processing algorithm, and debugs and optimizes the image processing algorithm according to the processing result. Since the calibration image file is pre-stored, the pre-stored calibration image file is processed. , without continuous shooting, it can avoid algorithm debugging obstacles caused by uncontrollable factors such as shooting environment and shooting angle changes, thereby improving the algorithm debugging efficiency while improving the accuracy of algorithm debugging, which has strong industrial applicability.

Claims

一种基于ISP的图像处理方法，其特征在于，包括：A kind of image processing method based on ISP, it is characterized in that, comprising:

确定当前拍摄场景模式；Determine the current shooting scene mode;

获取对应于所述当前拍摄场景模式的标定图像文件；Acquiring a calibration image file corresponding to the current shooting scene mode;

基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；Processing the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image.
根据权利要求1所述的基于ISP的图像处理方法，其中，获取对应于所述当前拍摄场景模式的标定图像文件，包括：The ISP-based image processing method according to claim 1, wherein obtaining a calibration image file corresponding to the current shooting scene mode comprises:

获取当前拍摄场景下拍摄的初始图像；Obtain the initial image captured in the current shooting scene;

将所述初始图像对应的数字图像文件替换为预设数字图像文件，将所述初始图像对应的数据信息替换为预设数据信息，以得到所述标定图像文件。The digital image file corresponding to the initial image is replaced with a preset digital image file, and the data information corresponding to the initial image is replaced with preset data information, so as to obtain the calibration image file.
根据权利要求1所述的基于ISP的图像处理方法，其中，根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，包括：The ISP-based image processing method according to claim 1, wherein optimizing the parameter configuration file of the image processing algorithm according to the simulated image comprises:

将所述仿真图像与预设图像进行对比，根据对比结果对所述图像处理算法的参数配置文件进行优化。The simulated image is compared with the preset image, and the parameter configuration file of the image processing algorithm is optimized according to the comparison result.
根据权利要求3所述的基于ISP的图像处理方法，其中，根据对比结果对所述图像处理算法的参数配置文件进行优化，包括：The image processing method based on ISP according to claim 3, wherein, optimizing the parameter configuration file of the image processing algorithm according to the comparison result comprises:

在所述仿真图像相对于所述预设图像存在缺陷的条件下，对所述图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于所述预存图像不存在缺陷。Under the condition that the simulated image has defects relative to the preset image, the parameter configuration file of the image processing algorithm is debugged until the simulated image processed according to the debugged image processing algorithm is relative to the pre-stored image There are no defects.
根据权利要求1所述的基于ISP的图像处理方法，其中，所述当前拍摄场景模式包括夜景模式、人像美颜模式、背景虚化模式及广角镜头模式中的任意一种。The image processing method based on ISP according to claim 1, wherein the current shooting scene mode includes any one of night scene mode, portrait beauty mode, background blur mode and wide-angle lens mode.
根据权利要求1所述的基于ISP的图像处理方法，其中，所述标定图像文件包括：数字图像缓冲寄存器和元数据；The image processing method based on ISP according to claim 1, wherein said calibration image file comprises: digital image buffer register and metadata;

其中，所述数字图像缓冲寄存器为摄像头传感器采集的原始数字图像文件，所述元数据为拍照时图像的属性信息，所述属性信息至少包括：感光度、曝光时间、白平衡增益及颜色校正矩阵。Wherein, the digital image buffer register is the original digital image file collected by the camera sensor, the metadata is the attribute information of the image when taking pictures, and the attribute information at least includes: sensitivity, exposure time, white balance gain and color correction matrix .
根据权利要求1所述的基于ISP的图像处理方法，其中，所述基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像，包括：The ISP-based image processing method according to claim 1, wherein the image processing algorithm corresponding to the current shooting scene mode in the ISP-based module processes the calibration image file to obtain a simulated image, including:

调取ISP模块中所述当前拍摄场景模式相应所述图像处理算法；calling the image processing algorithm corresponding to the current shooting scene mode in the ISP module;

基于所述当前拍摄场景模式相应的所述图像处理算法，对所述标定图像文件进行处理，以得到所述标定图像文件对应的所述仿真图像。Based on the image processing algorithm corresponding to the current shooting scene mode, the calibration image file is processed to obtain the simulation image corresponding to the calibration image file.
根据权利要求1所述的基于ISP的图像处理方法，其中，针对不同的所述仿真图像，分别对应不同的所述参数配置文件，且对应相同的预设因素，其中，所述预设因素至少包括：拍摄图像角度以及拍摄图像抖动。The ISP-based image processing method according to claim 1, wherein, for different simulated images, they correspond to different parameter configuration files and correspond to the same preset factors, wherein the preset factors are at least Including: shooting image angle and shooting image shaking.
根据权利要求1所述的基于ISP的图像处理方法，其中，所述获取对应于所述当前拍摄场景模式的标定图像文件；基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像，包括：The ISP-based image processing method according to claim 1, wherein said acquisition corresponds to the calibration image file of the current shooting scene mode; based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module, the The calibration image file is processed to obtain a simulation image; the parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image, including :

接收用户输入的图像调试指令；Receive image debugging instructions input by the user;

响应所述图像调试指令，获取对应于所述当前拍摄场景模式的标定图像文件；Responding to the image debugging instruction, acquiring a calibration image file corresponding to the current shooting scene mode;

基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；Processing the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image.
一种基于ISP的图像处理装置，其特征在于，包括：An ISP-based image processing device, characterized in that it comprises:

确定模块，配置成确定当前拍摄场景模式；A determination module configured to determine the current shooting scene mode;

获取模块，配置成获取对应于所述当前拍摄场景模式的标定图像文件；An acquisition module configured to acquire a calibration image file corresponding to the current shooting scene mode;

第一处理模块，配置成基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；The first processing module is configured to process the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

第二处理模块，配置成根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The second processing module is configured to optimize the parameter configuration file of the image processing algorithm according to the simulated image, and process the calibration image file according to the optimized image processing algorithm to obtain a target image.
根据权利要求12所述的基于ISP的图像处理装置，其中，所述获取模块，具体配置成：The ISP-based image processing device according to claim 12, wherein the acquisition module is specifically configured to:

获取当前拍摄场景下拍摄的初始图像；Obtain the initial image captured in the current shooting scene;

将所述初始图像对应的数字图像文件替换为预设数字图像文件，将所述初始图像对应的数据信息替换为预设数据信息，以得到所述标定图像文件。The digital image file corresponding to the initial image is replaced with a preset digital image file, and the data information corresponding to the initial image is replaced with preset data information, so as to obtain the calibration image file.
根据权利要求10所述的基于ISP的图像处理装置，其中，所述第二处理模块，具体配置成：The ISP-based image processing device according to claim 10, wherein the second processing module is specifically configured to:

将所述仿真图像与预设图像进行对比，根据对比结果对所述图像处理算法的参数配置文件进行优化。The simulated image is compared with the preset image, and the parameter configuration file of the image processing algorithm is optimized according to the comparison result.
根据权利要求10所述的基于ISP的图像处理装置，其中，所述第二处理模块，具体配置成：The ISP-based image processing device according to claim 10, wherein the second processing module is specifically configured to:

在所述仿真图像相对于所述预设图像存在缺陷的条件下，对所述图像处理算法的参数配置文件进行调试，直至根据调试后的图像处理算法处理得到的仿真图像相对于所述预存图像不存在缺陷。Under the condition that the simulated image has defects relative to the preset image, the parameter configuration file of the image processing algorithm is debugged until the simulated image processed according to the debugged image processing algorithm is relative to the pre-stored image There are no defects.
根据权利要求10所述的基于ISP的图像处理装置，其中，所述第一处理模块，具体配置成：The ISP-based image processing device according to claim 10, wherein the first processing module is specifically configured to:

调取ISP模块中所述当前拍摄场景模式相应图像处理算法；Calling the corresponding image processing algorithm of the current shooting scene mode described in the ISP module;

基于所述当前拍摄场景模式相应的图像处理算法，对所述标定图像文件进行处理，以得到所述标定图像文件对应的所述仿真图像。Based on the image processing algorithm corresponding to the current shooting scene mode, the calibration image file is processed to obtain the simulation image corresponding to the calibration image file.
根据权利要求10所述的基于ISP的图像处理装置，其中，所述装置还包括：The ISP-based image processing device according to claim 10, wherein said device further comprises:

图像调试指令接收模块，配置成接收用户输入的图像调试指令；An image debugging instruction receiving module configured to receive an image debugging instruction input by a user;

响应模块，配置成响应所述图像调试指令，获取对应于所述当前拍摄场景模式的标定图像文件；A response module, configured to respond to the image debugging instruction, and acquire a calibration image file corresponding to the current shooting scene mode;

基于ISP模块中对应于所述当前拍摄场景模式的图像处理算法对所述标定图像文件进行处理，得到仿真图像；Processing the calibration image file based on the image processing algorithm corresponding to the current shooting scene mode in the ISP module to obtain a simulation image;

根据所述仿真图像对所述图像处理算法的参数配置文件进行优化，根据优化后的图像处理算法对所述标定图像文件进行处理，得到目标图像。The parameter configuration file of the image processing algorithm is optimized according to the simulation image, and the calibration image file is processed according to the optimized image processing algorithm to obtain a target image.
一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有基于ISP的图像处理指令，所述基于ISP的图像处理指令被一个或多个处理器执行时实现如权利要求1-9任一项所述的基于ISP的图像处理方法。One or more non-volatile computer-readable storage media storing computer-readable instructions, wherein the computer-readable storage medium stores image processing instructions based on ISP, and the image processing instructions based on ISP When executed by one or more processors, the ISP-based image processing method according to any one of claims 1-9 is realized.
一种摄像设备，其特征在于，包括：An imaging device, characterized in that it comprises:

权利要求10-15任一项所述的基于ISP的图像处理装置；或者The ISP-based image processing device according to any one of claims 10-15; or

一个或多个处理器、存储器和存储在所述存储器上并可在所述处理器上运行的基于ISP的图像处理指令，所述基于ISP的图像处理指令被所述一个或多个处理器执行时实现如权利要求1-9任一项所述的基于ISP的图像处理方法。one or more processors, memory, and ISP-based image processing instructions stored on said memory and executable on said processors, said ISP-based image processing instructions being executed by said one or more processors When realizing the ISP-based image processing method as described in any one of claims 1-9.