CN113747145A - Image processing circuit, electronic device, and image processing method - Google Patents
Image processing circuit, electronic device, and image processing method Download PDFInfo
- Publication number
- CN113747145A CN113747145A CN202010478315.2A CN202010478315A CN113747145A CN 113747145 A CN113747145 A CN 113747145A CN 202010478315 A CN202010478315 A CN 202010478315A CN 113747145 A CN113747145 A CN 113747145A
- Authority
- CN
- China
- Prior art keywords
- image
- processor
- processing unit
- image signal
- application processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 242
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000012805 post-processing Methods 0.000 claims description 49
- 238000013528 artificial neural network Methods 0.000 claims description 35
- 238000007781 pre-processing Methods 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 11
- 230000003068 static effect Effects 0.000 claims description 11
- 239000000284 extract Substances 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 14
- 238000005457 optimization Methods 0.000 description 13
- 238000012549 training Methods 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 8
- 230000009467 reduction Effects 0.000 description 8
- 238000012937 correction Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013501 data transformation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/06—Diagnosis, testing or measuring for television systems or their details for recorders
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application provides an image processing circuit, electronic equipment and an image processing method, wherein an image sensor is connected with an image signal pre-processor and used for acquiring an external image according to configured acquisition parameters and transmitting the external image to the image signal pre-processor; the image signal preprocessor is connected with the application processor and is used for transmitting the indication information to the application processor when receiving the packet header of the first image data packet of the single-frame image in the external image transmitted by the image sensor; the application processor is connected with the image sensor and used for updating the acquisition parameters of the image sensor according to the indication information. According to the method and the device, the image signal preprocessor is used for indicating the application processor to update the acquisition parameters of the image sensor, so that the application processor can stably control the image acquisition of the image sensor. When the image processing circuit provided by the application is applied to the electronic equipment, the stability of image acquisition of the electronic equipment can be effectively improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing circuit, an electronic device, and an image processing method.
Background
At present, the quality of the shooting function becomes a key for measuring the performance of an electronic device (such as a smart phone, a tablet computer and the like). The quality of the shooting quality of the electronic equipment is closely related to the image processing capability of the electronic equipment, and whether the stable image acquisition is carried out influences whether the electronic equipment can normally and effectively carry out image processing.
Disclosure of Invention
The embodiment of the application provides an image processing circuit, an electronic device and an image processing method, which can improve the stability of image acquisition of the electronic device.
The application discloses an image processing circuit, comprising an image sensor, an image signal pre-processor and an application processor, wherein,
the image sensor is connected with the image signal pre-processor and used for acquiring an external image according to configured acquisition parameters and transmitting the external image to the image signal pre-processor;
the image signal preprocessor is connected with the application processor and is used for transmitting indication information to the application processor when receiving a packet header of a first image data packet of a single-frame image in the external image transmitted by the image sensor;
and the application processor is connected with the image sensor and used for updating the acquisition parameters of the image sensor according to the indication information.
The application also discloses an electronic device comprising the image processing circuit provided by the application.
The embodiment of the application also discloses an image processing method, which is suitable for an image processing circuit, wherein the image processing circuit comprises an image sensor, an image signal pre-processor and an application processor, and the image processing method comprises the following steps:
the image sensor collects external images according to configured collection parameters and transmits the external images to the image signal pre-processor;
the image signal preprocessor transmits indication information to the application processor when receiving a packet header of a first image data packet of a single frame image in the external image transmitted by the image sensor;
and the application processor updates the acquisition parameters of the image sensor according to the indication information.
According to the method and the device, the image signal preprocessor is used for indicating the application processor to update the acquisition parameters of the image sensor, so that the application processor can stably control the image acquisition of the image sensor. When the image processing circuit provided by the application is applied to the electronic equipment, the stability of image acquisition of the electronic equipment can be effectively improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 is a schematic diagram of a first structure of an image processing circuit according to an embodiment of the present disclosure.
Fig. 2 is a schematic diagram of a state machine of an RTL design in the embodiment of the present application.
Fig. 3 is a schematic diagram of a second structure of an image processing circuit according to an embodiment of the present disclosure.
Fig. 4 is a schematic diagram of a third structure of an image processing circuit according to an embodiment of the present application.
Fig. 5 is a schematic diagram of a fourth structure of an image processing circuit according to an embodiment of the present application.
Fig. 6 is a schematic diagram of a fifth structure of an image processing circuit according to an embodiment of the present application.
Fig. 7 is an exemplary diagram of an enhanced image obtained by enhancing a post-processing image in an embodiment of the present application.
Fig. 8 is a schematic diagram of a sixth structure of an image processing circuit according to an embodiment of the present application.
Fig. 9 is a schematic flowchart of an image processing method according to an embodiment of the present application.
Detailed Description
The technical solution provided by the embodiment of the present application can be applied to various scenarios requiring data communication, and the embodiment of the present application is not limited thereto.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating a first structure of an image processing circuit 100 according to an embodiment of the present disclosure. The image processing circuit 100 may be integrated in an electronic device (e.g., a processor-equipped device such as a smart phone, a tablet computer, a palm computer, a notebook computer, or a desktop computer) and includes an image sensor 110, an image signal pre-processor 120, and an application processor 130.
The image sensor 110, or referred to as a light-sensing element, is a device that converts an optical signal into an electrical signal, and compared with a light-sensing element of a "point" light source such as a photodiode or a phototransistor, the image sensor 110 divides an optical image sensed by the light-sensing element into a plurality of small units, and then converts the optical image into an available electrical signal, thereby obtaining original image data. It should be noted that, in the embodiment of the present application, the type of the image sensor 110 is not limited, and the image sensor may be a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor, a Charge Coupled Device (CCD) image sensor, or the like.
The application processor 130 is integrated with an image signal processing unit, and can process the image data collected by the image sensor 110, so as to improve the quality of the image data. For example, the image signal processor 130 can perform a dead pixel correction process on the image data to remove dead pixels in the image data.
The image signal pre-processor 120 performs some differentiation processing before the image signal processing unit integrated by the application processor 130 processes the image data, compared to the image signal processing unit integrated by the application processor 130.
In the embodiment of the present application, the image sensor 110 is connected to the image signal pre-processor 120, and is configured to acquire an external image according to the configured acquisition parameters, and transmit the acquired external image to the image signal pre-processor. For example, taking exposure parameters as an example, during automatic exposure, the application processor 130 calculates corresponding exposure parameters according to the collected relevant information required for automatic exposure by using a pre-configured automatic exposure control algorithm, and configures the exposure parameters to the image sensor 110, so that the image sensor 110 acquires an external image according to the exposure parameters, and transmits the external image to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, a connection manner of the image signal pre-Processor 120 and the image sensor 110 is not particularly limited, for example, the image signal pre-Processor 120 and the image sensor 110 are connected by an MIPI (Mobile Industry Processor Interface).
When transmitting the external image to the image signal pre-processor 120, the image sensor 110 encapsulates the external image into a plurality of image data packets for each frame, and transmits the image data packets to the image signal pre-processor. Illustratively, an image data packet includes a header field, an end-of-packet field and a data field, wherein the header field and the end-of-packet field are used for filling some necessary control information, such as synchronization information, address information, error control information, etc., and the data field is used for filling the actual image content.
On the other hand, the image signal pre-processor 120 receives an external image from the image sensor 110. In addition, the image signal pre-processor 120 is further connected to the application processor 130, wherein the connection manner between the application processor 130 and the image signal pre-processor 120 is not particularly limited in this embodiment, for example, the application processor 130 and the image signal pre-processor 120 may also be connected by MIPI.
When receiving the header of the first image data packet of a single frame image in the external image transmitted by the image sensor 110, the image signal pre-processor 120 transmits indication information to the application processor 130, where the indication information is used to instruct the application processor 130 to update the exposure parameters of the image sensor 110. It should be noted that, in the embodiment of the present application, the type of the indication information is not particularly limited, and may be selected by a person having ordinary skill in the art according to actual needs.
Illustratively, SOF is taken as the indication information.
The image signal pre-processor 120 and the application processor 130 are connected through MIPI, and in the process of designing MIPI by ASIC, the original RTL design of SoF (Start of Frame) needs to be changed, and a state machine thereof is shown in fig. 2.
The application processor 130 is further connected to the image sensor 110, and is configured to update the acquisition parameters of the image sensor 110 according to the indication information from the image signal pre-processor 120, so that the image sensor 110 performs image acquisition according to the updated acquisition parameters, thereby stably acquiring a high-quality image.
From the above, the image processing circuit provided by the application comprises an image sensor, an image signal pre-processor and an application processor, wherein the image sensor is connected with the image signal pre-processor and used for acquiring an external image according to configured acquisition parameters and transmitting the external image to the image signal pre-processor; the image signal preprocessor is connected with the application processor and is used for transmitting the indication information to the application processor when receiving the packet header of the first image data packet of the single-frame image in the external image transmitted by the image sensor; the application processor is connected with the image sensor and used for updating the acquisition parameters of the image sensor according to the indication information. According to the method and the device, the image signal preprocessor is used for indicating the application processor to update the acquisition parameters of the image sensor, so that the application processor can stably control the image acquisition of the image sensor. When the image processing circuit provided by the application is applied to the electronic equipment, the stability of image acquisition of the electronic equipment can be effectively improved.
Optionally, in an embodiment, in order to improve image quality, in this embodiment of the application, the image signal pre-processor 120 further performs pre-processing on a single frame image in an external image from the image sensor 110, so as to obtain a pre-processed image accordingly.
In order to avoid the original data of the external image from being damaged due to the preprocessing of the external image by the image signal preprocessing 120, the image signal preprocessing 120 further extracts the reference parameters of the single frame image before preprocessing the single frame image. The reference parameters include relevant parameters required by the application processor 130 for image processing, such as an auto-focus parameter, an auto-white balance parameter, an auto-exposure parameter, and a phase focus parameter.
It should be noted that, in the embodiment of the present application, the preprocessing performed by the image signal preprocessor 120 is not specifically limited, for example, when the electronic device is located in an extremely dark environment with an ambient illuminance lower than 1lux, the image content of the finally obtained imaging image is difficult to identify, and by using the image signal preprocessor 120 provided by the present application, the original external image collected by the image sensor 110 may be optimized, so that the image content of the finally obtained imaging image can be clearly identified, and the purpose of improving the image quality is achieved.
The application processor 130 is connected to the image signal pre-processor 120, and is configured to further perform post-processing on the pre-processed image obtained by pre-processing by the image signal pre-processor 120, so as to obtain a post-processed image accordingly. It should be noted that, in the embodiment of the present application, a connection manner between the application processor 130 and the image signal pre-processor 120 is not particularly limited, for example, the application processor 130 and the image signal pre-processor 120 may also be connected by an MIPI. In addition, in the embodiment of the present application, post-processing performed by the application processor 130 is not particularly limited, and may be configured by a person skilled in the art according to actual needs, or may be configured by the application processor 130 by default when the application processor is shipped from a factory.
In addition, after the application processor 130 processes the post-processed image, the post-processed image may be previewed and/or stored directly, or the post-processed image may be previewed and/or stored after being further processed.
In addition, the application processor 130 controls the image sensor 110 to start capturing an external image and to end capturing the external image based on its connection with the image sensor 110.
Optionally, in an embodiment, the acquisition parameter includes an exposure parameter, after the application processor 130 completes current update of the exposure parameter of the image sensor 110, the image sensor 110 acquires an external image by using the updated exposure parameter, and accordingly, the application processor 130 collects exposure statistical information required for automatic exposure control, and calculates a new exposure parameter by using a configured automatic exposure control algorithm, so as to update the exposure parameter of the image sensor 110 again when receiving the indication information of the image signal preprocessor 120 again. In this way, the exposure parameters of the image sensor 110 are continuously updated until the exposure parameters converge.
Compared with the related art, the image signal pre-processor 120 is additionally added, the image signal pre-processor 120 pre-processes the original image acquired by the image sensor 110, and the application processor 130 post-processes the pre-processed image, so that the image signal pre-processor 120 cooperates with the application processor 130 to perform image processing. In addition, the application instructs the application processor 130 to update the acquisition parameters of the image sensor 110 by using the image signal pre-processor 120, so that the application processor 130 can stably control the image acquisition of the image sensor 110. When the image processing circuit provided by the application is applied to the electronic equipment, the image processing capacity of the electronic equipment can be effectively improved.
Alternatively, in an embodiment, the image signal pre-processor 120 and the application processor 130 are directly connected through a GPIO (General-purpose input/output) interface.
In this embodiment, the image signal pre-processor 120 and the application processor 130 are directly connected by a GPIO interface, and accordingly, the second image signal processing unit 1302 sends the indication information to the application processor 130 through the GPIO interface between the second image signal processing unit 1302 and the application processor 130 when sending the indication information to the application processor 130.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating a second structure of the image processing circuit 100 according to the embodiment of the present disclosure. The image processing circuit 100 provided in the embodiment of the present application may include an image sensor 110, an image signal pre-processor 120, an application processor 130, and a register 140, where the image sensor 110, the image signal pre-processor 120, and the application processor 130 may refer to the image sensor 110, the image signal pre-processor 120, and the application processor 130 in fig. 1, respectively, and details are not repeated here. Among them, the register 140 is connected to the image signal pre-processor 120 and the application processor 130.
In the embodiment of the present application, the indication information of the image signal pre-processor 120 is not directly transmitted to the application processor 130, but is indirectly transmitted by using the third-party device "register 140", and accordingly, when the indication information is sent to the application processor 130, the image signal pre-processor 120 is configured to:
the preset flag bit of the register 140 is written as a preset value corresponding to the aforementioned indication information, so that the application processor 130 obtains the aforementioned indication information according to the preset value written by the preset flag bit of the register 140, and correspondingly updates the exposure parameter of the image sensor 110.
In the embodiment of the present application, it is not specifically limited to which flag bit of the register 140 is selected as the preset flag bit to indicate the transmission of information, and a person skilled in the art can select the flag bit according to actual needs. It should be noted that, for the preset flag bit in the register 140, the corresponding relationship between the preset value and the indication information is established in advance in the present application. In this embodiment, which value is used to represent the indication information is not specifically limited in this embodiment, and may be selected by a person of ordinary skill in the art according to actual needs, for example, for the preset flag bit of the register 140, a value "1" is used to represent the indication information in this embodiment, accordingly, when the image signal preprocessor 120 needs to send the indication information to the application processor 130, the value of the preset flag bit in the register 140 is written to "1", so that the register 140 sends an interrupt signal to the application processor 130 to notify that the application processor 130 currently has data written to, the application processor 130 correspondingly reads the value of the preset flag bit in the register 140, which is "1" at this time, so as to obtain the indication information, and update the exposure parameter of the image sensor 110.
It should be noted that, in the embodiment of the present application, a person skilled in the art may select a register type according to actual needs, for example, a PCIE register is used to implement transmission of the indication information in the embodiment of the present application.
Alternatively, in an embodiment, the image sensor 110 is configured to pack each line of image data of the single frame image into an image data packet, and sequentially transmit each image data packet to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, for one complete external image frame, the image sensor 110 does not directly transmit one complete external image frame to the image signal pre-processor 120, but transmits it to the image signal pre-processor 120 row by row.
The image sensor 110 is configured to encapsulate each line of image data of the single frame image into an image data packet, and sequentially transmit the encapsulated image data packet to the image signal pre-processor 120.
Alternatively, in an embodiment, the image sensor 110 is configured to, when acquiring image data of a row of external images, encapsulate the acquired row of image data into an image data packet, and transmit the encapsulated image data packet to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, the image sensor 110 does not transmit the complete external image to the image signal pre-processor 120 line by line after acquiring the complete external image of one frame, but transmits the acquired image data of one line of the external image to the image signal pre-processor 120 every time the image data of one line of the external image is acquired.
For example, after the image sensor 110 acquires the first line of image data of the single frame image, the acquired first line of image data is packaged into an image data packet and transmitted to the image signal pre-processor 120, and meanwhile, the next line of image data of the single frame image is continuously acquired and transmitted again, and the process is repeated until the last line of image data of the single frame image is acquired and packaged into an image data packet and transmitted to the second image signal processing unit 1302, so that the transmission of the single frame image is completed.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating a third structure of the image processing circuit 100 according to the embodiment of the present disclosure. The image processing circuit 100 includes an image sensor 110, an image signal pre-processor 120, and an application processor 130. The image signal pre-processor 120 includes a first image signal processing unit 1201 and a neural network processing unit 1202. In addition, the image sensor 110 and the application processor 130 can refer to the image sensor 110 and the application processor 130 in fig. 1, respectively, and are not described herein again.
In this embodiment, the first image signal processing unit 1201 is configured to extract a reference parameter of the single frame image, transmit the extracted reference parameter to the application processor 130, and perform optimization processing on the single frame image to obtain an optimized external image. After the reference parameters of the single frame image are extracted and obtained, the first image signal processing unit 1201 further performs optimization processing on the single frame image according to a pre-configured optimization strategy to obtain an optimized single frame image. It should be noted that, in the embodiment of the present application, the optimization processing performed by the first image signal processing unit 1201 is not particularly limited, and may include at least one of dead pixel correction processing, time domain noise reduction processing, 3D noise reduction processing, linearization processing, and black level correction processing, and may also include an optimization processing manner not listed in the present application.
The neural network processing unit 1202 is configured to perform pre-processing on the optimized single-frame image to obtain a pre-processed image. The neural network processing unit 1202 is fixed with a plurality of neural network algorithms (for example, a neural network-based video night scene algorithm, a video HDR algorithm, a video blurring algorithm, a video noise reduction algorithm, a video super-resolution algorithm, and the like), and after the first image signal processing unit 1201 completes optimization processing on the single-frame image, the neural network processing unit 1202 invokes a corresponding neural network algorithm according to a configured preprocessing strategy to further preprocess the optimized single-frame image, so as to obtain a preprocessed image.
After completing the pre-processing of the optimized single frame image to obtain a pre-processed image, the neural network processing unit 1202 further transmits the pre-processed image to the application processor 130.
For example, the neural network processing unit 1202 may transmit the preprocessed image to the first image signal processing unit 1201, and the first image signal processing unit 1201 transmits the extracted reference parameter and the preprocessed image processed by the neural network processing unit 1202 to the application processor 130.
Optionally, the first image signal processing unit 1201 is further configured to, when extracting the reference parameter, transmit the extracted reference parameter to the application processor 130.
Referring to fig. 5, fig. 5 is a schematic diagram illustrating a fourth structure of the image processing circuit 100 according to the embodiment of the present disclosure. The image processing circuit 100 includes an image sensor 110, an image signal pre-processor 120, and an application processor 130. The image signal pre-processor 120 includes a first image signal processing unit 1201 and a neural network processing unit 1202. In addition, the image signal pre-processor 120 further includes a central control processing unit 1203, and the image sensor 110, the application processor 130, the first image signal processing unit 1201 and the neural network processing unit 1202 may refer to the image sensor 110, the application processor 130, the first image signal processing unit 1201 and the neural network processing unit 1202 in fig. 4, which is not described herein again.
In this embodiment, the first image signal processing unit 1201 is further configured to synchronously send the instruction information to the central control processing unit 1203 when the instruction information is transmitted to the application processor 130.
On the other hand, the central control processing unit 1203 is configured to control the preprocessing of the neural network processing unit 1202 according to the aforementioned instruction information. The control of the neural network processing unit by the central control processing unit 1203 includes, but is not limited to, timing and type of preprocessing.
Referring to fig. 6, fig. 6 is a schematic diagram illustrating a fifth structure of the image processing circuit 100 according to the embodiment of the present disclosure. The image processing circuit 100 includes an image sensor 110, an image signal pre-processor 120, and an application processor 130. The image signal pre-processor 120 includes a first image signal processing unit 1201 and a neural network processing unit 1202, and the application processor 130 includes an application processing unit 1301 and a second image signal processing unit 1302.
The second image signal processing unit 1302 is connected to the first image signal processing unit 1201 and is configured to acquire the reference parameter and the pre-processed image transmitted by the first image signal processing unit 1201, and perform post-processing on the pre-processed image according to the reference parameter to obtain a post-processed image;
the application processing unit 1301 is configured to:
previewing the post-processing image or carrying out video coding according to the post-processing image when the post-processing image is a dynamic image; or,
and when the post-processing image is a static image, carrying out image coding according to the post-processing image.
In the embodiment of the present application, post-processing performed by the second image signal processing unit 1302 is not particularly limited, and may be configured by a person skilled in the art according to actual needs.
For example, the second image signal processing unit 1302 may perform image enhancement processing on the pre-processed image. The image enhancement is to perform data transformation on image data needing to be enhanced, selectively highlight interesting features in an image and inhibit some unnecessary features in the image, so that the visual effect of the enhanced image quality is improved. Supervised learning is a machine learning task that infers model parameters from a labeled training dataset. In supervised learning, a sample pair includes an input object and a desired output. In the embodiment of the application, the input object is a training image in a sample pair, and the expected output is a target image of the pair. The initial image enhancement model is an image enhancement model that requires further model training to adjust model parameters. The type of the initial image enhancement model can be set according to needs, and can be a deep convolutional neural network model or a residual convolutional network model, for example. The goal of model training is to obtain better model parameters to improve the image enhancement effect. When training is carried out, a training image is input into an initial image enhancement model to obtain an output model enhancement image, then model parameters are adjusted according to the difference between the model enhancement image and a matched target image, so that the model enhancement image obtained by image enhancement according to the adjusted model parameters is closer to the target image, for example, the model parameters are adjusted towards the direction of reducing the loss value corresponding to the model by adopting a gradient reduction method until convergence, and the image enhancement model is obtained.
In this embodiment, after acquiring the reference parameters, the second image signal processing unit 1302 calls a pre-trained image enhancement model to perform image enhancement on the pre-processed image, so as to obtain an enhanced image (i.e., a post-processed image).
For example, referring to fig. 7, the left side of fig. 7 shows a pre-processed image obtained by the second image signal processing unit 1302, and the second image signal processing unit 1302 invokes a pre-trained image enhancement model to enhance the brightness and the definition of the pre-processed image to obtain an enhanced image (i.e., a post-processed image) shown on the right side.
It should be noted that the image type does not change with the processing of the external image, that is, if the original external image is a static image, the post-processed image obtained by the corresponding processing is also a static image, and if the original external image is a dynamic image, the post-processed image obtained by the corresponding processing is also a dynamic image. The still image is, for example, a single frame image shot in real time, the dynamic image is, for example, one frame image in an image sequence acquired during preview, and one frame image in an image sequence acquired during video recording.
For example, when the image processing circuit 100 provided in the present application is applied to a photographing scene, an external image acquired by the image sensor 110 is a frame of static image;
when the image processing circuit 100 provided by the present application is applied to a preview scene, an external image acquired by the image sensor 110 is a frame of dynamic image in a preview image sequence;
when the image processing circuit 100 provided in the present application is applied to a video scene, an external image acquired by the image sensor 110 is a frame of dynamic image in a video image sequence.
In this embodiment of the application, after the second image signal processing unit 1302 finishes post-processing on the pre-processed image and obtains a post-processed image accordingly, the application processing unit 1301 is configured to acquire the post-processed image from the second image signal processing unit 1302 and execute a corresponding operation according to a type of the post-processed image. It should be noted that, in the embodiment of the present application, the post-processing performed on the pre-processed image does not change the type of the pre-processed image, for example, the pre-processed image is a moving image, and a post-processed image obtained by the post-processing is also a moving image.
When the post-processing image is a moving image, the application processing unit 1301 may preview the post-processing image or perform video encoding according to the post-processing image. For example, when the current processed image is a dynamic image of one frame in the preview image sequence, after the corresponding post-processing image is obtained, the application processing unit 1301 previews the post-processed image; when the current processed image is a dynamic image of one frame in the video image sequence, after the post-processed image is obtained through corresponding processing, the application processing unit 1301 performs video coding according to the post-processed image.
When the post-processing image is a still image, the application processing unit 1301 performs image encoding according to the post-processing image to obtain an encoded image. For example, the pre-processed image is an image in RAW format, the post-processed image obtained by corresponding post-processing is also an image in RAW format, and the application processing unit 1301 may perform JPEG image encoding according to the post-processed image to obtain an encoded image in JPEG format.
Referring to fig. 8, fig. 8 is a schematic diagram illustrating a sixth structure of the image processing circuit 100 according to the embodiment of the present disclosure. The difference from the illustrated image processing circuit 100 of fig. 6 is that the image signal pre-processor 120 further includes a memory 1204 in the embodiment of the present application.
It should be noted that, in the embodiment of the present application, the type of the memory 1204 is not limited, and can be selected by a person skilled in the art according to actual needs, and the memory may be a static memory or a dynamic memory. For example, in the embodiment of the present application, the memory 1204 is a DDR (Double Data Rate) type dynamic memory.
In the embodiment of the present application, the first image signal processing unit 1201 provides the preprocessed image and the reference parameter to the second image signal processing unit 1302 by using different transmission methods according to the image type of the preprocessed image.
When the current processed image is a moving image, the first image signal processing unit 1201 is configured to directly transmit the reference parameter and the preprocessed image to the second image signal processing unit 1302. Since the moving image needs to consider the real-time performance of the processing, the pre-processed image and the reference parameter can be directly transmitted to the second image signal processing unit 1302 by the first image signal processing unit 1201 based on a direct connection (such as an MIPI connection) between the first image signal processing unit and the second image signal processing unit 1302.
When the current processed image is a still image, the first image signal processing unit 1201 is configured to write the reference parameter and the preprocessed image into the memory 1204, and the second image signal processing unit 1302 is configured to extract the preprocessed image and the reference parameter from the memory 1204.
The application also provides an electronic device for processing image data, and the electronic device comprises the image processing circuit provided by the application.
The present application further provides an image processing method, please refer to fig. 9, the flow of the image processing method may be as follows:
in 201, the image sensor collects an external image according to the configured collection parameters, and transmits the external image to an image signal pre-processor;
in 202, the image signal pre-processor transmits indication information to the application processor when receiving a packet header of a first image data packet of an external image transmitted by the image sensor;
in 203, the application processor updates acquisition parameters of the image sensor according to the indication information.
The image processing method provided by the application is applied to the image processing circuit shown in fig. 1 as an example.
In the embodiment of the present application, the image sensor 110 is connected to the image signal pre-processor 120, and is configured to acquire an external image according to the configured acquisition parameters, and transmit the acquired external image to the image signal pre-processor. For example, taking exposure parameters as an example, during automatic exposure, the application processor 130 calculates corresponding exposure parameters according to the collected relevant information required for automatic exposure by using a pre-configured automatic exposure control algorithm, and configures the exposure parameters to the image sensor 110, so that the image sensor 110 acquires an external image according to the exposure parameters, and transmits the external image to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, a connection manner of the image signal pre-Processor 120 and the image sensor 110 is not particularly limited, for example, the image signal pre-Processor 120 and the image sensor 110 are connected by an MIPI (Mobile Industry Processor Interface).
When transmitting the external image to the image signal pre-processor 120, the image sensor 110 encapsulates the external image into a plurality of image data packets for each frame, and transmits the image data packets to the image signal pre-processor. Illustratively, an image data packet includes a header field, an end-of-packet field and a data field, wherein the header field and the end-of-packet field are used for filling some necessary control information, such as synchronization information, address information, error control information, etc., and the data field is used for filling the actual image content.
On the other hand, the image signal pre-processor 120 receives an external image from the image sensor 110. In addition, the image signal pre-processor 120 is further connected to the application processor 130, wherein the connection manner between the application processor 130 and the image signal pre-processor 120 is not particularly limited in this embodiment, for example, the application processor 130 and the image signal pre-processor 120 may also be connected by MIPI.
When receiving the header of the first image data packet of a single frame image in the external image transmitted by the image sensor 110, the image signal pre-processor 120 transmits indication information to the application processor 130, where the indication information is used to instruct the application processor 130 to update the exposure parameters of the image sensor 110. It should be noted that, in the embodiment of the present application, the type of the indication information is not particularly limited, and may be selected by a person having ordinary skill in the art according to actual needs.
Illustratively, SOF is taken as the indication information.
The image signal pre-processor 120 and the application processor 130 are connected through MIPI, and in the process of designing MIPI by ASIC, the original RTL design of SoF (Start of Frame) needs to be changed, and a state machine thereof is shown in fig. 2.
The application processor 130 is further connected to the image sensor 110, and is configured to update the acquisition parameters of the image sensor 110 according to the indication information from the image signal pre-processor 120, so that the image sensor 110 performs image acquisition according to the updated acquisition parameters, thereby stably acquiring a high-quality image.
Optionally, in an embodiment, the image processing method provided by the present application further includes:
the image signal preprocessor 120 extracts the reference parameters of the single frame image, and preprocesses the single frame image to obtain a preprocessed image; and
transmitting the reference parameters and the preprocessed image to the application processor 130;
the application processor 130 performs post-processing on the pre-processed image according to the reference parameters to obtain a post-processed image.
In order to improve the image quality, in the embodiment of the present application, the image signal pre-processor 120 performs pre-processing on the external image from the image sensor 110, so as to obtain a pre-processed image accordingly.
In order to avoid the original data of the external image from being damaged due to the preprocessing of the external image by the image signal preprocessing 120, the image signal preprocessing 120 further extracts the reference parameters of the external image before the preprocessing of the external image. The reference parameters include relevant parameters required by the application processor 130 for image processing, such as an auto-focus parameter, an auto-white balance parameter, an auto-exposure parameter, and a phase focus parameter.
It should be noted that, in the embodiment of the present application, the preprocessing performed by the image signal preprocessor 120 is not specifically limited, for example, when the electronic device is located in an extremely dark environment with an ambient illuminance lower than 1lux, the image content of the finally obtained imaging image is difficult to identify, and by using the image signal preprocessor 120 provided by the present application, the original external image collected by the image sensor 110 may be optimized, so that the image content of the finally obtained imaging image can be clearly identified, and the purpose of improving the image quality is achieved.
The application processor 130 is connected to the image signal pre-processor 120, and is configured to further perform post-processing on the pre-processed image obtained by pre-processing by the image signal pre-processor 120, so as to obtain a post-processed image accordingly. It should be noted that, in the embodiment of the present application, a connection manner between the application processor 130 and the image signal pre-processor 120 is not particularly limited, for example, the application processor 130 and the image signal pre-processor 120 may also be connected by an MIPI. In addition, in the embodiment of the present application, post-processing performed by the application processor 130 is not particularly limited, and may be configured by a person skilled in the art according to actual needs, or may be configured by the application processor 130 by default when the application processor is shipped from a factory.
In addition, after the application processor 130 processes the post-processed image, the post-processed image may be previewed and/or stored directly, or the post-processed image may be previewed and/or stored after being further processed.
In addition, the application processor 130 controls the image sensor 110 to start capturing an external image and to end capturing the external image based on its connection with the image sensor 110.
Optionally, in an embodiment, the acquisition parameter includes an exposure parameter, after the application processor 130 completes current update of the exposure parameter of the image sensor 110, the image sensor 110 acquires an external image by using the updated exposure parameter, and accordingly, the application processor 130 collects exposure statistical information required for automatic exposure control, and calculates a new exposure parameter by using a configured automatic exposure control algorithm, so as to update the exposure parameter of the image sensor 110 again when receiving the indication information of the image signal preprocessor 120 again. In this way, the exposure parameters of the image sensor 110 are continuously updated until the exposure parameters converge.
Optionally, referring to fig. 3, the image processing circuit 100 further includes a register 140, where the register 140 is connected to the image signal pre-processor 120 and the application processor 130, and transmits the indication information to the application processor, including:
the image signal preprocessor 120 writes the preset flag bit of the register 140 as a preset value corresponding to the indication information, so that the application processor 130 acquires the indication information according to the preset value.
In the embodiment of the present application, the indication information of the image signal pre-processor 120 is not directly transmitted to the application processor 130, but is indirectly transmitted by using the third-party device "register 140", and accordingly, when the indication information is sent to the application processor 130, the image signal pre-processor 120 is configured to:
the preset flag bit of the register 140 is written as a preset value corresponding to the aforementioned indication information, so that the application processor 130 obtains the aforementioned indication information according to the preset value written by the preset flag bit of the register 140, and correspondingly updates the exposure parameter of the image sensor 110.
The embodiment of the present application does not specifically limit which flag bit of the PCIE register is selected as the preset flag bit to indicate the transmission of information, and a person skilled in the art can select the flag bit according to actual needs. It should be noted that, for the preset flag bit in the register 140, the corresponding relationship between the preset value and the indication information is established in advance in the present application. In this embodiment, which value is used to represent the indication information is not specifically limited in this embodiment, and may be selected by a person of ordinary skill in the art according to actual needs, for example, for the preset flag bit of the register 140, a value "1" is used to represent the indication information in this embodiment, accordingly, when the image signal preprocessor 120 needs to send the indication information to the application processor 130, the value of the preset flag bit in the register 140 is written to "1", so that the register 140 sends an interrupt signal to the application processor 130 to notify that the application processor 130 currently has data written to, the application processor 130 correspondingly reads the value of the preset flag bit in the register 140, which is "1" at this time, so as to obtain the indication information, and update the exposure parameter of the image sensor 110.
It should be noted that, in the embodiment of the present application, a person skilled in the art may select a register type according to actual needs, for example, a PCIE register is used to implement transmission of the indication information in the embodiment of the present application.
Optionally, in an embodiment, the transmitting the external image to the image signal pre-processor 120 includes:
the image sensor 110 encapsulates each line of image data of the single frame image into an image data packet, and sequentially transmits each image data packet to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, for one complete external image frame, the image sensor 110 does not directly transmit one complete external image frame to the image signal pre-processor 120, but transmits it to the image signal pre-processor 120 row by row.
The image sensor 110 is configured to encapsulate each line of image data of the single frame image into an image data packet, and sequentially transmit the encapsulated image data packet to the image signal pre-processor 120.
Optionally, in an embodiment, encapsulating each line of image data of the aforementioned image into an image data packet, and sequentially transmitting each image data packet to the image signal pre-processor 120 includes:
when acquiring one line of image data of the single frame image, the image sensor 110 encapsulates the acquired one line of image data into an image data packet, and transmits the encapsulated image data packet to the image signal pre-processor 120.
It should be noted that, in the embodiment of the present application, the image sensor 110 does not transmit the complete external image to the image signal pre-processor 120 line by line after acquiring the complete external image of one frame, but transmits the acquired image data of one line of the external image to the image signal pre-processor 120 every time the image data of one line of the external image is acquired.
For example, after the image sensor 110 acquires the first line of image data of the single frame image, the acquired first line of image data is packaged into an image data packet and transmitted to the image signal pre-processor 120, and meanwhile, the next line of image data of the single frame image is continuously acquired and transmitted again, and the process is repeated until the last line of image data of the single frame image is acquired and packaged into an image data packet and transmitted to the second image signal processing unit 1302, so that the transmission of the single frame image is completed.
Referring to fig. 4, the image signal preprocessor 120 includes a first image signal processing unit 1201 and a neural network processing unit 1202, extracts the reference parameter of the external image, and preprocesses the external image to obtain a preprocessed image; and transmitting the reference parameters and the pre-processed image to the application processor comprises:
the first image signal processing unit 1201 extracts a reference parameter of the external image, and transmits the extracted reference parameter to the application processor 130; performing optimization processing on the external image to obtain an optimized external image;
the neural network processing unit is used for preprocessing the optimized external image to obtain a preprocessed image; and transmits the pre-processed image to the application processor 130.
In this embodiment, the first image signal processing unit 1201 is configured to extract a reference parameter of the single frame image, transmit the extracted reference parameter to the application processor 130, and perform optimization processing on the single frame image to obtain an optimized external image. After the reference parameters of the single frame image are extracted and obtained, the first image signal processing unit 1201 further performs optimization processing on the single frame image according to a pre-configured optimization strategy to obtain an optimized single frame image. It should be noted that, in the embodiment of the present application, the optimization processing performed by the first image signal processing unit 1201 is not particularly limited, and may include at least one of dead pixel correction processing, time domain noise reduction processing, 3D noise reduction processing, linearization processing, and black level correction processing, and may also include an optimization processing manner not listed in the present application.
The neural network processing unit 1202 is configured to perform pre-processing on the optimized single-frame image to obtain a pre-processed image. The neural network processing unit 1202 is fixed with a plurality of neural network algorithms (for example, a neural network-based video night scene algorithm, a video HDR algorithm, a video blurring algorithm, a video noise reduction algorithm, a video super-resolution algorithm, and the like), and after the first image signal processing unit 1201 completes optimization processing on the single-frame image, the neural network processing unit 1202 invokes a corresponding neural network algorithm according to a configured preprocessing strategy to further preprocess the optimized single-frame image, so as to obtain a preprocessed image.
After completing the pre-processing of the optimized single frame image to obtain a pre-processed image, the neural network processing unit 1202 further transmits the pre-processed image to the application processor 130.
For example, the neural network processing unit 1202 may transmit the preprocessed image to the first image signal processing unit 1201, and the first image signal processing unit 1201 transmits the extracted reference parameter and the preprocessed image processed by the neural network processing unit 1202 to the application processor 130.
Optionally, the first image signal processing unit 1201 is further configured to, when extracting the reference parameter, transmit the extracted reference parameter to the application processor 130.
Referring to fig. 5, the image signal pre-processor 120 further includes a central control processing unit 1203, and the image processing method provided in the present application further includes:
when the first image signal processing unit 1201 transmits the indication information to the application processor 130, the indication information is synchronously transmitted to the central control processing unit 1203;
the central control processing unit 1203 controls the preprocessing of the neural network processing unit 1202 according to the instruction information.
In this embodiment, the first image signal processing unit 1201 is further configured to synchronously send the instruction information to the central control processing unit 1203 when the instruction information is transmitted to the application processor 130.
On the other hand, the central control processing unit 1203 is configured to control the preprocessing of the neural network processing unit 1202 according to the aforementioned instruction information. The control of the neural network processing unit by the central control processing unit 1203 includes, but is not limited to, timing and type of preprocessing.
Referring to fig. 6, the application processor 130 includes an application processing unit 1301 and a second image signal processing unit 1302, and performs post-processing on the pre-processed image according to the reference parameter to obtain a post-processed image, including:
the second image signal processing unit 1302 is configured to obtain the reference parameter and the pre-processed image transmitted by the first image signal processing unit 1201, and perform post-processing on the pre-processed image according to the reference parameter to obtain a post-processed image;
the image processing method provided by the application further comprises the following steps:
when the post-processing image is a dynamic image, the application processing unit 1301 previews the post-processing image or performs video encoding according to the post-processing image; or,
when the post-processing image is a still image, the application processing unit 1301 performs image encoding according to the post-processing image.
The post-processing performed by the second image signal processing unit 1302 in the embodiment of the application is not particularly limited, and can be configured by a person skilled in the art according to actual needs.
For example, the second image signal processing unit 1302 may perform image enhancement processing on the pre-processed image. The image enhancement is to perform data transformation on image data needing to be enhanced, selectively highlight interesting features in an image and inhibit some unnecessary features in the image, so that the visual effect of the enhanced image quality is improved. Supervised learning is a machine learning task that infers model parameters from a labeled training dataset. In supervised learning, a sample pair includes an input object and a desired output. In the embodiment of the application, the input object is a training image in a sample pair, and the expected output is a target image of the pair. The initial image enhancement model is an image enhancement model that requires further model training to adjust model parameters. The type of the initial image enhancement model can be set according to needs, and can be a deep convolutional neural network model or a residual convolutional network model, for example. The goal of model training is to obtain better model parameters to improve the image enhancement effect. When training is carried out, a training image is input into an initial image enhancement model to obtain an output model enhancement image, then model parameters are adjusted according to the difference between the model enhancement image and a matched target image, so that the model enhancement image obtained by image enhancement according to the adjusted model parameters is closer to the target image, for example, the model parameters are adjusted towards the direction of reducing the loss value corresponding to the model by adopting a gradient reduction method until convergence, and the image enhancement model is obtained.
In this embodiment, after acquiring the reference parameters, the second image signal processing unit 1302 calls a pre-trained image enhancement model to perform image enhancement on the pre-processed image, so as to obtain an enhanced image (i.e., a post-processed image).
For example, referring to fig. 7, the left side of fig. 7 shows a pre-processed image obtained by the second image signal processing unit 1302, and the second image signal processing unit 1302 invokes a pre-trained image enhancement model to enhance the brightness and the definition of the pre-processed image to obtain an enhanced image (i.e., a post-processed image) shown on the right side.
It should be noted that the image type does not change with the processing of the external image, that is, if the original external image is a static image, the post-processed image obtained by the corresponding processing is also a static image, and if the original external image is a dynamic image, the post-processed image obtained by the corresponding processing is also a dynamic image. The still image is, for example, a single frame image shot in real time, the dynamic image is, for example, one frame image in an image sequence acquired during preview, and one frame image in an image sequence acquired during video recording.
For example, when a scene is photographed, an external image acquired by the image sensor 110 is a frame of static image;
when a scene is previewed, an external image acquired by the image sensor 110 is a frame of dynamic image in a preview image sequence;
when a scene is recorded, the external image acquired by the image sensor 110 is a frame of dynamic image in the video image sequence.
In this embodiment of the application, after the second image signal processing unit 1302 finishes post-processing on the pre-processed image and obtains a post-processed image accordingly, the application processing unit 1301 is configured to acquire the post-processed image from the second image signal processing unit 1302 and execute a corresponding operation according to a type of the post-processed image. It should be noted that, in the embodiment of the present application, the post-processing performed on the pre-processed image does not change the type of the pre-processed image, for example, the pre-processed image is a moving image, and a post-processed image obtained by the post-processing is also a moving image.
When the post-processing image is a moving image, the application processing unit 1301 may preview the post-processing image or perform video encoding according to the post-processing image. For example, when the current processed image is a dynamic image of one frame in the preview image sequence, after the corresponding post-processing image is obtained, the application processing unit 1301 previews the post-processed image; when the current processed image is a dynamic image of one frame in the video image sequence, after the post-processed image is obtained through corresponding processing, the application processing unit 1301 performs video coding according to the post-processed image.
When the post-processing image is a still image, the application processing unit 1301 performs image encoding according to the post-processing image to obtain an encoded image. For example, the pre-processed image is an image in RAW format, the post-processed image obtained by corresponding post-processing is also an image in RAW format, and the application processing unit 1301 may perform JPEG image encoding according to the post-processed image to obtain an encoded image in JPEG format.
Referring to fig. 8, the image signal pre-processor 120 further includes a memory 1204 for transmitting the reference parameters and the pre-processed image to the application processor, which includes:
when the current processed image is a dynamic image, the first image signal processing unit 1201 is configured to directly transmit the reference parameter and the pre-processed image to the second image signal processing unit;
when the pre-processed image is a still image, the first image signal processing unit 1201 is configured to write the reference parameters and the pre-processed image into the memory, so that the second image signal processing unit 1302 extracts the reference parameters and the pre-processed image from the memory 1204.
It should be noted that, in the embodiment of the present application, the type of the memory 1204 is not limited, and can be selected by a person skilled in the art according to actual needs, and the memory may be a static memory or a dynamic memory. For example, in the embodiment of the present application, the memory 1204 is a DDR (Double Data Rate) type dynamic memory.
In the embodiment of the present application, the first image signal processing unit 1201 provides the preprocessed image and the reference parameter to the second image signal processing unit 1302 by using different transmission methods according to the image type of the preprocessed image.
When the current processed image is a moving image, the first image signal processing unit 1201 is configured to directly transmit the reference parameter and the preprocessed image to the second image signal processing unit 1302. Since the moving image needs to consider the real-time performance of the processing, the pre-processed image and the reference parameter can be directly transmitted to the second image signal processing unit 1302 by the first image signal processing unit 1201 based on a direct connection (such as an MIPI connection) between the first image signal processing unit and the second image signal processing unit 1302.
When the current processed image is a still image, the first image signal processing unit 1201 is configured to write the reference parameter and the preprocessed image into the memory 1204, and the second image signal processing unit 1302 is configured to extract the preprocessed image and the reference parameter from the memory 1204.
The image processing circuit, the circuit board, the electronic device, and the image processing method provided in the embodiments of the present application are described in detail above. The principles and implementations of the present application are described herein using specific examples, which are presented only to aid in understanding the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (12)
1. An image processing circuit comprising an image sensor, an image signal pre-processor and an application processor, wherein,
the image sensor is connected with the image signal pre-processor and used for acquiring an external image according to configured acquisition parameters and transmitting the external image to the image signal pre-processor;
the image signal preprocessor is connected with the application processor and is used for transmitting indication information to the application processor when receiving a packet header of a first image data packet of a single-frame image in the external image transmitted by the image sensor;
and the application processor is connected with the image sensor and used for updating the acquisition parameters of the image sensor according to the indication information.
2. The image processing circuit of claim 1, wherein the image signal pre-processor is further configured to extract reference parameters of the single frame image, and pre-process the single frame image to obtain a pre-processed image; and
transmitting the reference parameters and the pre-processed image to the application processor;
and the application processor is also used for carrying out post-processing on the pre-processed image according to the reference parameters to obtain a post-processed image.
3. The image processing circuit of claim 1, further comprising a register coupled to the image signal pre-processor and the application processor, the image signal pre-processor configured to:
and writing a preset flag bit of the register into a preset value corresponding to the indication information, so that the application processor acquires the indication information according to the preset value.
4. The image processing circuit of claim 1, wherein the image sensor is configured to pack each line of image data of the single frame image into one image data packet, and to sequentially transmit each image data packet to the image signal pre-processor.
5. The image processing circuit of claim 1, wherein the image signal pre-processor comprises a first image signal processing unit and a neural network processing unit, the first image signal processing unit to:
extracting a reference parameter of the single-frame image, and transmitting the reference parameter to the application processor;
optimizing the single-frame image to obtain an optimized single-frame image;
the neural network processing unit is used for preprocessing the optimized single-frame image to obtain a preprocessed image;
and transmitting the pre-processed image to the application processor.
6. The image processing circuit of claim 5, wherein the image signal pre-processor further comprises a central processing unit, and the first image signal processing unit is further configured to synchronously send the indication information to the central processing unit when transmitting the indication information to the application processor;
and the central control processing unit is used for controlling the pretreatment of the neural network processing unit according to the indication information.
7. The image processing circuit of claim 5, wherein the application processor comprises an application processing unit and a second image signal processing unit, wherein,
the second image signal processing unit is used for carrying out post-processing on the pre-processed image according to the reference parameter to obtain a post-processed image;
the application processing unit is configured to:
when the post-processing image is a dynamic image, previewing the post-processing image or carrying out video coding according to the post-processing image; or,
and when the post-processing image is a static image, carrying out image coding according to the post-processing image.
8. The image processing circuit of claim 7, wherein the image signal pre-processor further comprises a memory, wherein,
when the pre-processed image is a dynamic image, the first image signal processing unit is used for directly transmitting the reference parameter to the second image signal processing unit;
when the pre-processed image is a static image, the first image signal processing unit is used for writing the reference parameter into the memory;
the second image signal processing unit is used for extracting the reference parameter from the memory.
9. An electronic device for processing image data, characterized in that the electronic device comprises an image processing circuit according to any of claims 1-7.
10. An image processing method applied to an image processing circuit, wherein the image processing circuit comprises an image sensor, an image signal pre-processor and an application processor, the image processing method comprising:
the image sensor collects external images according to configured collection parameters and transmits the external images to the image signal pre-processor;
the image signal preprocessor transmits indication information to the application processor when receiving a packet header of a first image data packet of a single frame image in the external image transmitted by the image sensor;
and the application processor updates the acquisition parameters of the image sensor according to the indication information.
11. The image processing method according to claim 10, further comprising:
the image signal preprocessor extracts the reference parameters of the single-frame image and preprocesses the single-frame image to obtain a preprocessed image; and
transmitting the reference parameters and the pre-processed image to the application processor;
and the application processor performs post-processing on the pre-processed image according to the reference parameters to obtain a post-processed image.
12. The image processing method according to claim 11, wherein the image signal pre-processor includes a first image signal processing unit and a neural network processing unit, wherein,
the first image signal processing unit extracts a reference parameter of the external image; and
optimizing the external image to obtain an optimized external image;
the neural network processing unit is used for preprocessing the optimized external image to obtain a preprocessed image;
the first image signal processing unit transmits the reference parameter and the preprocessed image to the application processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010478315.2A CN113747145B (en) | 2020-05-29 | 2020-05-29 | Image processing circuit, electronic apparatus, and image processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010478315.2A CN113747145B (en) | 2020-05-29 | 2020-05-29 | Image processing circuit, electronic apparatus, and image processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113747145A true CN113747145A (en) | 2021-12-03 |
| CN113747145B CN113747145B (en) | 2024-02-27 |
Family
ID=78724940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010478315.2A Active CN113747145B (en) | 2020-05-29 | 2020-05-29 | Image processing circuit, electronic apparatus, and image processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113747145B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710202A (en) * | 2023-08-22 | 2024-03-15 | 荣耀终端有限公司 | Image fusion method, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015081556A1 (en) * | 2013-12-06 | 2015-06-11 | 华为终端有限公司 | Photographing method for dual-camera device and dual-camera device |
| CN107578453A (en) * | 2017-10-18 | 2018-01-12 | 北京旷视科技有限公司 | Compressed image processing method, apparatus, electronic equipment and computer-readable medium |
| CN108364267A (en) * | 2018-02-13 | 2018-08-03 | 北京旷视科技有限公司 | Image processing method, device and equipment |
| CN109801209A (en) * | 2019-01-29 | 2019-05-24 | 北京旷视科技有限公司 | Parameter prediction method, artificial intelligence chip, equipment and system |
| CN110602415A (en) * | 2019-09-30 | 2019-12-20 | 杭州海康威视数字技术股份有限公司 | Exposure control device, method and camera |
-
2020
- 2020-05-29 CN CN202010478315.2A patent/CN113747145B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015081556A1 (en) * | 2013-12-06 | 2015-06-11 | 华为终端有限公司 | Photographing method for dual-camera device and dual-camera device |
| CN107578453A (en) * | 2017-10-18 | 2018-01-12 | 北京旷视科技有限公司 | Compressed image processing method, apparatus, electronic equipment and computer-readable medium |
| CN108364267A (en) * | 2018-02-13 | 2018-08-03 | 北京旷视科技有限公司 | Image processing method, device and equipment |
| CN109801209A (en) * | 2019-01-29 | 2019-05-24 | 北京旷视科技有限公司 | Parameter prediction method, artificial intelligence chip, equipment and system |
| CN110602415A (en) * | 2019-09-30 | 2019-12-20 | 杭州海康威视数字技术股份有限公司 | Exposure control device, method and camera |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117710202A (en) * | 2023-08-22 | 2024-03-15 | 荣耀终端有限公司 | Image fusion method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113747145B (en) | 2024-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109963083B (en) | Image processor, image processing method, photographing device, and electronic apparatus | |
| CN110290288B (en) | Image processor, image processing method, photographing device, and electronic apparatus | |
| US11290641B2 (en) | Electronic device and method for correcting image corrected in first image processing scheme in external electronic device in second image processing scheme | |
| CN110177215A (en) | Image processing method, image processor, filming apparatus and electronic equipment | |
| US10600170B2 (en) | Method and device for producing a digital image | |
| CN113810593B (en) | Image processing method, device, storage medium and electronic equipment | |
| CN108401110A (en) | Acquisition methods, device, storage medium and the electronic equipment of image | |
| CN113744120A (en) | Multimedia processing chip, electronic device and image processing method | |
| CN113747145B (en) | Image processing circuit, electronic apparatus, and image processing method | |
| CN112822371B (en) | Image processing chip, application processing chip, data statistical system and method | |
| CN104811601B (en) | A kind of method and apparatus for showing preview image | |
| CN111193867B (en) | Image processing method, image processor, photographing device and electronic equipment | |
| CN110266967B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113744139A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113744119A (en) | Multimedia processing chip and electronic equipment | |
| CN113873142B (en) | Multimedia processing chip, electronic device, and moving image processing method | |
| CN113873141B (en) | Electronic equipment | |
| CN111491101B (en) | Image processor, image processing method, photographing device, and electronic apparatus | |
| CN114298889A (en) | Image processing circuit and image processing method | |
| US20210125304A1 (en) | Image and video processing using multiple pipelines | |
| CN113766142B (en) | Image processing apparatus, image signal preprocessing module, device, and processing method | |
| CN113873178B (en) | Multimedia processing chip, electronic device and image processing method | |
| CN113873143B (en) | Multimedia processing chip and electronic equipment | |
| CN110049254A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN113744125B (en) | Image processing method, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |