CN113873142A - Multimedia processing chip, electronic device and dynamic image processing method - Google Patents
Multimedia processing chip, electronic device and dynamic image processing method Download PDFInfo
- Publication number
- CN113873142A CN113873142A CN202010621869.3A CN202010621869A CN113873142A CN 113873142 A CN113873142 A CN 113873142A CN 202010621869 A CN202010621869 A CN 202010621869A CN 113873142 A CN113873142 A CN 113873142A
- Authority
- CN
- China
- Prior art keywords
- processing chip
- image data
- state information
- preprocessing
- multimedia
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application provides a multimedia processing chip, an electronic device and a dynamic image processing method, wherein the multimedia processing chip comprises an image signal processor, a neural network processor and an interface, and the multimedia processing chip preprocesses image data at least based on the neural network processor to obtain a preprocessing result; the image signal processor is used for extracting phase focusing state information from the image data; the interface comprises N paths, wherein M paths in the N paths are used for transmitting the preprocessing result to an application processing chip, and one path in an idle state in the N paths is used for directly transmitting the phase focusing information to the application processing chip; wherein N is a natural number greater than 2, and M is a positive integer less than N. The embodiment of the application can save bandwidth.
Description
Technical Field
The present application relates to the field of electronic technologies, and in particular, to a multimedia processing chip, an electronic device, and a dynamic image processing method.
Background
Various multimedia devices (such as digital cameras, smart phones, tablet computers, etc.) capable of performing video shooting and photographing functions generally include an image sensor for acquiring an image, a multimedia processing chip for performing image processing, and an Application Processor (AP). The image sensor may be connected to the multimedia processing chip through a Mobile Industry Processor Interface (MIPI) line, and the multimedia processing chip may be connected to the AP through the MIPI line.
The multimedia processing chip generally processes an Image obtained by the Image sensor by using an Image Signal Processor (ISP), and after the Image is processed, the multimedia processing chip stores a processing result into the memory and transmits the processing result stored in the memory to the AP.
Disclosure of Invention
The embodiment of the application provides a multimedia processing chip, electronic equipment and a dynamic image processing method, which can save bandwidth.
In a first aspect, an embodiment of the present application provides a multimedia processing chip, including an image signal processor, a neural network processor, and an interface, where the multimedia processing chip preprocesses image data based on at least the neural network processor to obtain a preprocessing result;
the image signal processor is used for extracting phase focusing state information from image data;
the interface comprises N paths, wherein M paths in the N paths are used for transmitting the preprocessing result to an application processing chip, and one path in an idle state in the N paths is used for transmitting the phase focusing information to the application processing chip;
wherein N is a natural number greater than 2, and M is a positive integer less than N.
In a second aspect, an embodiment of the present application provides an electronic device, including:
the camera is used for collecting image data;
the multimedia processing chip is the multimedia processing chip;
an application processing chip configured to:
receiving phase focusing state information and a preprocessing result sent by the multimedia processing chip;
calculating a phase focusing parameter based on the phase focusing state information, and configuring the phase focusing parameter to the camera;
and carrying out post-processing on the pre-processing result.
In a third aspect, an embodiment of the present application provides a dynamic image processing method, including:
the multimedia processing chip acquires dynamic image data;
the multimedia processing chip extracts phase focusing state information from the dynamic image data and directly transmits the phase focusing state information to an application processing chip;
the multimedia processing chip preprocesses the dynamic image data to obtain a preprocessing result and transmits the preprocessing result to the application processing chip;
the application processing chip receives the phase focusing state information, calculates phase focusing parameters based on the phase focusing state information, and configures the phase focusing parameters to a camera;
and the application processing chip receives the preprocessing result and carries out post-processing on the preprocessing result.
The phase focusing state information extracted from the image data by the multimedia processing chip can be directly transmitted to the application processing chip without an intermediate storage process, so that the occupation of bandwidth can be reduced, and the possibility of blockage is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 is a first structural schematic diagram of an electronic device provided in an embodiment of the present application.
Fig. 2 is a second schematic structural diagram of an electronic device according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of a multimedia processing chip according to an embodiment of the present application.
Fig. 4 is a third structural schematic diagram of an electronic device provided in the embodiment of the present application.
Fig. 5 is a fourth structural schematic diagram of an electronic device provided in the embodiment of the present application.
Fig. 6 is a schematic flowchart of a dynamic image processing method according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a multimedia processing chip, electronic equipment and a dynamic image processing method. With reference to the drawings, the following description is made in view of processing image data by an electronic device provided in an embodiment of the present application. In which like reference numerals represent like elements throughout the several figures, the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.
Referring to fig. 1, fig. 1 is a schematic view of a first structure of an electronic device according to an embodiment of the present disclosure. The electronic device 20 may include a camera 600, a multimedia processing chip 200, and an application processing chip 400. The camera 600 is in communication connection with the multimedia processing chip 200, and the multimedia processing chip 200 is in communication connection with the application processing chip 400. A communication connection is understood to mean a communication connection between two components, such as a wireless connection, or a wired connection. Where a wired connection is to be understood as a signal wire such as a wire connection, or two components soldered directly together.
The camera 600 may collect image data, and may also interpret the image data as image information. The cameras 600 may be one or more, such as the camera 600 may include at least one of a main camera, a wide-angle camera, and a tele camera, and the camera 600 may also include at least one of a macro camera, a normal black and white camera. The camera 600 may be a front camera of the electronic device 20, or may be a rear camera of the electronic device 20.
The camera 600 may include an image sensor, which may be a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor, a Charge Coupled Device (CCD) image sensor, and a lens. The camera 600 may collect RAW image data such as RAW domain data, and may transmit the collected image data to the multimedia processing chip 200 for processing by an image processor inside the multimedia processing chip 200.
The image data collected by the camera 600 may be referred to as raw image data, or may be referred to as initial image data. It is understood that the image data corresponds to an image, such as image data captured by the camera 600 during a photographing process of a user can be previewed from a preview window of the electronic device 20. Specifically, the image data collected by the camera 600 is processed by the multimedia processing chip 200 and the application processing chip 400 in sequence, and is displayed on the display screen of the electronic device 20 after the processing is completed. So that the user can preview an image through a preview window of the electronic device 20, which can be defined as a preview image.
The image data may be still image data or moving image data. Such as data of pictures stored in a memory, still image data acquired by the camera 600. The dynamic image data can be data of preview images, data of video recording and data of video playing.
The multimedia processing chip 200 may process image data, and the multimedia processing chip 200 may process image data such as moving image data collected from the camera 600. The multimedia processing chip 200 may also process image data not captured by the camera 600, such as dynamic image data stored in a memory. The multimedia processing chip 200 can transmit the processing result to the application processing chip 400 through the interface thereof, so that the application processing chip 400 can perform processing based on the processing result of the multimedia processing chip 200. Thereby, the display quality of the image can be improved.
The processing of the image data by the multimedia processing chip 200 may be referred to as preprocessing, or the like, and the processing of the image data by the application processing chip 400 may be referred to as post-processing.
Referring to fig. 2, fig. 2 is a schematic view of a second structure of an electronic device according to an embodiment of the present application. The electronic device 20 further includes a display 800, and the display 800 can display a screen. After the multimedia processing chip 200 and the application processing chip 400 process the image data, the display screen 800 can display the image. Of course, the multimedia processing chip 200 and the application processing chip 400 can store the image in the memory of the electronic device 20 after processing the image data.
It should be noted that the image data processed by the multimedia processing chip 200 in the embodiment of the present application may be RAW image data, and the application processing chip 200 may process the RAW image data first, convert the format of the RAW image data into the YUV image data format, and finally store and display the YUV image data. RAW image data retains more detail than other image data such as YUV image data. The embodiment of the application can improve the image quality on more details by processing the RAW image data.
The multimedia processing chip 200 may be configured to: acquiring image data such as dynamic image data, extracting phase focusing state information from the acquired image data, and directly transmitting the extracted phase focusing state information to the application processing chip 400; and preprocesses the image data to obtain a preprocessing result, and outputs the preprocessing result to the application processing chip 400. In the process of processing the acquired image data, the multimedia processing chip 200 may extract or extract the phase focusing information from the image data, and then directly transmit the extracted phase focusing information to the application processing chip 400, without performing storage processing on the phase focusing state information, so as to save the bandwidth of the multimedia processing chip 200, and reduce the possibility of increasing delay and causing jamming due to insufficient bandwidth.
The image signal processor 210 may resolve the phase-focus state information through a data type (Datatype) flag.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a multimedia processing chip according to an embodiment of the present disclosure. The multimedia Processing chip 200 may include an Image Signal Processing (ISP) 210, a Neural-Network Processing Unit (NPU) 220, and a second interface 202.
The multimedia processing chip 200 preprocesses the image data based on at least the neural network processor 220 to obtain a preprocessing result. Such as the multimedia processing chip 200, performs neural network algorithm processing on the image data based on the neural network processor 220. For example, the multimedia processing chip 200 performs a first pre-processing on the image data based on the image signal processor 210, and then performs a second pre-processing on the image data by the neural network processor 220. For example, the multimedia processing chip 200 performs a first preprocessing on the image data based on the image signal processor 210, then performs a second preprocessing on the image data after the first preprocessing by the neural network processor 220, and then performs a third preprocessing on the image data after the second preprocessing by the neural network processor 210.
It should be noted that, the manner of preprocessing the image data by the multimedia processing chip 200 in the embodiment of the present application is not limited thereto, and the above are only some specific examples of preprocessing the image data in the embodiment of the present application, and do not form the contraction of preprocessing the image data in the embodiment of the present application.
The image data processed by the multimedia processing chip 200 may be from the image data collected by the camera 600.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The multimedia processing chip 200 may further include a first interface 201, the first interface 201 is communicatively connected to the camera 600, and image data such as moving image data collected by the camera 600 may be transmitted to the first interface 201. The first interface 201 is also communicatively connected to the image signal processor 210, and the first interface 201 can transmit the image data received by the camera head 600 to the image signal processor 210.
It should be noted that the multimedia processing chip 200 can also process other image data, such as the image data of the multimedia processing chip 200 is transmitted to the multimedia processing chip 200 from the memory of the electronic device 20. Such as by using a transmission over a Peripheral Component Interconnect Express (PCIE) and/or low speed interconnect bus interface.
The image signal processor 210 performs a first preprocessing on the image data including at least one of optimization processes such as a linearization process, a dead pixel compensation process, and a black level correction process. The image signal processor 210 may store its first pre-processed result in a memory, such as the memory 230. The image signal processor 210 may also transmit the result of its first preprocessing directly to the neural network processor 220.
The neural network processor 220 performs a second pre-processing of the image data, such as processing of the image data using a neural network algorithm. Neural network algorithms such as semantic segmentation algorithms, HDR algorithms, noise reduction algorithms, super-resolution algorithms, night scene algorithms, fuzzy algorithms, etc. The neural network processor 220 has high efficiency of processing image data, and obviously improves the image quality. The results of the neural network processor 220 processing the image data may be stored to a memory, such as memory 230.
It should be noted that, in the embodiment of the present application, the image signal processor 210 performs the first preprocessing on the image data, such as dead pixel compensation and linearization, to optimize the image, improve the image quality, and accelerate the convergence rate of the neural network processor 220. Thereby reducing the time required for the neural network processor 220 to process one frame of image data, and achieving the purpose of processing the image data quickly and in real time.
When the bit width of the image data processed by the multimedia processing chip 200 is different from the bit width of the image data processed by the application processing chip 400, the preprocessing of the image data by the multimedia processing chip 200 according to the embodiment of the present application further includes a third preprocessing of the image data after the second preprocessing by the image signal processor 210, where the third preprocessing includes a bit width adjustment processing.
For example, the bit width of the image data processed by the neural network processor 220 according to the embodiment of the present application may be a first bit width, such as 14 bits (bits) and 16 bits. The bit width for processing the image data by using the processing chip 400 in the embodiment of the present application may be a second bit width, such as 12 bits. When the bit width of the image data processed by the neural network processor 220 is different from the bit width of the image data processed by the application processing chip 400, the image signal processor 210 may perform bit width adjustment processing on the image data processed by the neural network processor 220 to adjust the bit width of the image data to be the same as the bit width of the image data required to be processed by the application processing chip 400. The image data is then transmitted to the application processing chip 400 for post-processing.
Note that the image signal processor 210 performs processing on the image data, such as clipping processing, reduction processing, and the like on the image data, without being limited thereto.
The image signal processor 210 and the neural network processor 220 are connected to the system bus 240 in the embodiment of the present application. The memory 230 is also connected to the system bus 240, and the memory 230 may be external or internal. The memory 230 may store various data, such as a system in which the memory 230 stores the multimedia processing chip 200, a preprocessing result of the multimedia processing chip 200 preprocessing image data, and the like.
It should be noted that, since the preprocessing of the image data by the multimedia processing chip 200 according to the embodiment of the present application is at least based on the preprocessing of the image data by the neural network processor 220, it can be understood that the processing of the image data such as the RAW image data by the neural network processor 220 tends to change the state information of the RAW image data. After the multimedia processing chip 200 finishes preprocessing the image data, the application processing chip 400 may not perform post-processing on the preprocessing result, such as calculation of focusing parameters, white balance processing, etc., due to damage of the status information.
Based on this, in the embodiment of the present application, before the neural network processor 220 processes the image data, the image signal processor 210 processes the image data to obtain the state information of the image data. Such as phase focus state information, auto exposure state information, auto white balance state information, lens shading correction state information, and the like. The image signal processor 210 may extract or extract the phase-in-focus state information directly from the image data. The image signal processor 210 may use a preset algorithm to statistically calculate automatic exposure state information, automatic white balance state information, and lens shading correction state information from the image data. It should be noted that the image signal processor 210 may also use a preset algorithm to count other focusing state information such as contrast focusing state information from the image data.
Therefore, in the embodiment of the present application, before the neural network processor 220 processes the image data, the image signal processor 210 counts and extracts the image data to obtain various state information of the image data. When the neural network processor 220 processes the image data again, various state information is still maintained, even if the neural network processor 220 changes the state information of the image data after processing. The application processing chip 400 can still perform various processes based on various state information obtained by the image signal processor 210.
It should be noted that the data of various kinds of state information, particularly phase focus state information, is very large. And full pixel focusing is increasingly applied. That is, the space (buffer) for storing the phase-in-focus state information of the memory 230 is as large as the space (buffer) for storing the image data. If all the data are stored in the memory 230, the bandwidth consumption of the memory 230 is relatively large, and if the bandwidth cannot meet the requirement, the delay may be increased, and a stuck phenomenon may occur. Based on this, in the embodiment of the present application, when the image signal processor 210 acquires various kinds of state information, in particular, phase focusing state information, the phase focusing state information is not stored in the memory 230, but the image signal processor 210 directly transmits the extracted phase focusing state information to the second interface 202 and the third interface 402 through the second interface 202, and the application processing chip 400 calculates the phase focusing state information to calculate the phase focusing parameter and configures the phase focusing parameter to the camera 600. So that the camera 600 can perform focusing based on the phase focusing parameter.
Wherein the image signal processor 210 can resolve the phase focus state information through the Datatype flag.
Therefore, the embodiment of the application can save the bandwidth, reduce the delay increase caused by insufficient bandwidth and reduce the possibility of jamming. Therefore, the quality of the image is improved, and the fluency of image preview is further improved.
The image signal processor 210 may also directly transmit other state information it acquires, such as automatic white balance state information and automatic exposure state information, to the second interface 202 without storing it in the memory 230, and to the third interface 402 via the second interface 202. The application processing chip 400 may calculate the automatic white balance state information to calculate an automatic white balance parameter, and perform white balance processing on the preprocessing result of the multimedia processing chip 200 based on the automatic white balance parameter to improve image quality. The application processing chip 400 may also calculate automatic exposure state information to calculate exposure parameters, and configure the automatic exposure parameters to the camera 600. The camera 600 may perform an exposure based on the auto-exposure parameters.
The image signal processor 210 may also directly transmit other state information such as lens shading correction state information, which it acquires, to the second interface 202 without storing it in the memory 230, and to the third interface 402 by the second interface 202. The application processing chip 400 may calculate the lens shading correction status information to calculate a lens shading correction parameter, and perform lens shading correction on the preprocessing result of the multimedia processing chip 200 based on the lens shading correction parameter to improve the image quality.
The first Interface 201, the second Interface 202, and the third Interface 402 may each be a Mobile Industry Processor Interface (MIPI). In this embodiment, the second interface 202 may include N paths, where N is a natural number greater than 2. Such as 3. It should be noted that, in order to save power consumption and prevent the temperature rise from being too high, the embodiment of the present application may use a part of the second interface 202, such as an M-way path, to transmit image data. Or, the embodiment of the present application may use a part of the second interface 202, such as the M-path, to transmit the pre-processing result of the multimedia processing chip 200.
Where M is a positive integer less than N, such as 2.
Therefore, in the process of transmitting the preprocessing result to the application processing chip 400, the multimedia processing chip 200 according to the embodiment of the present application may leave at least one (N-M) path out of the N paths of the second interface 202. N-M is such as 1. That is, in the embodiment of the present application, the phase focusing state information may be transmitted to the path corresponding to the third interface 402 through one path of the second interface 202 in the idle state.
In some embodiments, M of the N paths of the second interface 202 are used to transmit the preprocessing result to the application processing chip 400, and at least one of the N paths of the second interface 202 in an idle state is used to directly transmit state information such as phase focusing information to the application processing chip 400. For example, M of the N paths of the second interface 202 are used to transmit the pre-processing result stored in the memory 230 to the application processing chip 400, and one of the N paths of the second interface 202 in the idle state is used to directly transmit the phase focusing state information extracted by the image signal processor 210 to the application processing chip 40.
It should be noted that, if the second interface 202 has other paths in the idle state, other data may also be transmitted. For example, the lanes of the second interface 202 in the idle state may also transmit other state information such as auto-exposure state information, auto-white balance state information. It is understood that the automatic exposure state information, the automatic white balance state information, etc. may also be transmitted to the application processing chip 400 using other interfaces.
It is understood that the first interface 201 may also have a three-way path, and the third interface 402 may also have a three-way path.
The application processing chip 400 may perform processing such as white balance processing, lens shading correction, and the like on the RAW image data. The processed RAW image data is then subjected to format conversion, such as converting the RAW image data into the format of YUV image data. And may be displayed on the display screen 800 or stored in a memory of the electronic device 20. It is understood that the application processing chip 400 may also perform image processing on the YUV image data.
It should be noted that the components of the multimedia processing chip 200 are not limited thereto, such as the multimedia processing chip 200 may further include a direct memory access controller, which may implement the movement of the image data. For example, it can move the pre-processing result stored in the memory 230 to the second interface 202 and transmit the pre-processing result to the third interface 402 through the second piece of interface 202.
Referring to fig. 5, fig. 5 is a schematic view illustrating a fourth structure of an electronic device according to an embodiment of the present disclosure. The multimedia processing chip 200 may further include a digital signal processor 250, and the digital signal processor 250 may be connected to the system bus 240, and may process still image data, such as selecting one frame image from a plurality of frame images as a photographed image. In some embodiments, when the digital signal processor 250 processes still image data, moving image data may be processed by the neural network processor 220. It should be noted that the digital signal processor 250 defined in the embodiment of the present application is mainly used to assist the neural network processor 220 in processing the image data.
The application processing chip 400 and the multimedia processing chip 200 can perform differentiation processing on the image data, such as the multimedia processing chip 200 performing noise reduction processing on the image data, and the application processing chip 400 does not perform noise reduction processing on the image data. For example, the multimedia processing chip 200 performs semantic segmentation processing on the image data, and the application processing chip 400 performs other processing such as focusing processing without performing semantic segmentation processing on the image data. Therefore, the embodiment of the present application can greatly save the power consumption of processing the image data by the application processing chip 400. The image data can be processed specifically by the multimedia processing chip 200, so as to improve the quality of the image data while saving the power consumption of the application processing chip 400.
Of course, the application processing chip 400 and the multimedia processing chip 200 may not perform differentiation processing on the image data, such as performing noise reduction processing on the image data by both the application processing chip 400 and the multimedia processing chip 200.
It should be noted that, although not shown in fig. 1-2 and fig. 4-5, the electronic device 20 defined in the embodiments of the present application may also include other devices, such as a battery, a speaker, a receiver, and the like.
The following description will be made in view of a moving image processing method.
Referring to fig. 6, fig. 6 is a schematic flowchart of a dynamic image processing method according to an embodiment of the present application, and referring to fig. 1 to fig. 5.
6001, the multimedia processing chip 200 acquires moving image data. The multimedia processing chip 200 can acquire moving image data from the camera 600 through the first interface 201. The first interface 201 acquires moving image data and transmits the moving image data to the image signal processor 210.
6002, the multimedia processing chip 200 extracts the phase focus state information from the moving image data and directly transmits the phase focus state information to the application processing chip 200. The image signal processor 210 may resolve the phase focus state information through the Datatype flag. After acquiring the phase focus information, the image signal processor 210 does not store the phase focus state information in the memory 230, but directly transmits the phase focus state information to the second interface 202 and transmits the phase focus state information to the third interface 402 through a part of the path of the second interface 202.
It should be noted that the image signal processor 210 may also use a preset algorithm to count other state information such as automatic white balance state information, automatic exposure state information, and lens shading correction state information. It is understood that other status information acquired by the image signal processor 210 may also be directly transmitted to the second interface 202, and transmitted to the third interface 402 by a part of the path of the second interface 202.
6003, the multimedia processing chip 200 preprocesses the dynamic image data to obtain a preprocessing result, and transmits the preprocessing result to the application processing chip 400. It should be noted that, the preprocessing of the dynamic image data by the multimedia processing chip 200 may be performed based on the neural network processor 220 to preprocess the image data to obtain a preprocessing result, or may be performed based on the image data signal processor 210 to perform the first preprocessing of the image data, and the neural network processor 220 to perform the second preprocessing of the dynamic image data after the first preprocessing. The image data may be pre-processed for the first time by the image data signal processor 210, pre-processed for the second time by the neural network processor 220, and pre-processed for the third time by the image signal processor 210. The processing of the moving image data by the image signal processor 210 may refer to the processing of the image data by the image signal processor 210, and is not described herein again. The processing of the dynamic image data by the neural network processor 220 may refer to the processing of the image data by the neural network processor 220, and is not described herein again.
The multimedia processing chip 200 may store the preprocessing result to the memory 230 after preprocessing the moving image data by its internal processor. The pre-processed result stored in the memory 230 is then transmitted to the second interface 202, and the second interface 202 transmits to the third interface 402 through a part of the paths, such as two paths.
6004, the application processing chip 400 receives the phase focusing status information, calculates a phase focusing parameter based on the phase focusing status information, and configures the phase focusing parameter to the camera 600. The third interface 402 of the application processing chip 400 receives the phase focusing state information from the second interface 202, and the application processing chip 400 may analyze, calculate and process the phase focusing information through a phase focusing module, for example, analyze and calculate the phase focusing state information to obtain a phase focusing parameter, and then configure the phase focusing parameter to the camera 600.
6005, the application processing chip 400 receives the pre-processing result and performs post-processing on the pre-processing result to improve the image quality.
It is understood that the application processing chip 400 may also receive other state information, such as automatic white balance state information, automatic exposure state information, and the like. The application processing chip 400 may execute a correlation algorithm to calculate an automatic white balance parameter and an automatic exposure parameter, configure the automatic exposure parameter to the camera 600, and perform white balance processing on the preprocessing result based on the automatic white balance parameter.
Therefore, the image quality can be improved, the bandwidth occupation can be saved, and the blocking is reduced.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The foregoing describes in detail a multimedia processing chip, an electronic device, and a dynamic image processing method provided in the embodiments of the present application, and specific examples are applied herein to explain the principles and implementations of the present application, and the description of the foregoing embodiments is only used to help understand the method and core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (13)
1. The multimedia processing chip is characterized by comprising an image signal processor, a neural network processor and an interface, wherein the multimedia processing chip preprocesses image data at least based on the neural network processor to obtain a preprocessing result;
the image signal processor is used for extracting phase focusing state information from image data;
the interface comprises N paths, wherein M paths in the N paths are used for transmitting the preprocessing result to an application processing chip, and one path in an idle state in the N paths is used for directly transmitting the phase focusing information to the application processing chip;
wherein N is a natural number greater than 2, and M is a positive integer less than N.
2. The multimedia processing chip of claim 1, further comprising a memory for storing at least the pre-processing results;
m paths in the N paths are used for transmitting the preprocessing result stored in the memory to an application processing chip;
and one path in an idle state in the N paths is used for directly transmitting the phase focusing state information extracted by the image signal processor to an application processing chip.
3. The multimedia processing chip of claim 2, wherein N is 3 and M is 2.
4. The multimedia processing chip according to any of claims 1 to 3, wherein the multimedia processing chip is configured to process RAW image data.
5. The multimedia processing chip according to any of claims 1 to 3, wherein the image signal processor is further configured to count at least one of automatic white balance status information, automatic exposure status information, and lens shading correction status information from the image data.
6. The multimedia processing chip according to any of claims 1 to 3, wherein the preprocessing of the image data by the multimedia processing chip comprises at least:
the image signal processor carries out first preprocessing on image data;
and the neural network processor performs second preprocessing on the image data after the first preprocessing.
7. The multimedia processing chip according to any of claims 1 to 3, wherein the preprocessing of the image data by the multimedia processing chip comprises at least:
the image signal processor carries out first preprocessing on image data;
the neural network processor carries out second preprocessing on the image data after the first preprocessing;
and the image signal processor performs third preprocessing on the image data after the second preprocessing.
8. The multimedia processing chip according to any of claims 1 to 3, further comprising a digital signal processor for processing still image data and the neural network processor for processing moving image data.
9. An electronic device, comprising:
the camera is used for collecting image data;
a multimedia processing chip according to any one of claims 1 to 8;
an application processing chip configured to:
receiving phase focusing state information and a preprocessing result sent by the multimedia processing chip;
calculating a phase focusing parameter based on the phase focusing state information, and configuring the phase focusing parameter to the camera;
and carrying out post-processing on the pre-processing result.
10. The electronic device of claim 9, wherein the image signal processor is further configured to count at least one of auto white balance status information, auto exposure status information, lens shading correction status information from the image data;
the application processing chip is further configured to:
calculating a white balance parameter based on the automatic white balance state information, and performing white balance processing on the preprocessing result based on the white balance parameter; and/or
Calculating exposure parameters based on the automatic exposure state information, and configuring the exposure parameters to the camera; and/or
And calculating lens shading correction parameters based on the lens shading correction state information, and correcting the preprocessing result based on the lens shading correction parameters.
11. The electronic device of claim 10, wherein the application processing chip is further configured to:
firstly, processing white balance processing and/or correction is carried out on the RAW image data;
the format of the RAW image data is converted into the format of YUV image data.
12. A moving image processing method, characterized in that the method comprises:
the multimedia processing chip acquires dynamic image data;
the multimedia processing chip extracts phase focusing state information from the dynamic image data and directly transmits the phase focusing state information to an application processing chip;
the multimedia processing chip preprocesses the dynamic image data to obtain a preprocessing result and transmits the preprocessing result to the application processing chip;
the application processing chip receives the phase focusing state information, calculates phase focusing parameters based on the phase focusing state information, and configures the phase focusing parameters to a camera;
and the application processing chip receives the preprocessing result and carries out post-processing on the preprocessing result.
13. The method for processing dynamic images according to claim 12, wherein the multimedia processing chip preprocesses the dynamic image data to obtain a preprocessing result, and before outputting the preprocessing result to the application processing chip, the method further comprises:
the multimedia processing chip counts at least one of automatic white balance state information, automatic exposure state information and lens shading correction state information from the dynamic image data, and at least one of the automatic white balance state information, the automatic exposure state information and the lens shading correction state information is transmitted to the application processing chip;
the application processing chip calculates a white balance parameter based on the automatic white balance state information, and performs white balance processing on the preprocessing result based on the white balance parameter; and/or
The application processing chip calculates exposure parameters based on the automatic exposure state information and configures the exposure parameters to the camera; and/or
And the application processing chip calculates lens shading correction parameters based on the lens shading correction state information and corrects the preprocessing result based on the lens shading correction parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621869.3A CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621869.3A CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113873142A true CN113873142A (en) | 2021-12-31 |
| CN113873142B CN113873142B (en) | 2023-07-25 |
Family
ID=78981946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010621869.3A Active CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113873142B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225822A (en) * | 2022-09-20 | 2022-10-21 | 荣耀终端有限公司 | Data processing method and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030076992A1 (en) * | 2001-06-26 | 2003-04-24 | Banish Michele R. | Neural network based element, image pre-processor, and method of pre-processing using a neural network |
| CN101472062A (en) * | 2008-05-23 | 2009-07-01 | 嘉兴闻泰通讯科技有限公司 | Mobile terminal apparatus and method for implementing pick-up head function |
| CN101573695A (en) * | 2006-12-29 | 2009-11-04 | 诺基亚公司 | Method and system for image pre-processing |
| CN205333219U (en) * | 2016-01-28 | 2016-06-22 | 兰州大学 | Multichannel temperature measuring circuit based on quartz crystal temperature sensor |
| CN109286753A (en) * | 2018-09-29 | 2019-01-29 | 南昌黑鲨科技有限公司 | Image processing method, system and computer readable storage medium |
| CN110213480A (en) * | 2019-04-30 | 2019-09-06 | 华为技术有限公司 | A kind of focusing method and electronic equipment |
| WO2020073959A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Image capturing method, and electronic device |
| CN111147695A (en) * | 2019-12-31 | 2020-05-12 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
-
2020
- 2020-06-30 CN CN202010621869.3A patent/CN113873142B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030076992A1 (en) * | 2001-06-26 | 2003-04-24 | Banish Michele R. | Neural network based element, image pre-processor, and method of pre-processing using a neural network |
| CN101573695A (en) * | 2006-12-29 | 2009-11-04 | 诺基亚公司 | Method and system for image pre-processing |
| CN101472062A (en) * | 2008-05-23 | 2009-07-01 | 嘉兴闻泰通讯科技有限公司 | Mobile terminal apparatus and method for implementing pick-up head function |
| CN205333219U (en) * | 2016-01-28 | 2016-06-22 | 兰州大学 | Multichannel temperature measuring circuit based on quartz crystal temperature sensor |
| CN109286753A (en) * | 2018-09-29 | 2019-01-29 | 南昌黑鲨科技有限公司 | Image processing method, system and computer readable storage medium |
| WO2020073959A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Image capturing method, and electronic device |
| CN110213480A (en) * | 2019-04-30 | 2019-09-06 | 华为技术有限公司 | A kind of focusing method and electronic equipment |
| CN111147695A (en) * | 2019-12-31 | 2020-05-12 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225822A (en) * | 2022-09-20 | 2022-10-21 | 荣耀终端有限公司 | Data processing method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113873142B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109671106B (en) | Image processing method, device and equipment | |
| CN111028189A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| WO2021238522A1 (en) | Multimedia processing chip, electronic device, and image processing method | |
| CN105611177A (en) | Method for realizing multiple-camera simultaneous photographing of panorama camera and panorama camera | |
| CN110290325B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113810593B (en) | Image processing method, device, storage medium and electronic equipment | |
| WO2023077939A1 (en) | Camera switching method and apparatus, and electronic device and storage medium | |
| CN110278375B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113873142B (en) | Multimedia processing chip, electronic device, and moving image processing method | |
| CN110266967B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113744119A (en) | Multimedia processing chip and electronic equipment | |
| US20230086519A1 (en) | Multimedia processing chip, electronic device, and method for dynamic-image processing | |
| CN113744139A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN112822371A (en) | Image processing chip, application processing chip, data statistical system and method | |
| CN108307114B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN109309784B (en) | Mobile terminal | |
| CN113747145B (en) | Image processing circuit, electronic apparatus, and image processing method | |
| CN114143471B (en) | Image processing method, system, mobile terminal and computer readable storage medium | |
| CN113873141B (en) | Electronic equipment | |
| CN113873143B (en) | Multimedia processing chip and electronic equipment | |
| CN113873178B (en) | Multimedia processing chip, electronic device and image processing method | |
| CN113837937A (en) | Multimedia processing chip, electronic equipment image fusion method and image cutting method | |
| CN113781288A (en) | Electronic device and image processing method | |
| CN113923341A (en) | Electronic device | |
| CN113840075B (en) | Electronic equipment and image fusion method based on electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |