CN113873142B - Multimedia processing chip, electronic device, and moving image processing method - Google Patents
Multimedia processing chip, electronic device, and moving image processing method Download PDFInfo
- Publication number
- CN113873142B CN113873142B CN202010621869.3A CN202010621869A CN113873142B CN 113873142 B CN113873142 B CN 113873142B CN 202010621869 A CN202010621869 A CN 202010621869A CN 113873142 B CN113873142 B CN 113873142B
- Authority
- CN
- China
- Prior art keywords
- processing chip
- image data
- preprocessing
- state information
- multimedia processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application discloses a multimedia processing chip, electronic equipment and a dynamic image processing method, wherein the multimedia processing chip comprises an image signal processor, a neural network processor and an interface, and the multimedia processing chip is used for preprocessing image data at least based on the neural network processor so as to obtain a preprocessing result; the image signal processor is used for extracting phase focusing state information from image data; the interface comprises N paths, M paths in the N paths are used for transmitting the preprocessing result to an application processing chip, and one path in an idle state in the N paths is used for directly transmitting the phase focusing information to the application processing chip; wherein N is a natural number greater than 2, and M is a positive integer less than N. The embodiment of the application can save bandwidth.
Description
Technical Field
The application relates to the technical field of electronics, in particular to a multimedia processing chip, electronic equipment and a dynamic image processing method.
Background
Various multimedia devices capable of performing video photographing and photographing functions (such as a digital camera, a smart phone, a tablet computer, etc.), generally have an image sensor for acquiring an image, a multimedia processing chip capable of performing image processing, and an application processor (AP, application Processor). The image sensor may be connected to the multimedia processing chip through a MIPI (Mobile Industry Processor Interface ) line, and the multimedia processing chip may be connected to the AP through a MIPI line.
The multimedia processing chip generally processes the image acquired by the image sensor by using an image signal processor (Image Signal Processor, ISP), and after the image processing is completed, the multimedia processing chip stores the processing result in a memory, and transmits the processing result stored in the memory to the AP.
Disclosure of Invention
The embodiment of the application provides a multimedia processing chip, electronic equipment and a dynamic image processing method, which can save bandwidth.
In a first aspect, an embodiment of the present application provides a multimedia processing chip, including an image signal processor, a neural network processor, and an interface, where the multimedia processing chip performs preprocessing on image data based at least on the neural network processor to obtain a preprocessing result;
the image signal processor is used for extracting phase focusing state information from image data;
the interface comprises N paths, M paths in the N paths are used for transmitting the preprocessing result to an application processing chip, and one path in an idle state in the N paths is used for transmitting the phase focusing information to the application processing chip;
wherein N is a natural number greater than 2, and M is a positive integer less than N.
In a second aspect, an embodiment of the present application provides an electronic device, including:
the camera is used for collecting image data;
a multimedia processing chip, wherein the multimedia processing chip is the multimedia processing chip;
an application processing chip configured to:
receiving phase focusing state information and a preprocessing result sent by the multimedia processing chip;
calculating phase focusing parameters based on the phase focusing state information, and configuring the phase focusing parameters to the camera;
and carrying out post-treatment on the pretreatment result.
In a third aspect, an embodiment of the present application provides a moving image processing method, including:
the multimedia processing chip acquires dynamic image data;
the multimedia processing chip extracts phase focusing state information from the dynamic image data and directly transmits the phase focusing state information to the application processing chip;
the multimedia processing chip performs preprocessing on the dynamic image data to obtain a preprocessing result, and transmits the preprocessing result to the application processing chip;
the application processing chip receives the phase focusing state information, calculates phase focusing parameters based on the phase focusing state information, and configures the phase focusing parameters to a camera;
and the application processing chip receives the pretreatment result and performs post-processing on the pretreatment result.
According to the embodiment of the application processing chip, the phase focusing state information extracted from the image data by the multimedia processing chip can be directly transmitted to the application processing chip, the occupation of bandwidth can be reduced without an intermediate storage process, and the possibility of blocking is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments will be briefly described below.
Fig. 1 is a schematic diagram of a first structure of an electronic device according to an embodiment of the present application.
Fig. 2 is a second schematic structural diagram of the electronic device according to the embodiment of the present application.
Fig. 3 is a schematic structural diagram of a multimedia processing chip according to an embodiment of the present application.
Fig. 4 is a third schematic structural diagram of the electronic device according to the embodiment of the present application.
Fig. 5 is a fourth schematic structural diagram of an electronic device according to an embodiment of the present application.
Fig. 6 is a flowchart of a moving image processing method according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a multimedia processing chip, electronic equipment and a dynamic image processing method. The following describes, with reference to the accompanying drawings, processing image data by an electronic device provided in an embodiment of the present application. In which like reference numerals refer to like elements throughout, the principles of the present application are illustrated as embodied in a suitable computing environment. The following description is based on the illustrated embodiments of the present application and should not be taken as limiting other embodiments not described in detail herein.
Referring to fig. 1, fig. 1 is a schematic diagram of a first structure of an electronic device according to an embodiment of the present application. The electronic device 20 may include a camera 600, a multimedia processing chip 200, and an application processing chip 400. The camera 600 is communicatively connected to the multimedia processing chip 200, and the multimedia processing chip 200 and the application processing chip 400 are communicatively connected. Where a communication connection is understood to be a communication connection between two components, such as by way of a wireless connection, and such as by way of a wired connection. Where a wired connection is understood to mean a signal line such as a wire connection, or two components are soldered directly together.
The camera 600 may collect image data, or may understand the image data as image information. The camera 600 may be one or more, such that the camera 600 may include at least one of a main camera, a wide-angle camera, and a tele camera, and the camera 600 may further include at least one of a macro camera, a normal black-and-white camera. The camera 600 may be a front camera of the electronic device 20 or a rear camera of the electronic device 20.
The camera 600 may include an image sensor, which may be a Complementary Metal Oxide Semiconductor (CMOS) image sensor, a charge coupled device (Charge Coupled Device, CCD) image sensor, or the like, and a lens. The camera 600 may collect RAW image data such as RAW domain data and may transmit the image data collected thereof to the multimedia processing chip 200 for processing by an image processor inside the multimedia processing chip 200.
The image data acquired by the camera 600 may be referred to as raw image data or initial image data. It will be appreciated that the image data corresponds to an image, such as image data captured by the camera 600 during a photograph by a user, may be previewed from a preview window of the electronic device 20. Specifically, the image data collected by the camera 600 is processed by the multimedia processing chip 200 and the application processing chip 400, and is displayed on the display screen of the electronic device 20 after the processing is completed. So that a user can preview an image through the preview window of the electronic device 20, which can be defined as a preview image.
The image data may be still image data or moving image data. Such as data of pictures stored in a memory, still image data acquired by the camera 600. The dynamic image data can be data of preview images, video recording data and video playing data.
The multimedia processing chip 200 may process image data, and the multimedia processing chip 200 may process image data such as moving image data acquired from the camera 600. The multimedia processing chip 200 may also process image data not collected from the camera 600, such as moving image data stored in a memory. The multimedia processing chip 200 may transmit its processing result to the application processing chip 400 through its interface so that the application processing chip 400 performs processing based on the processing result of the multimedia processing chip 200. Thereby improving the display quality of the image.
Among them, the processing of the image data by the multimedia processing chip 200 may be referred to as preprocessing, etc., and the processing of the image data by the application processing chip 400 may be referred to as post-processing.
Referring to fig. 2, fig. 2 is a schematic diagram of a second structure of the electronic device according to the embodiment of the present application. The electronic device 20 also includes a display screen 800, and the display screen 800 may display a picture. The multimedia processing chip 200 and the application processing chip 400 can display a picture on the display screen 800 after processing the image data. Of course, the multimedia processing chip 200 and the application processing chip 400 may also store the image in the memory of the electronic device 20 after the image data is processed.
It should be noted that, the image data processed by the multimedia processing chip 200 in this embodiment of the present application may be RAW image data, the application processing chip 200 may process the RAW image data first, then convert the format of the RAW image data into a YUV image data format, and finally store and display the YUV image data. The RAW image data retains more detail than other image data such as YUV image data. According to the embodiment of the application, the RAW image data is processed, so that the image quality can be improved on more details.
The multimedia processing chip 200 may be configured to: acquiring image data such as moving image data, extracting phase focus state information from the image data acquired thereby, and directly transmitting the phase focus state information extracted thereby to the application processing chip 400; and preprocessing the image data to obtain a preprocessing result, and inputting the preprocessing result to the application processing chip 400. In the processing process of the acquired image data, the multimedia processing chip 200 can extract or extract the phase focusing information from the image data, and then directly transmit the extracted phase focusing information to the application processing chip 400 without storing and processing the phase focusing state information, so that the bandwidth of the multimedia processing chip 200 can be saved, and the possibility of blocking caused by delay increase due to insufficient bandwidth is reduced.
The image signal processor 210 may decompose the phase focus state information through a data type (Datatype) flag, among others.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a multimedia processing chip according to an embodiment of the present application. The multimedia processing chip 200 may include an image signal processor (Image Signal Processing, ISP) 210, a Neural Network Processor (NPU) 220, and a second interface 202.
The multimedia processing chip 200 performs preprocessing on the image data based at least on the neural network processor 220 to obtain a preprocessing result. Such as the multimedia processing chip 200, performs neural network algorithmic processing on the image data based on the neural network processor 220. For another example, the multimedia processing chip 200 performs a first preprocessing on the image data based on the image signal processor 210, and then performs a second preprocessing on the image data by the neural network processor 220. For example, the multimedia processing chip 200 performs first preprocessing on the image data based on the image signal processor 210, then performs second preprocessing on the image data after the first preprocessing by the neural network processor 220, and then performs third preprocessing on the image data after the second preprocessing by the neural network processor 210.
It should be noted that, the mode of preprocessing the image data by the multimedia processing chip 200 in the embodiment of the present application is not limited to this, and the above is only a few specific examples of preprocessing the image data in the embodiment of the present application, and does not constitute the limitation of preprocessing the image data in the embodiment of the present application.
The image data processed by the multimedia processing chip 200 may be from the image data collected by the camera 600.
Referring to fig. 4, fig. 4 is a schematic diagram of a third structure of an electronic device according to an embodiment of the present application. The multimedia processing chip 200 may further include a first interface 201, the first interface 201 being communicatively connected to the camera 600, and image data such as moving image data collected by the camera 600 may be transferred to the first interface 201. The first interface 201 is also communicatively coupled to the image signal processor 210, and the first interface 201 may transmit image data it receives from the camera 600 to the image signal processor 210.
It should be noted that, the multimedia processing chip 200 may also process other image data, such as the image data of the multimedia processing chip 200 is transferred to the multimedia processing chip 200 from the memory of the electronic device 20. Such as by using a high speed interconnect bus interface (perfermal component InterconnectExpress, PCIE) and/or a low speed interconnect bus interface.
The image signal processor 210 performs optimization processing including at least one of linearization processing, dead pixel compensation, and black level correction on the image data for the first time. The image signal processor 210 may store the result after its first preprocessing in a memory such as the memory 230. The image signal processor 210 may also transmit the result of its first preprocessing directly to the neural network processor 220.
The neural network processor 220 performs a second preprocessing on the image data such as processing the image data using a neural network algorithm. Neural network algorithms such as semantic segmentation algorithms, HDR algorithms, noise reduction algorithms, super resolution algorithms, night scene algorithms, blur algorithms, and the like. The neural network processor 220 processes the image data with high efficiency, and the improvement of the image quality is obvious. The results of the processing of the image data by the neural network processor 220 may be stored to a memory, such as memory 230.
It should be noted that, in the embodiment of the present application, the image signal processor 210 performs the first preprocessing on the image data, such as dead pixel compensation, linearization processing, and the like, so that not only can the image be optimized, the image quality can be improved, but also the convergence speed of the neural network processor 220 can be increased. So that the time required for the neural network processor 220 to process one frame of image data can be reduced for the purpose of fast, real-time processing of image data.
When the bit width of the image data processed by the multimedia processing chip 200 is different from the bit width of the image data processed by the application processing chip 400, the preprocessing of the image data by the multimedia processing chip 200 according to the embodiment of the present application further includes a third preprocessing of the image data after the second preprocessing by the image signal processor 210, where the third preprocessing includes a bit width adjustment process.
For example, the bit width of the image data processed by the neural network processor 220 of the present embodiment may be a first bit width, such as 14 bits (bits), 16 bits. The bit width of the image data processed by the processing chip 400 according to the embodiment of the present application may be the second bit width, such as 12 bits. When the bit width of the image data processed by the neural network processor 220 is different from the bit width of the image data processed by the application processing chip 400, the image signal processor 210 may perform a bit width adjustment process on the image data processed by the neural network processor 220 to adjust the bit width of the image data to be the same as the bit width of the image data required to be processed by the application processing chip 400. The image data is then transferred to the application processing chip 400 for post-processing.
The image signal processor 210 processes the image data, such as clipping processing, reduction processing, and the like, without being limited thereto.
The image signal processor 210 and the neural network processor 220 of the embodiment of the present application are all connected to the system bus 240. The memory 230 is also connected to the system bus 240, and the memory 230 may be external or internal. The memory 230 may store various data, such as a system in which the multimedia processing chip 200 is stored in the memory 230, a preprocessing result in which the multimedia processing chip 200 preprocesses image data, and the like.
It should be noted that, since the preprocessing of the image data by the multimedia processing chip 200 according to the embodiment of the present application is based on at least the preprocessing of the image data by the neural network processor 220, it is understood that the processing of the image data such as the RAW image data by the neural network processor 220 tends to change the state information of the RAW image data. After the preprocessing of the image data by the multimedia processing chip 200 is completed, the application processing chip 400 may not perform post-processing such as calculation of focus parameters, white balance processing, etc. on the preprocessing result due to the damage of the state information.
Based on this, the embodiment of the present application processes the image data by the image signal processor 210 before the neural network processor 220 processes the image data, so as to obtain the status information of the image data. Such as phase focus state information, auto exposure state information, auto white balance state information, lens shading correction state information, and the like. The image signal processor 210 may extract or extract the phase focus state information directly from the image data. The image signal processor 210 may calculate the auto-exposure status information, the auto-white balance status information, the lens shading correction status information from the image data using a preset algorithm. It should be noted that the image signal processor 210 may also use a preset algorithm to calculate other focusing status information, such as contrast focusing status information, from the image data.
Thus, before the neural network processor 220 processes the image data, the image signal processor 210 performs statistics and extraction on the image data to obtain various status information of the image data. When the neural network processor 220 processes the image data again, various state information is still maintained even though the neural network processor 220 changes the state information of the image data after processing. The application processing chip 400 can still perform various processes based on various status information obtained by the image signal processor 210.
The data ratio of the various state information, particularly the phase focus state information, is very large. And full pixel focusing is increasingly applied. I.e., the memory 230 is used to store the phase focus state information space (buffer) as large as the space (buffer) used to store the image data. If all the data is stored in the memory 230, the bandwidth consumption of the memory 230 is relatively large, and if the bandwidth is not enough, the delay may be increased, so that a clamping phenomenon occurs. Based on this, in the embodiment of the present application, when the image signal processor 210 acquires various status information, especially phase focusing status information, the phase focusing status information is not stored in the memory 230, but the image signal processor 210 directly transmits the extracted phase focusing status information to the second interface 202, and transmits the phase focusing status information to the third interface 402 through the second interface 202, and the application processing chip 400 calculates the phase focusing status information to calculate a phase focusing parameter, and configures the phase focusing parameter to the camera 600. So that the camera 600 can focus based on the phase focus parameter.
The image signal processor 210 may decompose the phase focus state information through the Datatype flag.
Therefore, the embodiment of the application can save the bandwidth, reduce delay increase caused by insufficient bandwidth and reduce the possibility of blocking. Therefore, the quality of the image is improved, and the smoothness of image preview is further improved.
The image signal processor 210 may also transmit other status information acquired by it, such as auto white balance status information and auto exposure status information, directly to the second interface 202 and from the second interface 202 to the third interface 402, without storing the status information in the memory 230. The application processing chip 400 may calculate the automatic white balance status information to calculate an automatic white balance parameter, and perform white balance processing on the preprocessing result of the multimedia processing chip 200 based on the automatic white balance parameter to improve image quality. The application processing chip 400 may also calculate automatic exposure state information to calculate exposure parameters and configure the automatic exposure parameters to the camera 600. The camera 600 may perform one exposure based on the auto-exposure parameter.
The image signal processor 210 may also transmit other status information acquired by it, such as lens shading correction status information, directly to the second interface 202 without being stored in the memory 230, and to the third interface 402 by the second interface 202. The application processing chip 400 may calculate the lens shading correction status information to calculate lens shading correction parameters, and perform lens shading correction on the preprocessing result of the multimedia processing chip 200 based on the lens shading correction parameters to improve the image quality.
The first interface 201, the second interface 202, and the third interface 402 may each be a mobile industry processor interface (Mobile Industry Processor Interface, MIPI). The second interface 202 of the embodiment of the present application may include N paths, where N is a natural number greater than 2. Such as 3. It should be noted that, in order to save power consumption and prevent excessive temperature rise, some of the second interfaces 202, such as an M-way path, may be used to transmit image data in the embodiments of the present application. Or the embodiments of the present application may employ a portion of the second interface 202, such as the M-way path, to transmit the preprocessing result of the multimedia processing chip 200.
Where M is a positive integer less than N, such as 2.
Therefore, in the process of transmitting the preprocessing result to the application processing chip 400, the multimedia processing chip 200 in the embodiment of the present application may be free of at least one (N-M) path from the N paths of the second interface 202. N-M is, for example, 1. That is, in the embodiment of the present application, the phase focusing state information may be transmitted to the corresponding path of the third interface 402 through the path of the second interface 202 in the idle state.
In some embodiments, M paths of the N paths of the second interface 202 are used to transmit the preprocessing result to the application processing chip 400, and at least one path of the N paths of the second interface 202 in an idle state is used to directly transmit status information such as phase focusing information to the application processing chip 400. For example, the M paths of the N paths of the second interface 202 are used to transmit the preprocessing result stored in the memory 230 to the application processing chip 400, and the path of the N paths of the second interface 202 in the idle state is used to directly transmit the phase focusing state information extracted by the image signal processor 210 to the application processing chip 40.
It should be noted that, if the second interface 202 further has other paths in the idle state, other data may also be transmitted. For example, the path in which the second interface 202 is in an idle state may also transmit other state information such as auto-exposure state information, auto-white balance state information. It will be appreciated that the auto-exposure status information, auto-white balance status information, etc. may also be transmitted to the application processing chip 400 using other interfaces.
It will be appreciated that the first interface 201 may also have a three-way path and the third interface 402 may also have a three-way path.
In this case, the application processing chip 400 may perform processing such as white balance processing, lens shading correction, and the like on RAW image data. The processed RAW image data is then subjected to format conversion, such as converting the RAW image data into a format of YUV image data. And may be displayed on the display 800 or stored in the memory of the electronic device 20. It is understood that the application processing chip 400 may also perform image processing on YUV image data.
It should be noted that, the components of the multimedia processing chip 200 are not limited thereto, and for example, the multimedia processing chip 200 may further include a direct memory access controller, which may implement the movement of the image data. For example, it may move the pre-processing results stored in the memory 230 to the second interface 202 and transmit the pre-processing results to the third interface 402 via the second interface 202.
Referring to fig. 5, fig. 5 is a schematic diagram of a fourth structure of an electronic device according to an embodiment of the present application. The multimedia processing chip 200 may further include a digital signal processor 250, and the digital signal processor 250 may be connected to the system bus 240, which may process still image data, such as selecting one frame image from a plurality of frame images as a photographed image. In some embodiments, the dynamic image data may be processed by the neural network processor 220 when the digital signal processor 250 processes the static image data. It should be noted that the digital signal processor 250 defined in the embodiments of the present application is mainly used to assist the neural network processor 220 in processing the image data.
In which the application processing chip 400 and the multimedia processing chip 200 may perform differentiation processing on the image data, such as the multimedia processing chip 200 performing noise reduction processing on the image data, and the application processing chip 400 no longer performs noise reduction processing on the image data. For example, the multimedia processing chip 200 performs semantic division processing on the image data, and the application processing chip 400 performs other processing such as focusing processing without performing semantic division processing on the image data. Therefore, the embodiment of the application can greatly save the power consumption of the application processing chip 400 for processing the image data. The image data may be subjected to specialized processing by the multimedia processing chip 200 to improve the quality of the image data while saving power consumption of the application processing chip 400.
Of course, the application processing chip 400 and the multimedia processing chip 200 may not perform the differentiation processing on the image data, such as the application processing chip 400 and the multimedia processing chip 200 each performing the noise reduction processing on the image data.
It should be noted that, although not shown in fig. 1-2 and fig. 4-5, the electronic device 20 defined in the embodiments of the present application may further include other devices, such as a battery, a speaker, a receiver, and so on.
The following describes a dynamic image processing method in terms of the dynamic image processing method.
Referring to fig. 6, fig. 6 is a flowchart of a moving image processing method according to an embodiment of the present disclosure, and fig. 1 to 5 are combined.
6001, the multimedia processing chip 200 acquires moving image data. The multimedia processing chip 200 may acquire moving image data from the camera 600 through the first interface 201. After the first interface 201 acquires the moving image data, the moving image data is transmitted to the image signal processor 210.
6002, the multimedia processing chip 200 extracts phase focus state information from the moving image data, and directly transmits the phase focus state information to the application processing chip 200. The image signal processor 210 can decompose the phase focus state information through the Datatype flag. The image signal processor 210 does not store the phase focusing state information in the memory 230 after acquiring the phase focusing information, but directly transmits the phase focusing state information to the second interface 202, and transmits the phase focusing state information to the third interface 402 through a part of the paths of the second interface 202.
It should be noted that the image signal processor 210 may also use a preset algorithm to calculate other status information, such as automatic white balance status information, automatic exposure status information, and lens shading correction status information. It will be appreciated that other status information acquired by the image signal processor 210 may also be transmitted directly to the second interface 202, and transmitted by a portion of the path of the second interface 202 to the third interface 402.
6003, the multimedia processing chip 200 performs preprocessing on the moving image data to obtain a preprocessing result, and transmits the preprocessing result to the application processing chip 400. It should be noted that, the preprocessing of the moving image data by the multimedia processing chip 200 may be performed on the image data based on the neural network processor 220 to obtain a preprocessing result, or may be performed on the image data based on the image data signal processor 210 for the first time, and the moving image data after the first preprocessing may be performed on the moving image data by the neural network processor 220 for the second time. The image data may be subjected to a first preprocessing based on the image data signal processor 210, the dynamic image data after the first preprocessing may be subjected to a second preprocessing by the neural network processor 220, and the dynamic image data after the second preprocessing may be subjected to a third preprocessing by the image signal processor 210. The processing of the dynamic image data by the image signal processor 210 may refer to the processing of the image data by the image signal processor 210, and will not be described herein. The processing of the dynamic image data by the neural network processor 220 may refer to the processing of the image data by the neural network processor 220, which is not described herein.
The multimedia processing chip 200 may store the preprocessing result to the memory 230 after preprocessing the moving image data by a processor therein. The preprocessing result stored in the memory 230 is then transferred to the second interface 202, and the second interface 202 is transferred to the third interface 402 through a part of the paths, such as two paths.
6004, the application processing chip 400 receives the phase focusing status information, calculates a phase focusing parameter based on the phase focusing status information, and configures the phase focusing parameter to the camera 600. The third interface 402 of the application processing chip 400 receives the phase focusing status information from the second interface 202, and the application processing chip 400 may parse, calculate and process the phase focusing status information through the phase focusing module, for example, parse, calculate the phase focusing status information to obtain a phase focusing parameter, and then configure the phase focusing parameter to the camera 600.
6005, the application processing chip 400 receives the preprocessing result and performs post-processing on the preprocessing result to improve the image quality.
It will be appreciated that the application processing chip 400 may also receive other status information, such as auto white balance status information, auto exposure status information, etc. The application processing chip 400 may perform a correlation algorithm to calculate an auto white balance parameter and an auto exposure parameter, and configure the auto exposure parameter to the camera 600, and perform a white balance process on the preprocessing result based on the auto white balance parameter.
Therefore, the embodiment of the application not only can improve the image quality, but also can save the bandwidth occupation so as to reduce the jamming.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
The foregoing has described in detail a multimedia processing chip, an electronic device and a dynamic image processing method provided in the embodiments of the present application, and specific examples have been applied to illustrate the principles and embodiments of the present application, where the foregoing description of the embodiments is only for helping to understand the method and core ideas of the present application; meanwhile, those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, and the present description should not be construed as limiting the present application in view of the above.
Claims (12)
1. The multimedia processing chip is characterized by comprising an image signal processor, a neural network processor, an interface and a memory, wherein the multimedia processing chip is used for preprocessing image data at least based on the neural network processor to obtain a preprocessing result and storing the preprocessing result in the memory;
the storage is used for storing the preprocessing result, and the image signal processor is used for extracting phase focusing state information from image data;
the interface comprises N paths, M paths in the N paths are used for transmitting the preprocessing result stored in the memory to an application processing chip, and one path in an idle state in the N paths is used for directly transmitting phase focusing information extracted by the image signal processor to the application processing chip so that the application processing chip calculates phase focusing parameters based on the phase focusing state information; wherein N is a natural number greater than 2, and M is a positive integer less than N.
2. The multimedia processing chip of claim 1, wherein N is 3 and M is 2.
3. The multimedia processing chip according to claim 1 or 2, wherein the multimedia processing chip is configured to process RAW image data.
4. The multimedia processing chip according to claim 1 or 2, wherein the image signal processor is further configured to count at least one of auto white balance status information, auto exposure status information, and lens shading correction status information from image data.
5. The multimedia processing chip according to claim 1 or 2, wherein the preprocessing of the image data by the multimedia processing chip comprises at least:
the image signal processor performs first preprocessing on image data;
the neural network processor performs a second preprocessing on the image data after the first preprocessing.
6. The multimedia processing chip according to claim 1 or 2, wherein the preprocessing of the image data by the multimedia processing chip comprises at least:
the image signal processor performs first preprocessing on image data;
the neural network processor performs second preprocessing on the image data after the first preprocessing;
the image signal processor performs a third preprocessing on the image data after the second preprocessing.
7. The multimedia processing chip according to claim 1 or 2, further comprising a digital signal processor for processing still image data, the neural network processor for processing moving image data.
8. An electronic device, comprising:
the camera is used for collecting image data;
a multimedia processing chip as claimed in any one of claims 1 to 7;
an application processing chip configured to:
receiving phase focusing state information and a preprocessing result sent by the multimedia processing chip;
calculating phase focusing parameters based on the phase focusing state information, and configuring the phase focusing parameters to the camera;
and carrying out post-treatment on the pretreatment result.
9. The electronic device of claim 8, wherein the image signal processor is further configured to count at least one of auto white balance status information, auto exposure status information, lens shading correction status information from image data;
the application processing chip is further configured to:
calculating a white balance parameter based on the automatic white balance state information, and performing white balance processing on the preprocessing result based on the white balance parameter; and/or
Calculating exposure parameters based on the automatic exposure state information, and configuring the exposure parameters to the camera; and/or
And calculating lens shading correction parameters based on the lens shading correction state information, and correcting the preprocessing result based on the lens shading correction parameters.
10. The electronic device of claim 9, wherein the application processing chip is further configured to:
firstly, carrying out white balance processing and/or correction on RAW image data;
the format of RAW image data is converted into a format of YUV image data.
11. A dynamic image processing method, characterized in that the method comprises:
the method comprises the steps that a multimedia processing chip obtains dynamic image data, wherein the multimedia processing chip comprises an image signal processor, a neural network processor, an interface and a memory, and the interface comprises an N-path passage;
the multimedia processing chip extracts phase focusing state information from the dynamic image data through the image signal processor, and directly transmits the phase focusing state information to the application processing chip through one path in an idle state in the N paths;
the multimedia processing chip preprocesses the dynamic image data through the neural network processor to obtain a preprocessing result, and stores the preprocessing result in the memory; transmitting the preprocessing result stored in the memory to the application processing chip through an M path in the N paths, wherein N is a natural number greater than 2, and M is a positive integer less than N;
the application processing chip receives the phase focusing state information, calculates phase focusing parameters based on the phase focusing state information, and configures the phase focusing parameters to a camera;
and the application processing chip receives the pretreatment result and performs post-processing on the pretreatment result.
12. The moving image processing method according to claim 11, wherein the multimedia processing chip performs preprocessing on the moving image data to obtain a preprocessing result, and before inputting the preprocessing result to the application processing chip, the method further comprises:
the multimedia processing chip calculates at least one of automatic white balance state information, automatic exposure state information and lens shading correction state information from the dynamic image data, and transmits the at least one of the automatic white balance state information, the automatic exposure state information and the lens shading correction state information to the application processing chip;
the application processing chip calculates a white balance parameter based on the automatic white balance state information, and performs white balance processing on the preprocessing result based on the white balance parameter; and/or
The application processing chip calculates exposure parameters based on the automatic exposure state information, and configures the exposure parameters to the camera; and/or
The application processing chip calculates lens shading correction parameters based on the lens shading correction state information, and corrects the preprocessing result based on the lens shading correction parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621869.3A CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621869.3A CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113873142A CN113873142A (en) | 2021-12-31 |
| CN113873142B true CN113873142B (en) | 2023-07-25 |
Family
ID=78981946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010621869.3A Active CN113873142B (en) | 2020-06-30 | 2020-06-30 | Multimedia processing chip, electronic device, and moving image processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113873142B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225822B (en) * | 2022-09-20 | 2023-03-31 | 荣耀终端有限公司 | Data processing method and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101472062A (en) * | 2008-05-23 | 2009-07-01 | 嘉兴闻泰通讯科技有限公司 | Mobile terminal apparatus and method for implementing pick-up head function |
| CN110213480A (en) * | 2019-04-30 | 2019-09-06 | 华为技术有限公司 | A kind of focusing method and electronic equipment |
| WO2020073959A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Image capturing method, and electronic device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2003003296A1 (en) * | 2001-06-26 | 2003-01-09 | Sy Technologies | Neural network based element image pre-processor and method of pre-processing using a neural network |
| US7973823B2 (en) * | 2006-12-29 | 2011-07-05 | Nokia Corporation | Method and system for image pre-processing |
| CN205333219U (en) * | 2016-01-28 | 2016-06-22 | 兰州大学 | Multichannel temperature measuring circuit based on quartz crystal temperature sensor |
| CN109286753A (en) * | 2018-09-29 | 2019-01-29 | 南昌黑鲨科技有限公司 | Image processing method, system and computer readable storage medium |
| CN111147695B (en) * | 2019-12-31 | 2022-05-13 | Oppo广东移动通信有限公司 | Image processing method, image processor, shooting device and electronic equipment |
-
2020
- 2020-06-30 CN CN202010621869.3A patent/CN113873142B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101472062A (en) * | 2008-05-23 | 2009-07-01 | 嘉兴闻泰通讯科技有限公司 | Mobile terminal apparatus and method for implementing pick-up head function |
| WO2020073959A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Image capturing method, and electronic device |
| CN110213480A (en) * | 2019-04-30 | 2019-09-06 | 华为技术有限公司 | A kind of focusing method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113873142A (en) | 2021-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4546565B2 (en) | Digital image processing | |
| EP2882184A2 (en) | Robust automatic exposure control using embedded data | |
| CN109993722B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| WO2021238522A1 (en) | Multimedia processing chip, electronic device, and image processing method | |
| CN110290325B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US20230086519A1 (en) | Multimedia processing chip, electronic device, and method for dynamic-image processing | |
| CN113810593B (en) | Image processing method, device, storage medium and electronic equipment | |
| CN113873142B (en) | Multimedia processing chip, electronic device, and moving image processing method | |
| CN110266965B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110266967B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US20110316971A1 (en) | Single pipeline stereo image capture | |
| CN113744119A (en) | Multimedia processing chip and electronic equipment | |
| CN113744139A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN108307114B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN116437222B (en) | Image processing method and electronic equipment | |
| CN113747145B (en) | Image processing circuit, electronic apparatus, and image processing method | |
| CN113873143B (en) | Multimedia processing chip and electronic equipment | |
| CN113873141B (en) | Electronic equipment | |
| CN113873178B (en) | Multimedia processing chip, electronic device and image processing method | |
| CN110049254B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN113837937A (en) | Multimedia processing chip, electronic equipment image fusion method and image cutting method | |
| CN113923341A (en) | Electronic device | |
| WO2023040540A1 (en) | Image processing chip, application processing chip, electronic device, and image processing method | |
| CN113781288A (en) | Electronic device and image processing method | |
| CN113744118A (en) | Multimedia processing chip, electronic device and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |