US20140146187A1

US20140146187A1 - Data processing apparatus for configuring camera interface based on compression characteristic of compressed multimedia data and related data processing method

Info

Publication number: US20140146187A1
Application number: US13/965,169
Authority: US
Inventors: Chi-cheng Ju; Tsu-Ming Liu
Original assignee: MediaTek Inc
Current assignee: MediaTek Inc
Priority date: 2012-11-23
Filing date: 2013-08-12
Publication date: 2014-05-29
Also published as: US9535489B2; US9568985B2; CN103841417B; CN103841317A; CN103841315B; US20140146188A1; CN103841318A; US20170034517A1; CN103841315A; US20140146186A1; US20140146195A1; US20170078567A1; US10200603B2; US20140146194A1; CN103841316B; CN103841316A; CN103841417A; CN103841415B; CN107277347A; US20140146874A1

Abstract

A data processing apparatus at a transmitter end has an output interface and a camera controller. The output interface packs a compressed multimedia data into an output bitstream transmitted via a camera interface. The camera controller refers to a compression characteristic of the compressed multimedia data to configure a transmission setting of the output interface over the camera interface. A data processing apparatus at a receiver end has an input interface and a controller. The input interface un-packs an input bitstream received via the camera interface into a compressed multimedia data. The controller configures a reception setting of the input interface over the camera interface in response to a compression characteristic of the compressed multimedia data. In addition, the data processing apparatus at the transmitter end may selectively enable a compression mode by checking the de-compression capability of the data processing apparatus at the receiver end.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional application No. 61/729,426, filed on Nov. 23, 2012 and incorporated herein by reference.

BACKGROUND

The disclosed embodiments of the present invention relate to transmitting and receiving multimedia data over a camera interface, and more particularly, to a data processing apparatus for configuring a camera interface based on a compression characteristic of a compressed multimedia data and related data processing method.
A camera interface is disposed between a first chip and a second chip to transmit multimedia data from the first chip to the second chip for further processing. For example, the first chip may include a camera module, and the second chip may include an image signal processor (ISP). The multimedia data may include image data (i.e., a single still image) or video data (i.e., a video sequence composed of images). When a camera sensor with a higher resolution is employed in the camera module, the multimedia data transmitted over the camera interface would have a larger data size/data rate, which increases the power consumption of the camera interface inevitably. If the camera module and the ISP are both located at a portable device (e.g., a smartphone) powered by a battery device, the battery life is shortened due to the increased power consumption of the camera interface. Thus, there is a need for an innovative design which can effectively reduce the power consumption of the camera interface.

SUMMARY

In accordance with exemplary embodiments of the present invention, a data processing apparatus for configuring a camera interface based on a compression characteristic of a compressed multimedia data and related data processing method are proposed.
According to a first aspect of the present invention, an exemplary data processing apparatus is disclosed. The exemplary data processing apparatus includes an output interface and a camera controller. The output interface is arranged for packing a compressed multimedia data into an output bitstream and outputting the output bitstream via a camera interface. The camera controller is arranged for referring to at least a compression characteristic of the compressed multimedia data to configure a transmission setting of the output interface over the camera interface.
According to a second aspect of the present invention, an exemplary data processing apparatus is disclosed. The exemplary data processing apparatus includes an input interface and an image signal processor (ISP) controller. The input interface is arranged for receiving an input bitstream via a camera interface, and un-packing the input bitstream into a compressed multimedia data that is transmitted over the camera interface. The controller is arranged for configuring a reception setting of the input interface over the camera interface in response to at least a compression characteristic of the compressed multimedia data.
According to a third aspect of the present invention, an exemplary data processing method is disclosed. The exemplary data processing method includes following steps: referring to at least a compression characteristic of a compressed multimedia data to configure a transmission setting of an output interface over the camera interface; and utilizing an output interface for packing the compressed multimedia data into an output bitstream and outputting the output bitstream via the camera interface.
According to a fourth aspect of the present invention, an exemplary data processing method is disclosed. The exemplary data processing method includes the following steps: configuring a reception setting of an input interface over a camera interface in response to at least a compression characteristic of a compressed multimedia data; and utilizing an input interface for receiving an input bitstream via the camera interface, and un-packing the input bitstream into the compressed multimedia data that is transmitted over the camera interface.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a data processing system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a first control and data flow of the data processing system shown in FIG. 1.

FIG. 3 is a flowchart illustrating a second control and data flow of the data processing system shown in FIG. 1.

FIG. 4 is a diagram illustrating the change of the blanking period after the interface compression is applied.

FIG. 5 is a flowchart illustrating a third control and data flow of the data processing system shown in FIG. 1.

FIG. 6 is a flowchart illustrating a method of setting the camera module by referring to a result of checking a de-compression capability of the ISP according to an embodiment of the present invention.

DETAILED DESCRIPTION

Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.
The present invention proposes applying data compression to a multimedia data and then transmitting a compressed multimedia data over a camera interface. As the data size/data rate of the compressed multimedia data is smaller than that of the original un-compressed multimedia data, the power consumption of the camera interface is reduced correspondingly. Besides, as the compressed multimedia data requires a smaller occupied bandwidth, the present invention further proposes configuring the camera interface based on the compression characteristic of the compressed multimedia data. Further details will be described as below.
FIG. 1 is a block diagram illustrating a data processing system according to an embodiment of the present invention. The data processing system 100 includes a plurality of data processing apparatuses such as a camera module 102 and an image signal processor (ISP) 104. The camera module 102 and the ISP 104 may be different chips, and the camera module 102 communicates with the ISP 104 via a camera interface 103 having a plurality of data lines (e.g., DL0, DL1, DL2, and DL3) (for example, differential pair, a pin or group of pins) and one clock line CLK. It should be noted that the number of data lines shown in FIG. 1 is for illustrative purposes only, and is not meant to be a limitation of the present invention. In this embodiment, the camera interface 103 may be a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).
The camera module 102 supports un-compressed data transmission and compressed data transmission. When the camera module 102 is used to transmit un-compressed data to the ISP 104, the camera module 102 generates the un-compressed multimedia data D1 according to an input multimedia data derived from capturing the incident light, and transmits the un-compressed multimedia data D1 over the camera interface 103. When the camera module 102 is used to transmit compressed data to the ISP 104, the camera module 102 generates a compressed multimedia data D1′ according to the input multimedia data derived from capturing the incident light, and transmits the compressed multimedia data D1′ over the camera interface 103. By way of example, but not limitation, the input multimedia data may be an image data or a video data.
As shown in FIG. 1, the camera module 102 includes a camera controller 112, an output interface 114 and other circuitry 116. The other circuitry 116 includes circuit elements required for capturing the incident light to obtain the input multimedia data which is used for generating the un-compressed data D1 or the compressed data D1′. For example, the other circuitry 116 may have a lens module, a camera sensor, a camera buffer, a compressor, multiplexer(s), etc. The lens module guides the incident light to the camera sensor, and the camera sensor obtains the input multimedia data correspondingly. The input multimedia data obtained by the camera sensor may be a single captured image or a video sequence composed of a plurality of captured images. Besides, the input multimedia data obtained by the camera sensor may be single view data for 2D display or multiple view data for 3D display. The camera buffer is arranged to buffer the input multimedia data obtained by the camera sensor. The compressor performs data compression. The multiplexer receives the un-compressed multimedia data D1 and the compressed multimedia data D1′, and selectively outputs the un-compressed multimedia data D1 or the compressed multimedia data D1′ according to the operation mode of the camera module 102. For example, when the camera module 102 is operated under a compression mode, the multiplexer outputs the compressed multimedia data D1′; and when the camera module 102 is operated under a non-compression mode, the multiplexer outputs the un-compressed multimedia data D1. As the present invention focuses on the control of the output interface 114, further description of the other circuitry 116 is omitted here for brevity.
The output interface 114 includes a packing unit 117 and a plurality of switches (e.g., 118_1, 118_2, 118_3 and 118_4). It should be noted that the number of the switches included in the output interface 114 is equal to the number of data lines included in the camera interface 103. Hence, each of the switches 118_1-118_4 controls whether a data line is used for data transmission. In this embodiment, the switches 118_1-118_4 are controlled by a plurality of enable signals EN0-EN3 generated from the camera controller 112, respectively. By way of example, the enable signals EN0-EN3 may be transmitted via an I²C (Inter-Integrated Circuit) interface, and/or programmed in an e-fuse device or an EPPROM (erasable programmable read only memory) by the camera controller 112. When an enable signal has a first logic value (e.g., ‘1’), a corresponding switch is enabled (i.e., switched on) to enable the data transmission over the corresponding data line; and when the enable signal has a second logic value (e.g., ‘0’), the corresponding switch is disabled (i.e., switched off) to disable the data transmission over the corresponding data line.
Regarding the ISP 104, it communicates with the camera module 102 via the camera interface 103. In this embodiment, the ISP 104 supports un-compressed data reception and compressed data reception. When the camera module 102 transmits the un-compressed data D1 to the ISP 104, the ISP 104 is operated under a non-decompression mode to receive an un-compressed data D2 from the camera interface 103, generate a processed multimedia data according to the un-compressed multimedia data D2, and send the processed multimedia data to one or multiple processors, such as a display processor 106_1, a video processor 106_2 and/or a graphic processor 106_3, for further processing. When the camera module 102 transmits the compressed data D1′ to the ISP 104, the ISP 104 is operated under a de-compression mode to receive a compressed multimedia data D2′ from the camera interface 103, generate a processed multimedia data according to a de-compressed multimedia data derived from de-compressing the compressed multimedia data D2′, and send the processed multimedia data to one or multiple processors, such as the display processor 106_1, the video processor 106_2 and/or the graphic processor 106_3, for further processing. If there is no error introduced during the data transmission, the un-compressed data D1 transmitted under the non-compression mode should be identical to the un-compressed data D2 received under the non-decompression mode, and the compressed data D1′ transmitted under the compression mode should be identical to the compressed data D2′ received under the de-compression mode.
As shown in FIG. 1, the ISP 104 includes an ISP controller 122, an input interface 124 and other circuitry 126. The other circuitry 126 may include circuit elements required for generating the processed multimedia data according to an internal transmission mode (e.g., an on-the-fly mode or an off-line mode). For example, the other circuitry 126 may have a de-compressor, a write direct memory access (DMA) controller, a read DMA controller, an image processor, multiplexers, etc. The de-compressor is used for performing data de-compression to obtain a de-compressed multimedia data. The write DMA controller is arranged for storing a multimedia data into an external image buffer of the ISP 105, where the multimedia data may be an un-compressed multimedia data, a compressed multimedia data or a de-compressed multimedia data. The read write DMA controller is arranged for reading a buffered multimedia data from the external image buffer of the ISP 104. The image processor is arranged to support several image processing operations, including resizing/scaling, rotation, quality enhancement, etc., where the processed multimedia data is generated from the image processor. The multiplexers control interconnections of the de-compressor, the image processor, the write DMA controller, and the read DMA controller. As the present invention focuses on the control of the input interface 124, further description of the other circuitry 126 is omitted here for brevity.
The input interface 124 includes an un-packing unit 127 and a plurality of switches (e.g., 128_1, 1 28_2, 128_3 and 128_4). It should be noted that the number of the switches included in the input interface 124 is equal to the number of data lines included in the camera interface 103. Hence, each of the switches 128_1-128_4 controls whether a data line is used for data reception. In this embodiment, the switches 128_1-128_4 are controlled by a plurality of enable signals EN0′-EN3′ generated from the ISP controller 122, respectively. By way of example, the enable signals EN0′-EN3′ may be transmitted via an I²C interface, and/or programmed in an e-fuse device or an EPPROM by the ISP controller 122. When an enable signal has a first logic value (e.g., ‘1’), a corresponding switch is enabled (i.e., switched on) to enable the data reception over the corresponding data line; and when the enable signal has a second logic value (e.g., ‘0’), the corresponding switch is disabled (i.e., switched off) to disable the data reception over the corresponding data line. It should be noted that, to make the data successfully transmitted from the camera module 102 and received by the ISP 104, switches located at different ends of one data line of the camera interface 103 should be both enabled. Further details of controlling the output interface 114 of the camera module 102 and the input interface 124 of the ISP 104 are described as below.
When the non-compression mode of the camera module 102 is enabled, the camera controller 112 controls the other circuitry 116 to generate the un-compressed multimedia data D1 to the output interface 114, and controls the output interface 114 to use all of the data lines DL0-DL3, each having a predetermined operating frequency, for data transmission. The predetermined operating frequency (i.e., a bit clock rate per line) may be set by using the following equations.
$\begin{matrix} Pixel Rate (Mhz) = \frac{Frame Resolution \times Frame Rate \times (1 + % of Blanking)}{10^{6}} & (1) \\ Bit Clock Rate per Line = \frac{Pixel Rate \times Bits per Pixel}{Number of Data Lines} & (2) \end{matrix}$
For example, considering a 20M (20-Mega pixels) sensor operating at 30 fps (frames per second) capture rate with 10% of blanking overhead, the pixel rate equals 660 MHz. When Bayer 10-bit color per pixel is transmitted over 4 data lines, the bit clock rate per line reaches 1.65 Gbps.
Besides, when the non-decompression mode of the ISP 104 is enabled, the ISP controller 122 controls the input interface 124 to use all of the data lines DL0-DL3, each having the predetermined operating frequency, for data reception, and controls the other circuitry 126 to generate the processed multimedia data based on the un-compressed multimedia data D2.
Regarding the output interface 114, the packing unit 117 is arranged for packing/packetizing the un-compressed multimedia data D1 based on the transmission protocol of the camera interface 103 and accordingly generating an output bitstream to the camera interface 103, wherein all of the switches 118_1-118_4 are enabled (i.e., switched on), and each data line is operated under the predetermined operating frequency (e.g., 1.65 Gbps) determined according to above equations. In other words, the output interface 114 has a transmission setting for packing/packetizing the un-compressed multimedia data D1 into an output bitstream and outputting the output bitstream via the camera interface 103.
Regarding the input interface 124, the un-packing unit 117 is arranged for un-packing/un-packetizing an input bitstream based on the transmission protocol of the camera interface 103 and accordingly generating the un-compressed multimedia data D2 to the other circuitry 126, wherein all of the switches 128_1-128_4 are enabled (i.e., switched on), and each data line is operated under the predetermined operating frequency (e.g., 1.65 Gbps) determined according to above equations. In other words, the input interface 124 has a reception setting for receiving an input bitstream via the camera interface 103 and un-packing/un-packetizing the input bitstream into the un-compressed multimedia data D1 that is transmitted over the camera interface 103.
In general, the aforementioned un-compressed data transmission is exploited and standardized by MIPI's CSI. However, it may not afford the high data rate, and may have potential problems in high power dissipation and low design yield. To alleviate the aforementioned problems, the other circuitry 116 in the camera module 102 is configured to have a compressor implemented therein, and the other circuitry 126 in the ISP 104 is configured to have a de-compressor implemented therein. Hence, the compressed data transmission over the camera interface 103 is realized through the compressor and the de-compressor.
When the compression mode of the camera module 102 is enabled, the camera controller 112 controls the other circuitry 116 to generate the compressed multimedia data D1′ to the output interface 114, wherein the compressor implemented in the other circuitry 116 may employ a lossy or lossless compression algorithm, depending upon actual design consideration/requirement. Regarding the output interface 114, the packing unit 117 packs/packetizes the compressed multimedia data D1′ based on the transmission protocol of the camera interface 103 and accordingly generates an output bitstream to the camera interface 103. In other words, when the camera module 102 is operated under the compression mode, the output interface 114 is arranged for packing/packetizing the compressed multimedia data D1′ into an output bitstream and outputting the output bitstream via the camera interface 103.
When the de-compression mode of the ISP 104 is enabled, the ISP controller 122 controls the other circuitry 126 to generate the processed multimedia data based on the compressed multimedia data D2′, wherein the de-compressor implemented in the other circuitry 126 may employ a lossy or lossless de-compression algorithm, depending upon actual design consideration/requirement. Regarding the input interface 124, the un-packing unit 127 un-packs/un-packetizes the input bitstream into the compressed multimedia data D2′ based on the transmission protocol of the camera interface 103. In other words, when the ISP 104 is operated under the de-compression mode, the input interface 124 is arranged for receiving an input bitstream via the camera interface 103, and un-packing/un-packetizing the input bitstream into the compressed multimedia data D2′ that is transmitted over the camera interface 103.
As the data size of the compressed multimedia data is smaller than that of the original un-compressed multimedia data, the output interface 114 is controlled by the camera controller 112 to have a different transmission setting, and the input interface 124 of the ISP 104 is controlled by the ISP controller 122 to have a different reception setting. In this embodiment of the present invention, the camera interface 103 provides a data line management layer and is scalable to the number of data lines according to the bandwidth and the compression requirement. The camera controller 112 is therefore arranged for referring to at least a compression characteristic of the compressed multimedia data D1′ to configure a transmission setting of the output interface 114 over the camera interface 103, and the ISP controller 122 is arranged for configuring a reception setting of the input interface 124 over the camera interface 103 in response to at least the compression characteristic of the compressed multimedia data D1′.
By way of example, but not limitation, the aforementioned compression characteristic may be a compression ratio M corresponding to the compressed multimedia data D1′, where M≦1.0. The compression ratio M may be expressed using the following equation.
$\begin{matrix} M = \frac{amount of compressed data}{amount of un - compressed data} & (3) \end{matrix}$
Take 32×1 Bayer raw 10-bit pixels as an example, the amount of un-compressed data reaches 320 bits (i.e., 32×1×10 bits). When the compression ratio M is equal 0.5, the compressed data would have 160 bits. As the data amount to be transmitted is reduced by data compression with the compression ratio M smaller than 1, the number of data lines, the operating frequency of each data line, and/or the behavior in the blanking period may be adjusted to improve the overall system performance.
In a first exemplary design, each of the camera controller 112 and the ISP controller 122 refers to the compression ratio M to configure the number of data lines enabled over the camera interface 103 for data transmission and reception when the camera module 102 is used to transmit the compressed multimedia data D1′ to the ISP 104, wherein an operating frequency of each data line (i.e., a bit clock rate of each data line) remains unchanged regardless of the configured number of data lines. As the data compression is performed on the camera module 102, the camera module 102 may inform the ISP 104 of the compression ratio M corresponding to the compressed multimedia data D1′. Suppose that all of the data lines DL0-DL3, each having the predetermined operating frequency, will be used for transmitting the un-compressed multimedia data D1. Thus, when the non-compression mode of the camera module 102 is enabled and the non-decompression mode of the ISP 104 is enabled, the camera controller 112 sets each of the enable signals EN0-EN3 by the first logic value (e.g., ‘1’) such that all of the switches 118_1-118_4 are enabled (i.e., switched on), and the ISP controller 122 sets each of the enable signals EN0′-EN3′ by the first logic value (e.g., ‘1’) such that all of the switches 128_1-128_4 are enabled (i.e., switched on).
However, when the compression mode of the camera module 102 is enabled and the de-compression mode of the ISP 104 is enabled, the number of data lines enabled over the camera interface 103 is set to a value equal to a product of the number of data lines DL0-DL3 (i.e., the number of data lines without compression) and the compression ratio M. For example, if M=0.5 and the number of data lines without compression is 4, the number of data lines with compression is equal to 2. As the number of data lines is reduced to 2, the camera controller 112 may set two of four enable signals (e.g., EN0 and EN1) by the first logic value (e.g., ‘1’) and set the remaining enable signals (e.g., EN2 and EN3) by the second logic value (e.g., ‘0’), and the ISP controller 122 may set two of four enable signals (e.g., EN0′ and EN1′) by the first logic value (e.g., ‘1’) and set the remaining enable signals (e.g., EN2′ and EN3′) by the second logic value (e.g., ‘0’). Thus, only a portion of the switches 118_1-118_4 are enabled (i.e., switched on) by the camera controller 112, and only a portion of the switches 128_1-128_4 are enabled (i.e., switched on) by the ISP controller 122. To put it simply, the bandwidth can be reduced by the interface compression, and the reduced number of data lines is related to the compression ratio M. Besides, due to the reduced number of data lines enabled over the camera interface 103, the transmission power dissipation and electromagnetic interference (EMI) can be alleviated.
Please refer to FIG. 2, which is a flowchart illustrating a first control and data flow of the data processing system 100 shown in FIG. 1. In this embodiment, the compression ratio M is referenced to configure the number of data lines enabled over the camera interface 103, and the operating frequency of each data line remains unchanged. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 2. The exemplary first control and data flow may be briefly summarized by following steps.
Step 200: Start.
Step 202: Check if a compression mode is enabled. If yes, go to step 210; otherwise, go to step 204.
Step 204: The other circuitry 116 generates the un-compressed multimedia data D1 according to an input multimedia data obtained by a camera sensor.
Step 206: The camera controller 112 controls the output interface 114 to switch on all of the switches 118_1-118_4. In this way, all of the data lines DL0-DL3 of the camera interface 103 would be used by the camera module 102 for data transmission.
Step 208: The packing unit 117 directly packs/packetizes the un-compressed multimedia data D1 into an output bitstream. Go to step 216.
Step 210: The other circuitry 116 generates the compressed multimedia data D1′ according to the input multimedia data obtained by the camera sensor.
Step 212: The camera controller 112 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to switch on a portion of the switches 118_1-118_4. In this way, only part of the data lines DL0-DL3 of the camera interface 103 would be used by the camera module 102 for data transmission.
Step 214: The packing unit 117 packs/packetizes the compressed multimedia data D1′ into an output bitstream.
Step 216: Transmit the output bitstream over the camera interface 103.
Step 218: Check if a de-compression mode is enabled. If yes, go to step 226; otherwise, go to step 220.
Step 220: The ISP controller 122 controls the input interface 124 to switch on all of the switches 128_1-128_4. In this way, all of the data lines DL0-DL3 of the camera interface 103 would be used by the ISP 104 for data reception.
Step 222: The un-packing unit 127 un-packs/un-packetizes an input bitstream into the un-compressed multimedia data D2.
Step 224: The other circuitry 126 generate a processed multimedia data according to the un-compressed multimedia data D2. Go to step 232.
Step 226: The ISP controller 122 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to switch on a portion of the switches 128_1-128_4. In this way, only part of the data lines DL0-DL3 of the camera interface 103 would be used by the ISP 104 for data reception.
Step 228: The un-packing unit 127 un-packs/un-packetizes the input bitstream into the compressed multimedia data D2′.
Step 229: The other circuitry 126 derives a de-compressed multimedia data from de-compressing the compressed multimedia data D2′.
Step 230: The other circuitry 126 generates the processed multimedia data according to the de-compressed multimedia data.
Step 232: End.
It should be noted that steps 202-216 are performed by the camera module 102, and steps 218-230 are performed by the ISP 104. As a person skilled in the art can readily understand details of each step shown in FIG. 2 after reading above paragraphs, further description is omitted here for brevity.
In a second exemplary design, each of the camera controller 112 and the ISP controller 122 refers to the compression ratio M to configure an operating frequency of each data line (i.e., a bit clock rate of each data line) when the camera module 102 is used to transmit the compressed multimedia data D1′ to the ISP 104, wherein the number of data lines enabled over the camera interface 103 remains unchanged regardless of the configured operating frequency. As the data compression is performed on the camera module 102, the camera module 102 may inform the ISP 104 of the compression ratio M corresponding to the compressed multimedia data D1′. Suppose that all of the data lines DL0-DL3, each having the predetermined operating frequency, will be used for transmitting the un-compressed multimedia data D1. Thus, when the non-compression mode of the camera module 102 is enabled and the non-decompression mode of the ISP 104 is enabled, each of the data lines DL0-DL3 is controlled to operate under the predetermined operating frequency, wherein all of the switches 118_1-118_4 are enabled (i.e., switched on) by the camera controller 112, and all of the switches 128_1-128_4 are enabled (i.e., switched on) by the ISP controller 122.
However, when the compression mode of the camera module 102 is enabled and the de-compression mode of the ISP 104 is enabled, the operating frequency of each data line is set by a value equal to a product of the predetermined operating frequency (i.e., a non-compression/non-decompression mode bit clock rate) and the compression ratio M. Thus, the compressed multimedia data D1′ is transmitted at a lower clock rate under the condition where the number of data lines enabled over the camera interface 103 remains unchanged. For example, if M=0.5, the compression/de-compression mode bit clock rate is half of the non-compression/non-decompression mode bit clock rate. In one exemplary implementation, the required clock frequency selection may be implemented using different clock generators which supply clocks with different clock rates. However, this is for illustrative purposes only, and is not meant to be a limitation of the present invention. To put it simply, the bandwidth can be reduced by the interface compression, and the reduced operating frequency is related to the compression ratio M. Besides, due to the reduced operating frequency of each data line enabled over the camera interface 103, the transmission power dissipation and EMI can be alleviated.
Please refer to FIG. 3, which is a flowchart illustrating a second control and data flow of the data processing system 100 shown in FIG. 1. In this embodiment, the compression ratio M is referenced to configure the operating frequency of each data line, and the number of data lines enabled over the camera interface 103 remains unchanged. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 3. The exemplary second control and data flow may be briefly summarized by following steps.
Step 300: Start.
Step 302: Check if a compression mode is enabled. If yes, go to step 310; otherwise, go to step 304.
Step 304: The other circuitry 116 generates the un-compressed multimedia data D1 according to an input multimedia data obtained by a camera sensor.
Step 306: The camera controller 112 controls the output interface 114 to set a predetermined operating frequency of data transmission on each data line.
Step 308: The packing unit 117 directly packs/packetizes the un-compressed multimedia data D1 into an output bitstream. Go to step 316.
Step 310: The other circuitry 116 generates the compressed multimedia data D1′ according to the input multimedia data obtained by the camera sensor.
Step 312: The camera controller 112 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to set a reduced operating frequency of data transmission on each data line.
Step 314: The packing unit 117 packs/packetizes the compressed multimedia data D1′ into an output bitstream.
Step 316: Transmit the output bitstream over the camera interface 103.
Step 318: Check if a de-compression mode is enabled. If yes, go to step 326; otherwise, go to step 320.
Step 320: The ISP controller 122 controls the input interface 124 to set a predetermined operating frequency of data reception on each data line.
Step 322: The un-packing unit 127 un-packs/un-packetizes an input bitstream into the un-compressed multimedia data D2.
Step 324: The other circuitry 126 generates a processed multimedia data according to the un-compressed multimedia data D2. Go to step 332.
Step 326: The ISP controller 122 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to set a reduced operating frequency for data reception on each data line.
Step 328: The un-packing unit 127 un-packs/un-packetizes the input bitstream into the compressed multimedia data D2′.
Step 329: The other circuitry 126 derives a de-compressed multimedia data from de-compressing the compressed multimedia data D2′.
Step 330: The other circuitry 126 generates the processed multimedia data according to the de-compressed multimedia data.
Step 332: End.
It should be noted that steps 302-316 are performed by the camera module 102, and steps 318-330 are performed by the ISP 104. As a person skilled in the art can readily understand details of each step shown in FIG. 3 after reading above paragraphs, further description is omitted here for brevity.
In a third exemplary design, when a compression mode of the camera module 102 is enabled and a de-compression mode of the ISP 104 is enabled, the camera controller 112 refers to the compression ratio M to configure a behavior of the output interface 114 during a blanking period between adjacent data transmissions, and the ISP controller 122 refers to the compression ratio M to configure a behavior of the input interface 124 during a blanking period between adjacent data transmissions, wherein the number of data lines enabled over the camera interface 103 and the operating frequency of each data line remain unchanged regardless of the configured behavior of the output interface 114 and the configured behavior of the input interface 124.
Please refer to FIG. 4, which is a diagram illustrating the change of the blanking period after the interface compression is applied. When no interface compression is applied (i.e., the un-compressed multimedia data D1 is transmitted over the camera interface 103 and the un-compressed multimedia data D2 is received from the camera interface 103), the banking period is generally shorter than the data transmission period. However, when interface compression is applied (i.e., the compressed multimedia data D1′ is transmitted over the camera interface 103 and the compressed multimedia data D2′ is received from the camera interface 103), the banking period is extended due to a shorter data transmission period. In other words, as the compression ratio M is smaller than 1 (e.g., M=0.5), the interface compression is capable of creating an extra blanking period. In one exemplary design, the extra blanking period may allow negotiation between the camera module 102 and the ISP 104 so as to alleviate electrostatic discharge (ESD), EMI and/or power consumption. For example, during the blanking period extended by the interface compression, the data transmission may be turned off and/or additional commands may be sent from the camera module 102 to the ISP 104. To put it simply, the bandwidth can be reduced by the interface compression, and the extended blanking period is related to the compression ratio M.
Please refer to FIG. 5, which is a flowchart illustrating a third control and data flow of the data processing system 100 shown in FIG. 1. In this embodiment, the compression ratio M is referenced to configure the behavior of the output interface 114 and the input interface 124 during the blanking period, and the number of data lines enabled over the camera interface 103 and the operating frequency of each data line remain unchanged. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 5. The exemplary third control and data flow may be briefly summarized by following steps.
Step 500: Start.
Step 502: Check if a compression mode is enabled. If yes, go to step 508; otherwise, go to step 504.
Step 504: The other circuitry 116 generates the un-compressed multimedia data D1 according to an input multimedia data obtained by a camera sensor.
Step 506: The packing unit 117 directly packs/packetizes the un-compressed multimedia data D1 into an output bitstream. Go to step 514.
Step 508: The other circuitry 116 generates the compressed multimedia data D1′ according to the input multimedia data obtained by the camera sensor.
Step 510: The camera controller 112 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to configure the behavior of the output interface 114 during the blanking period between adjacent data transmissions.
Step 512: The packing unit 117 packs/packetizes the compressed multimedia data D1′ into an output bitstream.
Step 514: Transmit the output bitstream over the camera interface 103.
Step 516: Check if a de-compression mode is enabled. If yes, go to step 522; otherwise, go to step 518.
Step 518: The un-packing unit 127 un-packs/un-packetizes an input bitstream into the un-compressed multimedia data D2.
Step 520: The other circuitry 126 generates a processed multimedia data according to the un-compressed multimedia data D2. Go to step 528.
Step 522: The ISP controller 122 refers to the compression ratio M corresponding to the compressed multimedia data D1′ to configure the behavior of the input interface 214 during the blanking period between adjacent data transmissions.
Step 524: The un-packing unit 127 un-packs/un-packetizes the input bitstream into the compressed multimedia data D2′.
Step 525: The other circuitry 126 derives a de-compressed multimedia data from de-compressing the compressed multimedia data D2′.
Step 526: The other circuitry 126 generates the processed multimedia data according to the de-compressed multimedia data.
Step 528: End.
It should be noted that steps 502-514 are performed by the camera module 102, and steps 516-526 are performed by the ISP 104. As a person skilled in the art can readily understand details of each step shown in FIG. 5 after reading above paragraphs, further description is omitted here for brevity.
The camera interface 103 may have at least one data line which is a bidirectional line. For example, the data line DL0 as shown in FIG. 1 is bidirectional. Hence, information transaction between the camera module 102 and the ISP 104 can be realized by using the bidirectional data line DL0. In this embodiment of the present invention, the camera controller 112 may further check a de-compression capability of the ISP 104 by sending a request to the ISP 104, and the ISP controller 122 may further inform the camera module 102 of the de-compression capability of the ISP 104 by sending a response to the camera module 102. In this way, the camera module 102 can detect whether the ISP 104 has the ability of performing data de-compression, and further detect what kinds of de-compression algorithms the ISP 104 supports if the ISP 104 is equipped with de-compression capability. In a case where the ISP 104 supports the interface compression, the ISP 104 may enable the de-compression mode after transmitting the de-compression capability information in response to the request issued by the camera module 102. Next, the camera module 102 enables the compression mode according to a checking result of the de-compression capability of the ISP 104, and configures the compressor in the other circuitry 116 to employ one of compression algorithms corresponding to de-compression algorithms supported by the ISP 104. However, when the ISP 104 does not support the interface compression, the ISP 104 simply operates under the non-decompression mode, and the camera module 102 would enable the non-compression mode according to the checking result of the de-compression capability of the ISP 104.
Please refer to FIG. 6, which is a flowchart illustrating a method of setting the camera module 102 by referring to a result of checking a de-compression capability of the ISP 104 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 6. The exemplary method may be briefly summarized by following steps.
Step 600: Start.
Step 602: Check a de-compression capability of the ISP 104 to see if the ISP 104 supports the proposed interface compression, and accordingly obtain a checking result.
Step 604: Refer to the checking result to determine if the de-compression mode in the ISP 104 is set? If yes, go to step 606; otherwise, go to step 608.
Step 606: Enable the compression mode in the camera module 102. Go to step 610.
Step 608: Enable the non-compression mode in the camera module 102.
Step 610: End.
As a person skilled in the art can readily understand details of each step shown in FIG. 6 after reading above paragraphs, further description is omitted here for brevity.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.

Claims

What is claimed is:

1. A data processing apparatus comprising:

an output interface, arranged for packing a compressed multimedia data into an output bitstream and outputting the output bitstream via a camera interface; and

a camera controller, arranged for referring to at least a compression characteristic of the compressed multimedia data to configure a transmission setting of the output interface over the camera interface.

2. The data processing apparatus of claim 1, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

3. The data processing apparatus of claim 1, wherein the compression characteristic is a compression ratio corresponding to the compressed multimedia data.

4. The data processing apparatus of claim 1, wherein the transmission setting of the output interface comprises a number of data lines enabled over the camera interface.

5. The data processing apparatus of claim 4, wherein an operating frequency of each data line remains unchanged regardless of the configured number of data lines.

6. The data processing apparatus of claim 1, wherein the transmission setting of the output interface comprises an operating frequency of each data line.

7. The data processing apparatus of claim 6, wherein a number of data lines enabled over the camera interface remains unchanged regardless of the configured operating frequency.

8. The data processing apparatus of claim 1, wherein the transmission setting of the output interface comprises a behavior of the output interface during a blanking period between adjacent data transmissions.

9. The data processing apparatus of claim 8, wherein a number of data lines enabled over the camera interface and an operating frequency of each data line remain unchanged regardless of the configured behavior of the output interface during the blanking period.

10. The data processing apparatus of claim 1, wherein the data processing apparatus is coupled to another data processing apparatus via the camera interface; and the camera controller is further arranged for checking a de-compression capability of the another data processing apparatus, where the data processing apparatus selectively enables a compression mode according to a checking result.

11. A data processing apparatus comprising:

an input interface, arranged for receiving an input bitstream via a camera interface, and un-packing the input bitstream into a compressed multimedia data that is transmitted over the camera interface; and

an image signal processor (ISP) controller, arranged for configuring a reception setting of the input interface over the camera interface in response to at least a compression characteristic of the compressed multimedia data.

12. The data processing apparatus of claim 11, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

13. The data processing apparatus of claim 11, wherein the compression characteristic is a compression ratio corresponding to the compressed multimedia data.

14. The data processing apparatus of claim 11, wherein the reception setting of the input interface comprises a number of data lines enabled over the camera interface.

15. The data processing apparatus of claim 14, wherein an operating frequency of each data line remains unchanged regardless of the configured number of data lines.

16. The data processing apparatus of claim 11, wherein the reception setting of the input interface comprises an operating frequency of each data line.

17. The data processing apparatus of claim 16, wherein a number of data lines enabled over the camera interface remains unchanged regardless of the configured operating frequency.

18. The data processing apparatus of claim 11, wherein the reception setting of the input interface comprises a behavior of the input interface during a blanking period between adjacent data transmissions.

19. The data processing apparatus of claim 18, wherein a number of data lines enabled over the camera interface and an operating frequency of each data line remain unchanged regardless of the configured behavior of the input interface during the blanking period.

20. The data processing apparatus of claim 11, wherein the data processing apparatus is coupled to another data processing apparatus via the camera interface, and the ISP controller is further arranged for informing the another data processing apparatus of a de-compression capability of the data processing apparatus.

21. A data processing method comprising:

referring to at least a compression characteristic of a compressed multimedia data to configure a transmission setting of an output interface over the camera interface; and

utilizing an output interface for packing the compressed multimedia data into an output bitstream and outputting the output bitstream via the camera interface.

22. The data processing method of claim 21, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

23. The data processing method of claim 21, wherein the compression characteristic is a compression ratio corresponding to the compressed multimedia data.

24. The data processing method of claim 21, wherein the transmission setting of the output interface comprises a number of data lines enabled over the camera interface.

25. The data processing method of claim 21, wherein the transmission setting of the output interface comprises an operating frequency of each data line.

26. The data processing method of claim 21, wherein the transmission setting of the output interface comprises a behavior of the output interface during a blanking period between adjacent data transmissions.

27. The data processing method of claim 21, wherein the output bitstream is transmitted to a data processing apparatus via the camera interface; and the data processing method further comprises:

checking a de-compression capability of the data processing apparatus, and selectively enabling a compression mode according to a checking result.

28. A data processing method comprising:

configuring a reception setting of an input interface over a camera interface in response to at least a compression characteristic of a compressed multimedia data; and

utilizing an input interface for receiving an input bitstream via the camera interface, and un-packing the input bitstream into the compressed multimedia data that is transmitted over the camera interface.

29. The data processing method of claim 28, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

30. The data processing method of claim 28, wherein the compression characteristic is a compression ratio corresponding to the compressed multimedia data.

31. The data processing method of claim 28, wherein the reception setting of the input interface comprises a number of data lines enabled over the camera interface.

32. The data processing method of claim 28, wherein the reception setting of the input interface comprises an operating frequency of each data line.

33. The data processing method of claim 28, wherein the reception setting of the input interface comprises a behavior of the input interface during a blanking period between adjacent data transmissions.

34. The data processing method of claim 28, wherein the input bitstream is received from a data processing apparatus via the camera interface; and the data processing method further comprises:

informing the data processing apparatus of a de-compression capability.