CN109472078B - 3D image sensor defect detection and repair method based on 2X 2 pixel subarray - Google Patents

Abstract

The invention discloses a 3D image sensor defect detection and repair method based on a 2X 2 pixel sub-array, which adopts a 4X 4 operation window, wherein 4 pixels at the center in the window are pixels to be detected, 12 pixels at the periphery are peripheral pixels for auxiliary repair, and the positions of each pixel in the window are numbered; the window shifts according to a pixel unit each time in a sequence from left to right and from top to bottom, and detection and repair operations are repeatedly performed; in each window operation, the peripheral pixels are utilized to carry out overall state judgment and restoration on 4 pixels to be detected. The algorithm can realize the detection and repair of the whole 2 x 2 defective pixel block by utilizing the 4*4 operation window, effectively avoids the situation of using defects to correct defects in the traditional correction mode, and enhances the repair effect of the 2 x 2 defective block.

Description

3D image sensor defect detection and repair method based on 2X 2 pixel subarray

Technical Field

The invention relates to the field of analog integrated circuit design, a 3D stacked fault-tolerant structure image sensor and an image correction method, in particular to a low-cost, efficient and robust defective pixel detection and repair method.

Background

The 3D stacked fault tolerant architecture image sensor uses micro bump and through silicon via (Through Silicon Vias, TSV) technology to achieve vertical interconnection of the chip layers. Typically the pixel array is segmented in the form of 2 x 2 (or other size) sub-arrays of pixels, each sub-array of pixels being connected by micro-bumps and signal lines to a different analog-to-digital converter ADC at the lower level, each ADC being in turn connected to an image processing module (Image Signal Processor, ISP) via a respective TSV via. If the signal line, the micro bump, the ADC or the TSV on the signal transmission path fails, the missing pixel of the output image presents 2 multiplied by 2 pixel block distribution. The conventional defect correction method aims at the defect of one pixel, and in this case, the defect is repaired by the defect, so that the correction effect is not ideal. Therefore, it is important to develop a defect detection and repair method capable of effectively correcting a 2×2 defective block.

Disclosure of Invention

The invention provides a defect detection and repair method of a 3D image sensor based on a 2X 2 pixel sub-array, which can effectively detect defective pixels caused by faults such as micro bumps and TSVs on a connecting path in an image processing module, and perform pixel repair operation by using adjacent pixels, so that an output image with higher quality can be obtained in a targeted manner under the condition of the fault existence in a low-cost manner.

In order to achieve the purpose of the invention, the invention provides a defect detection and repair method of a 3D image sensor based on a 2X 2 pixel sub-array, the method adopts a 4X 4 operation window, wherein 4 pixels at the center in the window are pixels to be detected, 12 pixels at the periphery are peripheral pixels for auxiliary repair, and the position of each pixel in the window is numbered; the window shifts according to a pixel unit each time in a sequence from left to right and from top to bottom, and detection and repair operations are repeatedly performed; in each window operation, the peripheral pixels are utilized to carry out overall state judgment and restoration on 4 pixels to be detected.

Compared with the prior art, the method has the beneficial effects that the method can realize the detection and repair of the whole 2 x 2 defective pixel block by utilizing the operation window of 4*4, effectively avoid the defect correction by using the defect in the traditional correction mode, and enhance the repair effect of the 2 x 2 defective block. When the 3D stacking fault-tolerant structure image sensor has faults and loses corresponding pixel information, the algorithm can still transmit the image with qualified quality, and the cost of maintaining the sensor is reduced.

Drawings

FIG. 1 is a schematic diagram of a 4×4 operation window of a defect detection repair method based on a 2×2 pixel sub-array according to an embodiment of the present application;

FIG. 2 is a flow chart of hot spot pixel detection in an embodiment of the present application;

FIG. 3 is a flow chart of hot spot pixel repair according to an embodiment of the present application;

FIG. 4 illustrates a filtering matrix for each direction in an embodiment of the present application;

in fig. 4, (1) a transversal filter Fh; (2) a longitudinal filter Fv; (3) 45 ° filter F45; (4) 135 ° filter F135.

Detailed Description

The invention is described in further detail below with reference to the drawings and the specific examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

It should be noted that the terms "coupled" and "connected," as used herein, include both directly coupled to and connected to another element or the like.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the singular is "a," an, "and/or" the "include" when used in this specification is taken to mean that there are features, steps, operations, components or modules, assemblies, and/or combinations thereof.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be capable of being practiced otherwise than as specifically illustrated and described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Spatially relative terms, such as "above … …," "above … …," "upper surface at … …," "above," and the like, may be used herein for ease of description to describe one component or module or feature's spatial location relative to another component or module or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations in use or operation in addition to the orientation of the components or modules depicted in the figures. For example, if a component or module in the figures is turned over, elements or modules described as "above" or "over" other components or modules or configurations would then be oriented "below" or "beneath" the other components or modules or configurations. Thus, the exemplary term "above … …" may include both orientations of "above … …" and "below … …". The component or module may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.

In a 2×2 pixel unit sub-array, in a 3D fault tolerant readout architecture in which adjacent sub-arrays are connected to different ADCs (for example, three consecutive pixel sub-arrays are respectively connected to three rows of ADCs, and every second pixel sub-array is connected to the same ADC), if any part of the signal transmission path fails, the signal of the pixel array is lost, which may result in serious quality impairment of the readout image. The yield of such image sensors has not yet reached expectations due to the limitations of the TSV technology level, and 2 x 2 defective pixel arrays are easily generated on the readout image. The invention provides a defect detection and repair algorithm based on a 2 multiplied by 2 pixel sub-array, which can effectively detect defective pixels caused by faults such as micro bumps, TSVs and the like on a connecting path in an image processing module, and utilizes adjacent pixels to carry out pixel repair operation, so that an output image with higher quality can be obtained in a targeted manner under the condition of the existence of the faults in a low-cost manner.

The invention provides a defect detection and repair method of a 3D image sensor based on a 2X 2 pixel sub-array, which adopts a 4X 4 operation window, as shown in figure 1, wherein the central 4 pixels (black) are pixels to be detected, the peripheral 12 pixels (white) are peripheral pixels for auxiliary repair, and the positions of each pixel in the window are numbered. The window is shifted one pixel unit at a time in a left to right, top to bottom order, and the detection and repair operations are repeated. In each window operation, the peripheral pixels are utilized to carry out overall state judgment and repair on 4 pixels to be detected, so that effective correction on the 2×2 defect block can be realized in a targeted manner.

Defective pixels may be classified into hot spot pixels and cold spot pixels. The hot spot pixels are brighter than the average level of the image and the cold spot pixels are darker than the average level of the image. The detection flow for the example of hot spot pixels is shown in fig. 2. And judging the pixel to be detected, wherein if all four pixels meet the following conditions, the pixel to be detected in the operation window is considered as a defect pixel caused by the fault.

1. In a 4 x 4 operating window, the detected pixels are significantly different from the peripheral pixels. The concrete steps are as follows:

hot spot pixels:

I(i,j)＞(1+M ₁ )*I _avg (i,j)(1)

cold spot pixel:

I(i,j)＜(1-M ₁ )*I _avg (i,j)(2)

2. in the window of 4*4, the local luminance difference for the hot spot pixel is much greater than the smallest local luminance difference in the window, and the local luminance difference for the cold spot pixel is correspondingly much less than the largest local luminance difference in the window. The concrete steps are as follows:

hot spot pixels:

cold spot pixel:

wherein I (I, j) represents the pixel value of the input original image I at the position (I, j), I ranging from [1,2, …, H]J is in the range of [1,2, …, W]Wherein H and W represent the height and width of the image I, respectively, I (I, j) ranges from [0.0,255.0 ]]。I _avg Represents the end-cut average (with the maxima and minima removed) of the peripheral pixels. M is M ₁ Is a parameter in the algorithm to represent the intensity of the test, which ranges from (0.0, 1.0). d, d _lb (i, j) is the local luminance difference at (i, j), defined by d _lb (i,j)=I(i,j)-I _avg (i, j) calculated. M is M ₂ Is a parameter for controlling the false detection rate, M ₂ In the range of [1.0, U]U represents an upper limit.

The repair flow for the example of a hot spot pixel is shown in fig. 3. Firstly, the response of the defect to the transverse, longitudinal, 45 DEG and 135 DEG filters is calculated to find the characteristic direction at the pixel position, namely, the correlation calculation is carried out on the pixel value in the operation window and each filter matrix, as shown in (5), rh, rv, R45 and R135 respectively represent the filter response of the pixel I (I, j) and the corresponding direction filter. The filter matrix is shown in fig. 4.

The direction of the most value in the calculated result is the characteristic direction of the pixel to be detected in the operation window (the hot pixel is the maximum value and the cold pixel is the minimum value). And if the calculated result has a unique maximum value, performing directional repair, and if the calculated result has a plurality of maximum values, performing non-directional repair. The specific calculation is as follows:

horizontal direction:

vertical direction:

45 ° direction:

135 ° direction:

if non-directional repair is performed, the pixel estimation value I of the defective pixel _ND (I, j) is the second largest or the second smallest value of the pixels at the periphery of the operation window, respectively using I _2max And I _2min The representation is as follows:

hot spot pixels:

cold spot pixel:

the algorithm can realize the detection and repair of the whole 2 x 2 defective pixel block by utilizing the 4*4 operation window, effectively avoids the situation of using defects to correct defects in the traditional correction mode, and enhances the repair effect of the 2 x 2 defective block. When the 3D stacking fault-tolerant structure image sensor has faults and loses corresponding pixel information, the algorithm can still transmit the image with qualified quality, and the cost of maintaining the sensor is reduced.

The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims (1)

1. The 3D image sensor defect detection and repair method based on the 2X 2 pixel sub-array is characterized in that the method adopts a 4X 4 operation window, wherein 4 pixels at the center in the window are pixels to be detected, 12 pixels at the periphery are peripheral pixels for auxiliary repair, and each pixel position in the window is numbered; the window shifts according to a pixel unit each time in a sequence from left to right and from top to bottom, and detection and repair operations are repeatedly performed; in each window operation, carrying out overall state judgment and restoration on 4 pixels to be detected by utilizing peripheral pixels;

judging the pixel to be detected, wherein the pixel to be detected in the operation window is considered as a defect pixel caused by a fault if all four pixels meet the following conditions:

(1) In the 4×4 operation window, the detected pixels are different from the peripheral pixels, which is expressed as follows:

hot spot pixels:

I(i,j)＞(1+M ₁ )*I _avg (i,j)(1)

cold spot pixel:

I(i,j)＜(1-M ₁ )*I _avg (i,j)(2)

(2) In the window of 4*4, the local luminance difference of the hot spot pixel is larger than the smallest local luminance difference in the window, and the local luminance difference of the cold spot pixel is correspondingly smaller than the largest local luminance difference in the window, which is specifically expressed as follows:

hot spot pixels:

cold spot pixel:

wherein I (I, j) represents the pixel value of the input original image I at the position (I, j), I ranging from [1,2, …, H]J is in the range of [1,2, …, W]Wherein H and W represent the height and width of the image I, respectively, I (I, j) ranges from [0.0,255.0 ]]，I _avg Represents the end-cut average value of peripheral pixels, M ₁ Is a parameter for representing the detection intensity, and is in the range of (0.0, 1.0), d _lb (i, j) is the local luminance difference at (i, j), defined by d _lb (i,j)＝I(i,j)-I _avg (i, j) calculated, M ₂ Is a parameter for controlling the false detection rate, M ₂ In the range of [1.0, U]U represents an upper limit;

the repair procedure for the example of hot spot pixels is as follows:

firstly, calculating the response of defects to transverse, longitudinal, 45 DEG and 135 DEG filters to find the characteristic direction at the pixel position, namely, respectively carrying out correlation calculation on pixel values in an operation window and each filter matrix, wherein Rh, rv, R45 and R135 respectively represent the filter response of a pixel I (I, j) and a corresponding direction filter, fh is a transverse filter, fv is a longitudinal filter, F45 is a 45 DEG filter and F135 is a 135 DEG filter;

the direction of the most value in the calculated result is the characteristic direction of the pixel to be detected in the operation window, the hot spot pixel is the maximum value, and the cold spot pixel is the minimum value; if the calculation result has a unique maximum value, directional restoration is performed, and if the calculation result has a plurality of maximum values, non-directional restoration is performed, wherein the specific calculation is as follows:

horizontal direction:

vertical direction:

45 ° direction:

135 ° direction:

if non-directional repair is performed, the pixel estimation value I of the defective pixel _ND (I, j) is the second largest or the second smallest value of the pixels at the periphery of the operation window, respectively using I _2max And I _2min The representation is as follows:

hot spot pixels:

cold spot pixel:

/>

Images