CN103150717B - The detection method of image highlight area, content detection algorithm and content detection device - Google Patents
The detection method of image highlight area, content detection algorithm and content detection device Download PDFInfo
- Publication number
- CN103150717B CN103150717B CN201110401299.8A CN201110401299A CN103150717B CN 103150717 B CN103150717 B CN 103150717B CN 201110401299 A CN201110401299 A CN 201110401299A CN 103150717 B CN103150717 B CN 103150717B
- Authority
- CN
- China
- Prior art keywords
- pixel
- intensity difference
- highlight regions
- threshold value
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Disclose a kind of content detection algorithm and device of image highlight area, and a kind of detection method of image highlight area and device.This content detection algorithm comprises: the highlighted factor of influence determining each pixel in highlight regions, and this highlighted factor of influence represents the intensity that highlight regions affects this pixel content; Highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions.The detection method of this image highlight area comprises: obtain diffuse reflection image from original image; Obtain the intensity difference of each pixel between original image and diffuse reflection image; Determine intensity difference threshold value adaptively; And according to the intensity difference of intensity difference threshold value and each pixel described, detect described highlight regions.
Description
Technical field
Relate generally to image procossing of the present invention, more specifically, relates to the detection method of image highlight area, content detection algorithm and content detection device.
Background technology
Often there is highlighted or retroreflective regions in the image taken in certain scenarios, as image captured in camera or projector-camera system.
Such as in speech or meeting, general use projector-camera system, wherein projector or large-screen display equipment are usually used for showing relevant material content, when such as teleconference, video camera can be used to take view field and captured image to be passed to the participant of teleconference, at this moment the surface in view field often occurs highlighted, cause the content in captured image highlight area invisible or cannot see clearly, therefore the participant of teleconference recognizes that the content of highlight regions has difficulties, and causes communication difficult.
It is generally acknowledged, imaging surface has two kinds of reflecting components, mirror-reflection and diffuse reflection component.Diffuse reflection (diffuse) refers to the light from a direction, through diffuse reflection, light is evenly propagated to all directions.Diffuse reflection is caused by the coarse injustice on surface, and have nothing to do with viewpoint, the space distribution diffused is uniform.Mirror-reflection (Specular) refers to for ideal mirror, and reflected light concentrates on a direction, and observes reflection law.To general smooth surface, reflected light concentrates within the scope of one, and the reflection direction light determined by reflection law is maximum.Therefore, for same point, be different from diverse location viewed mirror-reflection light intensity.Near reflection direction, form very bright hot spot, be called high optical phenomenon, the region that there is high optical phenomenon is called highlight regions.It is generally acknowledged that highlight regions causes due to specular components.Usually the approach of two kinds of highlighted impacts of removal of images is there is in prior art.Wherein one is, utilizes the texture information of reflecting surface to build model, thus estimates and strengthen the picture material of mirror-reflection highlight regions.Such as, but in the unconspicuous situation of imaging surface textural characteristics, in projector-camera system, the surface of view field is smooth, does not have texture information, and the method is difficult to be suitable for.
The another kind of common method that in prior art, removal of images is highlighted is from the original image that there is the figure image subtraction one width mirror-reflection of flash of light.Such as in US Patent No. 7027662, the method does not have image and the flashlight images of flash of light with regard to same object shooting, error image is obtained by deducting flashless image in flashlight images, using degree threshold value processes the image that error image obtains artificial treatment, and the image then deducting this artificial treatment from flashlight images is highlighted to eliminate.Shooting does not have the image glistened to limit the application of the method.
In addition, in the patent CN101146233 being entitled as " a kind of light source colour calculates and method for correcting image ", the colourity of pixel each in coloured image is normalized, utilizes the method for ballot to detect highlight regions fast according to the similarity of pixel; In addition, rgb color is projected in inverse intensity and chrominance space by the method, then by inverse intensity coordinate system by all Pixel fits on straight line, calculate the colourity of light source, based on the colouring information on the Color correction diffuse reflection image of light source.
In addition, being entitled as in the US Patent No. 7555159 of " ImagehighlightcorrectionusingilluminationspecificHSVcolo rcoordinate " at PishvaDavar etc., following method is proposed: on the HSV color space image projection under rgb color space becomed privileged to a kind of illumination; For each pixel prediction is without highlighted HSV coordinate system; In the reduction of RGB color space without highlighted image.The method needed intensity and the color of knowing light source before building new color coordinates system.
Summary of the invention
The present invention is desirable to provide a kind of method and apparatus detecting highlight regions.
The present invention it would also be desirable to provide a kind of method and apparatus of content of the detection highlight regions without using image texture information.
The present invention it would also be desirable to provide a kind of method and apparatus strengthening the testing result of highlight regions content.
The present invention it would also be desirable to provide a kind of method and apparatus of color information of reducing highlight regions.
For this reason, according to an aspect of the present invention, provide a kind of content detection algorithm of image highlight area, it can comprise: the highlighted factor of influence determining each pixel in highlight regions, and this highlighted factor of influence represents the intensity that highlight regions affects this pixel content; Highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions.
By considering the impact of highlight regions on each pixel in highlight regions content detection, various lighting condition can be adapted to, reducing the impact of mirror-reflection, the content of highlight regions can be detected more accurately.
In addition, the content come in detected image highlight regions based on the highlighted factor of influence of each pixel in highlight regions can comprise: determine first threshold according to the picture material outside highlight regions; For each pixel in highlight regions, based on the highlighted factor of influence of this each pixel, adjust this first threshold, to determine the threshold value for this each pixel; According to the threshold value for this each pixel, determine that this each pixel belongs to content or background, thus obtain the first testing result.
In addition, this content detection algorithm can also comprise: the information based on the regional area of multiple different scale carrys out Detection of content, thus obtains the second testing result; At least obtain final detection result based on the first testing result and the second testing result.The highlight regions content detection algorithm of this combination can obtain higher Detection accuracy.
In addition, the size of the yardstick of at least one regional area in the regional area of the plurality of different scale can be determined according to the highlighted factor of influence of each pixel.
In addition, this content detection algorithm can also comprise: be the pixel of content carrys out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result; The nd content pixel in highlight regions is predicted based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.Strengthen process by this, highlighted impact can be eliminated further, improve sharpness and the observability of highlight regions content.
In addition, this content detection algorithm can also comprise: use original image and diffuse reflection image to carry out cluster to the pixel color outside highlight regions respectively; Respectively the pixel color in the highlight regions of original image and diffuse reflection image is classified according to respective Color-based clustering result; And according to the result of color classification, the pixel color in highlight regions is adjusted.By the color rendition process in this highlight regions, sharpness and the observability of highlight regions content can be strengthened, eliminate highlighted impact.
According to a further aspect in the invention, provide a kind of detection method of image highlight area, comprising: obtain diffuse reflection image from original image; Obtain the intensity difference of each pixel between original image and diffuse reflection image; Determine intensity difference threshold value adaptively; And according to the intensity difference of intensity difference threshold value and each pixel described, detect described highlight regions.
In accordance with a further aspect of the present invention, provide a kind of content detection device of image highlight area, comprise: highlighted factor of influence calculating unit, it determines the highlighted factor of influence of each pixel in highlight regions, and this highlighted factor of influence represents the intensity that highlight regions affects this pixel content; And content detection parts, its highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions.
In accordance with a further aspect of the present invention, provide a kind of method strengthening the testing result of highlight regions content, the method can comprise: be the pixel of content carrys out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result; The nd content pixel in highlight regions is predicted based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.
In accordance with a further aspect of the present invention, provide a kind of device strengthening the testing result of highlight regions content, this device can comprise: COLOR COMPOSITION THROUGH DISTRIBUTION models fitting parts, for being the pixel of content to carry out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result; Highlight regions content pixel prediction unit, for predicting the nd content pixel in highlight regions based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.
In accordance with a further aspect of the present invention, provide a kind of method of color information of reducing highlight regions, the method can comprise: use original image and corresponding diffuse reflection image to carry out cluster to the pixel color outside highlight regions respectively; Respectively the pixel color in the highlight regions of original image and diffuse reflection image is classified according to respective Color-based clustering result; And according to the result of color classification, the pixel color in highlight regions is adjusted.
In accordance with a further aspect of the present invention, provide a kind of device of color information of reducing highlight regions, this device can comprise: cluster parts, carries out cluster for using original image and corresponding diffuse reflection image respectively to the pixel color outside highlight regions; Classification element, for classifying to the pixel color in the highlight regions of original image and diffuse reflection image respectively according to respective Color-based clustering result; And highlight regions color adjustment component, for the result according to color classification, the pixel color in highlight regions is adjusted.
Accompanying drawing explanation
Fig. 1 is the schematic diagram can applying projector-camera system of the present invention according to an embodiment of the invention;
Fig. 2 is the overall flow figure of the content detection algorithm of image highlight area according to an embodiment of the invention;
Fig. 3 shows the process flow diagram of highlight regions detection method according to an embodiment of the invention;
Fig. 4 is the schematic diagram that highlight regions testing process is shown;
Fig. 5 is the process flow diagram of self-adaptation intensity difference Threshold according to an embodiment of the invention;
Fig. 6 is the schematic diagram of a sample calculation of the highlight regions factor of influence that each pixel in highlight regions is described;
Fig. 7 illustrates according to an embodiment of the invention based on the process flow diagram of the method for highlighted factor of influence detected image content;
Fig. 8 is the process flow diagram of the multiple dimensioned regional area content detection algorithm that highlight regions is shown;
Fig. 9 shows the comparison schematic diagram of stroke testing result in highlight regions that various yardstick local region information analyzes.
Figure 10 shows the process flow diagram of the stroke testing result enhancing disposal route of the stroke testing result based on highlight regions being treated to example according to one embodiment of the invention with stroke;
Figure 11 shows the process flow diagram of the stroke color reduction treatment of the stroke testing result based on highlight regions being treated to example according to one embodiment of the invention with stroke;
Figure 12 shows the schematic diagram of a color rendition processing procedure and effect example;
Figure 13 shows the overall flow figure of highlighted according to an embodiment of the invention image procossing;
Figure 14 shows the schematic block diagram of the content detection device of image highlight area according to an embodiment of the invention;
Figure 15 shows the block diagram of highlighted according to an embodiment of the invention image processing apparatus 1500;
Figure 16 shows the block diagram of the device 1600 of the testing result strengthening highlight regions content according to an embodiment of the invention; And
Figure 17 shows and reduces the block diagram of device 1700 of color information of highlight regions according to an embodiment of the invention.
Embodiment
In order to make those skilled in the art understand the present invention better, below in conjunction with the drawings and specific embodiments, the present invention is described in further detail.
For the ease of understanding, Fig. 1 gives the schematic diagram can applying projector-camera system of the present invention according to an embodiment of the invention.To take hereinafter application scenarios as projector-camera system, the picture material that detect is strokes of characters be that example is described.But it is emphasized that above-mentioned situation is only example, may occur in fact any image that the situation of highlight area all can apply the present invention, and other picture materials such as face, scenery etc. all can apply the present invention.
Before stating in detail, in order to contribute to understanding the present invention better, generally set forth thought of the present invention.A thought of the present invention is, the pixel content generation adverse effect in highlight regions is recognized in the existence of highlight regions for people.Therefore, it is desirable to, in the process of Detection of content such as stroke, namely to consider this adverse effect, the threshold factor detected or area size etc. are adjusted adaptively, thus detects the stroke in highlight regions more accurately.Further, wish the undetected stroke estimated in highlight regions, and wish to be recovered for because of highlighted and unsharp color information.
Fig. 2 is the overall flow figure of the detection method 200 of content according to an embodiment of the invention in image highlight area.
As shown in Figure 2, in step S210, determine the highlighted factor of influence of each pixel in highlight regions.About highlight regions, its highlight regions detection method that hereinafter with reference Fig. 3 can be utilized to describe obtains, or is manually specified by user, and existing any highlight regions detection method also can be utilized to obtain.Highlighted factor of influence for a pixel represents the intensity that highlight regions affects this pixel content.Specifically, intuitively, a pixel, its surrounding pixel belongs to the more of highlight regions, then this pixel is larger by highlighted effect, and also namely more affect the observability of this pixel, therefore its highlight regions factor of influence is larger.Hereafter, a kind of example calculation method of highlight regions factor of influence is described with reference to Fig. 4.
In step S220, the highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions.Content in the highlight regions of acquisition like this can for process further, or can transmit out and user interactions.Illustrate based on highlighted factor of influence content detection threshold value determine hereinafter with reference to Fig. 7 and and then content detection, and with reference to figure 8, the multiple dimensioned regional area content detection based on the highlighted factor (optional) is described.
Fig. 3 shows the process flow diagram of highlight regions detection method 300 according to an embodiment of the invention.
In step S310, obtain diffuse reflection image from original image.For this reason, any method that can obtain diffuse reflection image from original image can be adopted.Such as, at " SeparatingReflectionComponentsBasedonChromaticityandNois eAnalysis ", RobbyT.Tan, etc.IEEETransactionsonPatternAnalysisandMachineIntellige nce, Vol26, describe a lot of methods being separated specular components from single image in No.10, October2004, and propose the method being separated diffuse reflection component and specular components based on colourity with noise analysis.Consider for efficiency, we have employed at document " Simpleandefficientmethodforspecularityremovalinanimage " H.L.ShenandQ.Y.Cai, AppliedOptics, 48 (14), 2711-2719, the method introduced in 2009 generates the image without specular components.The method by deducting the minimum rgb value of this pixel on each pixel, then the offset of a pixel dependent form is set so as to make amended without the image of specular components and the colourity of diffuse reflection image more close.But, the method is only example, and the method introduced in above-mentioned document and any method that can obtain diffuse reflection image from original image all may be used for the present invention.
Usually, in highlight regions, the intensity of the strength ratio diffuse reflection component of specular components is stronger.In the diffuse reflection image obtained in step S310, specular components has been removed substantially.So the intensity difference of original image and diffuse reflection image certainly exists in highlight regions.Intensity difference in highlight regions is higher than the intensity difference of its outside a lot.Therefore, can consider by suitable intensity difference threshold value, highlight regions to be split.
In step s 320, the intensity difference of each pixel between original image and diffuse reflection image is obtained.
In step S330, determine intensity difference threshold value adaptively.
Preferably, intensity difference threshold value needs to adapt to environment facies, and needs to change according to the intensity of highlight regions outside in original image.The method of the self-adaptation determination intensity difference threshold value according to the embodiment of the present invention described in hereinafter with reference Fig. 4 can be utilized to determine.But, anyly determine that the method for intensity difference threshold value all may be used for the present invention adaptively, such as, based on the neural network, genetic algorithm, support vector machine etc. of supervised learning.
In step S340, according to the intensity difference of intensity difference threshold value with each pixel, detect highlight regions.
Particularly, such as, if the intensity difference between the original image of a pixel and diffuse reflection image is greater than intensity difference threshold value, then think that this pixel belongs to highlight regions; Otherwise this pixel does not belong to highlight regions.Thus, highlight regions is detected.
May there is a lot of noise in the highlight regions so detected, or some highlight regions are because the color of background content is missed.So the method that can be strengthened by some aftertreatments such as noise reduction and detection alternatively improves highlight regions testing result.Preferably, following closed operation can be used and open operation stress release treatment and improve result: (1) eliminates the isolated not high bright spot and noise spot that are caused by written contents, because highlight regions should be closed usually; (2) highlight regions of closing on coupled together because of noise is separated.
The highlight regions testing process comprising aftertreatment is also schematically illustrated in Fig. 4.Wherein, original image 410 deducts diffuse reflection image 420 and obtains initial strength difference image 430, filter this initial strength difference image 430 with intensity difference threshold value and the image 440 that binary conversion treatment obtains initial detecting result is carried out to the result after filtering, strengthening and noise reduction process through detecting, obtaining the image 450 of final detection result.
Below with reference to Fig. 5, self-adaptation intensity difference Threshold 500 is according to an embodiment of the invention described.
As shown in Figure 5, in step S510, using the mean value of the intensity difference of each pixel as the first intensity difference threshold value T
1.
In step S520, according to the intensity difference of the first intensity difference threshold value with each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications.
In step S530, determine that the mean value of the intensity difference of high strength pixel is as the second intensity difference threshold value T
2.
In step S540, according to the intensity difference of the second intensity difference threshold value and each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications again.
In step S550, determine that the mean value of this intensity difference of sorted high strength pixel is again as the 3rd difference limen of intensity value T
3.
In step S560, determine intensity difference threshold value T based at least this first, second, third intensity difference threshold value.Such as, if the ratio molecule accounting for whole pixel through step S520 sorted high strength pixel is a, (1) final intensity difference threshold value T can be calculated as follows.
T＝T
3+(1-a)×(T
2-T
1)+a×(T
3-T
2)……(1)
But formula (1) is only example, can also such as adopt more complicated quadratic formula.In addition, the classification in above-mentioned steps S520-S550, the iterative process of intensity difference threshold value using the mean value of sorted high strength pixel as next classification process can also be continued, and therefore can based on more intensity difference threshold value T
4, T
5... determine last intensity difference threshold value.About above-mentioned weight coefficient factor a, can determine by rule of thumb, or be determined according to the number percent of whole pixel shared by high strength pixel after every subseries.
The sample calculation of the highlight regions factor of influence of each pixel in highlight regions is described below with reference to Fig. 6.
First, we create a mirror-reflection map to the reflection strength of each pixel in the highlight regions marking testing result.Figure 62 0 in the middle of in Fig. 6 shows a part for mirror-reflection map, wherein littlely squarely represent a pixel with each, mark H wherein represents that this pixel belongs to high luminance pixels, mark N represents that this pixel belongs to non-high luminance pixels, here the classification foundation of pixel can be used as according to the intensity difference image 440 (in figure 6, indicating this figure with label 630) as initial detecting result after the binaryzation shown in Fig. 4.And the testing result Figure 61 0 in left side in Fig. 6 is used to indicate which pixel to need to calculate highlight regions factor of influence for, this testing result Figure 61 0 can adopt the final detection result Figure 45 0 in Fig. 4.In this example, for each pixel belonging to highlight regions, calculate its highlight regions factor of influence, to characterize this pixel by highlight regions effect.
Mirror-reflection cartographic representation in Fig. 6 will calculate the highlight regions factor of influence of pixel P (x, y) at present.First determine the peripheral region size that will consider for this reason, namely determine the neighboring pixel that will consider, in the mirror-reflection map of Fig. 6, this neighboring pixel indicates with black background.This is a kind of conventional regional assignment method, represents in the scope of the radius r of neighboring pixel centered by object pixel, and in this example, r=2, its neighboring pixel is 12, the generally number N of neighboring pixel (2) expression with the formula
The position of pixel P (x, y) represents with (x, y), and in the consideration region of the radius r of pixel P (x, y), the number of high luminance pixels is expressed as N
h(x, y), then highlight regions factor of influence u (x, y) about pixel P (x, y) can be determined according to formula (3)
u(x，y)＝N
h(x，y)/N……(3)
In the example shown in Fig. 6, the neighboring pixel number N=12 of pixel P (x, y), the number N of high luminance pixels
h(x, y)=10, it is shown in the square frame at pixel P (x, y) place, then the highlight regions factor of influence u=N of pixel P (x, y)
h(x, y)/N=10/12 ≈ 0.8333.
The form of the neighboring area shown in Fig. 6 and the size of radius r are only example, and it can change according to design requirement.In addition, intensity difference Figure 63 0 of binaryzation is have employed in Fig. 6, but this is only example, as the implementation that another is more complicated, initial strength difference image 430 as shown in Figure 4 can be adopted, now, pixel P (x can be calculated, y) mean value of the intensity difference of neighboring pixel and the ratio of intensity difference, as the highlight regions factor of influence of pixel P (x, y).In a word, anyly a pixel periphery highlight regions all can be used for calculating this pixel highlight regions factor of influence to the algorithm of its influence degree can be reflected.
Below respectively with reference to figure 7 and Fig. 8, how the step S220 shown in exemplary illustration Fig. 2, namely carry out the content in detected image highlight regions based on the highlighted factor of influence of each pixel in highlight regions.Fig. 7 describe determine based on the content detection threshold value of highlighted factor of influence according to an embodiment of the invention and and then the method 700 of detected image content, and Fig. 8 illustrates the content detection analyzed based on multiple dimensioned local region information in highlight regions.
As shown in Figure 7, in step S710, determine first threshold or default threshold T according to the picture material outside highlight regions
0.
Be detected as example with stroke, usually need a threshold value such as gray threshold to split stroke and background, think if the gray scale of a pixel is less than this gray threshold, then belong to stroke, otherwise then belong to background.Any conventional stroke detection threshold defining method can be adopted, according to the first threshold that the stroke outside highlight regions is determined herein.
Determine that an example of first threshold is: calculate the average of all pixels outside highlight regions as T
n, as shown in formula (4), the value of pixel can adopt gray scale or colour here.
T
n=average (∑ P (x, y)) wherein (x, y)
highlight regions ... (4)
In addition, T can be less than by calculated value
nand the average of all pixels outside highlight regions is as first threshold or default threshold T
0, as shown in formula (5).Can think T
0what represent is the average belonging to the pixel of stroke outside highlight regions.
T
0=average (∑ P (x, y)) wherein p (x, y) < T
nand (x, y)
highlight regions ... (5)
In step S720, for each pixel in highlight regions, based on the highlighted factor of influence of this each pixel, adjust this first threshold, to determine the threshold value for this each pixel.
For this reason, both can consider the overall impact for threshold value of highlight regions, highlight regions factor of influence u (x, y) of each pixel self simultaneously considered in highlight regions determines the threshold value for each pixel.
Such as, can arrange a threshold value based on overall highlight regions and change parameter Δ t, it is for all pixels belonging to highlight regions.Following formula (6) calculated threshold can be utilized to change parameter Δ t.
Δt＝T
h-T
n……(6)
Wherein T
hrepresent the average of all pixels in highlight regions, namely as shown in formula (7)
T
h=average (∑ P (x, y)) wherein (x, y) ∈ highlight regions ... (7)
Here, Δ t reflects the difference between the region of highlight regions and highlight regions outside, reflects the impact of overall highlight regions for picture material to a certain extent.In principle, overall highlight regions is larger, also namely brighter, then threshold value change parameter is also large.
For pixel P (x, y), change parameter Δ t based on its highlight regions factor of influence u (x, y) and global threshold, the threshold value T (x, y) of pixel P (x, y) can calculate according to formula (8):
T(x，y)＝T
0+Δt×f(u(x，y))……(8)
Wherein, f (u (x, y)) represents a function, can be such as quadratic function, exponential function, logarithmic function etc., an example of its form as shown in formula (9),
f(u(x，y))＝u(x，y)
2+0.5……(9)
Visible, the threshold value of the content detection here in highlight regions is based on each pixel, and they are different with the highlight regions factor of influence of each pixel, therefore, it is possible to change with conforming better.
For the threshold value of each pixel in highlight regions, other constraint can be applied, such as T (x, y) < 250 etc.
In step S730, according to the threshold value T (x, y) for each pixel, determine that this each pixel in highlight regions belongs to content or background, thus obtain the first testing result.
Such as, for stroke, if be gray threshold, if then P (x, y) < T (x, y), think that this pixel belongs to stroke, otherwise belong to background.
Fig. 8 shows the process flow diagram of the content detection algorithm analyzed based on multiple dimensioned local region information.
Be detected as example with stroke, the thresholding method based on local region information analysis can adjust threshold value adaptively and carry out detecting pen portrait element, and be the general way in this area, such as, the smoothing processing of image is generally carried out based on regional area.But, there is highlighted image more complicated, highlighted part can change the color of regional area pixel to a great extent simultaneously.So the common thresholding method based on local region information analysis may not process highlighted image well.For this reason, for the pixel in highlight regions, stroke can be detected based on the information of the regional area of multiple different scale, thus obtain the second testing result.And, final detection result can be obtained by the first testing result and the second testing result combining method described in above-mentioned composition graphs 7.
As shown in Figure 8, in step S810, judge a pixel whether in highlight regions.If not in highlight regions, then process proceeds to step S820, it carries out mesoscale local region threshold and stroke detects, here mesoscale is large scale relatively hereafter in highlight regions situation and small scale, and in fact it can be the yardstick adopted in any current local region threshold method.
If pixel is in highlight regions, then perform the operation of step S820, S830 and S840, the thresholding method that the local region information namely simultaneously performing mesoscale, large scale and small scale is analyzed and detection stroke simultaneously.
Then step S850 is proceeded to, the thresholding method that the local region information under comprehensive different scale is analyzed and the result that stroke detects.
Above with three yardsticks, namely the regional area of mesoscale, large scale, small scale is local region threshold and the content detection that example describes multiple different scale.But the number of yardstick is not limited to three, but can arrange arbitrarily as required, or be set by supervised learning.
In addition, the size about yardstick can utilize above-mentioned highlight regions factor of influence u (x, y) coming to adjust.It is generally acknowledged, compared with the regional area of large scale, noise can be filtered better; And can more effectively information extraction compared with the regional area of small scale.Thus, in principle, highlight regions factor of influence u (x, y) is larger, and above-mentioned large scale can arrange larger, with filtered noise better; Highlight regions factor of influence u (x, y) is less, and above-mentioned large scale can arrange less, with information extraction better.But the size of yardstick is also conditional, such as, is detected as example with stroke, and yardstick should greatly to the size exceeding highlight regions, and yardstick also should not be less than the thickness of stroke.In one example, we are set to 20 pixels mesoscale, and large scale is set to 50 pixels, and small scale is set to 5 pixels.
In addition, the stroke detection threshold under different scale also can adjust according to the pixel color in regional area adaptively.
Fig. 9 shows the comparison schematic diagram of stroke testing result in the highlight regions based on various yardstick local region information analysis.Visible, the thresholding method using multiple dimensioned local region information to analyze, compares with the method for single yardstick, can improve testing result, reduces noise.
After go out the picture material in highlight regions through operation detection shown in Fig. 2, enhancing or completion process, the color rendition process etc. of such as content detection result can be carried out.Exemplary illustration is carried out below with reference to Figure 10 and Figure 11.
Figure 10 shows the example of the stroke testing result enhancing disposal route 1000 of the stroke testing result based on highlight regions being treated to example according to one embodiment of the invention with stroke.
As shown in Figure 10, in step S1010, be the pixel of content carrys out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result.
About this COLOR COMPOSITION THROUGH DISTRIBUTION model, can determine as follows, using a large amount of highlighted image as training dataset, utilize statistical method to analyze this training dataset, thus determine its pixel color distributed model.Exemplarily, after the stroke pixel analyzing a large amount of highlight regions, find that Gaussian distribution can reflect the COLOR COMPOSITION THROUGH DISTRIBUTION of stroke pixel preferably.But, Gaussian distribution is only example, and those skilled in the art can find according to application scenario and design requirement etc. and apply other suitable COLOR COMPOSITION THROUGH DISTRIBUTION models.
Exemplarily, after determining that Gaussian distribution is as pixel color distributed model, can utilize in current stroke testing result and carry out this Gaussian distribution of matching as the pixel of stroke as sample, thus determine the parameter in Gaussian distribution, such as, average mean and standard variance std.
In step S1020, predict the nd content pixel in highlight regions based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.
Be detected as example with stroke, after determining the parameter of Gaussian distribution model, this Gaussian distribution model can be utilized to estimate nd stroke pixel.Such as, the confidence space arranging Gaussian distribution model is D=[mean-kstd, mean+kstd] (K=1,2), if the variance of pixel color is in D interval, this pixel is just considered to undetected stroke pixel.
After supplementing undetected stroke pixel, can also carry out other operations, such as inspection and stress release treatment are to reduce the noise of above-mentioned enhancing process, can carry out post-processing step in addition and come according to hand-written stroke feature assay, such as, Skeleton algorithm can eliminate short stroke.
In addition, the color of the picture material in highlight regions may change or clear not, therefore can take into account the color reduction treatment of the circumstances in which people get things ready for a trip.Figure 11 shows the example of the stroke color reduction treatment 1100 of the stroke testing result based on highlight regions being treated to example according to one embodiment of the invention with stroke.
The color rendition process of Figure 11 is considered to propose for following: the colors in the region outside highlight regions and feature can as benchmark, can be reduced the color of the picture material in highlight regions by the color analyzing the region outside highlight regions.In addition, original image and diffuse reflection image can be utilized as handling object, to eliminate the impact of highlight regions on color simultaneously.For stroke process exemplarily, because stroke has usually, color distinction between stroke is clearly demarcated, the feature of clear border, limitednumber, is therefore particularly suited for adopting clustering method.
As shown in figure 11, in step S1110, original image and diffuse reflection image is used to carry out cluster to the pixel color outside highlight regions respectively.About the algorithm of cluster, conventional k mean cluster or k central cluster etc. can be adopted.Calculate can carry out on RGB and/or LAB color space about the distance may used in cluster.
In step S1120, respectively the pixel color in the highlight regions of original image and diffuse reflection image is classified according to respective Color-based clustering result.Equally, during classification, the distance of color can be calculated on RGB and/or LAB color space, and in original image and diffuse reflection image, compare the distance of color simultaneously.About the classification results of original image and diffuse reflection image, different weight can be used the two to be combined.
In step S1130, the result according to color classification adjusts the pixel color in highlight regions.Such as, if the color of a pixel is attributed to redness in highlight regions, the color of this pixel in highlight regions then can be replaced by the average of the color belonging to other all pixel of pink group, or in highlight regions this pixel color on add a regulation coefficient, to make the color of this pixel closer to the color character of belonging classification.
After having carried out above-mentioned color rendition process, according to circumstances and need, to the smoothing process of the background of highlight regions peripheral region, can also make it more natural.
Figure 12 shows the schematic diagram of a color rendition processing procedure and effect, wherein after simultaneously based on the cluster of original image and diffuse reflection image and classification, each pixel is classified as different classifications, and the image finally after color rendition process considerably improves the observability of pixel.
Figure 13 shows the overall flow figure of highlighted according to an embodiment of the invention image procossing 1300.
In step S1310, input original image.
In step S1320, detect highlight regions, and calculate highlight regions factor of influence.The calculating of the highlight regions factor of influence that the method for the detection highlight regions that the operation of this step can describe with reference to composition graphs 3 above and composition graphs 6 describe.
In step S1330, detected image content, wherein detects the picture material of highlight regions based on highlight regions factor of influence.About the operation of this step, can with reference to the image content detection method of composition graphs 7 and/or Fig. 8 description above.
In step S1340, carry out the enhancing of highlight regions content detection result.About the operation of this step, can with reference to the content detection result Enhancement Method described in conjunction with Figure 10.
In step S1350, by Color-based clustering and the color rendition carried out in highlight regions of classifying.About the operation of this step, can with reference to above in conjunction with Figure 11 describe color rendition method.
Figure 14 shows the schematic block diagram of the content detection device 1400 of image highlight area according to an embodiment of the invention.The content detection device 1400 of this image highlight area can comprise: highlighted factor of influence calculating unit 1410, and it determines the highlighted factor of influence of each pixel in highlight regions, and this highlighted factor of influence represents the intensity that highlight regions affects this pixel content; And content detection parts 1420, its highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions.Operation about the content detection device 1400 of image highlight area and its highlighted factor of influence calculating unit 1410, content detection parts 1420 can with reference to the foregoing teachings of the process flow diagram based on Fig. 2, no longer repeated description here.
Figure 15 shows the block diagram of highlighted according to an embodiment of the invention image processing apparatus 1500.This highlighted image processing apparatus 1500 can comprise: highlight regions detecting unit 1510, image content detection unit 1520, highlight regions content enhancement unit 1530, highlight regions color rendition unit 1540.About highlight regions detecting unit 1510, image content detection unit 1520, highlight regions content enhancement unit 1530, highlight regions color rendition unit 1540 concrete operations can with reference to above in conjunction with Figure 13 to step S1320, S1330, S1340, S1350 operation description.Here will repeat no more.
In accordance with a further aspect of the present invention, provide a kind of device 1600 strengthening the testing result of highlight regions content, this device 1600 can comprise: COLOR COMPOSITION THROUGH DISTRIBUTION models fitting parts 1610, for being the pixel of content to carry out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result; Highlight regions content pixel prediction unit 1620, for predicting the nd content pixel in highlight regions based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.With reference in conjunction with the description of Figure 10 to step S1010 and S1020, can repeat no more here about the function of above-mentioned COLOR COMPOSITION THROUGH DISTRIBUTION models fitting parts 1610 and highlight regions content pixel prediction unit 1620 and concrete operations.
Figure 17 shows a kind of device 1700 of color information of reducing highlight regions, and this device 1700 can comprise: cluster parts 1710, carries out cluster for using original image and corresponding diffuse reflection image respectively to the pixel color outside highlight regions; Classification element 1720, for classifying to the pixel color in the highlight regions of original image and diffuse reflection image respectively according to respective Color-based clustering result; And highlight regions color adjustment component 1730, for the result according to color classification, the pixel color in highlight regions is adjusted.With reference to above in conjunction with the description of Figure 11 to step S1110-S1130, can repeat no more here about the function of above-mentioned cluster parts 1710, classification element 1720 and highlight regions color adjustment component 1730 and concrete operations.
Variation
The present invention is using projector-camera system as an application scenarios of the present invention, but application of the present invention is not limited thereto, and may occur that the situation of highlight area all can apply the present invention in any image.
The present invention describes the detection of content, enhancing and recovery using word or stroke as the example of picture material.But the present invention is not limited thereto, other picture materials such as face, scenery etc. all can apply the present invention.
Although, when the present invention is particularly useful for the image being short of texture information, when the present invention also goes for the image with texture information.And highlighted factor of influence of the present invention also can take in when building the iconic model of texture information.
Below ultimate principle of the present invention is described in conjunction with specific embodiments, but, it is to be noted, for those of ordinary skill in the art, whole or any step or the parts of method and apparatus of the present invention can be understood, can in the network of any calculation element (comprising processor, storage medium etc.) or calculation element, realized with hardware, firmware, software or their combination, this is that those of ordinary skill in the art use their basic programming skill just can realize when having read explanation of the present invention.
Therefore, object of the present invention can also be realized by an operation program or batch processing on any calculation element.Described calculation element can be known fexible unit.Therefore, object of the present invention also can realize only by the program product of providing package containing the program code realizing described method or device.That is, such program product also forms the present invention, and the storage medium storing such program product also forms the present invention.Obviously, described storage medium can be any storage medium developed in any known storage medium or future.
Also it is pointed out that in apparatus and method of the present invention, obviously, each parts or each step can decompose and/or reconfigure.These decompose and/or reconfigure and should be considered as equivalents of the present invention.Further, the step performing above-mentioned series of processes can order naturally following the instructions perform in chronological order, but does not need necessarily to perform according to time sequencing.Some step can walk abreast or perform independently of one another.
Above-mentioned embodiment, does not form limiting the scope of the invention.It is to be understood that depend on designing requirement and other factors, various amendment, combination, sub-portfolio can be there is and substitute in those skilled in the art.Any amendment done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within scope.
Claims (8)
1. a content detection algorithm for image highlight area, comprising:
Determine the highlighted factor of influence of each pixel in highlight regions, this highlighted factor of influence represents the intensity that highlight regions affects this pixel content;
Highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions,
Wherein, described highlight regions is by following operation detection out:
Diffuse reflection image is obtained from original image;
Obtain the intensity difference of each pixel between original image and diffuse reflection image;
Determine intensity difference threshold value adaptively; And
According to the intensity difference of intensity difference threshold value and each pixel described, detect described highlight regions,
Wherein saidly determine that intensity difference threshold value comprises adaptively:
Using the mean value of the intensity difference of this each pixel as the first intensity difference threshold value;
According to the intensity difference of the first intensity difference threshold value with each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications;
Determine that the mean value of the intensity difference of high strength pixel is as the second intensity difference threshold value;
According to the intensity difference of the second intensity difference threshold value and each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications again;
Determine that the mean value of this intensity difference of high strength pixel of subseries is again as the 3rd difference limen of intensity value; And
Described intensity difference threshold value is determined based at least this first, second, third intensity difference threshold value.
2. content detection algorithm according to claim 1, the described content come in detected image highlight regions based on the highlighted factor of influence of each pixel in highlight regions comprises:
First threshold is determined according to the picture material outside highlight regions;
For each pixel in highlight regions, based on the highlighted factor of influence of this each pixel, adjust this first threshold, to determine the threshold value for this each pixel;
According to the threshold value for this each pixel, determine that this each pixel belongs to content or background, thus obtain the first testing result.
3., according to the content detection algorithm of claim 1 or 2, also comprise:
Information based on the regional area of multiple different scale carrys out Detection of content, thus obtains the second testing result;
At least obtain final detection result based on the first testing result and the second testing result.
4. content detection algorithm according to claim 3, wherein determines the size of the yardstick of at least one regional area in the regional area of the plurality of different scale according to the highlighted factor of influence of each pixel.
5., according to any one content detection algorithm in claim 1,2,4, also comprise:
It is the pixel of content carrys out the content pixel in matching highlight regions pre-color distributed model as sample in detected testing result;
The nd content pixel in highlight regions is predicted based on the content pixel COLOR COMPOSITION THROUGH DISTRIBUTION model after matching.
6. content detection algorithm according to claim 5, also comprises:
Original image and diffuse reflection image is used to carry out cluster to the pixel color outside highlight regions respectively;
Respectively the pixel color in the highlight regions of original image and diffuse reflection image is classified according to respective Color-based clustering result; And
Result according to color classification adjusts the pixel color in highlight regions.
7. a detection method for image highlight area, comprising:
Diffuse reflection image is obtained from original image;
Obtain the intensity difference of each pixel between original image and diffuse reflection image;
Determine intensity difference threshold value adaptively; And
According to the intensity difference of intensity difference threshold value and each pixel described, detect described highlight regions,
Wherein saidly determine that intensity difference threshold value comprises adaptively:
Using the mean value of the intensity difference of this each pixel as the first intensity difference threshold value;
According to the intensity difference of the first intensity difference threshold value with each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications;
Determine that the mean value of the intensity difference of high strength pixel is as the second intensity difference threshold value;
According to the intensity difference of the second intensity difference threshold value and each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications again;
Determine that the mean value of this intensity difference of high strength pixel of subseries is again as the 3rd difference limen of intensity value; And
Described intensity difference threshold value is determined based at least this first, second, third intensity difference threshold value.
8. a content detection device for image highlight area, comprising:
Highlighted factor of influence calculating unit, it determines the highlighted factor of influence of each pixel in highlight regions, and this highlighted factor of influence represents the intensity that highlight regions affects this pixel content; And
Content detection parts, its highlighted factor of influence based on each pixel in highlight regions carrys out the content in detected image highlight regions,
Wherein, described highlight regions is by the following operation detection of described highlighted factor of influence calculating unit out:
Diffuse reflection image is obtained from original image;
Obtain the intensity difference of each pixel between original image and diffuse reflection image;
Determine intensity difference threshold value adaptively; And
According to the intensity difference of intensity difference threshold value and each pixel described, detect described highlight regions,
Wherein saidly determine that intensity difference threshold value comprises adaptively:
Using the mean value of the intensity difference of this each pixel as the first intensity difference threshold value;
According to the intensity difference of the first intensity difference threshold value with each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications;
Determine that the mean value of the intensity difference of high strength pixel is as the second intensity difference threshold value;
According to the intensity difference of the second intensity difference threshold value and each pixel, be high strength pixel and hypo-intense pixels by each pixel classifications again;
Determine that the mean value of this intensity difference of high strength pixel of subseries is again as the 3rd difference limen of intensity value; And
Described intensity difference threshold value is determined based at least this first, second, third intensity difference threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110401299.8A CN103150717B (en) | 2011-12-06 | 2011-12-06 | The detection method of image highlight area, content detection algorithm and content detection device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110401299.8A CN103150717B (en) | 2011-12-06 | 2011-12-06 | The detection method of image highlight area, content detection algorithm and content detection device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103150717A CN103150717A (en) | 2013-06-12 |
| CN103150717B true CN103150717B (en) | 2016-02-10 |
Family
ID=48548771
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110401299.8A Expired - Fee Related CN103150717B (en) | 2011-12-06 | 2011-12-06 | The detection method of image highlight area, content detection algorithm and content detection device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103150717B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201414144D0 (en) * | 2014-08-08 | 2014-09-24 | Imagination Tech Ltd | Relightable texture for use in rendering an image |
| CN105405130B (en) * | 2015-11-02 | 2019-03-01 | 北京旷视科技有限公司 | License image highlight detection method and device based on cluster |
| CN109299633A (en) * | 2017-07-25 | 2019-02-01 | 上海中科顶信医学影像科技有限公司 | Wrinkle detection method, system, equipment and medium |
| CN109829480A (en) * | 2019-01-04 | 2019-05-31 | 广西大学 | The method and system of the detection of body surface bloom feature and material classification |
| CN116297463B (en) * | 2023-05-16 | 2023-08-01 | 四川省港奇电子有限公司 | Power adapter shell injection molding detection method, system and device |
| CN116503633B (en) * | 2023-07-03 | 2023-09-05 | 山东艾迈科思电气有限公司 | Intelligent detection control method for switch cabinet state based on image recognition |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0810551A1 (en) * | 1996-05-30 | 1997-12-03 | Agfa-Gevaert N.V. | Detection of and correction for specular reflections in digital image acquisition |
| CN101193202A (en) * | 2006-12-22 | 2008-06-04 | 陈珉 | Method for display highly dynamic image on the traditional output device |
| CN101646092A (en) * | 2008-08-05 | 2010-02-10 | 株式会社东芝 | Picture processing method and mobile communication terminal |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6859565B2 (en) * | 2001-04-11 | 2005-02-22 | Hewlett-Packard Development Company, L.P. | Method and apparatus for the removal of flash artifacts |
-
2011
- 2011-12-06 CN CN201110401299.8A patent/CN103150717B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0810551A1 (en) * | 1996-05-30 | 1997-12-03 | Agfa-Gevaert N.V. | Detection of and correction for specular reflections in digital image acquisition |
| CN101193202A (en) * | 2006-12-22 | 2008-06-04 | 陈珉 | Method for display highly dynamic image on the traditional output device |
| CN101646092A (en) * | 2008-08-05 | 2010-02-10 | 株式会社东芝 | Picture processing method and mobile communication terminal |
Non-Patent Citations (2)
| Title |
|---|
| 《Detection of Diffuse and Specular Interface Reflections and Inter-Reflections by Color Image Segmentation》;RUZENA BAJCSY et al;《International Journal of Computer Vision》;19961231(第3期);参见第242页右栏第2段，摘要 * |
| 《Separating Reflection Components Based on Chromaticity and Noise Analysis》;Robby T.Tan et al;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》;20041031;第26卷(第10期);1373-1379 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103150717A (en) | 2013-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569695B (en) | Image processing method and device based on loss assessment image judgment model | |
| CN103150717B (en) | The detection method of image highlight area, content detection algorithm and content detection device | |
| US11037291B2 (en) | System and method for detecting plant diseases | |
| CN108229526B (en) | Network training method, network training device, image processing method, image processing device, storage medium and electronic equipment | |
| US10062172B2 (en) | Automated tattoo recognition techniques | |
| US20200272866A1 (en) | Computer-executed method and apparatus for assessing vehicle damage | |
| US9773196B2 (en) | Utilizing deep learning for automatic digital image segmentation and stylization | |
| US20200066407A1 (en) | Method and System for Assisting Pathologist Identification of Tumor Cells in Magnified Tissue Images | |
| EP2959454B1 (en) | Method, system and software module for foreground extraction | |
| US10026004B2 (en) | Shadow detection and removal in license plate images | |
| US8023766B1 (en) | Method and system of processing an image containing undesirable pixels | |
| US20140050387A1 (en) | System and Method for Machine Vision Inspection | |
| US20150125074A1 (en) | Apparatus and method for extracting skin area to block harmful content image | |
| CN110222694B (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| US10373316B2 (en) | Images background subtraction for dynamic lighting scenarios | |
| CN110610509A (en) | Optimized matting method and system capable of assigning categories | |
| JP2017117019A (en) | Image processing device, image processing method, and program | |
| CN106709916A (en) | Image quality assessment method and device | |
| US7403636B2 (en) | Method and apparatus for processing an image | |
| CN111291778B (en) | Training method of depth classification model, exposure anomaly detection method and device | |
| CN116958113A (en) | Product detection method, device, equipment and storage medium | |
| Zou et al. | Statistical analysis of signal-dependent noise: application in blind localization of image splicing forgery | |
| CN111754491A (en) | Picture definition judging method and device | |
| JP2004246618A (en) | Method, device, and program for generating image used for collating in pattern recognition and pattern recognition using the image | |
| US20080199073A1 (en) | Red eye detection in digital images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160210 Termination date: 20201206 |