WO2017214848A1 - Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image - Google Patents

Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image Download PDF

Info

Publication number
WO2017214848A1
WO2017214848A1 PCT/CN2016/085680 CN2016085680W WO2017214848A1 WO 2017214848 A1 WO2017214848 A1 WO 2017214848A1 CN 2016085680 W CN2016085680 W CN 2016085680W WO 2017214848 A1 WO2017214848 A1 WO 2017214848A1
Authority
WO
WIPO (PCT)
Prior art keywords
intensity
mapped
output
pixel
image
Prior art date
Application number
PCT/CN2016/085680
Other languages
English (en)
Inventor
Xuhang LIAN
Original Assignee
Nokia Technologies Oy
Nokia Technologies (Beijing) Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Technologies Oy, Nokia Technologies (Beijing) Co., Ltd. filed Critical Nokia Technologies Oy
Priority to PCT/CN2016/085680 priority Critical patent/WO2017214848A1/fr
Publication of WO2017214848A1 publication Critical patent/WO2017214848A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G06T5/75—Unsharp masking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20004—Adaptive image processing
    • G06T2207/20012—Locally adaptive
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G06T2207/20028—Bilateral filtering

Definitions

  • Embodiments of the disclosure generally relate to information technologies, and, more particularly, to removing object in an image.
  • Image quality plays an important role in most applications based on image.
  • computer vision systems are broadly used for video surveillance, traffic surveillance, driver assistant systems, traffic monitoring, human identification, human-computer interaction, public security, event detection, tracking, frontier guards and the Customs, scenario analysis and classification, object detecting and identification, image indexing and retrieve, etc.
  • certain object such as haze, dark light, etc
  • haze and dark light are two common sources of degrading image quality. They hamper the visibility of the scene and its objects. The intensity, hue and saturation of the scene and its objects are also altered by the haze or dark light. Consequently, it gives arise difficulty in extracting haze-invariant or light-invariant features.
  • image features are key for computer vision tasks such as image matching, recognition, retrieval, and object detection, the existence of certain object such as haze and dark causes negative effect in the computer vision systems. Therefore, removing certain object in the image is necessary in many applications based on image.
  • the apparatus may comprise at least one processor; and at least one memory including computer program code, the memory and the computer program code configured to, working with the at least one processor, cause the apparatus to perform at least the following: determine input intensity of a pixel in an image; and determine output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • the method may comprise determining input intensity of a pixel in an image; and determining output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • a computer program product embodied on a distribution medium readable by a computer and comprising program instructions which, when loaded into a computer, execute at least the following: determine input intensity of a pixel in an image; and determine output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • a non-transitory computer readable medium having encoded thereon statements and instructions to cause a processor to execute at least the following: determine input intensity of a pixel in an image; and determine output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • an apparatus comprising means configured to determine input intensity of a pixel in an image; and means configured to determine output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • Figure 1 is a simplified block diagram showing an apparatus according to an embodiment
  • Figure 2 is a flow chart depicting a process of removing certain object in image in accordance with embodiments of the present disclosure
  • Figure 3 is a flow chart depicting a part of process of removing certain object in image in accordance with embodiments of the present disclosure
  • Figure 4 is a flow chart depicting a part of process of removing certain object in image in accordance with embodiments of the present disclosure
  • Figure 5 is a flow chart depicting a process of removing haze in image in accordance with embodiments of the present disclosure.
  • Figure 6 shows some results of methods according to embodiments of the present disclosure and Dark Channel Prior (DCP) .
  • circuitry refers to (a) hardware-only circuit implementations (e.g., implementations in analog circuitry and/or digital circuitry) ; (b) combinations of circuits and computer program product (s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor (s) or a portion of a microprocessor (s) , that require software or firmware for operation even if the software or firmware is not physically present.
  • This definition of 'circuitry' applies to all uses of this term herein, including in any claims.
  • the term 'circuitry' also includes an implementation comprising one or more processors and/or portion (s) thereof and accompanying software and/or firmware.
  • the term 'circuitry' as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network apparatus, other network apparatus, and/or other computing apparatus.
  • non-transitory computer-readable medium which refers to a physical medium (e.g., volatile or non-volatile memory device)
  • the embodiments are mainly described in the context of image dehazing, they are not limited to this but can be applied to remove any suitable object in the image, such as image dedarking. Moreover, the embodiments can be applied to video, though they are mainly discussed in the context of a single image. It is also noted that the embodiments may be applied to not only the processing of non-real time image or video but also the processing of real time image or video. In addition, it is further noted that the image as used herein may refer to a color image or a gray image.
  • Dehazing is a process of haze removal and dedarking is a process of dealing with dark (low light) image so that the content of the image is clear.
  • Existing dehazing methods work on the assumption of an idea imaging model. But the model is not guaranteed to perfectly fit the practical situation. In addition, it is difficult to precisely estimate the parameters of the imaging model.
  • Existing dedarking methods conduct dedarking by general and empirical image enhancement techniques which are not guaranteed to have minimum error between the dedarked image and ideal bright one.
  • State-of-the-art dehazing methods are characterized by estimation of medium transmission and airlight.
  • Representative methods include Dark Channel Prior (DCP) as described in “K. He, J. Sun, X. Tang, Single image haze removal using dark channel prior, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (12) (2011) 2341-2352” , and Haze Relevant Features (HRF) as described in “K. Tang, J. Yang, J. Wang, Investigating Haze-relevant Features in A Learning Framework for Image Dehazing, Proc. IEEE International Conference on Computer Vision and Pattern Recognition, 2014” .
  • DCP Dark Channel Prior
  • HRF Haze Relevant Features
  • the medium transmission is estimated by using the so-called dark channel prior.
  • HRF utilizes haze-relevant features extracted from synthetic hazy patches to train a regression model (Random Forest) .
  • the output of the regressor is an estimation of the medium transmission. It is noted that the most important haze-relevant feature is dark channel prior.
  • Some dedarking methods are characterized by tone mapping framework/function. However, the empirically designed tone-mapping function is not necessary to obtain optimal results.
  • Embodiments of the disclosure can at least solve or mitigate one or more of above issues.
  • a reconstruction framework for removing certain object in the image (such as dehazing and dedarking) is provided in the embodiments, where the output intensity is the multiplication of mapped input intensity and mapped detail intensity.
  • the embodiments do not rely on the imaging model so it is not required to estimate media transmission and air-light which are difficult to be exactly estimated.
  • the tone mapping functions provided by the embodiments are leaned according to an objective function and hence the parameters are optimal and are not empirical.
  • the embodiments propose to recover a degraded image by two tone mapping functions for image intensity and one tone mapping function for image saturation.
  • the embodiments propose an alternative optimization algorithm for estimating the optimal parameters with a reconstruction-error minimization strategy.
  • FIG. 1 is a simplified block diagram showing an apparatus, such as an electronic apparatus 10, in which various embodiments of the disclosure may be applied. It should be understood, however, that the electronic apparatus as illustrated and hereinafter described is merely illustrative of an apparatus that could benefit from embodiments of the disclosure and, therefore, should not be taken to limit the scope of the disclosure. While the electronic apparatus 10 is illustrated and will be hereinafter described for purposes of example, other types of apparatuses may readily employ embodiments of the disclosure.
  • the electronic apparatus 10 may be a portable digital assistant (PDAs) , a user equipment, a mobile computer, a desktop computer, a smart television, a gaming apparatus, a laptop computer, a media player, a camera, a video recorder, a mobile phone, a global positioning system (GPS) apparatus, a smart phone, a tablet, a server, a thin client, a cloud computer, a virtual server, a set-top box, a computing device, a distributed system, a smart glass, a vehicle navigation system and/or any other types of electronic systems.
  • the electronic apparatus 10 may run with any kind of operating system including, but not limited to, Windows, Linux, UNIX, Android, iOS and their variants.
  • the apparatus of at least one example embodiment need not to be the entire electronic apparatus, but may be a component or group of components of the electronic apparatus in other example embodiments.
  • the electronic apparatus may readily employ embodiments of the disclosure regardless of their intent to provide mobility.
  • embodiments of the disclosure may be utilized in conjunction with a variety of applications, both in the mobile communications industries and outside of the mobile communications industries.
  • the electronic apparatus 10 may comprise processor 11 and memory 12.
  • Processor 11 may be any type of processor, controller, embedded controller, processor core, and/or the like.
  • processor 11 utilizes computer program code to cause an apparatus to perform one or more actions.
  • Memory 12 may comprise volatile memory, such as volatile Random Access Memory (RAM) including a cache area for the temporary storage of data and/or other memory, for example, non-volatile memory, which may be embedded and/or may be removable.
  • the non-volatile memory may comprise an EEPROM, flash memory and/or the like.
  • Memory 12 may store any of a number of pieces of information, and data. The information and data may be used by the electronic apparatus 10 to implement one or more functions of the electronic apparatus 10, such as the functions described herein.
  • memory 12 includes computer program code such that the memory and the computer program code are configured to, working with the processor, cause the apparatus to perform one or more actions described herein.
  • the electronic apparatus 10 may further comprise a communication device 15.
  • communication device 15 comprises an antenna, (or multiple antennae) , a wired connector, and/or the like in operable communication with a transmitter and/or a receiver.
  • processor 11 provides signals to a transmitter and/or receives signals from a receiver.
  • the signals may comprise signaling information in accordance with a communications interface standard, user speech, received data, user generated data, and/or the like.
  • Communication device 15 may operate with one or more air interface standards, communication protocols, modulation types, and access types.
  • the electronic communication device 15 may operate in accordance with second-generation (2G) wireless communication protocols IS-136 (time division multiple access (TDMA) ) , Global System for Mobile communications (GSM) , and IS-95 (code division multiple access (CDMA) ) , with third-generation (3G) wireless communication protocols, such as Universal Mobile Telecommunications System (UMTS) , CDMA2000, wideband CDMA (WCDMA) and time division-synchronous CDMA (TD-SCDMA) , and/or with fourth-generation (4G) wireless communication protocols, wireless networking protocols, such as 802.11, short-range wireless protocols, such as Bluetooth, and/or the like.
  • Communication device 15 may operate in accordance with wireline protocols, such as Ethernet, digital subscriber line (DSL) , and/or the like.
  • Processor 11 may comprise means, such as circuitry, for implementing audio, video, communication, navigation, logic functions, and/or the like, as well as for implementing embodiments of the disclosure including, for example, one or more of the functions described herein.
  • processor 11 may comprise means, such as a digital signal processor device, a microprocessor device, various analog to digital converters, digital to analog converters, processing circuitry and other support circuits, for performing various functions including, for example, one or more of the functions described herein.
  • the apparatus may perform control and signal processing functions of the electronic apparatus 10 among these devices according to their respective capabilities.
  • the processor 11 thus may comprise the functionality to encode and interleave message and data prior to modulation and transmission.
  • the processor 11 may additionally comprise an internal voice coder, and may comprise an internal data modem. Further, the processor 11 may comprise functionality to operate one or more software programs, which may be stored in memory and which may, among other things, cause the processor 11 to implement at least one embodiment including, for example, one or more of the functions described herein. For example, the processor 11 may operate a connectivity program, such as a conventional internet browser.
  • the connectivity program may allow the electronic apparatus 10 to transmit and receive internet content, such as location-based content and/or other web page content, according to a Transmission Control Protocol (TCP) , Internet Protocol (IP) , User Datagram Protocol (UDP) , Internet Message Access Protocol (IMAP) , Post Office Protocol (POP) , Simple Mail Transfer Protocol (SMTP) , Wireless Application Protocol (WAP) , Hypertext Transfer Protocol (HTTP) , and/or the like, for example.
  • TCP Transmission Control Protocol
  • IP Internet Protocol
  • UDP User Datagram Protocol
  • IMAP Internet Message Access Protocol
  • POP Post Office Protocol
  • SMTP Simple Mail Transfer Protocol
  • WAP Wireless Application Protocol
  • HTTP Hypertext Transfer Protocol
  • the electronic apparatus 10 may comprise a user interface for providing output and/or receiving input.
  • the electronic apparatus 10 may comprise an output device 14.
  • Output device 14 may comprise an audio output device, such as a ringer, an earphone, a speaker, and/or the like.
  • Output device 14 may comprise a tactile output device, such as a vibration transducer, an electronically deformable surface, an electronically deformable structure, and/or the like.
  • Output Device 14 may comprise a visual output device, such as a display, a light, and/or the like.
  • the electronic apparatus may comprise an input device 13.
  • Input device 13 may comprise a light sensor, a proximity sensor, a microphone, a touch sensor, a force sensor, a button, a keypad, a motion sensor, a magnetic field sensor, a camera, a removable storage device and/or the like.
  • a touch sensor and a display may be characterized as a touch display.
  • the touch display may be configured to receive input from a single point of contact, multiple points of contact, and/or the like.
  • the touch display and/or the processor may determine input based, at least in part, on position, motion, speed, contact area, and/or the like.
  • the electronic apparatus 10 may include any of a variety of touch displays including those that are configured to enable touch recognition by any of resistive, capacitive, infrared, strain gauge, surface wave, optical imaging, dispersive signal technology, acoustic pulse recognition or other techniques, and to then provide signals indicative of the location and other parameters associated with the touch. Additionally, the touch display may be configured to receive an indication of an input in the form of a touch event which may be defined as an actual physical contact between a selection object (e.g., a finger, stylus, pen, pencil, or other pointing device) and the touch display.
  • a selection object e.g., a finger, stylus, pen, pencil, or other pointing device
  • a touch event may be defined as bringing the selection object in proximity to the touch display, hovering over a displayed object or approaching an object within a predefined distance, even though physical contact is not made with the touch display.
  • a touch input may comprise any input that is detected by a touch display including touch events that involve actual physical contact and touch events that do not involve physical contact but that are otherwise detected by the touch display, such as a result of the proximity of the selection object to the touch display.
  • a touch display may be capable of receiving information associated with force applied to the touch screen in relation to the touch input.
  • the touch screen may differentiate between a heavy press touch input and a light press touch input.
  • a display may display two-dimensional information, three-dimensional information and/or the like.
  • the keypad may comprise numeric (for example, 0-9) keys, symbol keys (for example, #, *) , alphabetic keys, and/or the like for operating the electronic apparatus 10.
  • the keypad may comprise a conventional QWERTY keypad arrangement.
  • the keypad may also comprise various soft keys with associated functions. Any keys may be physical keys in which, for example, an electrical connection is physically made or broken, or may be virtual. Virtual keys may be, for example, graphical representations on a touch sensitive surface, whereby the key is actuated by performing a hover or touch gesture on or near the surface.
  • the electronic apparatus 10 may comprise an interface device such as a joystick or other user input interface.
  • the media capturing element may be any means for capturing an image, video, and/or audio for storage, display or transmission.
  • the camera module may comprise a digital camera which may form a digital image file from a captured image.
  • the camera module may comprise hardware, such as a lens or other optical component (s) , and/or software necessary for creating a digital image file from a captured image.
  • the camera module may comprise only the hardware for viewing an image, while a memory device of the electronic apparatus 10 stores instructions for execution by the processor 11 in the form of software for creating a digital image file from a captured image.
  • the camera module may further comprise a processing element such as a co-processor that assists the processor 11 in processing image data and an encoder and/or decoder for compressing and/or decompressing image data.
  • the encoder and/or decoder may encode and/or decode according to a standard format, for example, a Joint Photographic Experts Group (JPEG) standard format, a moving picture expert group (MPEG) standard format, a Video Coding Experts Group (VCEG) standard format or any other suitable standard formats.
  • JPEG Joint Photographic Experts Group
  • MPEG moving picture expert group
  • VCEG Video Coding Experts Group
  • Figure 2 is a flow chart depicting a process 200 of removing object such as haze in the image according to embodiments of the present disclosure, which may be performed at an apparatus such as the electronic apparatus 10 of Figure 1.
  • the electronic apparatus 10 may provide means for accomplishing various parts of the process 200 as well as means for accomplishing other processes in conjunction with other components.
  • the object to be removed in the image may refer to certain object which may influence image quality.
  • the certain object may be an object in bad weather, such as haze, dark, dust, drizzle, fog, smoke, or other possible objects or particles; may be an object caused by an imaging device, for example object caused by turbid medium on lens of the imaging device; may be an object resulting from imaging condition, for example, the image may be captured in an imaging condition where a camera is behind a glass (such as a window glass) on which a lot of particles (such as fog) are located. In these cases, the irradiance received by the camera from the scene point is attenuated along the line of sight.
  • the incoming light is blended with the airlight (ambient light reflected into the line of sight by particles) .
  • the degraded images may lose the contrast and color fidelity. Therefore, certain object removal (such as dehazing) is highly desired in applications based on image.
  • the process 200 may start at block 201 where input intensity of a pixel in an image is determined.
  • the image may be pre-stored in a memory of the electronic apparatus 10, captured in real time by an image sensor, or retrieved from a network location or a local location.
  • the processor 11 may obtain the image from the memory 12 if the image is stored in the memory 12; obtain the image from the input device 13 such as from a removable storage device which has stored the image or from a camera; or obtain the image from a network location by means of the communication device 15.
  • the image may be received from a digital camera arranged in a monitoring location.
  • the image may be a color image or a gray image.
  • the image may be a single image or an image frame in a video.
  • the color image may be represented by any suitable color model, such as RGB color model, HSL color model, CMYK color model, or the like.
  • the image format may include but not limit to bmp, jpg, jpeg, tiff, gif, pcx, tga, exif, fpx, svg, psd, cdr, pcd, dxf, ufo, eps, ai, raw, or the like.
  • I i be the input color image represented by RGB color model.
  • R i (x, y) the red, green, and blue channels at location (x, y) of the input color image I i by R i (x, y) , G i (x, y) , andB i (x, y) , respectively.
  • the intensity L i (x, y) at a location (x, y) on the input color image I i is a quadratic combination of the three color channels:
  • L i (x, y) w r R i (x, y) +w g G i (x, y) +w b B i (x, y) (1)
  • the gray value of a pixel may be used as the input intensity of the pixel.
  • the input intensity may be determined by any suitable approaches.
  • the input intensity for each pixel in the image can be determined at block 201 and used as input in the following operations of method 200.
  • output intensity of the pixel may be determined based on mapped input intensity and mapped detail intensity.
  • a mapping fucntion can be determined, which can map the input intensity to the mapped input intensity
  • another mapping fucntion can be determined, which can determine the detail intensity based on the input intensity and map it to the mapped detail intensity, such that the output intensity determined by the mapped input intensity and the mapped detail intensity is equal to or approach to the intensity of the pixel of a corresponding image without the certain object (such as haze) .
  • mapping fucntions may be learned from a dataset including pairs of images (one with the certain object, and the other one without the certain object) or determined empirically.
  • determination of the output intensity may comprise: at block 212-1, determining mapped input intensity based on the input intensity; at block 212-5, determining mapped detail intensity based on the input intensity; and at block 212-8, determining the output intensity of the pixel based on the mapped input intensity and the mapped detail intensity.
  • the output intensity L o (x, y) may be calculated by multiplying the mapped input intensity L g (x, y) and the mapped detail intensity D (x, y) .
  • the process may be described as follows.
  • the intensity mapping function is g: L i (x, y) ⁇ L g (x, y) . It is noted that many forms of intensity mapping functions may be possible.
  • the intensity mapping function may be a polynomial function g (which is referred as first polynomial function herein) :
  • ⁇ j denote coefficients, and each of the terms comprises respective coefficient ⁇ j (which is referred as first coefficient herein) and the input intensity (L i (x, y) ) j .
  • A may be set as other suitable value for example empirically or by machine learning. The determination of the first coefficients will be discussed in detail in a subsequent section.
  • the image detail may be weaken or eliminated by certain object such as haze or low-light. Therefore, transforming intensity by a simple function g may be not enough for recovering the image detail.
  • the output intensity L o (x, y) may be calculated based on the mapped input intensity I g (x, y) and the mapped detail intensityD (x, y) .
  • the mapped detail intensity D (x, y) may be calculated by any suitable approaches.
  • determination of mapped detail intensityD (x, y) may comprise: at block 212-5-1, smoothing the input intensity to obtain a detail intensity; at block 212-5-5, normalizing the detail intensity; and at block 212-5-8, determining the mapped detail intensity based on the normalized detail intensity by a second polynomial function.
  • the detail intensity L D (x, y) may be computed by subtracting the input intensity L i (x, y) by its smoothed version L s (x, y) at block 212-5-1:
  • the detail intensity L D (x, y) may be normalized to L d (x, y) at block 212-5-5 so that L d (x, y) is within a standard range such as a range of [0, 255] .
  • the mapped detail intensity D (x, y) may be obtained by a function f: L d (x, y) ⁇ D (x, y) at block 212-5-8. It is noted that many forms of functions may be possible.
  • fucntion f is also a polynomial function (which is referred as second polynomial function herein) :
  • ⁇ j denotes coefficients, and each of the terms comprises respective coefficient ⁇ j (which is referred as second coefficient herein) and the normalized detail intensity (L d (x, y) ) j .
  • B may be set as other suitable value for example empirically or by machine learning.
  • removal of different type of object can use different first coefficients and second coefficients. The determination of the second coefficients will be discussed in connection with the first coefficients in a subsequent section.
  • the output intensity of the pixel may be determined based on the mapped input intensity and the mapped detail intensity at block 212-8. It is noted that many forms of functions may be possible, which are used for determining the output intensity based on the mapped input intensity and the mapped detail intensity.
  • the product of the mapped input intensity L g (x, y) and the mapped detail intensity D (x, y) may be used as the output intensity L o (x, y) :
  • the output intensity of the pixel may be used as the gray value of the output pixel. In this way, certain object in the gray image can be removed.
  • the image may be a color image. In this case, individual color channel of the output pixel should be determined. To determine the individual color channel of the output pixel, a transform coefficient may be used to determine the individual color channel of the output pixel.
  • the transform coefficient may be determined based on the input intensity and the output intensity.
  • the transform coefficient c (x, y) may be determined by the following formula:
  • An output pixel associated with the pixel may be determined based on the transform coefficient.
  • the image is a color image for example represented by RGB color model and the product of the transform coefficient and individual color channel of the pixel may be used as individual color channel of the output pixel:
  • R o (x, y) , G o (x, y) , and B o (x, y) are the red, green, and blue channels of the output image I o . It is straightforward that the output intensity L o (x, y) can be obtained by:
  • the saturation of the output pixel may be adjusted.
  • the saturation of the output pixel may be adjusted empirically or based on a learned saturation adjustment function or by other suitable approaches.
  • a saturation mapping function q (S (x, y) ) may be used to correct the saturation S (x, y) of the output pixel.
  • the output image is reprsented by RGB color model where R o (x, y) , G o (x, y) , and B o (x, y) are the red, green, and blue channels of the output image I o , R o (x, y) , G o (x, y) , and B o (x, y) may be transformed into a HSI (Hue, Saturation, Intensity) color space and the resulting components of hue, saturation, and intensity are H (x, y) , S (x, y) , and I (x, y) respectively.
  • the saturation mapping function may be defined as:
  • q (S (x, y) ) may be referred as saturation coefficient.
  • the saturation S (x, y) may be mapped to output saturation S o (x, y) by multiplying q (S (x, y)) :
  • H (x, y) , S o (x, y) , and I (x, y) may be transformed to RGB color space.
  • the output pixel may be normalized such that the individual color channel of the output pixel is within a standard range, such as [0-255] .
  • the normalization can be performed at any stage of method 200.
  • the first coefficients and the second coefficients may be determined based on a minimum-reconstruction-error criterion for learning the optimal parameters.
  • this embodiment will be discribed for example in the context of hazy images.
  • pairs of images for example one image with haze and the other one without haze in a pair of images.
  • the objective function is defined as:
  • Formula (13) may measure error of estimated when it is used for approximating the ground-truth haze-free image
  • ⁇ -step and ⁇ -step may be used for updating and respectively.
  • subscript ‘i’ is used to index pixel (x, y) and omit the summation over k.
  • N is the number of pixels.
  • Subscript ‘i’ is also used to index pixel (x, y) and omit the summation over k.
  • the algorithm iteratively run ⁇ -step and ⁇ -step until convergence. Finally, it can obtain the optimal first coefficients and the second coefficients.
  • Figure 5 shows a flow chart depicting a process of dehazing in accordance with embodiments of the present disclosure.
  • obtaining an input image I i which may be an input color image for example represented by RGB color model.
  • R i (x, y) the red, green, and blue channels at location (x, y) of the input image I i by R i (x, y) , G i (x, y) , andB i (x, y) , respectively.
  • mapped input intensity L g (x, y) may be determined based on the first polynomial function.
  • the input intensity L i (x, y) may be smoothed by a bilateral filter to obtain its smoothed version L s (x, y) .
  • detail intensity L D (x, y) may be computed by subtracting the input intensity L i (x, y) by its smoothed version L s (x, y) .
  • L D (x, y) may be normalized to L d (x, y) so that L d (x, y) is in the range of [0, 255] .
  • the mapped detail intensity D (x, y) may be obtained by the second polynomial function.
  • the product of the mapped input intensity L g (x, y) and the mapped detail intensity D (x, y) may be used as the output intensity L o (x, y) .
  • the dehazing (transform) coefficient c (x, y) may be determined by dividing the output intensity L o (x, y) by the input intensity L i (x, y) .
  • the output pixel may be determined based on the transform coefficient.
  • the saturation of the output pixel may be adjusted by the saturation mapping function.
  • the output pixel may be normalized such that individual color channel of the output pixel is within a standard range, such as [0-255] .
  • Figure 6 shows some results of methods according to the embodiments and DCP.
  • Figure 6 (a) and 6 (d) are two different hazy images which are both color images.
  • Figure 6 (b) and 6 (e) are the results of methods according to the embodiments, and
  • Figure 6 (c) and 6 (f) are the results of DCP. Comparing Figure 6 (b) with Figure 6 (c) , we can see that DCP incorrectly recovers greenish buildings in block 601.
  • Figure 6 (e) and (f) show that DCP is unable to correctly recover the green leafs in block 602. Methods according to the embodiments are thus significantly better than DCP.
  • the proposed methods do not rely on any model (such as imaging or low-light model) whose parameters are difficult to be estimated exactly. It proposes to enhance both intensity and saturation by novel tone mapping functions whose parameters are learned from degraded images and their ground truth. Existing methods do not deal with image saturation with tone mapping. The form of tone mapping functions is different from existing ones and the manner of obtained the parameters of the tone mapping function is also different. The parameters of the proposed tone mapping are optimal whereas those of existing ones are empirically chosen. Both image intensity and saturation are explicitly recovered whereas existing methods merely consider intensity. As shown in Figure 6, the proposed methods are significantly better than existing method such as DCP.
  • an apparatus for removing object in an image may comprise means configured to carry out the processes described above.
  • the apparatus comprises means configured to determine input intensity of a pixel in an image; and means configured to determine output intensity of the pixel based on mapped input intensity and mapped detail intensity.
  • the mapped input intensity is determined based on the input intensity and the mapped detail intensity is determined based on the input intensity.
  • the mapped input intensity is determined based on the input intensity by a first polynomial function.
  • the number of terms of the first polynomial function is 4, and each of the terms comprises respective first coefficient and the input intensity.
  • the mapped detail intensity is determined by: smoothing the input intensity to obtain detail intensity; normalizing the detail intensity; and determining the mapped detail intensity based on the normalized detail intensity by a second polynomial function.
  • the number of terms of the second polynomial function is 3, and each of the terms comprises respective second coefficient and the normalized detail intensity.
  • the apparatus may further comprise means configured to determine the first coefficients and the second coefficients based on a minimum-reconstruction-error criterion.
  • determination of the output intensity of the pixel based on the mapped input intensity and the mapped detail intensity comprises: using the product of the mapped input intensity and the mapped detail intensity as the output intensity.
  • the image is a color image
  • the apparatus may further comprise means configured to determine a transform coefficient based on the input intensity and the output intensity; and means configured to determine an output pixel associated with the pixel based on the transform coefficient.
  • determination of the transform coefficient based on the input intensity and the output intensity comprises: using the result of dividing the input intensity by the output intensity as the transform coefficient.
  • determination of the output pixel associated with the pixel based on the transform coefficient comprises: using the product of the transform coefficient and individual color channel of the pixel as individual color channel of the output pixel.
  • the apparatus may further comprise means configured to adjust the saturation of the output pixel
  • the apparatus may further comprise means configured to normalize the output pixel.
  • any of the components of the apparatus described above can be implemented as hardware or software modules.
  • software modules they can be embodied on a tangible computer-readable recordable storage medium. All of the software modules (or any subset thereof) can be on the same medium, or each can be on a different medium, for example.
  • the software modules can run, for example, on a hardware processor. The method steps can then be carried out using the distinct software modules, as described above, executing on a hardware processor.
  • an aspect of the disclosure can make use of software running on a general purpose computer or workstation.
  • a general purpose computer or workstation Such an implementation might employ, for example, a processor, a memory, and an input/output interface formed, for example, by a display and a keyboard.
  • the term “processor” as used herein is intended to include any processing device, such as, for example, one that includes a CPU (central processing unit) and/or other forms of processing circuitry. Further, the term “processor” may refer to more than one individual processor.
  • memory is intended to include memory associated with a processor or CPU, such as, for example, RAM (random access memory) , ROM (read only memory) , a fixed memory device (for example, hard drive) , a removable memory device (for example, diskette) , a flash memory and the like.
  • the processor, memory, and input/output interface such as display and keyboard can be interconnected, for example, via bus as part of a data processing unit. Suitable interconnections, for example via bus, can also be provided to a network interface, such as a network card, which can be provided to interface with a computer network, and to a media interface, such as a diskette or CD-ROM drive, which can be provided to interface with media.
  • computer software including instructions or code for performing the methodologies of the disclosure, as described herein, may be stored in associated memory devices (for example, ROM, fixed or removable memory) and, when ready to be utilized, loaded in part or in whole (for example, into RAM) and implemented by a CPU.
  • Such software could include, but is not limited to, firmware, resident software, microcode, and the like.
  • aspects of the disclosure may take the form of a computer program product embodied in a computer readable medium having computer readable program code embodied thereon.
  • computer readable media may be a computer readable signal medium or a computer readable storage medium.
  • a computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
  • a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Computer program code for carrying out operations for aspects of the disclosure may be written in any combination of at least one programming language, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • each block in the flowchart or block diagrams may represent a module, component, segment, or portion of code, which comprises at least one executable instruction for implementing the specified logical function (s) .
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • connection or coupling means any connection or coupling, either direct or indirect, between two or more elements, and may encompass the presence of one or more intermediate elements between two elements that are “connected” or “coupled” together.
  • the coupling or connection between the elements can be physical, logical, or a combination thereof.
  • two elements may be considered to be “connected” or “coupled” together by the use of one or more wires, cables and/or printed electrical connections, as well as by the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the radio frequency region, the microwave region and the optical region (both visible and invisible) , as several non-limiting and non-exhaustive examples.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne un appareil, un procédé, un produit-programme informatique et un support lisible par ordinateur permettant d'éliminer un objet dans une image. L'appareil comprend au moins un processeur; au moins une mémoire comportant un code de programme informatique, la mémoire et le code de programme informatique étant configurés, en fonctionnement avec ledit processeur, pour amener l'appareil à déterminer l'intensité d'entrée d'un pixel dans une image; et pour déterminer l'intensité de sortie du pixel sur la base de l'intensité d'entrée mappée et de l'intensité de détail mappée.
PCT/CN2016/085680 2016-06-14 2016-06-14 Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image WO2017214848A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/085680 WO2017214848A1 (fr) 2016-06-14 2016-06-14 Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/085680 WO2017214848A1 (fr) 2016-06-14 2016-06-14 Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image

Publications (1)

Publication Number Publication Date
WO2017214848A1 true WO2017214848A1 (fr) 2017-12-21

Family

ID=60662913

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/085680 WO2017214848A1 (fr) 2016-06-14 2016-06-14 Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image

Country Status (1)

Country Link
WO (1) WO2017214848A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070211049A1 (en) * 2006-03-08 2007-09-13 Sharp Laboratories Of America, Inc. Methods and systems for enhancing display characteristics with ambient illumination input
US20090184915A1 (en) * 2008-01-21 2009-07-23 National Taiwan University Low-backlight image visibility enhancement method and system
CN102231264A (zh) * 2011-06-28 2011-11-02 王洪剑 动态对比度增强装置和方法
CN103455979A (zh) * 2013-07-16 2013-12-18 大连理工大学 一种低照度视频图像增强方法
CN104882097A (zh) * 2015-06-08 2015-09-02 西安电子科技大学 基于环境光的图像显示方法及***
CN104902141A (zh) * 2015-06-08 2015-09-09 西安诺瓦电子科技有限公司 图像处理方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070211049A1 (en) * 2006-03-08 2007-09-13 Sharp Laboratories Of America, Inc. Methods and systems for enhancing display characteristics with ambient illumination input
US20090184915A1 (en) * 2008-01-21 2009-07-23 National Taiwan University Low-backlight image visibility enhancement method and system
CN102231264A (zh) * 2011-06-28 2011-11-02 王洪剑 动态对比度增强装置和方法
CN103455979A (zh) * 2013-07-16 2013-12-18 大连理工大学 一种低照度视频图像增强方法
CN104882097A (zh) * 2015-06-08 2015-09-02 西安电子科技大学 基于环境光的图像显示方法及***
CN104902141A (zh) * 2015-06-08 2015-09-09 西安诺瓦电子科技有限公司 图像处理方法及装置

Similar Documents

Publication Publication Date Title
CN107301624B (zh) 基于区域划分及浓雾预处理的卷积神经网络去雾方法
KR101756173B1 (ko) 전달률 하한치의 변형에 의한 안개영상 개선 시스템 및 그 방법
Li et al. Single image dehazing using the change of detail prior
US10970824B2 (en) Method and apparatus for removing turbid objects in an image
US20160154469A1 (en) Mid-air gesture input method and apparatus
Fu et al. Improved single image dehazing using dark channel prior
US20110299774A1 (en) Method and system for detecting and tracking hands in an image
US11800076B2 (en) Spatially varying reduction of haze in images
Kil et al. A dehazing algorithm using dark channel prior and contrast enhancement
KR20150130767A (ko) 사용자의 손 영역을 검출하기 위한 사용자 손 검출 장치 및 그 동작 방법
Yang et al. Coarse-to-fine luminance estimation for low-light image enhancement in maritime video surveillance
CN110298796B (zh) 基于改进Retinex与对数图像处理的低照度图像增强方法
US20140286535A1 (en) Methods and Apparatuses for Gesture Recognition
WO2018132961A1 (fr) Appareil, procédé et produit-programme d'ordinateur pour une détection d'objet
CN111192205A (zh) 一种图像去雾方法、***和计算机可读存储介质
Yu et al. Image and video dehazing using view-based cluster segmentation
Ling et al. Perception oriented transmission estimation for high quality image dehazing
Zhang et al. Image dehazing based on dark channel prior and brightness enhancement for agricultural remote sensing images from consumer-grade cameras
Chen et al. Improve transmission by designing filters for image dehazing
CN110991325A (zh) 一种模型训练的方法、图像识别的方法以及相关装置
US11605220B2 (en) Systems and methods for video surveillance
CN107424134B (zh) 图像处理方法、装置、计算机可读存储介质和计算机设备
CN112565674A (zh) 一种可远程视频监控和控制的展厅中控***
CN111062272A (zh) 基于色彩恢复的图像处理、行人识别方法、装置与可读存储介质
WO2017214848A1 (fr) Appareil, procédé et produit de programme informatique permettant d'éliminer un objet dans une image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16904966

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16904966

Country of ref document: EP

Kind code of ref document: A1