CN111131718A - Multi-exposure image fusion method and system with LED flicker compensation function - Google Patents

Multi-exposure image fusion method and system with LED flicker compensation function Download PDF

Info

Publication number
CN111131718A
CN111131718A CN201910638889.9A CN201910638889A CN111131718A CN 111131718 A CN111131718 A CN 111131718A CN 201910638889 A CN201910638889 A CN 201910638889A CN 111131718 A CN111131718 A CN 111131718A
Authority
CN
China
Prior art keywords
fusion
image
led flicker
unit
exposure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910638889.9A
Other languages
Chinese (zh)
Other versions
CN111131718B (en
Inventor
不公告发明人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Iwaysense Intelligent Co ltd
Original Assignee
Shenzhen Iwaysense Intelligent Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Iwaysense Intelligent Co ltd filed Critical Shenzhen Iwaysense Intelligent Co ltd
Priority to CN201910638889.9A priority Critical patent/CN111131718B/en
Publication of CN111131718A publication Critical patent/CN111131718A/en
Application granted granted Critical
Publication of CN111131718B publication Critical patent/CN111131718B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Picture Signal Circuits (AREA)

Abstract

The invention discloses a multi-exposure image fusion method and a system with an LED flicker compensation function, wherein the system comprises the following steps: the image sensor comprises an image sensor unit, an interface unit, a line buffer unit, a fusion unit and an output unit; the method and the system do not need an LFM image sensor, use a conventional image sensor, compensate LED flicker by fusion processing, improve image real-time performance, reduce hardware cost, only need a small amount of line cache in the processing process, directly process Bayer format data, reduce system delay, reduce storage overhead, and can be conveniently expanded to various programmable devices and special integrated circuits.

Description

Multi-exposure image fusion method and system with LED flicker compensation function
Technical Field
The invention discloses a multi-exposure image fusion method and a multi-exposure image fusion system with an LED flicker compensation function, and relates to the technical field of image processing.
Background
Because of the characteristics of energy conservation, environmental protection, safety, low power consumption, high brightness and the like, light emitting diodes (hereinafter referred to as LEDs) are widely applied to the fields of various daily illuminations, traffic indicator lamps, automobile headlamps, road illuminations and the like. The LED dimming uses a pulse width modulation mode, and the LED light emission in a daily scene has periodicity, namely, flicker, with the period being
Figure DEST_PATH_IMAGE001
To
Figure DEST_PATH_IMAGE002
The seconds are different, and no uniform standard is formed in all parts of the world at present. The human eye is hardly affected by the flickering of the LED due to the characteristic of persistence of vision.
In human daily life scenes, for example, when a high-brightness automobile LED headlamp appears at extremely dark nights, the brightness change range of different areas of the same scene exceeds 8 orders of magnitude, namely the dynamic range is larger than 160 dB. Human eyes can adapt to brightness change of more than 6 orders of magnitude due to long-term evolution, namely, the dynamic range is higher than 120 dB. The single exposure of the image sensor can only provide a dynamic range of about 70dB, and the requirement of reflecting and recording a high-dynamic scene in the real world can not be met far, so that a dynamic range of 100-144 dB can be provided by a multi-exposure method, a dark area of light in a scene is recorded by long exposure, a strong area of light in the scene is recorded by short exposure, and brightness information in the real world scene is reflected and recorded better.
In the prior art, in order to provide a dynamic range not inferior to that of human eyes, a conventional image sensor performs multiple exposures in a line-alternating manner at the same time, and the longest exposure time can be reached
Figure 883099DEST_PATH_IMAGE001
Second, the shortest exposure time can be
Figure DEST_PATH_IMAGE003
The short exposure time is far shorter than the LED light-emitting period, so that the flickering and the brightness rolling of the LED illumination area can be seen in the fused image, on one hand, a user feels uncomfortable, on the other hand, the real states of a traffic signal lamp and other LED illumination areas cannot be correctly reflected and recorded, and the application of automobile safety, automatic driving and the like is threatened fatally. To improve the LED flicker problem, the shorter exposure time is generally broken up and distributed uniformly over a longer period by improving the exposure timing of the image sensor, so as to reduce the LED flicker effect, which is called LED flicker reduction (LFM).
In the prior art, the line alternation and multiple exposure fusion method has the defects that the line alternation and multiple exposure fusion method is interfered by LED flicker with uncertain periods, so that visual discomfort of a user, identification errors of traffic lights and the like are caused.
In the prior art, the LFM has the disadvantages of prolonging the operation time of each exposure, increasing the time delay between multiple exposures, and finally causing the reduction of image real-time performance and dynamic range, etc.
In the prior art, the LFM has another disadvantage that the operation time of a single exposure is prolonged, and in order to increase the dynamic range by using multiple exposures, the image data which must be cached by a processing system is greatly increased, which causes the increase of the system overhead and the cost.
Disclosure of Invention
The invention aims to solve the technical problem of providing a multi-exposure image fusion method and system with LED flicker compensation function, which has low complexity, small operand, low cost and is suitable for a conventional image sensor.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a multi-exposure image fusion method with an LED flicker compensation function comprises the following steps:
s1: two sets of raw image data are acquired from an input in units of lines
Figure DEST_PATH_IMAGE004
And
Figure DEST_PATH_IMAGE005
；
s2: from input raw data
Figure 924873DEST_PATH_IMAGE004
And
Figure 766927DEST_PATH_IMAGE005
extracting luminance information therefrom
Figure DEST_PATH_IMAGE006
And
Figure DEST_PATH_IMAGE007
(ii) a The input image data can be from a line cache unit and also can be from the output of a previous-level fusion unit;
s3: according to the brightness information
Figure 286770DEST_PATH_IMAGE006
Calculating
Figure 101142DEST_PATH_IMAGE005
The fusion weight coefficient of each pixel
Figure DEST_PATH_IMAGE008
；
S4: according to the brightness information
Figure 45309DEST_PATH_IMAGE006
And
Figure 312342DEST_PATH_IMAGE007
calculating a reference value reflecting the severity of LED flicker occurring at each location
Figure DEST_PATH_IMAGE009
；
S5: according to reference values
Figure 573559DEST_PATH_IMAGE009
Calculating the LED flicker compensation coefficient
Figure DEST_PATH_IMAGE010
：
S6: according to the fusion weight coefficient
Figure 367072DEST_PATH_IMAGE008
And LED flicker compensation factor
Figure 360436DEST_PATH_IMAGE010
Calculating a fusion value of each pixel
Figure DEST_PATH_IMAGE011
And obtaining a final output image.
Preferably, in S2, the method specifically includes the following substeps:
s11: denote a 3 × 3 convolution kernel by h;
S12：
Figure 177082DEST_PATH_IMAGE004
and
Figure 976411DEST_PATH_IMAGE005
each being a plane of pixels arranged periodically in a Bayer (2x2) format;
s13: using said convolution kernel
Figure DEST_PATH_IMAGE012
To the above
Figure 562113DEST_PATH_IMAGE004
And
Figure 54274DEST_PATH_IMAGE005
performing a convolution operation, i.e.
Figure DEST_PATH_IMAGE013
Figure DEST_PATH_IMAGE014
Deriving structural luminance information
Figure 423463DEST_PATH_IMAGE006
And
Figure 698586DEST_PATH_IMAGE007
。
preferably, the division operation in the convolution operation described in S13 realizes fast calculation using a shift operation, that is
Figure DEST_PATH_IMAGE015
Figure DEST_PATH_IMAGE016
；
Figure DEST_PATH_IMAGE017
。
Preferably, in S3, the method specifically includes the following sub-steps:
s31: by using
Figure DEST_PATH_IMAGE018
Coordinates representing any element in the image and luminance value data;
s32: by using
Figure DEST_PATH_IMAGE019
Representing an input image
Figure 591325DEST_PATH_IMAGE004
The brightness value of the current position pixel;
s33: constructing a plane rectangular coordinate system, wherein the horizontal axis is brightness, and the vertical axis is a fusion weight coefficient;
s34: obtaining abscissa of piecewise polyline equation from system configuration
Figure DEST_PATH_IMAGE020
、
Figure DEST_PATH_IMAGE021
、
Figure DEST_PATH_IMAGE022
、
Figure DEST_PATH_IMAGE023
、
Figure DEST_PATH_IMAGE024
；
S35: obtaining the ordinate of a piecewise polyline equation from a system configuration
Figure DEST_PATH_IMAGE025
、
Figure DEST_PATH_IMAGE026
、
Figure DEST_PATH_IMAGE027
；
S36: using said abscissa
Figure 133401DEST_PATH_IMAGE020
、
Figure 924639DEST_PATH_IMAGE021
、
Figure 3453DEST_PATH_IMAGE022
、
Figure 563748DEST_PATH_IMAGE023
、
Figure 397712DEST_PATH_IMAGE024
And ordinate
Figure 348350DEST_PATH_IMAGE025
、
Figure 293172DEST_PATH_IMAGE026
、
Figure 710903DEST_PATH_IMAGE027
Constructing a piecewise polyline equation for said
Figure 653451DEST_PATH_IMAGE006
Coefficient mapping is carried out, i.e.
Figure DEST_PATH_IMAGE028
Computing an input image
Figure 153703DEST_PATH_IMAGE005
Fusion weight coefficient of current position
Figure 636636DEST_PATH_IMAGE008
。
Preferably, wherein the reference value in S4
Figure 171523DEST_PATH_IMAGE009
One calculation method of (2) is as follows:
obtaining current effective exposure time length from system configuration
Figure DEST_PATH_IMAGE029
And
Figure DEST_PATH_IMAGE030
ratio of
Figure DEST_PATH_IMAGE031
Calculating a reference value
Figure 409606DEST_PATH_IMAGE009
，
Figure DEST_PATH_IMAGE032
。
Preferably, in S5, the method specifically includes the following sub-steps:
s51: obtaining threshold value for starting to compensate LED flicker from system configuration
Figure DEST_PATH_IMAGE033
；
S52: obtaining threshold values for fully compensating LED flicker from system configuration
Figure DEST_PATH_IMAGE034
；
S53: calculated by the formula, i.e.
Figure DEST_PATH_IMAGE035
Obtaining the LED flicker compensation coefficient of the current position
Figure 581175DEST_PATH_IMAGE010
。
Preferably, in S6, the method specifically includes the following sub-steps:
s61: obtaining current effective exposure time length from system configuration
Figure 867800DEST_PATH_IMAGE029
And
Figure 929297DEST_PATH_IMAGE030
ratio of
Figure 275964DEST_PATH_IMAGE031
；
S62: by using
Figure 750808DEST_PATH_IMAGE004
、
Figure 778807DEST_PATH_IMAGE005
、
Figure 22706DEST_PATH_IMAGE008
、
Figure 274696DEST_PATH_IMAGE010
、
Figure 236836DEST_PATH_IMAGE031
Calculating a fusion value of a current position pixel
Figure 802947DEST_PATH_IMAGE011
I.e. by
Figure DEST_PATH_IMAGE036
。
A multi-exposure image fusion system with LED flicker compensation, which is described below by taking only 4 exposures as an example, the system comprising: an image sensor unit, an interface unit, line buffer units (1 to 4), a fusion unit (1 to 3), and an output unit;
an image sensor unit for generating raw data of 4 different exposure time lengths in a line-alternating manner, the exposure time lengths being respectively used
Figure DEST_PATH_IMAGE037
、
Figure DEST_PATH_IMAGE038
、
Figure DEST_PATH_IMAGE039
、
Figure DEST_PATH_IMAGE040
Represents;
the interface unit is used for connecting the image sensor unit, separating the line alternation image data generated by the image sensor unit and inputting the line alternation image data into the system;
line buffer units (1 to 4) for buffering and aligning 4 sets of exposure image data generated by the image sensor, respectively
Figure DEST_PATH_IMAGE041
、
Figure DEST_PATH_IMAGE042
、
Figure DEST_PATH_IMAGE043
、
Figure DEST_PATH_IMAGE044
Represents;
fusion unit 1 for detection and compensation
Figure 543763DEST_PATH_IMAGE042
LED flicker in (1), fusion
Figure 966654DEST_PATH_IMAGE041
And
Figure 413160DEST_PATH_IMAGE042
image data, output fusion image
Figure DEST_PATH_IMAGE045
；
A fusion unit 2 for detecting and compensating
Figure 845279DEST_PATH_IMAGE043
The LED in (1) flickers and is fused
Figure 798191DEST_PATH_IMAGE045
And
Figure 657563DEST_PATH_IMAGE043
image data, output fusion image
Figure DEST_PATH_IMAGE046
；
A fusion unit 3 for detecting and compensating
Figure 328715DEST_PATH_IMAGE044
The LED in (1) flickers and is fused
Figure 564525DEST_PATH_IMAGE046
And
Figure 637523DEST_PATH_IMAGE044
image data, output fusion image
Figure DEST_PATH_IMAGE047
；
An output unit for fusing the final image according to a certain interface format
Figure 667796DEST_PATH_IMAGE047
And outputting to an external or next-stage processing unit.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The present invention will be described in detail below with reference to the accompanying drawings so that the above advantages of the present invention will be more apparent. Wherein the content of the first and second substances,
FIG. 1 is a block diagram of a pipeline of a multiple exposure image fusion system with LED flicker compensation according to the present invention.
FIG. 2 is a pipeline diagram of the multiple exposure image fusion method with LED flicker compensation according to the present invention.
Fig. 3 is a schematic diagram illustrating the principle of extracting luminance information by performing a convolution operation of 3 × 3 on a Bayer format according to the present invention.
FIG. 4 is a schematic diagram illustrating the principle of piecewise polygonal line mapping in the process of calculating the fusion coefficient by the multi-exposure image fusion method with the LED flicker compensation function according to the present invention.
Detailed Description
The following detailed description of the embodiments of the present invention will be provided with reference to the drawings and examples, so that how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented. It should be noted that, as long as there is no conflict, the embodiments and the features of the embodiments of the present invention may be combined with each other, and the technical solutions formed are within the scope of the present invention.
As shown in fig. 1-4, wherein,
a multiple exposure image fusion system with LED flicker compensation, which is described by taking a 4-exposure image fusion system as an example, the system comprising: the system comprises an image sensor unit, a pipeline structure consisting of an interface unit, line buffer units (1 to 4), fusion units (1 to 3) and an output unit, wherein the image sensor unit generates 4 kinds of original image data with different exposure durations according to the flow sequence of the data, the original image data passes through the interface unit in a line-alternating mode, is separated and flows into the line buffer units (1 to 4), further flows into the fusion units (1 to 3), and finally flows out of the system through the output unit; all or part (at least including fusion units 1 to 3) of the units except the image sensor unit are realized in the form of a programmable device, wherein one expression is to run software on a certain general processing chip (CPU or DSP), the other expression is to realize a special image processing pipeline on a Field Programmable Gate Array (FPGA), and the other expression is to realize the special image processing pipeline by using an application specific integrated circuit chip (ASIC) or a system on a chip (SoC); the interface unit is used for realizing a communication protocol with the image sensor unit, establishing a data transmission channel between the image sensor and the system, and separating the line alternation image data generated by the data transmission channel into 4 groups of exposure image data to be input into the system; a line buffer unit (1 to 4) for buffering and aligning 4 sets of exposure image data generated by the image sensor; and the fusion units (1 to 3) are used for detecting and compensating LED flicker in each group of short exposure image data in a cascading mode, fusing each group of image data to obtain a final fusion image, and outputting the final fusion image to an external or next-stage processing unit through the output unit according to a certain interface format.
The multi-exposure image fusion method with the LED flicker compensation function comprises the following steps:
taking the exposure duration from the input as a unit of line
Figure 829174DEST_PATH_IMAGE029
Image data of (1) by
Figure 603095DEST_PATH_IMAGE004
Means that the exposure time length is obtained as
Figure 468283DEST_PATH_IMAGE030
Image data of (1) by
Figure 669457DEST_PATH_IMAGE005
Is shown in which
Figure DEST_PATH_IMAGE048
；
The input image data can be from a line cache unit and also can be from the output of a previous-level fusion unit;
extracting brightness information from the input original data according to a certain algorithm;
preferably, a calculation method for extracting luminance information from the input raw data is as follows:
denote a 3 × 3 convolution kernel by h;
preferably, the first and second electrodes are formed of a metal,
Figure 315202DEST_PATH_IMAGE004
and
Figure 627235DEST_PATH_IMAGE005
each being a plane of pixels arranged periodically in a Bayer (2x2) format;
further, using the convolution kernel
Figure 674825DEST_PATH_IMAGE012
To the above
Figure 46901DEST_PATH_IMAGE004
And
Figure 117625DEST_PATH_IMAGE005
performing a convolution operation, i.e.
Figure 233349DEST_PATH_IMAGE013
Figure 132516DEST_PATH_IMAGE014
Deriving structural luminance information
Figure 613176DEST_PATH_IMAGE006
And
Figure 967934DEST_PATH_IMAGE007
。
preferably, the division operation in the convolution operation uses a shift operation to achieve fast calculation, i.e.
Figure RE-DEST_PATH_IMAGE030A
Figure RE-DEST_PATH_IMAGE098
。
Further, using the luminance information
Figure 706268DEST_PATH_IMAGE006
And a set of parameters obtained from the system configuration, calculating
Figure 420147DEST_PATH_IMAGE005
The fusion weight coefficient of each pixel in the image;
preferably, one method for calculating the fusion weight coefficient is as follows:
by using
Figure 527780DEST_PATH_IMAGE018
Coordinates representing any element in the image and luminance value data;
by using
Figure 922989DEST_PATH_IMAGE019
Representing an input image
Figure 534099DEST_PATH_IMAGE004
The brightness value of the current position pixel;
constructing a plane rectangular coordinate system, wherein the horizontal axis is brightness, and the vertical axis is a fusion weight coefficient;
obtaining abscissa of piecewise polyline equation from system configuration
Figure 687387DEST_PATH_IMAGE020
、
Figure 16738DEST_PATH_IMAGE021
、
Figure 215638DEST_PATH_IMAGE022
、
Figure 681254DEST_PATH_IMAGE023
、
Figure 2514DEST_PATH_IMAGE024
；
Obtaining the ordinate of a piecewise polyline equation from a system configuration
Figure DEST_PATH_IMAGE051
、
Figure DEST_PATH_IMAGE052
、
Figure DEST_PATH_IMAGE053
；
Further, using said abscissa
Figure 943794DEST_PATH_IMAGE020
、
Figure 680806DEST_PATH_IMAGE021
、
Figure 266508DEST_PATH_IMAGE022
、
Figure 779177DEST_PATH_IMAGE023
、
Figure 817540DEST_PATH_IMAGE024
And ordinate
Figure 358243DEST_PATH_IMAGE025
、
Figure 64031DEST_PATH_IMAGE026
、
Figure 461514DEST_PATH_IMAGE027
Constructing a piecewise polyline equation for said
Figure 252753DEST_PATH_IMAGE006
Coefficient mapping is carried out, i.e.
Figure 393884DEST_PATH_IMAGE028
Computing an input image
Figure 954178DEST_PATH_IMAGE005
Fusion weight coefficient of current position
Figure 460246DEST_PATH_IMAGE008
。
Further, using the luminance information
Figure 738781DEST_PATH_IMAGE006
And
Figure 686532DEST_PATH_IMAGE007
and calculating a reference value reflecting the severity of the LED flicker at the current position:
preferably, one method of calculating the reference value is as follows:
obtaining current effective exposure time length from system configuration
Figure 101333DEST_PATH_IMAGE029
And
Figure 43882DEST_PATH_IMAGE030
ratio of
Figure 544133DEST_PATH_IMAGE031
,
Figure DEST_PATH_IMAGE054
。
Further, a group of parameters are obtained from the system configuration, and the LED flicker compensation coefficient of the current position is calculated as follows:
preferably, one method of calculating the flicker compensation coefficient is as follows:
obtaining threshold value for starting to compensate LED flicker from system configuration
Figure DEST_PATH_IMAGE055
；
Obtaining threshold values for fully compensating LED flicker from system configuration
Figure DEST_PATH_IMAGE056
；
Calculated by the formula, i.e.
Figure 151701DEST_PATH_IMAGE035
Obtaining the LED flicker compensation coefficient of the current position
Figure 686587DEST_PATH_IMAGE010
。
Further, calculating a fusion value of the pixel at the current position according to the fusion weight coefficient, the LED flicker compensation coefficient and a group of parameters obtained from system configuration;
preferably, one method of calculating the fusion value is as follows:
obtaining current effective exposure time length from system configuration
Figure 596774DEST_PATH_IMAGE029
And
Figure 581392DEST_PATH_IMAGE030
ratio of
Figure 805700DEST_PATH_IMAGE031
,
Use of
Figure 195093DEST_PATH_IMAGE004
、
Figure 541761DEST_PATH_IMAGE005
、
Figure DEST_PATH_IMAGE057
、
Figure DEST_PATH_IMAGE058
、
Figure 78922DEST_PATH_IMAGE031
Calculating a fusion value of a current position pixel
Figure 903658DEST_PATH_IMAGE011
I.e. by
Figure DEST_PATH_IMAGE059
。
In the invention, the image generation algorithm with high dynamic range can be conveniently realized in various integrated circuit forms, including ASIC, FPGA and the like, the whole system has good expansibility, is convenient for integrating other image algorithms, and can ensure the real-time operation of processing.
In the embodiment of the present invention, the embodiment based on an integrated circuit is implemented by taking an FPGA as an example, and those skilled in the art can easily extend the embodiment to other integrated circuits to implement the embodiment.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (8)

1. A multi-exposure image fusion method with an LED flicker compensation function is characterized by comprising the following steps:
s1: two sets of raw image data are acquired from an input in units of lines
Figure 583283DEST_PATH_IMAGE001
And
Figure 168330DEST_PATH_IMAGE002
；
s2: from input raw data
Figure 512723DEST_PATH_IMAGE001
And
Figure 73018DEST_PATH_IMAGE002
extracting luminance information therefrom
Figure 579085DEST_PATH_IMAGE003
And
Figure 857620DEST_PATH_IMAGE004
；
s3: according to the brightness information
Figure 802442DEST_PATH_IMAGE003
Calculating
Figure 217243DEST_PATH_IMAGE002
The fusion weight coefficient of each pixel
Figure 894212DEST_PATH_IMAGE005
；
S4: according to the brightness information
Figure 456780DEST_PATH_IMAGE003
And
Figure 208223DEST_PATH_IMAGE004
calculating a reference value reflecting the severity of LED flicker occurring at each location
Figure 415214DEST_PATH_IMAGE006
；
S5: according to reference values
Figure 590980DEST_PATH_IMAGE006
Calculating the LED flicker compensation coefficient
Figure 578528DEST_PATH_IMAGE007
：
S6: according to the fusion weight coefficient
Figure 865152DEST_PATH_IMAGE005
And LED flicker compensation factor
Figure 192228DEST_PATH_IMAGE007
Calculating a fusion value of each pixel
Figure 538896DEST_PATH_IMAGE008
And obtaining a final output image.
2. The multi-exposure image fusion method with the LED flicker compensation function according to claim 1, wherein in S2, the method specifically comprises the following sub-steps:
s11: denote a 3 × 3 convolution kernel by h;
S12：
Figure 748161DEST_PATH_IMAGE001
and
Figure 838476DEST_PATH_IMAGE002
each being a plane of pixels arranged periodically in a Bayer (2x2) format;
s13: using said convolution kernel
Figure 20059DEST_PATH_IMAGE009
To the above
Figure 537628DEST_PATH_IMAGE001
And
Figure 496838DEST_PATH_IMAGE002
performing a convolution operation, i.e.
Figure 125266DEST_PATH_IMAGE010
Figure 223672DEST_PATH_IMAGE011
Deriving structural luminance information
Figure 849825DEST_PATH_IMAGE003
And
Figure 33682DEST_PATH_IMAGE004
。
3. the multi-exposure image fusion method with LED flicker compensation function according to claim 2, wherein the division operation in the convolution operation in S13 uses a shift operation to realize fast calculation, i.e. the method is characterized in that
Figure 465800DEST_PATH_IMAGE012
Figure 684292DEST_PATH_IMAGE013
；
Figure 481346DEST_PATH_IMAGE014
。
4. The multi-exposure image fusion method with the LED flicker compensation function according to claim 1, wherein in S3, the method specifically comprises the following sub-steps:
s31: by using
Figure 152499DEST_PATH_IMAGE015
Coordinates representing any element in the image and luminance value data;
s32: by using
Figure 122729DEST_PATH_IMAGE016
Representing an input image
Figure 198657DEST_PATH_IMAGE001
The brightness value of the current position pixel;
s33: constructing a plane rectangular coordinate system, wherein the horizontal axis is brightness, and the vertical axis is a fusion weight coefficient;
s34: obtaining abscissa of piecewise polyline equation from system configuration
Figure 166613DEST_PATH_IMAGE017
、
Figure 325062DEST_PATH_IMAGE018
、
Figure 98983DEST_PATH_IMAGE019
、
Figure 292067DEST_PATH_IMAGE020
、
Figure 430924DEST_PATH_IMAGE021
；
S35: obtaining the ordinate of a piecewise polyline equation from a system configuration
Figure 811090DEST_PATH_IMAGE022
、
Figure 388702DEST_PATH_IMAGE023
、
Figure 436292DEST_PATH_IMAGE024
；
S36: using said abscissa
Figure 808368DEST_PATH_IMAGE017
、
Figure 613513DEST_PATH_IMAGE018
、
Figure 726307DEST_PATH_IMAGE019
、
Figure 628404DEST_PATH_IMAGE020
、
Figure 171380DEST_PATH_IMAGE021
And ordinate
Figure 791718DEST_PATH_IMAGE022
、
Figure 383236DEST_PATH_IMAGE023
、
Figure 405419DEST_PATH_IMAGE024
Constructing a piecewise polyline equation for said
Figure 119297DEST_PATH_IMAGE003
Coefficient mapping is carried out, i.e.
Figure 961351DEST_PATH_IMAGE025
Computing an input image
Figure 356560DEST_PATH_IMAGE002
Fusion weight coefficient of current position
Figure 233249DEST_PATH_IMAGE005
。
5. The multi-exposure image fusion method with LED flicker compensation function according to claim 1, wherein the reference value in S4
Figure 120958DEST_PATH_IMAGE006
One calculation method of (2) is as follows:
obtaining current effective exposure time length from system configuration
Figure 450308DEST_PATH_IMAGE026
And
Figure 649209DEST_PATH_IMAGE027
ratio of
Figure 380404DEST_PATH_IMAGE028
Calculating a reference value
Figure 436085DEST_PATH_IMAGE006
，
Figure 252731DEST_PATH_IMAGE029
。
6. The multi-exposure image fusion method with LED flicker compensation function according to claim 1, wherein the reference value in S4
Figure 255322DEST_PATH_IMAGE006
One calculation method of (2) is as follows:
obtaining current effective exposure time length from system configuration
Figure 841024DEST_PATH_IMAGE026
And
Figure 67606DEST_PATH_IMAGE027
ratio of
Figure 371549DEST_PATH_IMAGE028
Calculating a reference value
Figure 807576DEST_PATH_IMAGE006
，
Figure 247785DEST_PATH_IMAGE029
。
7. The multi-exposure image fusion method with the LED flicker compensation function according to claim 1, wherein in S6, the method specifically comprises the following sub-steps:
s61: obtaining current effective exposure time length from system configuration
Figure 676542DEST_PATH_IMAGE001
And
Figure 65935DEST_PATH_IMAGE002
ratio of
Figure 84707DEST_PATH_IMAGE003
；
S62: by using
Figure 559551DEST_PATH_IMAGE004
、
Figure 649866DEST_PATH_IMAGE005
、
Figure 893766DEST_PATH_IMAGE006
、
Figure 349018DEST_PATH_IMAGE007
、
Figure 45579DEST_PATH_IMAGE003
Calculating a fusion value of a current position pixel
Figure 674006DEST_PATH_IMAGE008
I.e. by
Figure 40921DEST_PATH_IMAGE009
。
8. A multi-exposure image fusion system with LED flicker compensation, comprising: the device comprises an image sensor unit, an interface unit, a line cache unit, a fusion unit and an output unit;
the image sensor unit is used for generating a plurality of groups of Bayer raw data with different exposure time lengths in a line-alternating mode;
the interface unit is connected with the image sensor unit, and is used for separating data generated by the image sensor and inputting the data into the system;
the line buffer unit is used for buffering and aligning a plurality of groups of exposure image data generated by the image sensor;
the fusion unit is used for detecting and compensating LED flicker in the shorter exposure image data, fusing each group of image data and outputting a fused image;
and the output unit is used for outputting the final fusion image to an external or next-stage processing unit.
CN201910638889.9A 2019-07-16 2019-07-16 Multi-exposure image fusion method and system with LED flicker compensation function Active CN111131718B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910638889.9A CN111131718B (en) 2019-07-16 2019-07-16 Multi-exposure image fusion method and system with LED flicker compensation function

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910638889.9A CN111131718B (en) 2019-07-16 2019-07-16 Multi-exposure image fusion method and system with LED flicker compensation function

Publications (2)

Publication Number Publication Date
CN111131718A true CN111131718A (en) 2020-05-08
CN111131718B CN111131718B (en) 2021-05-14

Family

ID=70495189

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910638889.9A Active CN111131718B (en) 2019-07-16 2019-07-16 Multi-exposure image fusion method and system with LED flicker compensation function

Country Status (1)

Country Link
CN (1) CN111131718B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113038026A (en) * 2021-03-01 2021-06-25 维沃移动通信有限公司 Image processing method and electronic device
CN113242389A (en) * 2021-07-12 2021-08-10 深圳市艾为智能有限公司 Multi-frame dynamic range extension method and system for RCCB (Rich communication Circuit Board) image sensor

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1250570A (en) * 1997-11-12 2000-04-12 皇家菲利浦电子有限公司 Method and apparatus for scaling and reducing flicker with dynamic coefficient weighting
JP2011160090A (en) * 2010-01-29 2011-08-18 Sony Corp Image processing device and signal processing method, and program
CN103379288A (en) * 2012-04-12 2013-10-30 索尼公司 Image processing device, image processing method, and program
US20140153839A1 (en) * 2012-12-04 2014-06-05 Samsung Techwin Co., Ltd. Apparatus and method for calculating flicker-evaluation value
CN103975578A (en) * 2011-12-08 2014-08-06 索尼公司 Image processing device, image processing method, and program
US20140267815A1 (en) * 2013-03-15 2014-09-18 Canon Kabushiki Kaisha Image capturing apparatus and method of controlling the same
US20150195441A1 (en) * 2012-09-20 2015-07-09 Huawei Technologies Co., Ltd. Image Processing Method and Apparatus
CN104780358A (en) * 2014-01-03 2015-07-15 联发科技（新加坡）私人有限公司 Method for flicker detection and associated circuit
CN105659583A (en) * 2013-12-04 2016-06-08 索尼公司 Image processing device, image processing method, electronic apparatus, and program
CN106303274A (en) * 2016-08-01 2017-01-04 凌云光技术集团有限责任公司 A kind of high dynamic-range image synthesis method and device
CN108353130A (en) * 2015-11-24 2018-07-31 索尼公司 Image processor, image processing method and program

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1250570A (en) * 1997-11-12 2000-04-12 皇家菲利浦电子有限公司 Method and apparatus for scaling and reducing flicker with dynamic coefficient weighting
JP2011160090A (en) * 2010-01-29 2011-08-18 Sony Corp Image processing device and signal processing method, and program
CN103975578A (en) * 2011-12-08 2014-08-06 索尼公司 Image processing device, image processing method, and program
CN103379288A (en) * 2012-04-12 2013-10-30 索尼公司 Image processing device, image processing method, and program
US20150195441A1 (en) * 2012-09-20 2015-07-09 Huawei Technologies Co., Ltd. Image Processing Method and Apparatus
US20140153839A1 (en) * 2012-12-04 2014-06-05 Samsung Techwin Co., Ltd. Apparatus and method for calculating flicker-evaluation value
US20140267815A1 (en) * 2013-03-15 2014-09-18 Canon Kabushiki Kaisha Image capturing apparatus and method of controlling the same
CN105659583A (en) * 2013-12-04 2016-06-08 索尼公司 Image processing device, image processing method, electronic apparatus, and program
CN104780358A (en) * 2014-01-03 2015-07-15 联发科技（新加坡）私人有限公司 Method for flicker detection and associated circuit
CN108353130A (en) * 2015-11-24 2018-07-31 索尼公司 Image processor, image processing method and program
CN106303274A (en) * 2016-08-01 2017-01-04 凌云光技术集团有限责任公司 A kind of high dynamic-range image synthesis method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113038026A (en) * 2021-03-01 2021-06-25 维沃移动通信有限公司 Image processing method and electronic device
CN113242389A (en) * 2021-07-12 2021-08-10 深圳市艾为智能有限公司 Multi-frame dynamic range extension method and system for RCCB (Rich communication Circuit Board) image sensor

Also Published As

Publication number Publication date
CN111131718B (en) 2021-05-14

Similar Documents

Publication Publication Date Title
CN103778887B (en) The brightness correcting method of LED display unit and device
CN101790270B (en) LED backlight brightness dynamic control method and system adopting same
CN111131718B (en) Multi-exposure image fusion method and system with LED flicker compensation function
CN104702850B (en) A kind of complementary metal oxide semiconductor video camera and its light compensation method
CN106470336A (en) Camera color temperature compensating system and intelligent terminal
CN110164366A (en) A kind of information processing method, electronic equipment
CN110728944B (en) Display device and display method
JP2010250320A (en) Method for correcting pixel data, and display apparatus for performing the method
JP2007193295A (en) Led driving integrated circuit device capable of pulse width modulation
CN101105912A (en) Display quality automatic regulation device and method based on environment induction
CN104639920A (en) Wide dynamic fusion method based on single-frame double-pulse exposure mode
CN104575405A (en) Method for adjusting backlight brightness of display device, display device
CN106782385A (en) A kind of light-dimming method and its device for wearing display device
CN103489411B (en) The color of display screen maintains method and system
JP2021168465A5 (en)
CN117274107B (en) End-to-end color and detail enhancement method, device and equipment under low-illumination scene
WO2018225516A1 (en) Image processing device
US8670660B2 (en) Image display device, brightness control method and brightness control program
TW202133601A (en) A low-light imaging system
CN202487127U (en) Backlight source and display device
JP4749142B2 (en) Visual status measurement device
KR100961675B1 (en) Traffic light lamp using light emitting diode
JP2012023459A (en) On-vehicle monitor camera and imaging method thereof
US20230237948A1 (en) Display screen adjustment method and apparatus, and device
TW202013332A (en) Local dimming system adaptable to a backlight of a display

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant