CN109191441A - Image processing method, equipment, system and storage medium - Google Patents
Image processing method, equipment, system and storage medium Download PDFInfo
- Publication number
- CN109191441A CN109191441A CN201810987724.8A CN201810987724A CN109191441A CN 109191441 A CN109191441 A CN 109191441A CN 201810987724 A CN201810987724 A CN 201810987724A CN 109191441 A CN109191441 A CN 109191441A
- Authority
- CN
- China
- Prior art keywords
- image
- light
- camera
- field camera
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 239000012141 concentrate Substances 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 27
- 230000015654 memory Effects 0.000 claims description 15
- 230000007423 decrease Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000004744 fabric Substances 0.000 claims 1
- 230000003287 optical effect Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 10
- 239000011159 matrix material Substances 0.000 description 7
- 230000010287 polarization Effects 0.000 description 6
- 239000011521 glass Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 235000013350 formula milk Nutrition 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 244000283207 Indigofera tinctoria Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000000571 coke Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000005304 optical glass Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- -1 showcase Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
A kind of image processing method provided by the invention, equipment, system and storage medium, by the first image set for receiving the shooting of light-field camera array, and optionally wherein a camera is standard camera, then the intrinsic parameter demarcated according to each camera and outer parameter by the first image concentrate each image project respectively in the corresponding image physical coordinates system of the standard camera to obtain the second image set, the unification of focal plane corresponding to each image in second image set is focused to bright face rear any plane to obtain third image set again, last each image is calculated by difference completes image.The present invention is able to solve the high optical issue that shooting photo occurs, and restores the scene of removal bloom.
Description
Technical field
The present invention relates to field of image processings.It is situated between more particularly, to a kind of image processing method, equipment, system and storage
Matter.
Background technique
We can frequently encounter in daily shooting, if occur glass type objects in scene, such as vehicle keeps out the wind glass
Glass, showcase, glasses etc., shooting the photo come often will appear the very high region of a piece of brightness, lead to object thereafter
Body details can not obtain well, this very high region of brightness, and region brighter for pixel tone in the picture is presented, leads to
Often referred to as high optical issue.The method of the existing high optical issue of removal is more, but effect is all unsatisfactory, these methods are often to height
Object behind light region is difficult clearly to restore, and still has loss of detail or texture information to lose.Therefore, a kind of energy is on the one hand needed
On the other hand the equipment that enough acquisitions do not have high optical issue at least partly also needs to effectively remove when correcting in the later period
Highlight area in photo, but also can efficiently restore the technology that defendant solely blocks scene.
Summary of the invention
In view of the foregoing deficiencies of prior art, the purpose of the present invention is to provide image processing method, equipment, systems
And storage medium, for solving the high optical issue that shooting photo occurs in the prior art, and restore the clear of not no bloom
The problem of scene.
In order to achieve the above objects and other related objects, the present invention provides a kind of image processing method, and method includes: to receive
First image set of light-field camera array shooting, and optionally wherein a camera is standard camera；It is each in the light-field camera array
Light-field camera is equipped with polariscope, and setting angle is equipped between each light-field camera and its described polariscope installed,
The setting angle between each light-field camera is different；The intrinsic parameter demarcated according to each camera and outer parameter are by described
In one image set each image project respectively in the corresponding image physical coordinates system of the standard camera to obtain the second image set；
By the unified focusing in focal plane corresponding to each image in second image set to bright face rear any plane to obtain third figure
Image set；Each image is calculated by difference in the third image set completes image.
In one embodiment of the invention, the intrinsic parameter demarcated according to each camera and outer parameter are by first figure
It includes: according to each camera that each image, which is projected respectively to the method in the corresponding image physical coordinates system of the standard camera, in image set
The corresponding image physical coordinates of each camera are converted to camera coordinates by the intrinsic parameter demarcated；The outer ginseng demarcated according to each camera
Number converts the corresponding camera coordinates of each camera to world coordinate system；It is closed according to the corresponding of each camera in the world coordinate system
System concentrates the first image in each image projection to the corresponding image physical coordinates system of the standard camera.
In one embodiment of the invention, the method for unified focusing to the bright face rear any plane includes: to judge institute
State whether focal plane corresponding to each image in the second image set is parallel to the light-field camera array plane；If so, focusing
It is parallel to focal plane to the bright face rear any plane of the light-field camera array plane；If it is not, then adjustment is non-parallel to described
The focal plane of light-field camera array plane is extremely parallel with the light-field camera array plane, then focuses any flat to bright face rear
Face.
In one embodiment of the invention, it is described adjustment non-parallel to the light-field camera array plane focal plane to
The parallel method of the light-field camera array plane includes: to choose the coke non-parallel to the light-field camera array plane to put down
On face a bit, it translates the focal plane and is overlapped to the point with the light-field camera array origin；By the focal plane rotate to
The light-field camera array plane is parallel；It is moved back to the focal plane.
In one embodiment of the invention, the difference calculation method includes: to take two in defocused image again
Image is opened, subtracts each other to obtain the first error image in pixel value under same pixel coordinate by each；By first error image
In each pixel coordinate under pixel value compared with preset threshold, if being less than threshold value, labeled as invalid；If it is not, being then labeled as
Effectively；By in two images as minuend subtracted image and first error image it is each under same pixel coordinate
Pixel value subtracts each other to obtain the second error image；Wherein, corresponding to label in first error image is part, value
For the difference for being subtracted image Yu each pixel value under same pixel coordinate of first error image；Corresponding to described
Mark the part for being in error image, value is described to be subtracted pixel value in image under each pixel coordinate；By described
Two error images are put back in the third image set, then appoint and two images is taken to repeat the above steps；When the third image set is surplus
Remaining an image when, then obtain the completion image.
In order to achieve the above objects and other related objects, the present invention provides a kind of computer readable storage medium, deposits thereon
Image processing program is contained, the image processing method is realized when which is executed by processor.
In order to achieve the above objects and other related objects, the present invention provides a kind of image processing equipment, comprising: communicator,
Processor and memory；The communicator communicates to connect external equipment；The memory is for storing image processing program；Institute
It states processor operation image processing routine and realizes the image processing method.
In order to achieve the above objects and other related objects, the present invention provides a kind of light-field camera array, comprising: multiple light fields
Camera；The multiple light-field camera forms array by certain arrangement mode；Each light-field camera is equipped with polariscope, Mei Gesuo
It states and is equipped with setting angle between light-field camera and its described polariscope installed, the installation between each light-field camera
Angle is different.
In one embodiment of the invention, the polariscope is circular polarizer.
In one embodiment of the invention, the installation that is equipped between each light-field camera and its polariscope installed
Angle is that ecto-entad successively successively decreases by a certain percentage, and the setting angle is distributed within the scope of 0-180 degree.
In one embodiment of the invention, the angle degree with bright face respectively of plane locating for the camera lens of each light-field camera
Meet Brewster angular region.
In order to achieve the above objects and other related objects, the present invention provides a kind of image processing system, comprising: the figure
As processing equipment and the light-field camera array；Described image processing equipment communicates to connect the light-field camera array, with reality
The existing image processing method.
As described above, image processing method of the invention, equipment, system and storage medium, by receiving light-field camera battle array
Arrange the first image set of shooting, and optionally wherein a camera is standard camera, the intrinsic parameter then demarcated according to each camera and
Outer parameter by the first image concentrate each image project respectively in the corresponding image physical coordinates system of the standard camera with
The second image set is obtained, then the unified focusing in focal plane corresponding to each image in second image set is any to bright face rear
For plane to obtain third image set, last each image is calculated by difference completes image.It has the advantages that
It is able to solve the high optical issue that shooting photo occurs, and the clearly scene of reduction removal bloom.
Detailed description of the invention
Fig. 1 is shown as the flow diagram of image processing method of the present invention in an embodiment.
Fig. 2 is shown as image processing equipment schematic diagram of the present invention in an embodiment.
Fig. 3 is shown as light-field camera array and its schematic diagram of a scenario of the present invention in an embodiment.
Fig. 4 is shown as each camera and its polariscopic structural schematic diagram for being installed of the present invention in an embodiment.
Fig. 5 is shown as image processing system schematic diagram of the present invention in an embodiment.
Component label instructions
S101~S104 method and step
200 image processing equipments
201 memories
202 processors
203 communicators
300 light-field camera arrays
301 light-field cameras
401 light-field cameras
402 polariscopes
501 image processing equipments
502 light-field camera arrays
Specific embodiment
Illustrate embodiments of the present invention below by way of specific specific example, those skilled in the art can be by this specification
Other advantages and efficacy of the present invention can be easily understood for disclosed content.The present invention can also pass through in addition different specific realities
The mode of applying is embodied or practiced, the various details in this specification can also based on different viewpoints and application, without departing from
Various modifications or alterations are carried out under spirit of the invention.It should be noted that in the absence of conflict, following embodiment and implementation
Feature in example can be combined with each other.
It should be noted that illustrating the basic structure that only the invention is illustrated in a schematic way provided in following embodiment
Think, only shown in schema then with related component in the present invention rather than component count, shape and size when according to actual implementation
Draw, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel
It is likely more complexity.
As shown in Figure 1, the flow diagram of image processing method of the present invention in an embodiment is shown, at described image
Reason method includes:
Step S101: the first image set of light-field camera array shooting is received, and optionally wherein a camera is standard camera；
Each light-field camera is equipped with polariscope, each light-field camera and its described polarization installed in the light-field camera array
Setting angle is equipped between mirror, the setting angle between each light-field camera is different.
In one embodiment of the invention, each camera and its polariscopic angle installed be by a certain percentage by
Incremented by successively outside to inside, the angular distribution is within the scope of 0-180 degree.
Wherein, the polariscopic angle that each light-field camera is installed with it is different, and by a certain percentage according to
Secondary variation, including ecto-entad are incremented by successively, and the purpose is to polarization light of the reception from different directions as much as possible.
For example, choosing one camera of light-field camera array center is No. 0 camera, the polarized lenses angle of this camera
Degree is 0 °, it is assumed that light-field camera array share n platform, then the distance of basis to No. 0 camera, and ecto-entad is by the polarization of camera around
Successively according to 180/n °, 180/n*2 °, 180/n*3 ° ... is configured mirror setting angle.Last polarised light is in different cameral
Different bright dark variations can be presented in intensity.
Step S102: the first image is concentrated each image to distinguish by the intrinsic parameter demarcated according to each camera and outer parameter
To obtain the second image set in projection to the corresponding image physical coordinates system of the standard camera.
In one embodiment of the invention, the intrinsic parameter of the calibration forms an internal reference matrix are as follows:
Wherein, K is Intrinsic Matrix；Fx, fy respectively indicate the normalization focal length in x-axis and y-axis；Wherein, fx=f/dx,
Fy=f/dy, f are the focal length of camera, and dx, dy respectively indicate length unit, i.e., one shared by a pixel in the direction x and the direction y
The size dimension for the actual physics value that a pixel represents；X0, y0 respectively indicate the center pixel coordinate and image origin picture of image
The horizontal and vertical pixel number differed between plain coordinate, s indicate warping factor or reference axis tilt parameters, are under normal circumstances 0.
For example, such as the focal length of a camera: f=35mm, highest resolution: 4256 × 2832, size sensor: 36.0
×23.9mm。
According to the corresponding intrinsic parameter of the above- mentioned information cameras are as follows: x0=4256/2=2128, y0=2832/2=1416, dx=
36.0/4256 dy=23.9/2832, fx=f/dx=4137.8, fy=f/dy=4147.3.
In one embodiment of the invention, the outer parameter of the calibration forms an outer parameter matrix are as follows: [R T].
Wherein, R is spin matrix, indicates image around x, y, the degree of z-axis rotation.Such as it rotates around x axis:
T is to appraise matrix through discussion, indicates image along x, y, the distance of z-axis translation.Such as:
It should be noted that the intrinsic parameter of camera calibration and outer parameter, are the processes for seeking final projection matrix.Wherein wrap
Containing four basic coordinate systems: world coordinate system, camera coordinates system, image physical coordinates system (imaging plane coordinate system) and
Pixel coordinate system.
In one embodiment of the invention, the intrinsic parameter demarcated according to each camera and outer parameter are by first figure
It includes: according to each camera that each image, which is projected respectively to the method in the corresponding image physical coordinates system of the standard camera, in image set
The corresponding image physical coordinates of each camera are converted to camera coordinates by the intrinsic parameter demarcated；The outer ginseng demarcated according to each camera
Number converts the corresponding camera coordinates of each camera to world coordinate system；It is closed according to the corresponding of each camera in the world coordinate system
System concentrates the first image in each image projection to the corresponding image physical coordinates system of the standard camera.
As an example it is assumed that No. 1 camera is standard camera, then for the image projection of m camera shooting to standard phase
The transformation for mula of the corresponding image physical coordinates system of machine are as follows:
Wherein, xm, ymIt is the two-dimensional coordinate under m camera correspondence image physical coordinates system, the d on the right of equation is (xm, ym)
This depth under m camera perspective, but depth information can not be actually obtained from two dimensional image, therefore can only assume institute
Depth value a little is the same.
The d ' on the equation left side is the depth value after projection, is different for each point.xm', ym' it is by throwing
The plane coordinates of movie queen, w ' expression normalized parameter.
In one embodiment of the invention, after above-mentioned formula projects, all images under the same coordinate system, with
Focusing again in light field rendering.
Step S103: the unified focusing in focal plane corresponding to each image in second image set is any to bright face rear
Plane is to obtain third image set.
In one embodiment of the invention, the bright face indicates to be formed with brightness upper zone or height in captured scene
The object plane in light area, is such as formed with the front truck glass pane of highlight area, or is formed with the glazing plate face of highlight area.
In one embodiment of the invention, the method for unified focusing to the bright face rear any plane includes: to judge institute
State whether focal plane corresponding to each image in the second image set is parallel to the light-field camera array plane；If so, focusing
To bright face rear any plane；If it is not, then choose it is described non-parallel on the focal plane of the light-field camera array plane a bit,
It translates the focal plane and is overlapped to the point with the light-field camera array origin；By the focal plane rotate to the light field phase
Machine array plane is parallel；It is moved back to the focal plane；It focuses to bright face rear any plane.
It should be noted that focusing refers to set depth when handling by changing light field to change the reality of virtual camera again
Depth of focusing and physical location, so that the image information of most required objects is obtained, specifically by the bigger distance of progress
Displacement to eliminate the relative distance between respective pixel, then superposition obtain new focusing in image.And needs of focusing again
All images are under the same coordinate system.
In one embodiment of the invention, in the case where focal plane is parallel to the light-field camera array plane, foundation
Setting focusing depth value, i.e., changeable focusing depth.
But in the case where focal plane is not parallel to the light-field camera array plane, then need first to adjust to parallel, then
Adjustment focusing depth.The process of above-mentioned adjustment can be expressed with formula are as follows:
Wherein, it under world coordinate system is really coordinate that XYZ, which is each camera,.
Step S104: each image is calculated by difference in the third image set completes image.
In one embodiment of the invention, the difference calculation method includes: to take two in defocused image again
Image is opened, subtracts each other to obtain the first error image in pixel value under same pixel coordinate by each；By first error image
In each pixel coordinate under pixel value compared with preset threshold, if being less than threshold value, labeled as invalid；If it is not, being then labeled as
Effectively；By in two images as minuend subtracted image and first error image it is each under same pixel coordinate
Pixel value subtracts each other to obtain the second error image；Wherein, corresponding to label in first error image is part, value
For the difference for being subtracted image Yu each pixel value under same pixel coordinate of first error image；Corresponding to described
Mark the part for being in error image, value is described to be subtracted pixel value in image under each pixel coordinate；By described
Two error images are put back in the third image set, then appoint and two images is taken to repeat the above steps；When the third image set is surplus
Remaining an image when, then obtain the completion image.
In one embodiment of the invention, the premise calculated by difference is that each image is needed to be in same image physics
Under coordinate.Multiple images can be overlapped under same coordinate, and be corresponding with unified image pixel coordinates under the image physical coordinates
Being includes the pixel coordinate of each image under the pixel coordinate system.
Wherein, wherein each pixel coordinate include multiple pixels, each pixel by r, g, b triple channel pixel value group
At the pixel value arbitrarily chosen under two each pixel coordinates of image subtracts each other one by one, and subtraction here is that two images are corresponding
R, the pixel value of g, b triple channel subtracts, and the difference of the pixel value in three channels of pixel can be obtained, to further may be used
Obtain error image.
For example, two image im are being taken in defocused image again1、im2, by the picture under same pixel coordinate
Plain value subtracts each other to obtain the first error image imd1, i.e. imd1=im1–im2。
By each pixel value under the pixel coordinate in first error image compared with preset threshold e, if being less than threshold value,
Labeled as invalid；If it is not, then labeled as effective.
Image im will be subtracted as minuend in two images1With the first error image imd1Same pixel sit
Pixel value under mark subtracts each other to obtain the second error image imd2。
By the second error image imd2It puts back in the third image set, then appoints and two images is taken to repeat the above steps；
When the remaining image of the third image set, then the completion image im is obtainedresult, as remove bloom
Image.
In one embodiment of the invention, because appointing two images taken not put back to image set, only by second difference
Image imd2The third image set is put back to, therefore image can gradually decrease in the third image set, so termination condition is figure
All images only remain an image by processing in image set in the third image set when that is, last time is handled.
It should be noted that it includes triple channel that preset threshold e, which is also, wherein the threshold value of triple channel must not in a pixel
So connection, needs the illumination according to actual scene, reflective intensity and white balance to adjust accordingly.
As an example it is assumed that preset threshold e is 0, then calculated by above-mentioned difference, different images subtract each other two-by-two, can obtain
Respective image lower part of pixel value under pixel coordinate, will eventually get the completion image of composite pixel value reduction.
And during difference calculates, if preset threshold e is more like sensitivity coefficient, and e value is bigger, more insensitive.Separately
For example, when illumination condition is good, in reflective strong scene, in bright face, the brightness of non-highlight area is also very high, such as
It is lower that fruit sets preset threshold e, i.e., more sensitive, it is likely that calculates effectively and accurately to remove by difference and wants removal
Highlight area, instead may non-highlight area because brightness it is higher, also do not remove, to influence image overall picture.
So in general, the preset threshold that illumination condition is good, reflective strong scene needs are bigger.
In order to achieve the above objects and other related objects, the present invention provides a kind of computer readable storage medium, deposits thereon
Image processing program is contained, realizes that Fig. 1 such as shows the present invention at the image in an embodiment when which is executed by processor
Reason method.
The computer readable storage medium, those of ordinary skill in the art will appreciate that: realize that above-mentioned each method is implemented
The all or part of the steps of example can be completed by the relevant hardware of computer program.Image processing program above-mentioned can be deposited
It is stored in a computer readable storage medium.When being executed, execution includes the steps that above-mentioned each method embodiment to the program；And it is preceding
The storage medium stated includes: the various media that can store program code such as ROM, RAM, magnetic or disk.
These computer program programs also can be loaded onto a computer or other programmable data processing device, and make calculating
On machine or other programmable devices execute series of operation steps to generate computer implemented processing, thus computer or its
Instruction that he executes on programmable device is provided for realizing in one or more flows of the flowchart and/or one, block diagram
The step of function of being specified in box or multiple boxes.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape type disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitorymedia), such as the data-signal and carrier wave of modulation.
As shown in Fig. 2, showing image processing equipment schematic diagram of the present invention in an embodiment, described image processing equipment
200 include: communicator 203, processor 202 and memory 201；The communicator 203 communicates to connect external equipment；It is described to deposit
Reservoir 201 is for storing image processing program；The 202 operation image processing routine of processor realizes that Fig. 1 such as shows the present invention
Image processing method in an embodiment.
The communicator 203 is for realizing database access device and other equipment (such as client, read-write library and read-only
Library) between communication connection, described communicate to connect can be any suitable group of one or more wired and or wireless networks
Conjunction form.For example, communication mode may include internet, Intranet, wide area network (WAN), local area network (LAN), wireless network, number
Word user line (DSL) network, frame-relay network, asynchronous transfer mode (ATM) network, Virtual Private Network (VPN) and/or any
Any one or more of other suitable communication networks.
The memory 201 may include random access memory (RandomAccessMemory, abbreviation RAM), can also
It can further include nonvolatile memory (non-volatilememory), a for example, at least magnetic disk storage.
The processor 202 can be general processor, including central processing unit (CentralProcessingUnit, letter
Claim CPU), network processing unit (NetworkProcessor, abbreviation NP) etc.；It can also be digital signal processor
(DigitalSignalProcessing, abbreviation DSP), specific integrated circuit (ApplicationSpecificIntegrated
Circuit, abbreviation ASIC), field programmable gate array (Field-ProgrammableGateArray, abbreviation FPGA) or
Other programmable logic device, discrete gate or transistor logic, discrete hardware components.
As shown in figure 3, showing light-field camera array and its schematic diagram of the present invention in an embodiment.As shown, institute
Stating light-field camera array 300 includes: multiple light-field cameras 301；The multiple light-field camera 301 forms battle array by certain arrangement mode
Arrange
In one embodiment of the invention, the light-field camera 301 is preferably plural rgb light field camera.
Wherein, the color mode of RGB is a kind of color standard of industry, is by red (R), green (G), indigo plant (B) three
The variation of a Color Channel and their mutual superpositions obtain miscellaneous color, RGB be represent it is red,
The color in green, blue three channels, this standard almost include all colours that human eyesight can perceive, and are current with most
One of wide color system.
In one embodiment of the invention, the light-field camera array 300 is equipped with the bracket of a rigidity, to guarantee to take
It does not move relative position after having built between each light-field camera 301.
In one embodiment of the invention, 301 arrangement mode of each light-field camera of the light-field camera array 300 is cleverer
It is living, it can be in the matrix arrangement of m × n, can be in the honeycomb arrangement of m × n, it can also be in the arrangement around center, i.e. a light field phase
Machine forms the light-field camera array 300 of a n ring around its arrangement at center, then with other light-field cameras.
It should be noted that the spacing size of every two light-field cameras 301 will affect it is last go bloom effect, if light
Field 301 spacing of camera is larger, then the object of non-setting focal plane depth will obscure when light field reconstruction, but for different angle
Light better adaptability, and arrangement meeting of intensive camera is so that the non-object for setting focal plane depth is apparent, to expand
Big data available range.So the spacing of light-field camera 301 is determined according to specific usage scenario, generally, for
Outdoor solar light, it should choose camera spacing as small as possible.
In one embodiment of the invention, the angle with bright face respectively of plane locating for the camera lens of each light-field camera 301
Degree meets Brewster angular region.
It should be noted that plane locating for the camera lens of each light-field camera 301 meets with the angle degree in bright face respectively
Brewster angular region, the characteristic that the purpose is to guarantee to have polarization into most of light of camera.
It is real in the present invention one although the Brewster's angle chosen has uncertainty in one embodiment of the invention
It applies in example, angle degree of the plane respectively with bright face locating for the camera lens of each light-field camera 301 should be consistent.Entirety can be passed through
Adjustment light-field camera array 300 and the bright face angle realize plane locating for the camera lens of each light-field camera 301 respectively with bright face
Angle degree be consistent.
Wherein, Brewster's angle is also known as polarizing angle, and when incidence natural lights inject interface at this angle, reflected light is
Linearly polarized light, and it is orthogonal with refracted light.When light injects medium from air, the tangent value of Brewster's angle is equal to
The refractive index n of medium.Due to the refractive index of medium be it is related with optical wavelength, to same medium, the size of Brewster's angle
It is also related with optical wavelength.
It for example, is 1.4-1.9 calculating with optical glass refractive index, Brewster's angle is about 54-62 degree or so.
As shown in figure 4, showing each camera and its polariscopic structural representation for being installed of the present invention in an embodiment
Figure.As shown, each light-field camera 401 is equipped with polariscope 402.
In one embodiment of the invention, the preferred circular polarizer of the polariscope 402.
It should be noted that general polariscope is divided into linear polarizer and circular polarizer.Usual linear polarizer and circular polarizer
The basic demand of photography can be met, but linear polarizer will affect and survey light and automatic focusing, and it is higher to be not suitable for the degree of automation
Digital camera use.Because present digital camera all has the function of that auto-focusing and automatic exposure, linear polarizer may be led
Cause automatic exposure inaccurate, auto-focusing is possible to out of control, however circular polarizer on this kind of camera but can place of working it is fine, so
The polariscope 402 is preferably circular polarizer in embodiments of the present invention.
It should be noted that direction of vibration is called polarization for the asymmetry of the direction of propagation, only shear wave could be generated
Polarization phenomena.The phenomenon that spatial distribution of light wave electric vector vibration loses symmetry for the direction of propagation of light is called the inclined of light
Vibration.Light wave includes the lateral vibration in all possible directions, but the amplitude on different directions differs, in two mutually perpendicular directions do so
Amplitude has maximum value and minimum value, and this light is known as partial poolarized light.Natural light and partial poolarized light are actually by many
The different linearly polarized light composition of direction of vibration.
In one embodiment of the invention, between each light-field camera 401 and its described polariscope 402 installed
Equipped with setting angle, the setting angle between each light-field camera 401 is different.
In one embodiment of the invention, it is equipped between each light-field camera 401 and its polariscope installed 402
Setting angle be that ecto-entad successively successively decreases by a certain percentage, the setting angle is distributed within the scope of 0-180 degree.
Wherein, each light-field camera 401 and the setting angle of its polariscope 402 installed are different, and press
Certain proportion successively changes, including ecto-entad is incremented by successively, and the purpose is to receptions as much as possible from the inclined of different directions
Shake light.
For example, choosing one light-field camera 401 of light-field camera array center is No. 0 camera, the polariscope of this camera
Piece setting angle is 0 °, it is assumed that light-field camera array share n platform, then the distance of basis to No. 0 camera, and ecto-entad is by ambient light
Successively according to 180/n °, 180/n*2 °, 180/n*3 ° ... is configured the polariscope setting angle of field camera.Last polarised light exists
Different bright dark variations can be presented in intensity in different cameral.
As shown in figure 5, showing image processing system schematic diagram of the present invention in an embodiment, comprising: as Fig. 2 shows this
Image processing equipment 501 of the invention in an embodiment and the light-field camera array such as Fig. 3 displaying present invention in an embodiment
502；Described image processing equipment communicates to connect the light-field camera array, to realize image processing method as described in Figure 1.
In one embodiment of the invention, described image processing equipment 501 and Fig. 2 show the present invention in an embodiment
Image processing equipment it is identical, the light-field camera array 502 and Fig. 3 shows the light-field camera of the invention in an embodiment
Array is identical, therefore which is not described herein again.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
It should be noted that the terms "include", "comprise" or its any other variant are intended to the packet of nonexcludability
Contain, so that the process, method, product or the equipment that include a series of elements not only include those elements, but also including
Other elements that are not explicitly listed, or further include for this process, method, product or the intrinsic element of equipment.
In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including the element
Process, method, there is also other identical elements in product or equipment.
In conclusion a kind of image processing method, equipment, system and storage medium of the invention, by receiving light field phase
First image set of machine array shooting, and optionally wherein a camera is standard camera, the internal reference then demarcated according to each camera
The first image is concentrated each image to project image physical coordinates system corresponding to the standard camera respectively by number with outer parameter
It is interior to obtain the second image set, then by the unified focusing in focal plane corresponding to each image in second image set to bright face rear
For any plane to obtain third image set, last each image is calculated by difference completes image.
The present invention is able to solve the high optical issue that shooting photo occurs, and clearly restores the scene of removal bloom.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.It is any ripe
The personage for knowing this technology all without departing from the spirit and scope of the present invention, carries out modifications and changes to above-described embodiment.Cause
This, institute is complete without departing from the spirit and technical ideas disclosed in the present invention by those of ordinary skill in the art such as
At all equivalent modifications or change, should be covered by the claims of the present invention.
Claims (12)
1. a kind of image processing method, which is characterized in that method includes:
The first image set of light-field camera array shooting is received, and optionally wherein a camera is standard camera；The light-field camera
Each light-field camera is equipped with polariscope in array, and peace is equipped between each light-field camera and its described polariscope installed
Angle is filled, the setting angle between each light-field camera is different；
The first image is concentrated each image to be projected respectively to the mark by the intrinsic parameter demarcated according to each camera and outer parameter
To obtain the second image set in the corresponding image physical coordinates system of quasi- camera；
The unified focusing in focal plane corresponding to each image in second image set is obtained the to bright face rear any plane
Three image sets；
Each image is calculated by difference in the third image set completes image.
2. image processing method according to claim 1, which is characterized in that the intrinsic parameter demarcated according to each camera
Each image is concentrated to be projected to the corresponding image physical coordinates system of the standard camera respectively the first image with outer parameter
Method include:
The corresponding image physical coordinates of each camera are converted to camera coordinates by the intrinsic parameter demarcated according to each camera；
The outer parameter demarcated according to each camera converts the corresponding camera coordinates of each camera to world coordinate system；
Concentrate each image projection to the mark the first image according to the corresponding relationship of each camera in the world coordinate system
In the corresponding image physical coordinates system of quasi- camera.
3. image processing method according to claim 1, which is characterized in that the unified focusing is any flat to bright face rear
The method in face includes:
Judge whether focal plane corresponding to each image is parallel to the light-field camera array plane in second image set；
If so, focusing is parallel to focal plane to the bright face rear any plane of the light-field camera array plane；
If it is not, then adjustment is put down non-parallel to the focal plane of the light-field camera array plane to the light-field camera array plane
Row, then focus to bright face rear any plane.
4. image processing method according to claim 3, which is characterized in that the adjustment is non-parallel to the light-field camera
The focal plane of array plane includes: to the method parallel with the light-field camera array plane
Choose it is described non-parallel to a bit, translated on the focal plane of the light-field camera array plane focal plane to the point with
The light-field camera array origin is overlapped；
The focal plane is rotated to parallel with the light-field camera array plane；
It is moved back to the focal plane.
5. image processing method according to claim 1, which is characterized in that the difference calculation method includes:
Again two images are being taken to appointing in defocused image, is subtracting each other to obtain the in pixel value under same pixel coordinate for each
One error image；
By the pixel value under each pixel coordinate in first error image compared with preset threshold, if being less than threshold value, mark
It is invalid to be denoted as；If it is not, then labeled as effective；
By in two images as minuend subtracted image and first error image it is each under same pixel coordinate
Pixel value subtracts each other to obtain the second error image；Wherein, corresponding to label in first error image is part, value
For the difference for being subtracted image Yu each pixel value under same pixel coordinate of first error image；Corresponding to described
Mark the part for being in error image, value is described to be subtracted pixel value in image under each pixel coordinate；
Second error image is put back in the third image set, then appoints and two images is taken to repeat the above steps；
When the remaining image of the third image set, then the completion image is obtained.
6. a kind of computer readable storage medium, is stored thereon with image processing program, which is characterized in that the program is by processor
Image processing method described in any one of claims 1 to 5 is realized when execution.
7. a kind of image processing equipment characterized by comprising communicator, processor and memory；
The communicator communicates to connect external equipment；The memory is for storing image processing program；The processor operation
Image processing program realizes image processing method described in any one of claims 1 to 5.
8. a kind of light-field camera array characterized by comprising multiple light-field cameras；The multiple light-field camera is by certain row
Mode for cloth forms array；Each light-field camera is equipped with polariscope, and each light-field camera is installed described inclined with it
Setting angle is equipped between galvanometer, the setting angle between each light-field camera is different.
9. light-field camera array according to claim 8, which is characterized in that the polariscope is circular polarizer.
10. light-field camera array according to claim 8, which is characterized in that each light-field camera is installed with it
Polariscope between the setting angle that is equipped be that ecto-entad successively successively decreases by a certain percentage, the setting angle is distributed in 0-
Within the scope of 180 degree.
11. light-field camera array according to claim 8, which is characterized in that put down locating for the camera lens of each light-field camera
Face meets Brewster angular region with the angle degree in bright face respectively.
12. a kind of image processing system characterized by comprising image processing equipment as claimed in claim 7 and such as power
Benefit require 8 described in light-field camera array；Described image processing equipment communicates to connect the light-field camera array, to realize right
It is required that image processing method described in any one of 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810987724.8A CN109191441B (en) | 2018-08-28 | 2018-08-28 | Image processing method, apparatus, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810987724.8A CN109191441B (en) | 2018-08-28 | 2018-08-28 | Image processing method, apparatus, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109191441A true CN109191441A (en) | 2019-01-11 |
| CN109191441B CN109191441B (en) | 2022-03-01 |
Family
ID=64916493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810987724.8A Active CN109191441B (en) | 2018-08-28 | 2018-08-28 | Image processing method, apparatus, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109191441B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132773A (en) * | 2019-06-24 | 2020-12-25 | 曜科智能科技（上海）有限公司 | Airplane head cover riveting point defect detection method, device, equipment and storage medium |
| CN113706398A (en) * | 2020-05-22 | 2021-11-26 | 西北工业大学 | Device and method for generating high dynamic image in motion scene |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968453A (en) * | 2009-12-01 | 2011-02-09 | 北京理工大学 | Polarization detection method and device for white and colorless foreign matters in cotton |
| US8345144B1 (en) * | 2009-07-15 | 2013-01-01 | Adobe Systems Incorporated | Methods and apparatus for rich image capture with focused plenoptic cameras |
| US20150215529A1 (en) * | 2014-01-24 | 2015-07-30 | Goodrich Corporation | Random imaging |
| CN108028892A (en) * | 2015-09-30 | 2018-05-11 | 索尼公司 | Information acquisition equipment and information obtaining method |
-
2018
- 2018-08-28 CN CN201810987724.8A patent/CN109191441B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8345144B1 (en) * | 2009-07-15 | 2013-01-01 | Adobe Systems Incorporated | Methods and apparatus for rich image capture with focused plenoptic cameras |
| CN101968453A (en) * | 2009-12-01 | 2011-02-09 | 北京理工大学 | Polarization detection method and device for white and colorless foreign matters in cotton |
| US20150215529A1 (en) * | 2014-01-24 | 2015-07-30 | Goodrich Corporation | Random imaging |
| CN108028892A (en) * | 2015-09-30 | 2018-05-11 | 索尼公司 | Information acquisition equipment and information obtaining method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132773A (en) * | 2019-06-24 | 2020-12-25 | 曜科智能科技（上海）有限公司 | Airplane head cover riveting point defect detection method, device, equipment and storage medium |
| CN112132773B (en) * | 2019-06-24 | 2024-04-12 | 曜科智能科技（上海）有限公司 | Method, device, equipment and storage medium for detecting riveting point defect of aircraft head cover |
| CN113706398A (en) * | 2020-05-22 | 2021-11-26 | 西北工业大学 | Device and method for generating high dynamic image in motion scene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109191441B (en) | 2022-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101356546B (en) | Image high-resolution upgrading device, image high-resolution upgrading method image high-resolution upgrading system | |
| Papadhimitri et al. | A new perspective on uncalibrated photometric stereo | |
| Papadhimitri et al. | Uncalibrated near-light photometric stereo | |
| CN110111262A (en) | A kind of projector distortion correction method, device and projector | |
| CN110336987A (en) | A kind of projector distortion correction method, device and projector | |
| JP4324238B2 (en) | Image processing system, method and apparatus | |
| CN104330074B (en) | Intelligent surveying and mapping platform and realizing method thereof | |
| CN105898282A (en) | Light-field camera | |
| CN107993276A (en) | The generation method and device of a kind of panoramic picture | |
| Sajadi et al. | Scalable Multi‐view Registration for Multi‐Projector Displays on Vertically Extruded Surfaces | |
| CN109191441A (en) | Image processing method, equipment, system and storage medium | |
| CN109685879A (en) | Determination method, apparatus, equipment and the storage medium of multi-view images grain distribution | |
| Gu et al. | Omni-nerf: neural radiance field from 360 image captures | |
| Hung et al. | Photometric stereo in the wild | |
| Einabadi et al. | Discrete Light Source Estimation from Light Probes for Photorealistic Rendering. | |
| Wang et al. | Polarimetric camera calibration using an LCD monitor | |
| CN113763480A (en) | Multi-lens panoramic camera combination calibration method | |
| CN109325912A (en) | Reflective separation method and calibration split system based on polarised light light field | |
| Huang et al. | 3D Browsing of Wide‐Angle Fisheye Images Under View‐Dependent Perspective Correction | |
| Li et al. | NeISF: Neural Incident Stokes Field for Geometry and Material Estimation | |
| Wu et al. | A camera calibration method based on OpenCV | |
| US11410378B1 (en) | Image processing for generating three-dimensional shape and spatially-varying reflectance of the object using a deep neural network | |
| CN109118425A (en) | A kind of pair of fish-eye image mosaic parameter correcting method and picture pick-up device | |
| Rodriguez et al. | Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization | |
| Ahmed et al. | Projector primary-based optimization for superimposed projection mappings |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |