CN101276415A - Apparatus and method for realizing multi-resolutions image acquisition with multi-focusing video camera - Google Patents

Abstract

A device and a method for using a multi-focusing camera to realize collection of multi-resolution images are disclosed, comprising five focusing cameras having restraining relationship with others, wherein four of them are located at the same horizontal plane and on four top points of the same square, and the four cameras have the identical focus which is set as f<s>; the rest 1 camera is located in the direction of the four cameras, axes line of the camera is perpendicular to the surfaces of four cameras and passes a central point O of the square, and the distance from the fifth camera to the rest four cameras is d, the focus of the camera is set as f<t>, and initial setting is performed to the parameters and locations of the five focusing cameras with the location relationship satisfying the following restraint: f<t>=f<s>+d. The invention achieves the function of lossless zooming film of the multi-focusing camera with adjustable details of scenes and images, and with flexible and controllable structure, the invention can rapidly collect the multi-resolution images in the large visual scene.

Description

Realize the apparatus and method that multi-resolution image is gathered with many focusing video cameras

Technical field

The invention belongs to field of video processing, the apparatus and method that specifically a kind of scene image that is applied to computer vision obtains.

Background technology

The multi-resolution image collection is meant the process of the view data of Same Scene being carried out consistent collection on different resolution details.What emphasize is according to the demand of user to image detail, artificial adjusting photographed scene image resolution ratio.Existent method and device are to use Zoom camera to realize the multi-resolution image collection.

Zoom camera is meant that the lens focus of video camera can regulate change, can obtain the different visions and the video camera of shooting effect by changing lens focus like this, comprises that the single camera optical zoom is taken, the single camera digital zoom is taken.

The single camera optical zoom is to rely on the optical lens structure to realize zoom, and the eyeglass by camera moves to amplify with dwindling needs the scenery taken, and the optical zoom multiple is big more, and the scenery that can take is just far away more.General optical zoom video camera costs an arm and a leg, and behind zoom, though scene resolution improves, visible scene reduces at double.

The principle of single camera digital zoom is different with optical zoom, and it is to use a part of pixel on the original CCD image sensor " interpolation " processing means to do amplification, and the pixel on the CCD image sensor is amplified to whole image with interpolation algorithm with picture.The electronics that is actually picture amplifies, though the scene of taking is amplified, its scene resolution does not have the raising on the essential meaning.Simultaneously, also there is the deficiency that visible scene diminishes behind the zoom in single camera digital zoom method.

Some patents also propose the apparatus and method that multi-resolution image is gathered both at home and abroad, use glass mirror, circuit, lens bar and light sensor arrays composite set that the total optical path changeable device that the reflector group that the alliteration optical illumination transmitter that Chinese patent CN1176351C proposes, CN2476040Y propose, CN1164962C propose, U.S. Patent application 20060077475 propose, all attempt to solve the multi-resolution image collection, but because its device belongs to the single camera type, and the unresolved visible scene problem of dwindling automatically.U.S. Patent application 20050111754 proposes the method that a kind of multi-resolution image merges, but does not provide image collecting device and controlled variable restriction relation.

In addition, according to the deficiency that the single camera optical zoom exists visible scene to diminish, can consider the situation of many optical zooms video camera shooting, collecting.Though can solve the problem of visible scene size, the uppity shortcoming of the image resolution ratio of introducing is arranged, and installation cost can be more expensive.

Generally speaking, existing multi-resolution image harvester and method exist visible scene little or be difficult for the deficiency of acquisition controlling.

Summary of the invention

One of purpose of the present invention is: overcome the deficiencies in the prior art, a kind of multi-resolution image harvester of realizing big visible scene is provided; Two of purpose is to propose the control method that multi-resolution image is gathered on the said apparatus basis.

Technical solution of the present invention: a kind of device that adopts many focusing video cameras to realize the multi-resolution image collection, comprise 5 focusing video cameras with position constraint relation, wherein 4 video cameras are on the same surface level, be positioned on same foursquare 4 summits, and 4 focus of camera are identical, are made as f _sOther 1 position for video camera is in a side of above-mentioned 4 video cameras, the axial line of this video camera is perpendicular to 4 shooting machine sides and through foursquare central point O, and the 5th video camera is d to the distance on all the other planes, 4 video camera places, and establishing this focus of camera is f _t, the initialization setting is done in the parameter and the position of 5 focusing video cameras, its position relation satisfies following constraint: f _t=f _s+ d.

The principle of work of apparatus of the present invention is: 5 focusing video cameras are divided into two groups of whole audience scape acquisition camera and details acquisition camera.1 of whole audience scape acquisition camera and stationkeeping, the image information of whole audience scape is obtained in assurance, and scene domain can not change because of the image resolution ratio adjustment; 4 of details acquisition camera, gather the image of 4 sub-square field scene area respectively, according to geometry site between video camera imaging principle and video camera, in the image that the data map of details acquisition camera is taken to whole audience scape acquisition camera, realize the multi-resolution image harvester of big visible scene.If details acquisition camera images acquired coordinate is (u ₁, v ₁), (u ₂, v ₂), (u ₃, v ₃), (u ₄, v ₄), be mapped to whole audience scape image coordinate and become $(u_1,{\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="A20081010124100112.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+\sqrt{2}m),(u_2+\sqrt{2}m,v_2+\sqrt{2}m),(u_3+\sqrt{2}m,v_3),(u_4,v_4),$ Wherein m is the vertical range of details acquisition camera according to whole audience scape acquisition camera.

A kind of control method of using many focusing video cameras to realize the collection of multiresolution scene that the present invention proposes may further comprise the steps:

(1) the visual target object in the automatic given scenario is perhaps by the visual target object in the artificial given scenario of manually-operated;

(2) carry out background according to the scope of destination object and reject, obtain the destination object prospect;

(3), determine that the destination object imaging shows subregion, and in subregion, arrange scene details acquisition camera and whole audience scape acquisition camera according to the destination object prospect;

(4), generate high resolving power scene overview by the camera acquisition view data in the subregion; (5), specify the harmless resolution enlargement factor of destination object wherein to high resolving power scene overview;

(6) destination object edge and background difference after amplifying are merged;

(7) judge that whether the fusion goal object moves, if move then change step (2); If move then change step (4), repeated acquisition image information.

The principle of the inventive method is: image processing methods such as use background segment are determined destination object, by the details acquisition camera image data of arranging in the subregion according to pixels displacement formula change, finishing the harmless resolution of destination object amplifies, use the interpolation blending algorithm destination object and scene image are merged, realize multiresolution scene acquisition controlling.

The present invention's beneficial effect compared with prior art is:

(1) apparatus of the present invention are provided with whole audience scape acquisition camera and the details acquisition camera is carried out image acquisition to same photographed scene, the destination object information that obtains comprises scene overview information and sub-scene detailed information, use the scene overview information can guarantee that the visible scene scope does not change, when effectively avoiding the single camera zoom shot, the deficiency that visible scene diminishes when obtaining high-definition picture realizes many focusing video cameras " zoom " shoot function in big visible scene.And have two parts of image acquisition information to backup each other to Same Scene, image quality is more stable.

(2) the details acquisition camera of apparatus of the present invention setting obtains the sub-scene information of destination object, can do further the detailed information of scene and describe, and be real image data, the Zooming method that has overcome the amplification of digital zoom video camera electronics can influence the shortcoming of the degree of accuracy of image.

(3) the inventive method is with respect to other image-pickup method, need not manual adjustment focus of camera or aperture, only need manual or automatic selected target object in the 1st step of inventive method, in the 5th step, set harmless resolution enlargement factor, just can finish the multiresolution scene image automatically gathers fast, it is simple to have manual steps, and destination object is flexibly selected, and image detail can be adjusted, manageable advantage.

Description of drawings

Fig. 1 is many focusing video cameras device synoptic diagram of the present invention;

Fig. 2 is 5 formed geometric relationship figure of camera focus in the example of the present invention;

Fig. 3 is the synoptic diagram that concerns of video camera imaging resolution of the present invention and focal length;

Fig. 4 is a multi-resolution image acquisition method process flow diagram of the present invention.

Embodiment

As shown in Figure 1, the present invention is provided with 5 focusing video cameras, and 4 video cameras in right side are on the same surface level, be positioned on same foursquare 4 summits, and B, C, D, E 4 points in the difference corresponding diagram, and 4 focus of camera are identical, are made as f _sAn other position for video camera is in the left of above-mentioned 4 video cameras, and A point in the corresponding diagram, its axial line be perpendicular to 4 shooting machine sides and through foursquare central point, O point in the corresponding diagram, and be d to the distance of this point, establishing this focus of camera is f _tThe initialization setting is done in parameter and position to 5 focusing video cameras, and its position relation satisfies following constraint: f _t=f _s+ d.

The principle of work and the process of apparatus of the present invention are: 5 focusing video cameras are divided into two groups of whole audience scape acquisition camera and details acquisition camera.1 of whole audience scape acquisition camera and stationkeeping, the image information of whole audience scape is obtained in assurance, and scene domain can not change because of the image resolution ratio adjustment; 4 of details acquisition camera, gather the image of 4 sub-square field scene area respectively, according to geometry site between video camera imaging principle and video camera, in the image that the data map of details acquisition camera is taken to whole audience scape acquisition camera, realize the multi-resolution image harvester of big visible scene.Whole audience scape acquisition camera at first begins images acquired, and remaining 4 details acquisition camera obtains subject surface different resolution pixel thus and represents according to multi-resolution image collection control method utonomous working, finishes multiresolution and regulates.

As shown in Figure 2, after step 1 initialization is provided with, the focus coplane of 5 video cameras in scene, and have geometric relationship shown in Figure 2.The imaging plane lower left corner of getting 4 video cameras in right side is the image coordinate system initial point, corresponding diagram mid point O ₁, O ₂, O ₃, O ₄, and every video camera viewpoint center is m to the distance of initial point.But the visible scene space of 5 focusing video cameras be 4 video cameras in right side from focus in the depth of field interval, back awl platform union and the common factor space of left video camera cone platform.

As shown in Figure 3, according to video camera imaging principle: O _c-X _cY _cZ _cBe camera coordinate system, O-XY is the imaging plane coordinate system, and O '-UV is a plane of delineation coordinate system.The coordinate of P point under camera coordinate system is (X _c, Y _c, Z _c), P ' point coordinate be (x, y, f); P ' some coordinate under image coordinate system be (u, v).Have according to the solid geometry proportionate relationship:

$\frac{f}{Z_c}=\frac{x}{X_c}=\frac{y}{Y_c}---\left(0.1\right)$

If the visual angle of video camera on the x axle is θ _x, resolution is u _x, the visual angle on the y axle is θ _y, resolution is u _y, then the pixel coordinate of P ' on image be (u v) is:

$\left\{\begin{array}{c}u=\frac{x-x_0}{f_x\&CenterDot;{\mathrm{tg\&theta;}}_x\&CenterDot;u_x}=\frac{x}{\mathrm{dx}}+u_0\\ v=\frac{y-y_0}{f_y\&CenterDot;{\mathrm{tg\&theta;}}_y\&CenterDot;u_y}=\frac{y}{\mathrm{dy}}+v_0\end{array}\right.---\left(02\right)$

Wherein, f _x, f _yRepresent video camera at x, the focal length on the y both direction, dx, dy are the physical size of each pixel on plane of delineation x axle and y axle.(u ₀, v ₀) be the physics intersection point of the camera optical axis and the plane of delineation.

As shown in Figure 2, the plane of delineation coordinate origin of establishing 4 video cameras in right side is respectively O ₁, O ₂, O ₃, O ₄, the plane of delineation coordinate origin of right-hand video camera is O ₃, and to make it be the true origin of unified coordinate system.If 4 video cameras in right side are m according to upper strata camera optical axis distance, then any point (the u of 4 video cameras in right side under former coordinate system ₁, v ₁), (u ₂, v ₂), (u ₃, v ₃), (u ₄, v ₄) respectively the coordinate of corresponding unified coordinate system be $({\mathrm{<figure-callout\; id="1"\; label="u"\; filenames="A20081010124100112.png"\; state="\{\{state\}\}">u</figure-callout>}}_1,{\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="A20081010124100112.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+\sqrt{2}m),(u_2+\sqrt{2}m,v_2+\sqrt{2}m),(u_3+\sqrt{2}m,v_3),(u_4,v_4),$ With this pixel corresponding relation, obtain pretreated foundation as multi-resolution image.

As shown in Figure 4, provide a kind of schematic flow sheet of multi-resolution image acquisition method, realize the adjusting and the control of scene image resolution, comprise the destination object that (1) automatic or manual appointment is observed; (2) background of destination object is rejected; (3) imaging of division destination object shows subregion; (4) subregion camera acquisition view data generates high resolving power scene overview; (5) the harmless resolution of intended target object is amplified; (6) destination object edge and background difference merge; (7) judge that whether destination object moves, the repeated acquisition image information is totally seven steps, below launches explanation respectively:

Step 4.1 is by the visual target object in the automatic given scenario of the method for pattern-recognition, perhaps by the visual target object in the artificial given scenario of manually-operated.When background image and destination object image aberration are obvious, adopt the software filter consistent that background is rejected with background color, select remaining visual target object images automatically.When background is comparatively complicated, or destination object is when close with the image aberration, and artificial is provided with key point in the scene overview image, and key point formation one convex polygon that is connected with each other is as the visual target object of appointment.

Step 4.2 according to the 4.1 destination object scopes of determining, is separated destination object prospect and background.If, background is rejected with automatic intended target object method in step 4.1, this step can be omited; If artificial intended target object method is rejected the image of convex polygon outside as a setting, the remainder image is the destination object prospect.

Step 4.3 according to the 4.2 destination object prospects of determining, by in the image block shown in Figure 2, is determined the demonstration subregion of destination object imaging.In the scene overview image, be the square sub-scene that 4 block sizes equate symmetry by Fig. 2 with image division, and guarantee that each sub-scene has an estrade scene detail pictures acquisition camera to be responsible for.The destination object image also is divided into 4 and shows subregions in the process that scene is divided, certainly, comprise that also the parton zone do not contain the situation of destination object pixel.

Step 4.4 shows subregion according to the 4.3 destination object imagings of calculating, and determines that the video camera of being responsible for this regional imaging begins image data.Based on scene overview image pixel coordinate, it is (u that sub-scene image acquisition camera is gathered the gained image coordinate ₁, v ₁), (u ₂, v ₂), (u ₃, v ₃), (u ₄, v ₄), correspond to scene overview image coordinate position and be respectively $({\mathrm{<figure-callout\; id="1"\; label="u"\; filenames="A20081010124100112.png"\; state="\{\{state\}\}">u</figure-callout>}}_1,{\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="A20081010124100112.png"\; state="\{\{state\}\}">v</figure-callout>}}_1+\sqrt{2}m),(u_2+\sqrt{2}m,v_2+\sqrt{2}m),(u_3+\sqrt{2}m,v_3),(u_4,v_4),$ And the details pixel is inserted in the scene overview image by coordinate position, obtain high resolving power scene overview.

Step 4.5 is amplified or is adjusted the 4.4 destination object imaging region picture elements of determining according to input parameter.The enlargement factor α of input destination object, minimum is 1, expression is not amplified.Center with the destination object zone is an initial point, adjusts the distance of destination object area pixel point, and each former pixel descends α times of the original picture element distance of left and right sides four direction translation up.If the distance of two neighbor pixels is m up and down; About the distance of two neighbor pixels be n, the distance after then amplifying becomes α m and α n respectively.

Step 4.6 is amplified or is adjusted according to the 4.4 pairs of destination objects, when the α multiple greater than 1 the time, will inevitably block former having powerful connections.The destination object that amplifies as the fresh target object, is used linear interpolation algorithm commonly used in the graphics, and the interpolation of doing fresh target target edges and new background merges computing, realizes seamlessly transitting of amplification target object and background.If the RGB information of fresh target object pixel point is N _R, N _G, N _B, the RGB information of the background pixel point that is adjacent is N _R', N _G', N _B', then the n-1 of Cha Ruing RGB information that merges pixel is followed successively by:

$(\frac{((n-1)N_R+N_R^{\&prime;})}{n},\frac{((n-1)N_G+N_G^{\&prime;})}{n},\frac{((n-1)N_B+N_B^{\&prime;})}{n}),$

$(\frac{((n-2)N_R+{2N}_R^{\&prime;})}{n},\frac{((n-2)N_G+{2N}_G^{\&prime;})}{n},\frac{((n-2)N_B+2N_B^{\&prime;})}{n}),$

…

$(\frac{(N_R+(n-1)N_R^{\&prime;})}{n},\frac{(N_G+(n-1)N_G^{\&prime;})}{n},\frac{(N_B+(n-1)N_B^{\&prime;})}{n}).$

Step 4.7 is done poor method by frame of video and is judged whether the intended target object moves, if in two two field pictures of front and back, the picture element position of destination object is offset, be frame of video when being poor result, illustrate that destination object moves, then forward step 4.2 to greater than certain threshold values (as N=10); Otherwise forward step 4.4 to.If the RGB value of information of the corresponding picture element of two two field pictures is N before and after the video of gathering _R, N _G, N _B", then frame of video is poor result for N=(N _R'-N _R)+(N _G'-N _G)+(N _B'-N _B).

Because image acquisition process is a cyclic process, in Fig. 4, do not provide schematically illustrating of flow process end.

It should be noted that at last; the above only is a preferred implementation of the present invention; should be understood that; for those skilled in the art; utilize under the principle prerequisite that many focusing video cameras realize that multi-resolution images gather not breaking away from the present invention; can also make some improvement or be equal to replacement, these improvement and be equal to replacement and also should be considered as protection scope of the present invention.

Claims (9)

1, a kind of device that adopts many focusing video cameras to realize the multi-resolution image collection, it is characterized in that: comprise 5 focusing video cameras with position constraint relation, wherein 4 video cameras are on the same surface level, be positioned on same foursquare 4 summits, and 4 focus of camera are identical, are made as f _sOther 1 position for video camera is in a side of above-mentioned 4 video cameras, the axial line of this video camera is perpendicular to 4 shooting machine sides and through foursquare central point O, and the 5th video camera is d to the distance on all the other planes, 4 video camera places, and establishing this focus of camera is f _t, the initialization setting is done in the parameter and the position of 5 focusing video cameras, its position relation satisfies following constraint: f _t=f _s+ d.

2, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered is characterized in that step is as follows:

(1) the visual target object in the automatic given scenario is perhaps by the visual target object in the artificial given scenario of manually-operated;

(2) carry out background according to the scope of destination object and reject, obtain the destination object prospect;

(3), determine that the destination object imaging shows subregion, and in subregion, arrange scene details acquisition camera and whole audience scape acquisition camera according to the destination object prospect;

(4), generate high resolving power scene overview by the camera acquisition view data in the subregion;

(5), specify the harmless resolution enlargement factor of destination object wherein to high resolving power scene overview;

(6) destination object edge and background difference after amplifying are merged;

(7) judge that whether the fusion goal object moves, if move then change step (2); If move then change step (4), repeated acquisition image information.

3, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the method for the destination object that the automatic or manual appointment in the described step (1) is observed is: when the image aberration of background image and destination object is obvious, adopt the software filter consistent that background is rejected, selected automatically remaining visual target object images with background color; When background is comparatively complicated, or destination object is when close with the image aberration, by manually in the scene overview image key point being set, as the visual target object of appointment.

4, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the background elimination method of the destination object in the described step (2) is: for artificial appointment target object method, key point is connected to a convex polygon, rejects the image background of convex polygon outside.

5, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the imaging of the division destination object of described step (3) shows subregion, and the method for arranging scene details acquisition camera and panoramic camera in subregion is: the destination object image is divided into 4 equal and opposite in directions in the process that scene is divided, symmetry, and be shaped as foursquare demonstration subregion, and arrange that 5 have the focusing video cameras that position constraint concerns, wherein 4 video cameras are on the same surface level, be positioned on same foursquare 4 summits, and 4 focus of camera are identical, are made as f _sOther 1 position for video camera is in a side of above-mentioned 4 video cameras, the axial line of this video camera is perpendicular to 4 shooting machine sides and through foursquare central point O, and the 5th video camera is d to the distance on all the other planes, 4 video camera places, and establishing this focus of camera is f _t, the initialization setting is done in the parameter and the position of 5 focusing video cameras, its position relation satisfies following constraint: f _t=f _s+ d.

6, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the subregion camera acquisition view data of described step (4), the method that generates high resolving power scene overview is: the corresponding relation of determining scene overview image coordinate and sub-scene detail pictures coordinate, based on scene overview image pixel coordinate, it is (u that sub-scene image acquisition camera is gathered the gained image coordinate ₁, v ₁), (u ₂, v ₂), (u ₃, v ₃), (u ₄, v ₄), correspond to scene overview image coordinate position and be respectively $(u_1,v_1+\sqrt{2}m),(u_2+\sqrt{2}m,v_2+\sqrt{2}m),(u_3+\sqrt{2}m,v_3),(u_4,v_4),$ And the details pixel is inserted in the scene overview image by coordinate position, obtain high resolving power scene overview.

7, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the method that the harmless resolution of the intended target object of described step (5) is amplified is: establishing the resolution enlargement factor is α, the distance of two neighbor pixels is m up and down, about the distance of two neighbor pixels be n, the distance after then amplifying becomes α m and α n respectively.

8, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: the method that amplification target target edges and background difference are merged in the described step (6) is: the interpolation of doing fresh target target edges and new background merges computing, realizes seamlessly transitting of amplification target object and background.If the RGB information of fresh target object pixel point is N _R, N _G, N _B, the RGB information of the background pixel point that is adjacent is N _R', N _G', N _B', then the n-1 of Cha Ruing RGB information that merges pixel is followed successively by:

$(\frac{((n-1)N_R+N_R^{\&prime;})}{n},\frac{((n-1)N_G+N_G^{\&prime;})}{n},\frac{((n-1)N_B+N_B^{\&prime;})}{n}),$

$(\frac{((n-2)N_R+{2N}_R^{\&prime;})}{n},\frac{((n-2)N_G+{2N}_G^{\&prime;})}{n},\frac{((n-2)N_B+2N_B^{\&prime;})}{n}),$

…

$(\frac{(N_R+(n-1)N_R^{\&prime;})}{n},\frac{(N_G+(n-1)N_G^{\&prime;})}{n},\frac{(N_B+(n-1)N_B^{\&prime;})}{n}).$

9, a kind of acquisition method that adopts many focusing video cameras to realize the device that multi-resolution image is gathered according to claim 2, it is characterized in that: judge in the described step (7) that the method whether destination object moves is: be poor N as a result as the foundation of judging whether destination object moves with frame of video, the RGB value of information of establishing the corresponding picture element of video front and back two two field pictures of collection is N _R, N _G, N _BAnd N _R', N _G', N _B', then frame of video is the poor (N of N=as a result _R'-N _R)+(N _G'-N _G)+(N _B'-N _B).

Images