CN101577795A

CN101577795A - Method and device for realizing real-time viewing of panoramic picture

Info

Publication number: CN101577795A
Application number: CNA200910147255XA
Authority: CN
Inventors: 李凯
Original assignee: Shenzhen Huawei Communication Technologies Co Ltd
Current assignee: Huawei Device Shenzhen Co Ltd
Priority date: 2009-06-17
Filing date: 2009-06-17
Publication date: 2009-11-11
Also published as: WO2010145499A1

Abstract

The invention discloses a method and a device for realizing real-time viewing of panoramic pictures, belonging to the field of electronics. The method comprises the following steps: two-dimensional images of the current scene are obtained; a depth map of the two-dimensional images is obtained; pre-stitching is carried out according to the two-dimensional images of the current scene, the two-dimensional images of the last scene of the current scene, the depth map of two-dimensional images of the current scene and the depth map of two-dimensional images of last scene of the current scene to generate and display pre-stitched panoroma three-dimensional images; when confirmation information from the users is received, the broad-viewing three-dimensional images are stitched and the three-dimensional images of the last scene of the current scene are continuously obtained until the panoramic three-dimensional images are obtained; in the invention images, especially the three-dimensional images are stitched in real time, displayed in real time and previewed in real time, so that panoramic images appear wherever the camera aperture goes and image-stitching mistakes can be corrected anytime, thus improving user experience.

Description

A kind of method and apparatus of realizing the live preview of panoramic picture

Technical field

The present invention relates to electronic applications, particularly a kind of method and apparatus of realizing the live preview of panoramic picture.

Background technology

Along with developing rapidly of digital imaging technology, digital imaging apparatus has obtained application more and more widely.Yet, because the restriction of digital imaging apparatus self makes the visual field width of digital picture and video can not satisfy the desired application requirements that obtains wide angle picture even panoramic picture of user.For set up seamless wide angle picture people use method towards hardware (as instead/dioptric system, fish-eye lens etc.), thereby obtain wide angle picture by newly-increased hardware auxiliary figure imaging device, adopt this mode, because fish-eye lens etc. is normally caught information as much as possible on limited imaging surface, will cause the serious distortion of image like this.Prior art also provides a kind of digital jointing image technique, can set up the wide-angle form of expression of scenery or scene by the input picture set intelligently, can set up the stitching image of a panel height resolution under this mode by several original input pictures apace, this stitching image not only has the overall image of scenery preferably, but also has kept the detailed information in the original image.But in original input picture,, reasons such as variations such as illumination condition or camera automatic exposure mechanism obtain to exist in the stitching image slit owing to causing, in order effectively to solve such gap problem, prior art has proposed a kind of Panoramagram montage (Panoramic Mosaic) technology, and soon the image sequence of taking at a fixed observer point is stitched together and forms the images in a wide-angle even the 360 degree visuals field.This method is normally used for the virtual roaming on the Internet, and the visitor who logins the website can browse the full-view image of scenery in the real world.

The inventor is realizing process discovery of the present invention, and there is following shortcoming and defect at least in above-mentioned prior art:

The splicing technology of panorama drawing that prior art provides can only realize the splicing at two dimensional image, along with user's instructions for use is more and more higher, this technology can not reach the purpose that the user wishes to splice three-dimensional image, and then can't satisfy the demand that the user obtains the panorama three-dimensional image.Moreover the prior art is a post processing of image process, and whether user's scene that can not know for sure enough is taken or covers, and the effect that can only just know panoramic picture after having taken all images how; For splicing, if in splicing zero lap zone between the original image, will occur the band ditch in the panoramic picture that then finally obtains, and this is with ditch in the end just can see, will cause user's photographed scene again again, splice again, waste time and energy; And because shooting or technical reason cause splicing effect not good, can not proofread and correct at any time during splicing, splicing brings sorry to user experience at any time.

Summary of the invention

For the splicing (particularly realizing the splicing of three-dimensional image) that realizes image, and can realize the real-time splicing of image, real-time demonstration, live preview, thereby can freely selected panorama three-dimensional image for the user, the embodiment of the invention provides a kind of method and apparatus of live preview of panoramic picture.Described technical scheme is as follows:

On the one hand, provide a kind of method that realizes the live preview of panoramic picture, described method comprises:

Obtain the two dimensional image of current scene;

Obtain the depth map of described two dimensional image;

Two dimensional image according to the last scene of the two dimensional image of described current scene, described current scene, and the depth map of the two dimensional image of a last scene of the depth map of the two dimensional image of described current scene, described current scene, carry out the pre-splicing of a last scene of described current scene and described current scene, generate the wide scape three-dimensional image of pre-splicing; And the wide scape three-dimensional image that shows described pre-splicing is for described user's preview;

Behind the affirmation signal that receives the user, then with described wide scape three-dimensional image splicing, and continue to obtain the three-dimensional image of the next scene of described current scene, till getting access to the panorama three-dimensional image.

On the one hand, provide a kind of method that realizes the live preview of panoramic picture again, described method comprises:

Obtain the two dimensional image of current scene;

The two dimensional image of the last scene of the two dimensional image of described current scene and described current scene is carried out the pre-in real time splicing of a last scene of described current scene and described current scene, generate the wide scape two dimensional image of pre-splicing; And the wide scape two dimensional image that shows described pre-splicing is for described user's preview;

Behind the affirmation signal that receives the user,, continue to obtain the two dimensional image of the next scene of described current scene, till getting access to the panorama two dimensional image then with described wide scape two dimensional image splicing.

On the one hand, provide a kind of device of realizing the live preview of panoramic picture again, described device comprises:

First acquisition module is used to obtain the two dimensional image of current scene;

Second acquisition module is used to obtain the depth map of the two dimensional image that described first acquisition module obtains;

Pre-concatenation module, the depth map of two dimensional image that is used for a last scene of the depth map of the two dimensional image of the two dimensional image of a last scene of the two dimensional image according to described current scene, described current scene and described current scene, described current scene, carry out the pre-in real time splicing of a last scene of described current scene and described current scene, generate the wide scape three-dimensional image of pre-splicing;

Display module, the wide scape three-dimensional image of the pre-splicing that is used to show that described pre-concatenation module obtains is for described user's preview;

Concatenation module is used for behind the affirmation signal that receives the user, then with described wide scape three-dimensional image splicing, and continues to obtain the three-dimensional image of the next scene of described current scene, till getting access to the panorama three-dimensional image.

Acquisition module is used for the demand according to the user, obtains the two dimensional image of current scene;

Pre-concatenation module is used for the two dimensional image of the last scene of the two dimensional image of described current scene and described current scene is carried out the pre-in real time splicing of a last scene of described current scene and described current scene, generates the wide scape two dimensional image of pre-splicing;

Display module is used to show that the wide scape two dimensional image of described pre-splicing is for described user's preview;

Concatenation module is used for behind the affirmation signal that receives the user, then with described wide scape two dimensional image splicing, continues to obtain the two dimensional image of the next scene of described current scene, till getting access to the panorama two dimensional image.

The beneficial effect of the technical scheme that the embodiment of the invention provides is:

The present invention has realized the real-time splicing of image (particularly three-dimensional image), real-time demonstration, live preview, realize where view-finder photographed, where just enter in the panoramic picture automatically, and the mistake of remedial frames splicing in time, thereby user's experience improved.

Description of drawings

In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.

Fig. 1 is the method flow diagram of the live preview of the realization panoramic picture that provides of the embodiment of the invention;

Fig. 2 is the method flow diagram of the live preview of the realization panoramic picture that provides of the embodiment of the invention;

Fig. 3 be the embodiment of the invention 1 provide realize the method flow diagram of the live preview of panoramic picture based on the range finding video camera;

Fig. 4 is a kind of range finding video camera schematic diagram that the embodiment of the invention 1 provides;

Fig. 5 be the embodiment of the invention 2 provide realize the method flow diagram of the live preview of panoramic picture based on binocular camera;

Fig. 6 is a kind of binocular camera schematic diagram that the embodiment of the invention 2 provides;

Fig. 7 is the another kind of binocular camera schematic diagram that the embodiment of the invention 2 provides;

Fig. 8 be the embodiment of the invention 3 provide realize the method flow diagram of the live preview of panoramic picture based on monocular-camera;

Fig. 9 is the mobile schematic diagram of the monocular-camera that provides of the embodiment of the invention 3;

Figure 10 is the device schematic diagram of the live preview of the realization panoramic picture that provides of the embodiment of the invention 4;

Figure 11 is the device detailed maps of the live preview of the realization panoramic picture that provides of the embodiment of the invention 4;

Figure 12 is another detailed maps of device of the live preview of the realization panoramic picture that provides of the embodiment of the invention 4;

Figure 13 is the device schematic diagram of the live preview of the realization panoramic picture that provides of the embodiment of the invention 5;

Figure 14 is the device detailed maps of the live preview of the realization panoramic picture that provides of the embodiment of the invention 5;

Figure 15 is another detailed maps of device of the live preview of the realization panoramic picture that provides of the embodiment of the invention 5.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.

For the method that the invention described above embodiment is provided is elaborated, see also following embodiment:

In order to realize the splicing of image, particularly realize the splicing of three-dimensional image, and can realize the real-time splicing of image, real-time demonstration, live preview, thereby can freely selected panoramic picture for the user, the embodiment of the invention provides a kind of method that realizes the live preview of panoramic picture, referring to Fig. 1, when panoramic picture three-dimensional image that the user wishes to obtain, this method thes contents are as follows

S1:, obtain the two dimensional image of current scene according to user's demand.

S2: the depth map that obtains this two dimensional image.

S3: according to the two dimensional image of the last scene of the two dimensional image of current scene, current scene, and the depth map of the two dimensional image of a last scene of the depth map of the two dimensional image of current scene, current scene, carry out the pre-in real time splicing of a last scene of current scene and current scene, generate the wide scape three-dimensional image of pre-splicing.

S4: the wide scape three-dimensional image that shows pre-splicing is for user's preview.

S5: behind the affirmation signal that receives the user, then with the splicing of wide scape three-dimensional image, and continue to obtain the three-dimensional image of the next scene of current scene, till getting access to the panorama three-dimensional image.

Further, after carrying out above-mentioned steps S3, this method can also comprise: whether the wide scape three-dimensional image of judging pre-splicing satisfies the splicing requirement, if, two dimensional image according to the last scene of the two dimensional image of current scene, current scene, and the depth map of the two dimensional image of a last scene of the depth map of the two dimensional image of current scene, current scene, splice the wide scape three-dimensional image that generates pre-splicing in advance; If not, then remind the user to obtain the two dimensional image of current scene once more.

To sum up,, realized the splicing of three-dimensional image, and can realize the real-time splicing of three-dimensional image, real-time demonstration, live preview, thereby can freely select the panorama three-dimensional image for the user by above-mentioned steps S1-S5.

Referring to Fig. 2, when the user wished that the panoramic picture that obtains is two dimensional image, this method comprised in addition:

C1:, obtain the two dimensional image of current scene according to user's demand.

C2: the two dimensional image of the last scene of the two dimensional image of current scene and current scene is carried out the pre-in real time splicing of a last scene of current scene and current scene, generate the wide scape two dimensional image of pre-splicing.

C3: the wide scape two dimensional image that shows pre-splicing is for user's preview.

C4: behind the affirmation signal that receives the user,, continue to obtain the two dimensional image of the next scene of current scene, till getting access to the panorama two dimensional image then with the splicing of wide scape two dimensional image.

To sum up,, realized the splicing of two dimensional image, and can realize the real-time splicing of two dimensional image, real-time demonstration, live preview, thereby can freely select the panorama two dimensional image for the user by above-mentioned steps C1-C4.

For the method that the invention described above embodiment is provided is described in detail, see also following embodiment.

Embodiment 1

The embodiment of the invention provides a kind of method that realizes the live preview of panoramic picture, the embodiment of the invention for convenience of explanation, wishing to obtain the panorama three-dimensional image with the user is that example is carried out, and referring to Fig. 3, thes contents are as follows:

101:, obtain the two dimensional image of first scene by the range finding video camera according to user's demand.

Wherein, present embodiment hypothesis user uses the range finding video camera, and then correspondingly, the demands such as hobby that this user can basis get access to the two dimensional image of first scene by this range finding video camera.Referring to Fig. 4, be a kind of range finding video camera schematic diagram that the embodiment of the invention provides, wherein, this range finding video camera comprises two parts at least: as shown in Figure 4, A represents to be used to take the colour TV camera of two dimensional image; B represents to be used for measuring the distance mearuring equipment of each scenery of the captured scene of colour TV camera and this range finding video camera distance, and this distance mearuring equipment can be realized (present embodiment does not limit this) by radar, mode such as infrared when realizing distance measurement function.Wherein, the described image of present embodiment can also be a video etc.

102:, obtain the depth information of this two dimensional image according to the two dimensional image of first scene of obtaining.

Wherein, as previously mentioned, the user utilizes this range finding video camera can measure the distance of each scenery and this range finding video camera in the two dimensional image of first scene of being obtained, according to this distance, can obtain the depth information of this two dimensional image that the user obtains, those skilled in the art are as can be known, distance is more little, then the degree of depth is more little, and parallax is big more, and the level of half-tone information is high more; Otherwise if distance is big more, then the degree of depth is big more, and parallax is more little, and the level of half-tone information is low more.For example, comprise at least 2 objects in first scene that the user obtains, suppose one of them object X, another is object Y, and object X and user's close together, object Y distance with the user for object X is far away, and then correspondingly, the level of object X use half-tone information is 255 in this that obtains first scene; It is 150 that object X uses the level of half-tone information.Owing to according to distance, obtain depth information and belong to techniques well known, do not repeat them here.

103:, obtain the depth map of the two dimensional image of this first scene according to the depth information of the two dimensional image of first scene that obtains.

Wherein, as previously mentioned, because scene is made up of each object, thus can be according to the depth information of each object that obtains in first scene, and then obtain the depth map of the two dimensional image of this first scene.

104:, obtain the two dimensional image of second scenario according to user's demand.

Wherein, the user holds the range finding video camera, and according to the mobile range finding video camera of demand (supposition is in order to carry out the splicing of panorama three-dimensional image) of oneself, then the colour TV camera in this range finding video camera photographs the two dimensional image of second scenario.

105:, obtain the depth information of this two dimensional image according to the two dimensional image of the second scenario of obtaining.

Wherein, this step and above-mentioned steps 102 are similar, repeat no more.

106:, obtain the depth map of the two dimensional image of this second scene according to the depth information of the two dimensional image of second scene that obtains.

107: the depth map of the depth map of the two dimensional image of the two dimensional image of first scene of obtaining according to step 101 and the two dimensional image of second scene that step 104 is obtained and this first scene of obtaining according to step 103 and the two dimensional image of second scene that step 106 is obtained, first scene and second scene are carried out pre-in real time splicing, obtain the pre-spliced three-dimensional image of first scene and second scene.

Wherein, the depth map based on the two dimensional image of the depth map of the two dimensional image of first scene and second scenario after shining upon in real time, obtains the three-dimensional image to the pre-splicing of first scene and second scene.

Further, before obtaining the pre-spliced three-dimensional image of first scene and second scene, this method also comprises:

The two dimensional image of first scene of obtaining according to step 101 (for convenience of explanation, abbreviate first two dimensional image as) and the two dimensional image of second scene obtained of step 104 is (for convenience of explanation, abbreviate second two dimensional image as), judge whether pre-splicing satisfies default splicing requirement, if satisfy, then carry out this step 107; Otherwise, remind the user, return execution in step 104, till satisfying default splicing requirement.

Wherein, above-mentioned default splicing requires to include but not limited to:

1, whether there is the overlay region between first two dimensional image and second two dimensional image, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement.Wherein, the pore belt ditch occurs if obtain image after not existing the overlay region then to splice between two dimensional image, splicing effect is poor.

2, whether there is the overlay region between first two dimensional image and second two dimensional image, if not, do not satisfy default splicing requirement; If judge then whether the overlay region is enough big, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement.Wherein, if having the overlay region but the overlay region is too little between two dimensional image, then can't splice.

Wherein, at above-mentioned 1,2, can be according to the method for feature point detection, some logarithm according to first two dimensional image and second two dimensional image coupling judges whether to exist the overlay region, and if there is the overlay region, whether the overlay region is enough big, and then can remind the user to take again.

3, whether there is the overlay region between first two dimensional image and second two dimensional image, if not, do not satisfy default splicing requirement; If, judge then whether the overlay region lacks obvious characteristics information, if, then not satisfying default splicing requirement, can't splice; If not, then satisfy default splicing requirement.

4, whether there is the overlay region between first two dimensional image and second two dimensional image, if not, do not satisfy default splicing requirement; If judge then whether the edge matching effect precision of overlay region reaches the required precision of splicing, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement.Wherein and since about (or up and down) two width of cloth images carry out solid coupling time, can cause the edge matching precision not high enough owing to reasons such as image motion are fuzzy, easily prospect when background or background when the prospect processing, the edge matching effect that causes occurring the overlay region is poor.

Wherein, at above-mentioned 3,4, when doubling of the image district lacks some obvious characteristics information or image when very fuzzy, be difficult to detect some characteristic informations, cause to find correct overlapping match information, make and to realize that splicing alignment and corresponding color merge, so need to remind the user this moment, in time adjusting video camera has the scenario objects of obvious characteristic information with shooting or adjusts focal length, adjustment taking moving speed, makes the clear picture of obtaining to be suitable for finding obvious characteristics information.

5, whether the focal length between first two dimensional image and second two dimensional image changes excessive (or too fast), if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement.Wherein, when carrying out the image splicing, tend to usually because the focal length problem causes splicing effect not good.

When 6, the range finding video camera is taken first two dimensional image or second two dimensional image, judge whether shooting environmental satisfies shooting condition, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement.Wherein, tend to phenomenons such as ghost image/ghost occur, and then cause splicing effect not good because shooting environmental does not reach shooting condition.

When 7, the range finding video camera is taken first two dimensional image or second two dimensional image, judge whether to exist ambient light (or video camera light tone parameter) to change, if, then do not satisfy default splicing requirement, otherwise, satisfy default splicing requirement.Wherein, tend to because when taking, ambient light or video camera light tone parameter change and cause splicing effect not good.

Wherein, at above-mentioned 5-7, when doubling of the image district lacks some obvious characteristics information or image when very fuzzy, be difficult to detect some characteristic informations, cause to find correct overlapping match information, make and to realize that splicing alignment and corresponding color merge, so need inform the user this moment, in time adjusting video camera has the scenario objects of obvious characteristic information with shooting or adjusts focal length, adjustment taking moving speed, makes the clear picture of obtaining to be suitable for finding obvious characteristics information.

8, judge whether the first captured two dimensional image and second two dimensional image are non-texture region (as sky), if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement.

Wherein,, then can cause follow-up three-dimensional coupling to lose efficacy, need to remind the user adjust photographed scene, avoid photographed scene the scene of similar sky to occur as far as possible if the user takes is non-texture region such as sky.

9, the characteristics of splicing in view of three-dimensional image, according to the depth map of the two dimensional image of first scene (for convenience of explanation, abbreviate first depth map as) and the depth map of the two dimensional image of second scene is (for convenience of explanation, abbreviate second depth map as), judge whether object (the particularly object in the overlay region) exists degree of depth level inconsistent phenomenon, if then do not satisfy default splicing requirement, otherwise, satisfy default splicing requirement.Wherein, be that level is taken certain scene when originally the user takes, subsequently, to overlook or certain anglec of rotation when taking same scene, tend to cause the degree of depth level of the image that obtained to differ, make follow-up splicing difficulty carry out, can remind the user to take same scene with a kind of shooting attitude as much as possible this moment, like this, could obtain better spliced panoramic or wide scape figure, further, could obtain better panorama or wide scape 3 dimensional drawing.

10, in view of the characteristics of three-dimensional image splicing, according to first depth map and second depth map, judge to take (or cavity) problem whether occurs blocking, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement.

Wherein, owing to block problems such as (or cavities), it is the peculiar problem of three-dimensional image, what is called block be meant since some o'clock in the scene that scenery in the scene and the mutual alignment between video camera relation are caused in a video camera as seen, and in the another one video camera sightless situation.Block the phenomenon discontinuous place of change in depth in scene of mostly occurring.When video camera during the closer to object scene, it is obvious more to block phenomenon; Otherwise video camera blocks phenomenon and weakens more more away from object scene, that is, to appear at the possibility of two or multiple-camera simultaneously big more for object in the scene.So, when carrying out solid when coupling, when finding occlusion issue, can remind the user, video camera is away from object scene or focus.

In sum, the embodiment of the invention is during to pre-splicing, may occur some cause the generation reason of splicing effect difference to be described, and provided corresponding prompting scheme according to each reason, thereby to guarantee that the user can obtain splicing effect preferably, after all taking, effective grade that has just overcome prior art just can splice, if problem, has only the defective that afterwards could correct, thereby by the method that the embodiment of the invention provides, make the splicing problem to occur as early as possible, solve as early as possible, improve user's experience, avoided the unnecessary trouble in later stage.Above-mentioned prompting function can be realized by written form, still can realize by speech form, and present embodiment is not done any requirement to this.

108: in display window, the three-dimensional image of the pre-splicing of first scene that obtains and second scene is shown, carry out preview for the user.

Wherein, in order to improve user's experience, first scene that obtains and the real-time pre-spliced three-dimensional image of second scene are offered the user by display window, carry out preview for the user.Wherein, during this display window specific implementation, can be 3 D stereo LCD display window (this 3 D stereo LCD display window user watches three-dimensional image by wearing spectacles), can also be 3 D stereo LCD display window of three-dimensional stereo grating or the like, and present embodiment does not limit this.

Wherein, three-dimensional image in the pre-splicing of related first scene that will obtain of this step 108 and second scene shows, when carrying out preview, be by second scene being mapped on the 3 D stereo LCD display unit with respect on the corresponding position of the preview image of first scene that just is being shown on this display unit for the user.Promptly to the location map of the three-dimensional image of current real-time demonstration of catching, carry out in real time, need not to store this three-dimensional image during actual the realization, can watch next Zhang San that will take to tie up stereo-picture in real time and be in and where will be more suitable for need not this three-dimensional image under the actual photographed in panorama 3 D stereo preview image.Like this, just make the user can catch the three-dimensional image content of final needed those scenes of panorama three-dimensional image at an easy rate, just can view the content that whether photographs required scene very easily, also watch captured content whether can be stitched together preferably with the 3 D stereo panorama sketch that has spliced correctly, preferably in real time and form bigger, the wider panorama three-dimensional image in the visual field.

Further, for better prompting is provided to the user, the suture (this suture is defined as splicing line) of can also be in the display window real-time three-dimensional image that demonstrates current second scene that obtains and the three-dimensional image (at present embodiment, 3 dimensional drawing of this previous splicing similarly is the three-dimensional image of first scene) of previous splicing; And alternatively, can also further connect the formation closed curve highlighted user of being shown in real time to the three-dimensional image of suture and current picked-up other borders except suture.

109: judge whether to receive the splicing confirmation signal of user's input, if then execution in step 110; Otherwise, return execution in step 104, till the splicing confirmation signal that receives user's input.

Wherein, the related splicing confirmation signal of this step 109 can be for after the user watches display window, after the splicing effect of the pre-splicing of approval, by user's input; Further, also can be non-user's input, be to discern automatically by the employed range finding video camera of user that the embodiment of the invention provides to obtain, thereby can increase intelligent and convenience that present embodiment does not limit this.

Step 110: the pre-spliced three-dimensional image that step 107 is obtained splices, and obtains first scene and the spliced three-dimensional image of second scene.

Step 111: repeat above-mentioned steps 101 to step 110, realize the splicing of the three-dimensional image of a plurality of scenes, till satisfying the user and getting access to the panorama three-dimensional image.

Wherein, so far pass through above-mentioned steps 101 to step 111, realized real-time splicing, the live preview of three-dimensional image, self adaptation realizes that the user obtains the panorama three-dimensional image of being wanted; Initiatively obtain and show in real time by the user, pre-in real time splicing, and the three-dimensional image that pre-splicing is in real time obtained shows, for the three-dimensional image that should pre-splicing obtains that shows, if the user confirm when not required, then remove; If when the user need to confirm, preserve.In other words, the method that the embodiment of the invention provides has realized which the view-finder of video camera photographed, and which just enters into the purpose of panorama three-dimensional image.Further, also reach the real-time reminding user and adjust image pickup method, proofread and correct the purpose of the three-dimensional image of the poor effect that is generated by above-mentioned prompting function.

What pay particular attention to is, in the real-time splicing of the three-dimensional image that carries out a plurality of scenes, because the restriction of the size of display window, the increasing of number along with the three-dimensional image of splicing, need in time adjust, to satisfy the user by the purpose of this fixing display window preview to the stitching image that is generated, promptly along with panorama three-dimensional image field range is increasing, the 3 D stereo LCD display window of fixed size just little by little can not once intactly show whole panorama three-dimensional image, this moment need be to the real-time auto zoom of panorama three-dimensional image, so that can intactly watch the panorama three-dimensional image that is obtained, also can according to user's needs by translation mode or scalable manner watch splice certain part of the panorama three-dimensional image that obtains, also can come translation to watch the three-dimensional image of 3 D stereo LCD display window by the direction of motion that built-in direction or motion sensor induction obtain panorama three-dimensional image live preview terminal.

Further, in order to save memory space, better realize the splicing of image, the method that the embodiment of the invention provides, clipping function also is provided, promptly indicate panorama 3 D stereo preview image will avoid that part of clipping rectangle to the user, by analyzing pre-spliced three-dimensional image, on this three-dimensional image, to draw along the scope of any frame of panorama 3 D stereo preview image maximum rectangle all very close to each other, finish in case calculate, then shear, correspondingly, the preview image that need provide to the user is provided image after the shearing, during specific implementation, can shear window and just be shown as covering on the panorama 3 D stereo preview image.

The invention described above embodiment is the explanation that example is carried out with the range finding video camera, not restricted passage should realize by the range finding video camera during specific implementation, can also realize that depth camera comprises and is used for the measuring instrument taking the colour TV camera of two dimensional image and be used to measure captured each scene depth information of scene of colour TV camera by types such as depth camera shootings.Method is similar, repeats no more.

Further, the method that the embodiment of the invention provides can also include but not limited to: will store the three-dimensional image or the panorama three-dimensional image that generate; The three-dimensional image or the panorama three-dimensional image that generate are controlled (for example delete, revise or the like); The three-dimensional image or the panorama three-dimensional image that generate are exported, to offer external equipment (as offer outside other as the USB memory device or the network equipment).

Wherein, art technology can be known, present embodiment is the explanation that example is carried out with real-time splicing, live preview, the real-time reminding of three-dimensional image, because the difference of three-dimensional image and two dimensional image is the depth information problem, therefore, the method that the embodiment of the invention provides can also be applied to real-time splicing, live preview, the real-time reminding of two dimensional image; Method is similar, repeats no more.

In sum, the method that the embodiment of the invention provides has realized real-time splicing, the live preview of three-dimensional image, and self adaptation realizes that the user obtains the panorama three-dimensional image of being wanted; Initiatively obtain and show in real time by the user, pre-in real time splicing, and the three-dimensional image that pre-splicing is in real time obtained shows, for the three-dimensional image that should pre-splicing obtains that shows, if the user confirm when not required, then remove; If when the user need to confirm, preserve.In other words, the method that the embodiment of the invention provides has realized which the view-finder of video camera photographed, and which just enters into the purpose of panorama three-dimensional image.Further, also reach the real-time reminding user and adjust image pickup method, proofread and correct the purpose of the three-dimensional image of the poor effect that is generated by above-mentioned prompting function.

Embodiment 2

The foregoing description 1 is the explanation that example is carried out with range finding video camera (or depth camera), be different from the mode that range finding video camera (or depth camera) obtains depth information of passing through that embodiment 1 provides, the embodiment of the invention provides a kind of method that realizes the live preview of panoramic picture based on binocular camera, referring to Fig. 5, the contents are as follows:

201:,, take left image and the right image that obtains this scene respectively by left camera lens in the binocular camera and right camera lens at first scene according to user's demand.

Present embodiment for convenience of explanation, use binocular camera to take with the user and be example, wherein, its operation principle of binocular camera is carried out the imaging of three-dimensional image for using two platform independent video cameras simulation human eye, at a certain scene, take this scene respectively by left camera lens in the binocular camera and right camera lens, obtain left image and right image at this scene.Referring to Fig. 6 and Fig. 7, the binocular camera schematic diagram that provides for the embodiment of the invention, Figure 6 shows that binocular camera is parallel double camera chain (promptly two camera optical axis is parallel), wherein, f represents focal length, Z is a distance to imaging plane, and d is that the corresponding imaging point of image pair transforms to conplane distance, and B is the spacing of two video camera photocentres.Referring to Fig. 7, binocular camera shown in Figure 7 specifically is to converge dual camera system (promptly two camera optical axis have angle) in addition.Fig. 7 and Fig. 6 principle are similar, repeat no more.

202: left image and right image are carried out the three-dimensional image coupling, obtain this first scene depth information, and, obtain the depth map of the two dimensional image of this first scene correspondence according to the depth information of this first scene.

Wherein, the core of three-dimensional stereoscopic visual is how to obtain the 3 D stereo information of imaging object, needs two-dimensional image information and depth information at least and constitute 3 D stereo information, and depth information must obtain from the multiple image at a certain imaging object.Referring to as follows:

At first, obtain depth information and need find certain some imaging point corresponding in multiple image in the scene;

Wherein, find in the scene certain a bit in different images process of corresponding imaging point finish by images match.In three-dimensional stereoscopic visual, the correlation between the multiple image and the relation on attributes of view data are very big, can utilize a lot of constraintss to carry out images match.Solid matching method has: based on the method for provincial characteristics with based on the method for feature.And comprise based on the method for other features: more typical optimizing matching algorithm, dynamic programming method, relaxative iteration method, regularization method or figure cutting method etc.

Secondly, obtain its coordinate in the space according to this coordinate in multiple image again, thereby obtain the depth information of this point.

So-called three-dimensional coupling be to about the public part in two width of cloth doublings of the image zone pursue pixel and mate (promptly according to a series of constraint mapping relations of binocular camera, left camera lens and the captured image of right camera lens are carried out the solid coupling), and then acquisition Same Scene point poor in picture position, the left and right sides (being parallax), according to the mapping relations of the parallax and the degree of depth, can obtain the degree of depth (i.e. the depth information of this scene) of left and right sides image by further calculating.

203:, obtain the three-dimensional image of this first scene according to the depth map of the two dimensional image of this first scene correspondence that obtains and the two dimensional image of this first scene.

204:,, take left image and the right image that obtains this scene respectively by left camera lens in the binocular camera and right camera lens at second scene according to user's demand.

205: left image and right image are carried out the three-dimensional image coupling, obtain this second scene depth information, and, obtain the depth map of the two dimensional image of this second scene correspondence according to the depth information of this second scene.

206:, obtain the three-dimensional image of this second scene according to the depth map of the two dimensional image of this second scene correspondence that obtains and the two dimensional image of this second scene.

207: in order to obtain the panorama three-dimensional image, with the two dimensional image of second scene and the two dimensional image splicing of first scene, obtain a wide scape two dimensional image, and, obtain wide scape depth map the depth map of the two dimensional image of second scene and the two dimensional image depth map splicing of first scene.

Wherein, wide scape depth map that this step obtained and wide scape two dimensional image are by the pixel correspondence.

208: wide scape two dimensional image and wide scape depth map according to above-mentioned steps obtains, carry out the pre-splicing of wide scape three-dimensional image, obtain the three-dimensional image of the pre-splicing of first scene and second scene.

209: in display window, the three-dimensional image of the pre-splicing of first scene that obtains and second scene is shown, carry out preview for the user.

210: judge whether to receive the splicing confirmation signal of user's input, if then execution in step 211; Otherwise, return execution in step 204, till the splicing confirmation signal that receives user's input.

Wherein, return execution in step 204, till the splicing confirmation signal that receives user's input, promptly the three-dimensional image of the pre-splicing of first scene that ought will obtain and second scene shows, when carrying out preview to offer the user, then according to the splicing confirmation signal that judges whether to receive user's input, to judge whether the three-dimensional image that shows reaches user's requirement, if do not receive the splicing confirmation signal, then the user can return and take second scene again, till the splicing that reaches the user requires.

Further, the method that the embodiment of the invention provides in carrying out above-mentioned pre-splicing, can provide prompting function to the user equally, includes but not limited to the prompting function that the foregoing description 1 is provided, and present embodiment repeats no more.Be in the process of the binocular camera live preview of realizing the panorama three-dimensional image, if when finding that the panorama generated or wide scape image occur as the described splicing problem of embodiment 1, need make timely adjustment, wherein, this adjustment strategy can point out the user whether to need to make according to the splicing prompting adjustment of further 3 D stereo panorama or wide scape image by the panorama that live preview generated or the effect quality of wide scape three-dimensional image.Also the spliced panoramic or the wide scape image that can generate in real time by the image that a video camera in the binocular camera is caught in real time adjusts, that is to say, whether unnecessary making by panorama that live preview generated or wide scape three-dimensional image is adjusted, and only is by judging whether panorama or wide scape image that image that a video camera is caught generates in real time have the splicing problem.Can adjust the splicing problem that is run into when splicing earlier, faster like this.

Step 211: the pre-spliced three-dimensional image that step 209 is obtained splices, and obtains first scene and the spliced three-dimensional image of second scene.

Step 212: repeat above-mentioned steps 201 successively to step 211, realize the splicing of the three-dimensional image of a plurality of scenes, till satisfying the user and getting access to the panorama three-dimensional image.

Further, the method that the embodiment of the invention provides can also be to generate the panoramic picture that panoramic picture that the left camera lens of this binocular camera catches and corresponding right video camera are caught earlier in real time when generating the panorama three-dimensional image; Again the left and right panoramic picture of being caught is carried out the solid coupling and obtain depth map (or parallax), finally just can obtain the panorama three-dimensional image.

Further, the method that the embodiment of the invention provides not only can show pre-splicing and spliced panorama three-dimensional image on display window, if comprise step 203 and step 206, then can also show the three-dimensional image of arbitrary scene.

Further, the method that the embodiment of the invention provides, similar to Example 1, clipping function equally also can be provided, splice highlighted prompt facility, convergent-divergent, rotation and translation functions, method is similar, repeats no more.

Embodiment 3

The foregoing description 2 is the explanation that example is carried out with the binocular camera, when specific implementation, can also be that the user passes through the live preview that the simplest monocular-camera is realized panoramic picture, correspondingly, the embodiment of the invention provides a kind of method that realizes the live preview of panoramic picture based on monocular-camera, referring to Fig. 8, the contents are as follows:

Step 301: at first scene, the user takes this scene by monocular-camera at position A.

Step 302: this monocular-camera is moved to position B from position A, take this scene once more at position B by monocular-camera.

Wherein, referring to Fig. 9, for the embodiment of the invention provide realize the method for live preview of panoramic picture based on monocular-camera the time, the mobile schematic diagram of this monocular-camera when taking a certain scene, as shown in Figure 9, when monocular-camera is positioned at position A and position B, gather the image of the target signature point that comprises first scene to be taken respectively, wherein, position A and position B are at a distance of d, definition d is the length of base, and this length of base d is relevant with the displacement of this monocular-camera.If two positions that this monocular-camera moves are in advance decided, then only need demarcation once can constitute simulation, approximate binocular vision and measure camera system.The method based on the panorama three-dimensional image live preview of binocular camera that so just can utilize that embodiment 2 provided is carried out obtaining of panorama three-dimensional image.Preferably, this monocular-camera moves along the x direction, does not move along other directions, does not also rotate, and does not limit the move mode and the displacement of this monocular-camera when practical application.

Step 303: the image of the scene of taking respectively according to the image of the scene of taking, at position B at position A, carry out solid and mate, obtain the two dimensional image of overlapping region and the depth map of this two dimensional image.

Wherein, carry out the solid coupling, obtain the degree of depth or the disparity map of this overlapping region by image to the different cameras position.

Wherein, described three-dimensional coupling be meant to about the public part in two width of cloth doublings of the image zone pursue the pixel coupling, and then acquisition Same Scene point is poor in picture position, the left and right sides, is exactly said parallax, can obtain the degree of depth of left and right sides image easily by further calculating.

Step 304: the two dimensional image of the overlapping region that obtains according to step 303 and the depth map of this two dimensional image, three-dimensional image (for convenience of explanation, being called first three-dimensional image) that must this two dimensional image.

Step 305: this monocular-camera is moved to position C from position B, take this scene once more at position B by monocular-camera.

Step 306: the image of the scene of taking respectively according to the image of the scene of taking, at position C at position B, carry out solid and mate, obtain the two dimensional image of overlapping region and the depth map of this two dimensional image.

Step 307: the two dimensional image of the overlapping region that obtains according to step 306 and the depth map of this two dimensional image, three-dimensional image (for convenience of explanation, being called second three-dimensional image) that must this two dimensional image.

Step 308: the three-dimensional image (i.e. second three-dimensional image) that three-dimensional image (i.e. first three-dimensional image) that step 304 is obtained and step 307 obtain, splice in advance, obtain spliced three-dimensional image.

Step 309: in display window, the three-dimensional image of the pre-splicing that obtains is shown, carry out preview for the user.

Step 310: judge whether to receive the splicing confirmation signal of user's input, if then execution in step 311; Otherwise, return execution in step 305, till the splicing confirmation signal that receives user's input.

Further, the method that the embodiment of the invention provides in carrying out above-mentioned pre-splicing, can provide prompting function to the user equally, includes but not limited to the prompting function that the foregoing description 1 is provided, and present embodiment repeats no more.

Step 311: the pre-spliced three-dimensional image that step 308 is obtained splices, and obtains spliced three-dimensional image.

Step 312: repeat above-mentioned steps 301 successively to step 311, realize the splicing of the three-dimensional image of a plurality of scenes, till satisfying the user and getting access to the panorama three-dimensional image.

To sum up, by one by one, repeat equidirectional mobile monocular-camera, obtain the different piece of whole scene, thereby photographed panorama or wide scape image and the corresponding degree of depth or the disparity map that is generated, realize that finally the user gets access to the panorama three-dimensional image.Promptly the present frame coloured image (i.e. a width of cloth two dimensional image) by obtaining scene with and pairing depth map or disparity map after, in order to obtain the panorama three-dimensional image, splicing next frame cromogram and the above-mentioned present frame coloured image that obtains become a wide scape coloured image, simultaneously pairing depth map of next frame coloured image and the pairing depth map of present frame coloured image are spliced, obtain one and above-mentioned wide scape coloured image wide scape depth map or disparity map by the pixel correspondence, so far, obtained wide scape three-dimensional image, similarly, by mating the next frame image frame by frame, splicing just can obtain the panorama three-dimensional image in real time.

Further, the method that the embodiment of the invention provides not only can show pre-splicing and spliced panorama three-dimensional image on display window, can also show the three-dimensional image of arbitrary scene.

Further, the method that the embodiment of the invention provides, similar to Example 1, clipping function equally also can be provided, splice highlighted prompt facility, convergent-divergent, rotation and translation functions.Method is similar, repeats no more.

Wherein, art technology can be known, present embodiment is the explanation that example is carried out with real-time splicing, live preview, the real-time reminding of three-dimensional image, since the difference of three-dimensional image and two dimensional image with the depth information problem, therefore, the method that the embodiment of the invention provides can also be applied to real-time splicing, live preview, the real-time reminding of two dimensional image; Method is similar, repeats no more.

The various embodiments described above are only done the example explanation to the method for the live preview of panoramic picture, when specific implementation, include but not limited to realize the live preview of panoramic picture based on parabola folding emission panoramic camera, and then generate the panorama three-dimensional image in real time, realize the live preview of panoramic picture based on the cylindrical panoramic 3 D stereo video camera of hyperboloid folding emission video camera, and then generate the panorama three-dimensional image in real time, realize the live preview of panoramic picture based on many viewpoints video camera of the video camera array of a plurality of camera lens lens, and then generate panorama three-dimensional image or the like in real time.

Embodiment 4

In order to realize the splicing of image, particularly realize the splicing of three-dimensional image, and can realize the real-time splicing of image, real-time demonstration, live preview, thereby can freely selected panorama three-dimensional image for the user, the embodiment of the invention provides a kind of device of realizing the live preview of panoramic picture, referring to Figure 10, this device comprises:

First acquisition module 1001 is used to obtain the two dimensional image of current scene;

Second acquisition module 1002 is used to obtain the depth map of the two dimensional image that first acquisition module 1001 obtains;

Pre-concatenation module 1003, the two dimensional image that is used for a last scene of two dimensional image, current scene according to current scene, and the depth map of the two dimensional image of a last scene of the depth map of the two dimensional image of current scene, current scene, carry out the pre-in real time splicing of a last scene of current scene and current scene, generate the wide scape three-dimensional image of pre-splicing;

Display module 1004, the wide scape three-dimensional image of the pre-splicing that is used to show that pre-concatenation module 1003 obtains is for user's preview;

Concatenation module 1005 is used for behind the affirmation signal that receives the user, then with the splicing of wide scape three-dimensional image, and continues to obtain the three-dimensional image of the next scene of current scene, till getting access to the panorama three-dimensional image.

Further, referring to Figure 11, the pre-concatenation module 1003 that the embodiment of the invention provides comprises:

Judging unit 10031 is used to judge whether the wide scape three-dimensional image of pre-splicing satisfies the splicing requirement;

Pre-concatenation unit 10032, the result who specifically is used for judging when judging unit 10031 is for being, then according to the two dimensional image of the last scene of the two dimensional image of current scene, current scene, and the depth map of the two dimensional image of a last scene of the depth map of the two dimensional image of current scene, current scene, carry out the pre-in real time splicing of a last scene of current scene and current scene, generate the wide scape three-dimensional image of pre-splicing;

Reminding unit 10033, the result who is used for judging when judging unit 10031 then reminds the user to obtain the two dimensional image of current scene once more for not.

Further, referring to Figure 12, the device that the embodiment of the invention provides also comprises:

Cutting module 1006, the wide scape three-dimensional image that is used for pre-splicing that pre-concatenation module 1003 is obtained carries out cutting; Or,

Memory module 1007, the wide scape three-dimensional image that is used for pre-splicing that pre-concatenation module 1003 is obtained is stored; Or,

Removing module 1008, the wide scape three-dimensional image that is used for pre-splicing that pre-concatenation module 1003 is obtained is deleted; Or,

Modified module 1009, the wide scape three-dimensional image that is used for pre-splicing that pre-concatenation module 1003 is obtained is made amendment.

Further, the display module 1004 in the device that provides of the embodiment of the invention also is used to show the splicing line of the wide scape three-dimensional image of pre-splicing.

To sum up, the device that the embodiment of the invention is passed through has been realized real-time splicing, the live preview of three-dimensional image, and self adaptation realizes that the user obtains the panorama three-dimensional image of being wanted; Initiatively obtain and show in real time by the user, pre-in real time splicing, and the three-dimensional image that pre-splicing is in real time obtained shows, for the three-dimensional image that should pre-splicing obtains that shows, if the user confirm when not required, then remove; If when the user need to confirm, preserve.In other words, the device that the embodiment of the invention provides has realized which the view-finder of video camera photographed, and which just enters into the purpose of panorama three-dimensional image.Further, also reach the real-time reminding user and adjust filming apparatus, proofread and correct the purpose of the three-dimensional image of the poor effect that is generated by above-mentioned prompting function.

Embodiment 5

In order to realize the splicing of image, and can realize the real-time splicing of image, real-time demonstration, live preview, thereby can freely selected panorama three-dimensional image for the user, the embodiment of the invention provides a kind of device of realizing the live preview of panoramic picture, referring to Figure 13, when the user wished that the panoramic picture that obtains is two dimensional image, device comprised:

Acquisition module 1301 is used to obtain the two dimensional image of current scene;

Pre-concatenation module 1302 is used for the two dimensional image of the last scene of the two dimensional image of current scene and current scene is carried out the pre-in real time splicing of a last scene of current scene and current scene, generates the wide scape two dimensional image of pre-splicing;

Display module 1303 is used to show that the wide scape two dimensional image of pre-splicing is for user's preview;

Concatenation module 1304 is used for behind the affirmation signal that receives the user, then with the splicing of wide scape two dimensional image, continues to obtain the two dimensional image of the next scene of current scene, till getting access to the panorama two dimensional image.

Referring to Figure 14, the pre-concatenation module 1302 that the embodiment of the invention provides comprises:

Judging unit 13021 is used to judge whether the wide scape two dimensional image of pre-splicing satisfies the splicing requirement;

Pre-concatenation unit 13022 specifically is used for the result that judges when judging unit 13021 for being, then according to the two dimensional image of the last scene of the two dimensional image of current scene, current scene, carries out pre-in real time splicing, generates the wide scape two dimensional image that splices in advance;

Reminding unit 13023, the result who is used for judging when judging unit 13021 then reminds the user to obtain the two dimensional image of current scene once more for not.

Referring to Figure 15, the device that the embodiment of the invention provides also comprises:

Cutting module 1305, the wide scape two dimensional image that is used for pre-splicing that pre-concatenation module 1302 is obtained carries out cutting; Or,

Memory module 1306, the wide scape two dimensional image that is used for pre-splicing that pre-concatenation module 1302 is obtained is stored; Or,

Removing module 1307, the wide scape two dimensional image that is used for pre-splicing that pre-concatenation module 1302 is obtained is deleted; Or,

Modified module 1308, the wide scape two dimensional image that is used for pre-splicing that pre-concatenation module 1302 is obtained is made amendment.

Further, the display module 1303 in the device that provides of the embodiment of the invention also is used to show the splicing line of the wide scape two dimensional image of pre-splicing.

To sum up, the device that the embodiment of the invention is passed through has been realized real-time splicing, the live preview of two dimensional image, and adaptive realization user obtains the panorama two dimensional image wanted; Initiatively obtain and show in real time by the user, pre-in real time splicing, and the two dimensional image that pre-splicing is in real time obtained shows, for the two dimensional image that should pre-splicing obtains that shows, if the user confirm when not required, then remove; If when the user need to confirm, preserve.In other words, the device that the embodiment of the invention provides has realized which the view-finder of video camera photographed, and which just enters into the purpose of panorama two dimensional image.Further, also reach the real-time reminding user and adjust filming apparatus, proofread and correct the purpose of the two dimensional image of the poor effect that is generated by above-mentioned prompting function.

In sum, the technical scheme that the embodiment of the invention provided, self adaptation provides the video camera user to obtain the panorama three-dimensional image of being wanted, rather than passive obtaining, initiatively obtain and demonstration in real time pre-in real time splicing preview by the user, generate three-dimensional image according to required cutting automatically, when when not required, remove, preserve when needing.Just: which view-finder photographed, and which just enters into the panorama three-dimensional image.Can the real-time reminding user adjust style of shooting, to proofread and correct the image (two dimension or three-dimensional image) of the poor effect that is generated

Consider that monocular-camera generates the 3 D stereo video in real time, binocular camera generates the 3 D stereo video in real time, the 3 D stereo video camera of (or the degree of depth or parallax) of can finding range generates the 3 D stereo video in real time, many viewpoints video camera of the video camera array of a plurality of even camera lens lens, further, considered the live preview of 3 D stereo or multi-view point video, generate or generate in real time panorama 3 D stereo or multi-view point video or image in real time, and live preview, in time correct produced when generating these three-dimensional images or video in real time such as some mistakes that generated when the two-dimensional panoramic figure, as be with ditch, ghost image, the peculiar problem of colour-difference and three-dimensional image as the cavity or block, these mistakes are in time embodied, corrigendum in time, when bringing the 3 D stereo sense organ to experience to the user, also bring use for the user, easily obtain, easily use, experience easily, evaded the obstacle when professional technique is used relative technological products to unprofessional user greatly.The scheme that the embodiment of the invention provides can be applied on the mobile phone, palmtop PC, notebook, digital camera, Digital Video, PC, work station etc. with shoot function and real screen.

" reception " speech in the embodiment of the invention can be understood as and initiatively obtains from other modules also can be to receive the information that other modules are sent.

It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, module in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.

It will be appreciated by those skilled in the art that the module in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from present embodiment.The module of the foregoing description can be merged into a module, also can further split into a plurality of submodules.

The invention described above embodiment sequence number is not represented the quality of embodiment just to description.

Part steps in the embodiment of the invention can utilize software to realize that corresponding software programs can be stored in the storage medium that can read, as CD or hard disk etc.

The above only is preferred embodiment of the present invention, and is in order to restriction the present invention, within the spirit and principles in the present invention not all, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims

1, a kind of method that realizes the live preview of panoramic picture is characterized in that, described method comprises:

Obtain the two dimensional image of current scene;

Obtain the depth map of described two dimensional image;

Depth map according to the two dimensional image of the last scene of the depth map of the two dimensional image of the two dimensional image of the last scene of the two dimensional image of described current scene, described current scene and described current scene, described current scene, carry out the pre-in real time splicing of a last scene of described current scene and described current scene, generate the wide scape three-dimensional image of pre-splicing; And the wide scape three-dimensional image that shows described pre-splicing is for described user's preview;

2, the method for claim 1 is characterized in that, after the step of the pre-splicing of a described last scene of carrying out described current scene and described current scene, described method also comprises:

Whether the wide scape three-dimensional image of judging described pre-splicing satisfies the splicing requirement, if, according to the depth map of the two dimensional image of the last scene of the depth map of the two dimensional image of the two dimensional image of the last scene of the two dimensional image of described current scene, described current scene and described current scene, described current scene, generate the wide scape three-dimensional image of pre-splicing; If not, then remind described user to obtain the two dimensional image of described current scene once more.

3, method as claimed in claim 2 is characterized in that, described splicing requires to include but not limited to:

Judge between the two dimensional image of a last scene of the two dimensional image of described current scene and described current scene whether have the overlay region, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement; Or,

Judge between the two dimensional image of a last scene of the two dimensional image of described current scene and described current scene whether have the overlay region, if not, then do not satisfy default splicing requirement; If judge then whether described overlay region satisfies default size, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement; Or,

Judge between the two dimensional image of a last scene of the two dimensional image of described current scene and described current scene whether have the overlay region, if not, then do not satisfy default splicing requirement; If judge then whether described overlay region lacks obvious characteristics information, if then do not satisfy default splicing requirement; If not, then satisfy default splicing requirement; Or,

Judge between the two dimensional image of a last scene of the two dimensional image of described current scene and described current scene whether have the overlay region, if not, then do not satisfy default splicing requirement; If judge then whether the edge matching effect precision of described overlay region reaches the required precision of splicing, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement; Or,

It is excessive or too fast to judge whether focal length between the two dimensional image of a last scene of the two dimensional image of described current scene and described current scene changes, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement; Or,

Judge whether described shooting environmental satisfies shooting condition, if then satisfy default splicing requirement; Otherwise, do not satisfy default splicing requirement; Or,

Judge whether to exist ambient light or the light tone parameter phenomenon that changes, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement; Or,

Whether the two dimensional image of judging a last scene of the two dimensional image of described current scene and described current scene is non-texture region, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement; Or,

Depth map according to the two dimensional image of the last scene of the depth map of the two dimensional image of described current scene, described current scene judges whether to exist degree of depth level inconsistent phenomenon, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement; Or,

Depth map according to the two dimensional image of the last scene of the depth map of the two dimensional image of described current scene, described current scene judges whether to occur blocking or cavitation, if then do not satisfy default splicing requirement; Otherwise, satisfy default splicing requirement.

4, the method for claim 1 is characterized in that, when using the range finding video camera, described two dimensional image and the described step of obtaining the depth map of described two dimensional image of obtaining current scene comprises:

Obtain the two dimensional image of current scene by the colour TV camera of described range finding video camera;

Obtain the depth information of described two dimensional image by the distance mearuring equipment of described range finding video camera; According to described depth information, obtain the depth map of the two dimensional image of described current scene.

5, the method for claim 1 is characterized in that, when using binocular camera, the step of the depth map of described two dimensional image that obtains current scene and described two dimensional image comprises:

Left camera lens and right camera lens by described binocular camera are taken the second from left dimension image and the right two dimensional image that obtains described current scene respectively;

According to described the second from left dimension image and right two dimensional image, obtain the two dimensional image of described current scene;

Described the second from left dimension image and right two dimensional image are carried out the stereo-picture coupling, obtain disparity map; According to described disparity map, obtain the depth information of the two dimensional image of described current scene;

According to described depth information, obtain the depth map of the two dimensional image of described current scene.

6, the method for claim 1 is characterized in that, when using monocular-camera, the step of the depth map of described two dimensional image that obtains current scene and described two dimensional image comprises:

Take current scene at position A and position B respectively by described monocular-camera; Wherein, described position A and described position B are at a distance of the length of base;

The image of the current scene of taking according to the image of the current scene of taking at described position A, at position B carries out the stereo-picture coupling, obtains the two dimensional image of the overlapping region captured two positions;

According to the described two dimensional image and the described length of base, obtain the depth map of described two dimensional image.

7, the method for claim 1 is characterized in that, after the step of the wide scape three-dimensional image of the pre-splicing of described generation, described method also comprises:

The wide scape three-dimensional image of described pre-splicing is carried out cutting; Or,

The wide scape three-dimensional image of described pre-splicing is stored; Or,

The wide scape three-dimensional image of described pre-splicing is deleted; Or,

The wide scape three-dimensional image of described pre-splicing is made amendment; Or,

The splicing line that shows the wide scape three-dimensional image of described pre-splicing.

8, a kind of method that realizes the live preview of panoramic picture is characterized in that, described method comprises:

Obtain the two dimensional image of current scene;

The two dimensional image of the last scene of the two dimensional image of described current scene and described current scene is carried out pre-in real time splicing, generate the wide scape two dimensional image of pre-splicing of a last scene of described current scene and described current scene; And the wide scape two dimensional image that shows described pre-splicing is for described user's preview;

9, method as claimed in claim 8 is characterized in that, after the step of the pre-splicing of a described last scene of carrying out described current scene and described current scene, described method also comprises:

Whether the wide scape two dimensional image of judging described pre-splicing satisfies the splicing requirement, if according to the two dimensional image of the last scene of the two dimensional image of described current scene, described current scene, generate the wide scape two dimensional image of pre-splicing; If not, then remind described user to obtain the two dimensional image of current scene once more.

10, method as claimed in claim 8 is characterized in that, after the step of the wide scape two dimensional image of the pre-splicing of described generation, described method also comprises:

The wide scape two dimensional image of described pre-splicing is carried out cutting; Or,

The wide scape two dimensional image of described pre-splicing is stored; Or,

The wide scape two dimensional image of described pre-splicing is deleted; Or,

The wide scape two dimensional image of described pre-splicing is made amendment; Or,

The splicing line that shows the wide scape two dimensional image of described pre-splicing.

11, a kind of device of realizing the live preview of panoramic picture is characterized in that, described device comprises:

12, device as claimed in claim 11 is characterized in that, described pre-concatenation module comprises:

Judging unit is used to judge whether the wide scape three-dimensional image of described pre-splicing satisfies the splicing requirement;

Pre-concatenation unit, be used for working as the result of described judgment unit judges for being, then according to the depth map of the two dimensional image of the last scene of the depth map of the two dimensional image of the two dimensional image of the last scene of the two dimensional image of described current scene, described current scene and described current scene, described current scene, carry out the pre-in real time splicing of a last scene of described current scene and described current scene, generate the wide scape three-dimensional image of pre-splicing;

Reminding unit, the result who is used for when described judgment unit judges is not, then reminds described user to obtain the two dimensional image of described current scene once more.

13, device as claimed in claim 11 is characterized in that, described device also comprises:

The cutting module, the wide scape three-dimensional image that is used for pre-splicing that described pre-concatenation module is obtained carries out cutting; Or,

Memory module, the wide scape three-dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is stored; Or,

Removing module, the wide scape three-dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is deleted; Or,

Modified module, the wide scape three-dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is made amendment.

14, device as claimed in claim 11 is characterized in that, described display module also is used to show the splicing line of the wide scape three-dimensional image of described pre-splicing.

15, a kind of device of realizing the live preview of panoramic picture is characterized in that, described device comprises:

Acquisition module is used to obtain the two dimensional image of current scene;

16, device as claimed in claim 15 is characterized in that, described pre-concatenation module comprises:

Judging unit is used to judge whether the wide scape two dimensional image of described pre-splicing satisfies the splicing requirement;

Pre-concatenation unit, be used for working as the result of described judgment unit judges for being, then according to the two dimensional image of the last scene of the two dimensional image of described current scene, described current scene, carry out the pre-in real time splicing of a last scene of described current scene and described current scene, generate the wide scape two dimensional image of pre-splicing;

Reminding unit, the result who is used for when described judgment unit judges is not, then reminds described user to obtain the two dimensional image of current scene once more.

17, device as claimed in claim 15 is characterized in that, described device also comprises:

The cutting module, the wide scape two dimensional image that is used for pre-splicing that described pre-concatenation module is obtained carries out cutting; Or,

Memory module, the wide scape two dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is stored; Or,

Removing module, the wide scape two dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is deleted; Or,

Modified module, the wide scape two dimensional image that is used for pre-splicing that described pre-concatenation module is obtained is made amendment.

18, device as claimed in claim 15 is characterized in that, described display module also is used to show the splicing line of the wide scape two dimensional image of described pre-splicing.