CN103985103A - Method and device for generating panoramic picture - Google Patents
Method and device for generating panoramic picture Download PDFInfo
- Publication number
- CN103985103A CN103985103A CN201310049193.5A CN201310049193A CN103985103A CN 103985103 A CN103985103 A CN 103985103A CN 201310049193 A CN201310049193 A CN 201310049193A CN 103985103 A CN103985103 A CN 103985103A
- Authority
- CN
- China
- Prior art keywords
- image
- key frame
- interval
- frame
- attitude information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Studio Devices (AREA)
- Stereoscopic And Panoramic Photography (AREA)
Abstract
The invention provides a method and device for generating a panoramic picture. The method includes the following steps: S1. obtaining a video fragment which is shoot through using a point where the user is located as the center, wherein the visible range of the video fragment is larger than a fixed-point shooting visible range of a shooting device, and obtaining and recording a sensor parameter sequence in a shooting process; S2. according to direction information of shooting points in the sensor parameter sequence, extracting more than two key frames from the video fragment; S3. splicing the extracted key frames so as to obtain a panoramic picture. The method and device for generating a panoramic picture are capable of improving the effect of the panoramic picture and helping the user to realize obtaining of the panoramic picture rapidly.
Description
[technical field]
The present invention relates to Computer Applied Technology field, particularly a kind of method and apparatus that generates panoramic pictures.
[background technology]
Panoramic pictures refers to the expression means mode that shows surrounding environment as much as possible by wide-angle, and usually said panoramic pictures is the concept of a broad sense, and it comprises wide scape, 360 degree panoramas etc.Panoramic pictures brings very the visual effect of shock obtain gradually liking of user and be widely used to people because of its visual angle broadness, can generate panoramic pictures in conjunction with picture processing software again by capture apparatus or the mobile device with shoot function, and can in the application such as social activity, map, tourism, be promoted well.
The mode of existing generation panoramic pictures is to take after the picture of multiple different visual angles by user mostly, after being spliced, the picture of these different visual angles obtains, but the panoramic pictures that this mode obtains often can not produce good effect, such as when user's pictures taken owing to there is the reasons such as shake in attitude run-off the straight or hand, can cause the quality of picture of different angles of shooting own unexcellent, the effect of the panoramic pictures that follow-up splicing obtains is unexcellent yet.
[summary of the invention]
In view of this, the invention provides a kind of method and apparatus that generates panoramic pictures, so that improve the effect of panoramic pictures.
Concrete technical scheme is as follows:
A method for generating panorama image, the method comprises:
S1, obtain the video segment that user takes centered by loca, the visual range of this video segment is greater than the fixed point of capture apparatus and takes visual range, obtain and records photographing process in sensor parameters sequence;
S2, according to the orientation information of each shooting point in sensor parameters sequence, from video segment, extract more than 2 key frames;
S3, the key frame of extraction is spliced, obtain panoramic pictures.
According to one preferred embodiment of the present invention, described step S2 specifically comprises:
From described video segment, extract set N towards corresponding image as key frame, N is more than 2 integer; Or,
Described video segment is divided into N1 interval according to orientation information, and N1 is more than 2 integer, picks out respectively 1 image as key frame from a described N1 interval.
According to one preferred embodiment of the present invention, from a described N1 interval, picking out respectively 1 image specifically comprises as key frame:
S21, extract respectively N2 open image from a described N1 interval, N2 is more than 2 integer;
S22, according at least one in capture apparatus attitude information, acceleration information and picture quality corresponding in sensor parameters sequence, open image and select respectively an image as key frame from each interval N2.
According to one preferred embodiment of the present invention, described S22 specifically comprises at least one in following mode:
Mode 1) determine that the N2 in each interval in described sensor parameters sequence opens the capture apparatus attitude information that image is corresponding, open and image, pick out respectively image that corresponding attitude information departs from three-axis reference minimum as key frame from the N2 in each interval;
Mode 2) determine that the N2 in each interval in described sensor parameters sequence opens the acceleration information that image is corresponding, opens the image of picking out corresponding acceleration minimum image as key frame from the N2 in each interval;
Mode 3) open and image, pick out respectively image that picture quality is the highest as key frame from the N2 in each interval;
Mode 4) according to the combination in any in corresponding attitude information, acceleration information and picture quality, the N2 in each interval is opened to image and give respectively weights, wherein to depart from the image weights that three-axis reference is less larger for corresponding attitude information, the image weights that corresponding acceleration is less are larger, the image weights that picture quality is higher are larger, open the image of picking out respectively weights maximum image as key frame from the N2 in each interval.
According to one preferred embodiment of the present invention, between described step S2 and step S3, also comprise:
S4, according to the capture apparatus attitude information that in sensor parameters sequence, each key frame is corresponding, key frame is rotated to correction.
According to one preferred embodiment of the present invention, described step S4 specifically comprises at least one in following operation:
Operation 1) determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
each key frame is rotated frame by frame in the plane perpendicular to image
degree, γ is the angle with respect to Z axis deflection in the attitude information that key frame is corresponding;
Operation 2) determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, α is the angle with respect to X-axis deflection in the attitude information that key frame is corresponding;
Operation 3) determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, β is the angle with respect to Y-axis deflection in the attitude information that key frame is corresponding.
A device for generating panorama image, this device comprises:
Video segment acquiring unit, the video segment of taking centered by loca for obtaining user;
Sensor parameters record cell, for obtaining and record the sensor parameters sequence of shooting process of described video segment;
Key-frame extraction unit for the orientation information according to the each shooting point of sensor parameters sequence, extracts more than 2 key frames from video segment;
Image Mosaics unit, for described key frame is spliced, obtains panoramic pictures.
According to one preferred embodiment of the present invention, described key-frame extraction unit concrete configuration is:
From described video segment, extract set N towards corresponding image as key frame, N is more than 2 integer; Or,
Described video segment is divided into N1 interval according to orientation information, and N1 is more than 2 integer, picks out respectively 1 image as key frame from a described N1 interval.
According to one preferred embodiment of the present invention, described key-frame extraction unit, pick out respectively 1 image as key frame from a described N1 interval time, is specifically carried out:
Operation 21, extract respectively N2 open image from a described N1 interval, N2 is more than 2 integer;
Operation 22, according at least one in capture apparatus attitude information, acceleration information and picture quality corresponding in sensor parameters sequence, open image and select respectively an image as key frame from each interval N2.
According to one preferred embodiment of the present invention, described key-frame extraction unit in the time of executable operations 22, concrete at least one in the following ways:
Mode 1) determine that the N2 in each interval in described sensor parameters sequence opens the capture apparatus attitude information that image is corresponding, open and image, pick out respectively image that corresponding attitude information departs from three-axis reference minimum as key frame from the N2 in each interval;
Mode 2) determine that the N2 in each interval in described sensor parameters sequence opens the acceleration information that image is corresponding, opens the image of picking out corresponding acceleration minimum image as key frame from the N2 in each interval;
Mode 3) open and image, pick out respectively image that picture quality is the highest as key frame from the N2 in each interval;
Mode 4) according to the combination in any in corresponding attitude information, acceleration information and picture quality, the N2 in each interval is opened to image and give respectively weights, wherein to depart from the image weights that three-axis reference is less larger for corresponding attitude information, the image weights that corresponding acceleration is less are larger, the image weights that picture quality is higher are larger, open the image of picking out respectively weights maximum image as key frame from the N2 in each interval.
According to one preferred embodiment of the present invention, this device also comprises:
Key frame correcting unit, be used for according to capture apparatus attitude information corresponding to the each key frame of sensor parameters sequence, the key frame that described key-frame extraction unit is extracted is rotated correction, the key frame after rotation correction is offered to described Image Mosaics unit and splice.
According to one preferred embodiment of the present invention, described key frame correcting unit is being rotated timing, concrete at least one that carry out in following operation:
Operation 1) determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
each key frame is rotated frame by frame in the plane perpendicular to image
degree, γ is the angle with respect to Z axis deflection in the attitude information that key frame is corresponding;
Operation 2) determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, α is the angle with respect to X-axis deflection in the attitude information that key frame is corresponding;
Operation 3) determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, β is the angle with respect to Y-axis deflection in the attitude information that key frame is corresponding.
As can be seen from the above technical solutions, in the present invention, the video segment of taking is carried out the extraction of key frame centered by user's loca, as the splicing basis that generates panoramic pictures, make on the one hand user's faster and more convenient operation, only need disposable video capture, and take the picture of a few width different visual angles without artificial fixed point, on the other hand, the mode of extracting key frame from video segment provides basis for selecting preferred picture to be spliced, and needn't necessarily be limited to user's limited picture of taking of fixing a point, thereby improve the effect of panoramic pictures.
[brief description of the drawings]
The method flow diagram of the generation panoramic pictures that Fig. 1 provides for the embodiment of the present invention one;
(a) in Fig. 2 is (b) and (c) schematic diagram of the visual range of the video segment that provides in the embodiment of the present invention one;
The schematic diagram that Fig. 3 a is a kind of capture apparatus run-off the straight of providing in the embodiment of the present invention one;
The schematic diagram that Fig. 3 b is the another kind of capture apparatus run-off the straight that provides in the embodiment of the present invention one;
The structure drawing of device of the generating panorama image that Fig. 4 provides for the embodiment of the present invention two.
[embodiment]
In order to make the object, technical solutions and advantages of the present invention clearer, describe the present invention below in conjunction with the drawings and specific embodiments.
Embodiment mono-,
The method flow diagram of the generation panoramic pictures that Fig. 1 provides for the embodiment of the present invention one, as shown in Figure 1, the method mainly comprises the following steps:
Step 101: obtain the video segment that user takes centered by loca, and sensor parameters sequence in records photographing process.
The video segment centered by loca that user takes in this step, the visual range of this video segment is at least greater than the fixed point of capture apparatus and takes visual range, in Fig. 2 shown in (a) is exactly the fixed point shooting visual range of capture apparatus, the angular range that when capture apparatus fixed point is taken, maximum can be taken.If finally generate 360 degree panoramic pictures, the visual range of the video segment of herein taking is more than or equal to 360 degree, and as shown in (c) in Fig. 2, capture apparatus is spent taking loca as center of circle rotary taking 360.Take visual range if the visual range of video segment is greater than the fixed point of capture apparatus, but be less than 360 degree, the final panoramic pictures generating is wide-angle picture, as shown in (b) in Fig. 2.
In shooting process, the sensor that is arranged at capture apparatus can get the orientation information of capture apparatus, also may further get attitude information, acceleration information etc.Orientation information can be presented as capture apparatus towards the angle that departs from preset direction, for example depart from the angle of direct north, attitude information can be presented as the angle of capture apparatus with respect to three-axis reference X, Y and Z axis deflection, the three-axis reference relating in the embodiment of the present invention is by forming X-axis and the Y-axis of surface level and form perpendicular to the Z axis of surface level, also can adopting geomagnetic coordinate system.Wherein, orientation information can be by obtaining direction sensors such as compass, attitude information can be by calculating rear acquisition to the parameter of obtaining such as the sensor such as gyroscope, gravity sensor, concrete computing method are prior arts, do not describe in detail at this, for example, can be expressed as (α, β at attitude information sometime, γ), wherein α, β and γ are respectively the angle of capture apparatus with respect to X-axis, Y-axis and Z axis deflection.Acceleration information can be by directly obtaining such as acceleration transducer, also can be to calculating rear acquisition by the parameter obtaining such as angular-rate sensor.
In shooting process, the corresponding one group of sensor parameters of each shooting point, after whole like this shooting completes, the sensor parameters of each shooting point has just formed sensor parameters sequence, and this sensor parameters sequence is carried out to record.
Step 102: according to the orientation information in sensor parameters sequence, extract key frame from video segment.
Extracting when key frame, the most basic principle is exactly that the key frame extracting can cover all image range of shooting substantially, can from video segment, extract set N individual towards corresponding image as key frame, wherein N is more than 2 integer.For example can by initially towards, with initially towards rotation angle be 90 degree, 180 degree, 270 degree, the image in video segment corresponding to this four direction is as key frame.
Also there is following mode:
First video segment is divided into N1 interval according to orientation information, N1 is more than 2 integer, taking four intervals as example, for example using initially towards as for initially towards rotation angle be that 90 degree are as first interval, using with initially towards rotation angle be that 90 degree to 180 degree are as second interval, using with initially towards rotation angle be 180 degree to 270 degree as the 3rd interval, using with initially towards rotation angle be that 270 degree to 360 degree are as the 4th interval.
From this N1 interval, pick out respectively 1 image as key frame, if rely on merely orientation information to extract key frame, can adopt the mode of selecting arbitrarily, preferably, can also select key frame further combined with at least one in attitude information, acceleration information and picture quality:
First from this N1 interval, extract respectively N2 and open image, N2 is more than 2 integer, can from each interval, extract arbitrarily N2 and open image, also can from each interval, open image etc. according to taking out N2 towards equalization, also can open image according to extracting N2 with other.The N2 extracting in each interval again extracts one as key frame in opening image, particularly, if in conjunction with attitude information, can open the attitude information that image is corresponding based on this N2, determines the image that departs from three-axis reference minimum as key frame.If in conjunction with acceleration information, can determine image that this N2 opens acceleration minimum in image as key frame, that is to say to choose and wherein shake minimum image as key frame.If combining image quality, can determine this N2 and open image that in image, picture quality is the highest as key frame, wherein picture quality can adopt the parameters such as histogram, sharpening angle or exposure to weigh.If simultaneously in conjunction with attitude information, acceleration information and picture quality, according to attitude information, acceleration information and picture quality, the N2 in each interval is opened to image respectively and give weights, the weights that attitude information departs from the image that three-axis reference is less are larger, the image weights that acceleration is less are larger, the weights of the image that quality is higher are larger, then pick out image that this N2 opens weights maximum in image as key frame.
Step 103: according to the capture apparatus attitude information that in sensor parameters sequence, each key frame is corresponding, key frame is rotated to correction.
Because user needs rotary taking equipment in whole shooting process, normally user is manual carries out in this rotation, inevitably can there is capture apparatus run-off the straight in shooting process, this inclination makes each key frame may exist picture angle to tilt, and this just need to be rotated correction to key frame.The mode tilting while shooting in view of capture apparatus may be different, the main rotation correction that has following two kinds of situations:
The first situation: if capture apparatus in shooting process with respect to Z axis run-off the straight, as shown in Fig. 3 a, capture apparatus in the vertical direction skew certain angle, needs to carry out the rotation correction with respect to Z axis deflection angle.
In this case, there is deviation in the image of taking out, need on depression angle, be rotated (in the plane perpendicular to image, being rotated) on depression angle: first determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
then each key frame is rotated frame by frame in the plane perpendicular to image
spend, so just obtained the key frame of same frame angle.
Certainly, in the time that key frame is rotated, can first judge whether this key frame exists the deviation with respect to Z axis, that is to say and judge in the attitude information that this key frame is corresponding, whether the angle with respect to Z axis exceedes preset range, if so, rotation
degree; Otherwise illustrate this key frame at the inclined degree of Z-direction in tolerance interval, without being rotated correction.
The second situation: if capture apparatus in shooting process with respect to X-axis or Y-axis run-off the straight, as shown in Figure 3 b, take so image out and have in the plane inclination at image, need in this plane, carry out rotation correction.
Conventionally in X-axis or Y-axis, carrying out rotation correction is to select one, and the screen that depends on picture pick-up device is that horizontal screen is taken or perpendicular screen is taken.Rotation correction in X-axis is: determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
then each key frame is rotated in the plane at image
degree.Rotation correction in Y-axis is: determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
then each key frame is rotated in the plane at image
degree.
Equally, before key frame is rotated, first judge in the attitude information that this key frame is corresponding, whether the angle with respect to X-axis or Y-axis exceedes preset range, if it is rotation
degree; Otherwise illustrate this key frame at the inclined degree of X-axis or Y direction in tolerance interval, without being rotated correction.
It should be noted that, this step is the further optimization process to key frame, and the panoramic picture effect generating after making is better, but is not necessary step of the present invention.
Step 104: key frame is spliced, obtain panoramic pictures.
It is that the picture of multiple different visual ranges is spliced into a pictures that key frame is spliced to object, mainly comprises: image pre-service, image registration and three steps of image co-registration reconstruct.
Wherein image pre-service comprises key frame is carried out to denoising, feature point extraction, overlapping detection and edge extracting etc.Image registration refers to the position that adopts certain matching strategy to find out the unique point correspondence in reference picture in image to be spliced (the present embodiment middle finger key frame), and then determine the transformation relation between two width images, carry out unified coordinate transform to treat stitching image.Merging reconstruct is that the smooth and seamless panoramic picture that obtains splicing reconstruct is merged in the overlapping region of image to be spliced.It is prior art that multiple images to be spliced are spliced to the above-mentioned steps that obtains panoramic picture, and the present invention does not do concrete restriction, is not described in detail in this.
Be more than the detailed description that method provided by the present invention is carried out, below in conjunction with embodiment bis-, device provided by the invention be described in detail.
Embodiment bis-,
The structure drawing of device of the generating panorama image that Fig. 4 provides for the embodiment of the present invention two, as shown in Figure 4, this device can comprise: video segment acquiring unit 01, sensor parameters record cell 02, key-frame extraction unit 03 and Image Mosaics unit 04.
The video segment that video segment acquiring unit 01 is taken centered by loca for obtaining user, the visual range of this video segment is at least greater than the fixed point of capture apparatus and takes visual range, if finally generate 360 degree panoramic pictures, the visual range of the video segment of herein taking is more than or equal to 360 degree, and capture apparatus is spent taking loca as center of circle rotary taking 360.Take visual range if the visual range of video segment is greater than the fixed point of capture apparatus, but be less than 360 degree, the final panoramic pictures generating is wide-angle picture.Wherein, capture apparatus can be equipment independently, and for example video camera, can be also the mobile device with camera function, such as mobile phone, panel computer etc.
The device providing in picture pick-up device and the present embodiment can be arranged in same mobile device, and for example this device is arranged in the mobile phone or panel computer with camera function with the form of software app.The device providing in picture pick-up device and the present embodiment also can arrange from different equipment, for example video camera or the mobile device with camera function have been taken after video segment, video segment are sent in the equipment at this device place and process and generate panoramic pictures.
The sensor that is arranged at capture apparatus in shooting process can get the orientation information of capture apparatus, also may further get attitude information or acceleration information, orientation information can be presented as capture apparatus towards the angle that departs from preset direction, for example depart from the angle of direct north, attitude information can be presented as the angle of capture apparatus with respect to three-axis reference X, Y and Z axis deflection.Wherein, orientation information can be by obtaining direction sensors such as compass, and attitude information can be by calculating rear acquisition to the parameter of obtaining such as the sensor such as gyroscope, gravity sensor.Acceleration information can be by directly obtaining such as acceleration transducer, also can be to calculating rear acquisition by the parameter obtaining such as angular-rate sensor.The corresponding one group of sensor parameters of each shooting point in shooting process like this, the sensor parameters of each shooting point has just formed sensor parameters sequence, by sensor parameters record cell 02 obtain and the shooting process of recording of video fragment in sensor parameters sequence.
Then, key-frame extraction unit 03, according to the orientation information of each shooting point in sensor parameters sequence, extracts more than 2 key frames from video segment.
In the time extracting key frame, must there are following two kinds of modes:
Mode one, from video segment, extract set N towards corresponding image as key frame, N is more than 2 integer.For example can by initially towards, with initially towards rotation angle be 90 degree, 180 degree, 270 degree, the image in video segment corresponding to this four direction is as key frame.
Mode two, by video segment according to orientation information be divided into N1 interval, N1 is more than 2 integer, picks out respectively 1 image as key frame from N1 interval.In this mode, can from N1 interval, pick out arbitrarily 1 image as key frame, but preferably, can from N1 interval, pick out respectively 1 image as key frame in conjunction with at least one in attitude information, acceleration information and picture quality.Below can specifically carrying out pick out respectively 1 image as key frame from N1 interval time, operate 1 and operation 2:
Operation 1, extract respectively N2 open image from N1 interval, N2 is more than 2 integer.
If operation 2 is selected key frame in conjunction with attitude information, determine that the N2 in each interval opens the capture apparatus attitude information that image is corresponding in sensor parameters sequence, open and image, pick out respectively image that corresponding attitude information departs from three-axis reference minimum as key frame from the N2 in each interval.If select key frame in conjunction with acceleration information, determine that the N2 in each interval in sensor parameters sequence opens the acceleration information that image is corresponding, open the image of picking out corresponding acceleration minimum image as key frame from the N2 in each interval.If combining image quality is selected key frame, open and image, pick out respectively image that picture quality is the highest as key frame from the N2 in each interval.If in conjunction with attitude information, combination in any in acceleration information and picture quality is selected key frame, the N2 that determines each interval in sensor parameters sequence opens the capture apparatus attitude information that image is corresponding, acceleration information, according to corresponding attitude information, combination in any in acceleration information and picture quality is opened image to the N2 in each interval and is given respectively weights, wherein to depart from the image weights that three-axis reference is less larger for corresponding attitude information, the image weights that corresponding acceleration is less are larger, the image weights that picture quality is higher are larger, open the image of picking out respectively weights maximum image as key frame from the N2 in each interval.
Last Image Mosaics unit 04 splices key frame, obtains panoramic pictures.This Image Mosaics unit 04 adopts existing function, mainly comprises image pre-service, image registration and three functions of image co-registration reconstruct, and wherein image pre-service comprises key frame is carried out to denoising, feature point extraction, overlapping detection and edge extracting etc.Image registration refers to the position that adopts certain matching strategy to find out the unique point correspondence in reference picture in image to be spliced (the present embodiment middle finger key frame), and then determine the transformation relation between two width images, carry out unified coordinate transform to treat stitching image.Merging reconstruct is that the smooth and seamless panoramic picture that obtains splicing reconstruct is merged in the overlapping region of image to be spliced.Multiple images to be spliced are spliced to the above-mentioned functions that obtains panoramic picture in view of being prior art, and the present invention does not do concrete restriction, is not described in detail in this.
Because user needs rotary taking equipment in whole shooting process, normally user is manual carries out in this rotation, the image inclination that inevitably can exist capture apparatus to tilt to cause, this just need to be rotated correction to key frame, now this device can also comprise: key frame correcting unit 05, be used for according to capture apparatus attitude information corresponding to the each key frame of sensor parameters sequence, the key frame that key-frame extraction unit 03 is extracted is rotated correction, the key frame after rotation correction is offered to Image Mosaics unit 04 and splice.
Particularly, key frame correcting unit 05 is being rotated timing, can carry out at least one in following operation:
Operation 1) determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
each key frame is rotated frame by frame in the plane perpendicular to image
degree, γ is the angle with respect to Z axis deflection in the attitude information that key frame is corresponding.This operation is corresponding to capture apparatus situation with respect to Z axis run-off the straight in shooting process.
Operation 2) determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, α is the angle with respect to X-axis deflection in the attitude information that key frame is corresponding.This operation is corresponding to capture apparatus situation with respect to X-axis run-off the straight in shooting process.
Operation 3) determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, β is the angle with respect to Y-axis deflection in the attitude information that key frame is corresponding.This operation is corresponding to capture apparatus situation with respect to Y-axis run-off the straight in shooting process.
In several embodiment provided by the present invention, should be understood that, disclosed method and apparatus can be realized by another way.For example, device embodiment described above is only schematically, and for example, the division of described unit, is only that a kind of logic function is divided, and when actual realization, can have other dividing mode.In addition, the each functional unit in each embodiment of the present invention can be integrated in a processing unit, can be also that the independent physics of unit exists, and also can be integrated in a unit two or more unit.
The integrated unit that the above-mentioned form with SFU software functional unit realizes, can be stored in a computer read/write memory medium.Above-mentioned SFU software functional unit is stored in a storage medium, comprise that some instructions (can be personal computers in order to make a computer equipment, server, or the network equipment etc.) or processor (processor) carry out the part steps of method described in each embodiment of the present invention.And aforesaid storage medium comprises: various media that can be program code stored such as USB flash disk, portable hard drive, ROM (read-only memory) (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disc or CDs.
Can be found out by above description, method and apparatus provided by the invention possesses following advantage:
1) video segment of user being taken centered by loca carries out the extraction of key frame, as the splicing basis that generates panoramic pictures, make on the one hand user's faster and more convenient operation, only need disposable video capture, and take the picture of a few width different visual angles without artificial fixed point, on the other hand, the mode of extracting key frame from video segment provides basis for selecting preferred picture to be spliced, and needn't necessarily be limited to user's limited picture of taking of fixing a point, thereby improve the effect of panoramic pictures.
2) the capture apparatus attitude information that the present invention can collect further combined with sensor is rotated correction to key frame, reduce because of user's adverse effect that capture apparatus tilts the effect of panoramic pictures is produced in shooting process, further improved the effect of panoramic pictures.
3) implementation provided by the invention can be applied to existing Intelligent mobile equipment, user is browsed and share the picture representing and risen to space navigation from plane, pictorial information represents more abundant true, increase substantially user's experience, can be widely used in the application such as social activity, map, tourism.
The foregoing is only preferred embodiment of the present invention, in order to limit the present invention, within the spirit and principles in the present invention not all, any amendment of making, be equal to replacement, improvement etc., within all should being included in the scope of protection of the invention.
Claims (12)
1. a method for generating panorama image, is characterized in that, the method comprises:
S1, obtain the video segment that user takes centered by loca, the visual range of this video segment is greater than the fixed point of capture apparatus and takes visual range, obtain and records photographing process in sensor parameters sequence;
S2, according to the orientation information of each shooting point in sensor parameters sequence, from video segment, extract more than 2 key frames;
S3, the key frame of extraction is spliced, obtain panoramic pictures.
2. method according to claim 1, is characterized in that, described step S2 specifically comprises:
From described video segment, extract set N towards corresponding image as key frame, N is more than 2 integer; Or,
Described video segment is divided into N1 interval according to orientation information, and N1 is more than 2 integer, picks out respectively 1 image as key frame from a described N1 interval.
3. method according to claim 2, is characterized in that, picks out respectively 1 image and specifically comprise as key frame from a described N1 interval:
S21, extract respectively N2 open image from a described N1 interval, N2 is more than 2 integer;
S22, according at least one in capture apparatus attitude information, acceleration information and picture quality corresponding in sensor parameters sequence, open image and select respectively an image as key frame from each interval N2.
4. method according to claim 3, is characterized in that, described S22 specifically comprises at least one in following mode:
Mode 1) determine that the N2 in each interval in described sensor parameters sequence opens the capture apparatus attitude information that image is corresponding, open and image, pick out respectively image that corresponding attitude information departs from three-axis reference minimum as key frame from the N2 in each interval;
Mode 2) determine that the N2 in each interval in described sensor parameters sequence opens the acceleration information that image is corresponding, opens the image of picking out corresponding acceleration minimum image as key frame from the N2 in each interval;
Mode 3) open and image, pick out respectively image that picture quality is the highest as key frame from the N2 in each interval;
Mode 4) according to the combination in any in corresponding attitude information, acceleration information and picture quality, the N2 in each interval is opened to image and give respectively weights, wherein to depart from the image weights that three-axis reference is less larger for corresponding attitude information, the image weights that corresponding acceleration is less are larger, the image weights that picture quality is higher are larger, open the image of picking out respectively weights maximum image as key frame from the N2 in each interval.
5. method according to claim 1, is characterized in that, between described step S2 and step S3, also comprises:
S4, according to the capture apparatus attitude information that in sensor parameters sequence, each key frame is corresponding, key frame is rotated to correction.
6. method according to claim 5, is characterized in that, described step S4 specifically comprises at least one in following operation:
Operation 1) determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
each key frame is rotated frame by frame in the plane perpendicular to image
degree, γ is the angle with respect to Z axis deflection in the attitude information that key frame is corresponding;
Operation 2) determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, α is the angle with respect to X-axis deflection in the attitude information that key frame is corresponding;
Operation 3) determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, β is the angle with respect to Y-axis deflection in the attitude information that key frame is corresponding.
7. a device for generating panorama image, is characterized in that, this device comprises:
Video segment acquiring unit, the video segment of taking centered by loca for obtaining user;
Sensor parameters record cell, for obtaining and record the sensor parameters sequence of shooting process of described video segment;
Key-frame extraction unit for the orientation information according to the each shooting point of sensor parameters sequence, extracts more than 2 key frames from video segment;
Image Mosaics unit, for described key frame is spliced, obtains panoramic pictures.
8. device according to claim 7, is characterized in that, described key-frame extraction unit concrete configuration is:
From described video segment, extract set N towards corresponding image as key frame, N is more than 2 integer; Or,
Described video segment is divided into N1 interval according to orientation information, and N1 is more than 2 integer, picks out respectively 1 image as key frame from a described N1 interval.
9. device according to claim 8, is characterized in that, described key-frame extraction unit, pick out respectively 1 image as key frame from a described N1 interval time, is specifically carried out:
Operation 21, extract respectively N2 open image from a described N1 interval, N2 is more than 2 integer;
Operation 22, according at least one in capture apparatus attitude information, acceleration information and picture quality corresponding in sensor parameters sequence, open image and select respectively an image as key frame from each interval N2.
10. device according to claim 9, is characterized in that, described key-frame extraction unit in the time of executable operations 22, concrete at least one in the following ways:
Mode 1) determine that the N2 in each interval in described sensor parameters sequence opens the capture apparatus attitude information that image is corresponding, open and image, pick out respectively image that corresponding attitude information departs from three-axis reference minimum as key frame from the N2 in each interval;
Mode 2) determine that the N2 in each interval in described sensor parameters sequence opens the acceleration information that image is corresponding, opens the image of picking out corresponding acceleration minimum image as key frame from the N2 in each interval;
Mode 3) open and image, pick out respectively image that picture quality is the highest as key frame from the N2 in each interval;
Mode 4) according to the combination in any in corresponding attitude information, acceleration information and picture quality, the N2 in each interval is opened to image and give respectively weights, wherein to depart from the image weights that three-axis reference is less larger for corresponding attitude information, the image weights that corresponding acceleration is less are larger, the image weights that picture quality is higher are larger, open the image of picking out respectively weights maximum image as key frame from the N2 in each interval.
11. devices according to claim 7, is characterized in that, this device also comprises:
Key frame correcting unit, be used for according to capture apparatus attitude information corresponding to the each key frame of sensor parameters sequence, the key frame that described key-frame extraction unit is extracted is rotated correction, the key frame after rotation correction is offered to described Image Mosaics unit and splice.
12. devices according to claim 11, is characterized in that, described key frame correcting unit is being rotated timing, concrete at least one that carry out in following operation:
Operation 1) determine the angle mean value with respect to Z axis deflection in the attitude information that key frame is corresponding
each key frame is rotated frame by frame in the plane perpendicular to image
degree, γ is the angle with respect to Z axis deflection in the attitude information that key frame is corresponding;
Operation 2) determine the angle mean value with respect to X-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, α is the angle with respect to X-axis deflection in the attitude information that key frame is corresponding;
Operation 3) determine the angle mean value with respect to Y-axis deflection in the attitude information that key frame is corresponding
each key frame is rotated in the plane at image frame by frame
degree, β is the angle with respect to Y-axis deflection in the attitude information that key frame is corresponding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310049193.5A CN103985103A (en) | 2013-02-07 | 2013-02-07 | Method and device for generating panoramic picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310049193.5A CN103985103A (en) | 2013-02-07 | 2013-02-07 | Method and device for generating panoramic picture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103985103A true CN103985103A (en) | 2014-08-13 |
Family
ID=51277061
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310049193.5A Pending CN103985103A (en) | 2013-02-07 | 2013-02-07 | Method and device for generating panoramic picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103985103A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104486553A (en) * | 2014-12-31 | 2015-04-01 | 广东欧珀移动通信有限公司 | Method and terminal for taking panoramic photos |
| CN105120155A (en) * | 2015-08-21 | 2015-12-02 | 小米科技有限责任公司 | Panoramic photograph generation method and device |
| CN105791669A (en) * | 2014-12-26 | 2016-07-20 | 联想(北京)有限公司 | Electronic device and control method of the electronic device |
| CN105791695A (en) * | 2016-03-30 | 2016-07-20 | 沈阳泰科易科技有限公司 | Panoramic image generation method and panoramic image generation device |
| CN106791437A (en) * | 2017-01-20 | 2017-05-31 | 维沃移动通信有限公司 | A kind of panoramic picture image pickup method and mobile terminal |
| CN107545538A (en) * | 2016-06-24 | 2018-01-05 | 清华大学深圳研究生院 | A kind of Panorama Mosaic method and device based on unmanned plane |
| CN105611169B (en) * | 2015-12-31 | 2019-06-25 | 联想(北京)有限公司 | Image acquiring method and electronic equipment |
| CN110019886A (en) * | 2017-08-28 | 2019-07-16 | 富泰华工业（深圳）有限公司 | Full-view image generating means and method |
| CN110236579A (en) * | 2019-06-14 | 2019-09-17 | 湖南省华芯医疗器械有限公司 | Medical imaging direction calibration system and its calibration method |
| CN112218158A (en) * | 2019-07-12 | 2021-01-12 | 华为技术有限公司 | Video processing method and device |
| CN112637564A (en) * | 2020-12-18 | 2021-04-09 | 中标慧安信息技术股份有限公司 | Indoor security method and system based on multi-picture monitoring |
| CN113038010A (en) * | 2021-03-12 | 2021-06-25 | Oppo广东移动通信有限公司 | Video processing method, video processing device, storage medium and electronic equipment |
| CN114827749A (en) * | 2022-04-21 | 2022-07-29 | 应急管理部天津消防研究所 | Method for seamless switching and playing of multi-view panoramic video |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080024614A1 (en) * | 2006-07-25 | 2008-01-31 | Hsiang-Tsun Li | Mobile device with dual digital camera sensors and methods of using the same |
| CN101231456A (en) * | 2008-01-31 | 2008-07-30 | 北京航空航天大学 | Method for correcting panorama video jitter using angle sensor |
| CN101345843A (en) * | 2008-08-28 | 2009-01-14 | 中兴通讯股份有限公司 | Method and system for implementing full view video of visible mobile terminal |
| CN101588451A (en) * | 2008-05-20 | 2009-11-25 | 索尼株式会社 | Image pickup apparatus, image pickup method, playback control apparatus, playback control method, and program |
| CN102201115A (en) * | 2011-04-07 | 2011-09-28 | 湖南天幕智能科技有限公司 | Real-time panoramic image stitching method of aerial videos shot by unmanned plane |
| CN102420898A (en) * | 2011-09-27 | 2012-04-18 | 惠州Tcl移动通信有限公司 | Mobile phone-based panoramic photographing realization method and mobile phone |
| CN102645837A (en) * | 2012-04-09 | 2012-08-22 | 广东步步高电子工业有限公司 | Panoramic picture taking and processing method applied to movable handheld equipment provided with front and back cameras |
-
2013
- 2013-02-07 CN CN201310049193.5A patent/CN103985103A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080024614A1 (en) * | 2006-07-25 | 2008-01-31 | Hsiang-Tsun Li | Mobile device with dual digital camera sensors and methods of using the same |
| CN101231456A (en) * | 2008-01-31 | 2008-07-30 | 北京航空航天大学 | Method for correcting panorama video jitter using angle sensor |
| CN101588451A (en) * | 2008-05-20 | 2009-11-25 | 索尼株式会社 | Image pickup apparatus, image pickup method, playback control apparatus, playback control method, and program |
| CN101345843A (en) * | 2008-08-28 | 2009-01-14 | 中兴通讯股份有限公司 | Method and system for implementing full view video of visible mobile terminal |
| CN102201115A (en) * | 2011-04-07 | 2011-09-28 | 湖南天幕智能科技有限公司 | Real-time panoramic image stitching method of aerial videos shot by unmanned plane |
| CN102420898A (en) * | 2011-09-27 | 2012-04-18 | 惠州Tcl移动通信有限公司 | Mobile phone-based panoramic photographing realization method and mobile phone |
| CN102645837A (en) * | 2012-04-09 | 2012-08-22 | 广东步步高电子工业有限公司 | Panoramic picture taking and processing method applied to movable handheld equipment provided with front and back cameras |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105791669B (en) * | 2014-12-26 | 2019-12-24 | 联想(北京)有限公司 | Electronic device and control method thereof |
| CN105791669A (en) * | 2014-12-26 | 2016-07-20 | 联想(北京)有限公司 | Electronic device and control method of the electronic device |
| CN104486553A (en) * | 2014-12-31 | 2015-04-01 | 广东欧珀移动通信有限公司 | Method and terminal for taking panoramic photos |
| CN104486553B (en) * | 2014-12-31 | 2018-09-11 | 广东欧珀移动通信有限公司 | A kind of distant view photograph image pickup method and terminal |
| CN105120155A (en) * | 2015-08-21 | 2015-12-02 | 小米科技有限责任公司 | Panoramic photograph generation method and device |
| WO2017031890A1 (en) * | 2015-08-21 | 2017-03-02 | 小米科技有限责任公司 | Panoramic photo generation method and apparatus |
| RU2664674C2 (en) * | 2015-08-21 | 2018-08-21 | Сяоми Инк. | Method and apparatus for creating a panorama |
| CN105611169B (en) * | 2015-12-31 | 2019-06-25 | 联想(北京)有限公司 | Image acquiring method and electronic equipment |
| CN105791695A (en) * | 2016-03-30 | 2016-07-20 | 沈阳泰科易科技有限公司 | Panoramic image generation method and panoramic image generation device |
| CN105791695B (en) * | 2016-03-30 | 2019-06-11 | 沈阳泰科易科技有限公司 | The generation method and device of panoramic picture |
| CN107545538A (en) * | 2016-06-24 | 2018-01-05 | 清华大学深圳研究生院 | A kind of Panorama Mosaic method and device based on unmanned plane |
| CN106791437A (en) * | 2017-01-20 | 2017-05-31 | 维沃移动通信有限公司 | A kind of panoramic picture image pickup method and mobile terminal |
| CN106791437B (en) * | 2017-01-20 | 2020-02-14 | 维沃移动通信有限公司 | Panoramic image shooting method and mobile terminal |
| CN110019886A (en) * | 2017-08-28 | 2019-07-16 | 富泰华工业（深圳）有限公司 | Full-view image generating means and method |
| CN110236579A (en) * | 2019-06-14 | 2019-09-17 | 湖南省华芯医疗器械有限公司 | Medical imaging direction calibration system and its calibration method |
| CN112218158A (en) * | 2019-07-12 | 2021-01-12 | 华为技术有限公司 | Video processing method and device |
| CN112218158B (en) * | 2019-07-12 | 2021-12-28 | 华为技术有限公司 | Video processing method and device |
| CN112637564A (en) * | 2020-12-18 | 2021-04-09 | 中标慧安信息技术股份有限公司 | Indoor security method and system based on multi-picture monitoring |
| CN112637564B (en) * | 2020-12-18 | 2021-10-26 | 中标慧安信息技术股份有限公司 | Indoor security method and system based on multi-picture monitoring |
| CN113038010A (en) * | 2021-03-12 | 2021-06-25 | Oppo广东移动通信有限公司 | Video processing method, video processing device, storage medium and electronic equipment |
| CN113038010B (en) * | 2021-03-12 | 2022-11-29 | Oppo广东移动通信有限公司 | Video processing method, video processing device, storage medium and electronic equipment |
| CN114827749A (en) * | 2022-04-21 | 2022-07-29 | 应急管理部天津消防研究所 | Method for seamless switching and playing of multi-view panoramic video |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103985103A (en) | Method and device for generating panoramic picture | |
| US11222440B2 (en) | Position and pose determining method, apparatus, smart device, and storage medium | |
| US10250800B2 (en) | Computing device having an interactive method for sharing events | |
| US20200302615A1 (en) | Repositioning method and apparatus in camera pose tracking process, device, and storage medium | |
| WO2019205842A1 (en) | Method and device for repositioning in camera orientation tracking process, and storage medium | |
| US9635251B2 (en) | Visual tracking using panoramas on mobile devices | |
| US11276183B2 (en) | Relocalization method and apparatus in camera pose tracking process, device, and storage medium | |
| US20120300020A1 (en) | Real-time self-localization from panoramic images | |
| US11102413B2 (en) | Camera area locking | |
| CN106534692A (en) | Video image stabilization method and device | |
| US10051180B1 (en) | Method and system for removing an obstructing object in a panoramic image | |
| CN111931877B (en) | Target detection method, device, equipment and storage medium | |
| WO2022042425A1 (en) | Video data processing method and apparatus, and computer device and storage medium | |
| CN108776822B (en) | Target area detection method, device, terminal and storage medium | |
| US9843724B1 (en) | Stabilization of panoramic video | |
| CN111258467A (en) | Interface display method and device, computer equipment and storage medium | |
| KR101868740B1 (en) | Apparatus and method for generating panorama image | |
| US20230362317A1 (en) | Method for generating rotation direction of gyroscope and computer device | |
| US20210289147A1 (en) | Images with virtual reality backgrounds | |
| WO2023165535A1 (en) | Image processing method and apparatus, and device | |
| Zhen et al. | Multi-image motion deblurring aided by inertial sensors | |
| US20220345621A1 (en) | Scene lock mode for capturing camera images | |
| KR102084161B1 (en) | Electro device for correcting image and method for controlling thereof | |
| CN104135622B (en) | A kind of method, device and mobile terminal of combination gyroscope optimization snap | |
| CN116170689A (en) | Video generation method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140813 |
|
| RJ01 | Rejection of invention patent application after publication |