Summary of the invention
In view of above-mentioned, the present invention provides a kind of image processing method and device, computer installation and readable storage medium storing program for executing, can
Promote the feeling of immersion that user watches panoramic video in reality environment.
On the one hand the embodiment of the present invention provides a kind of image processing method, be applied in virtual panoramic video, the method
Including:
The pixel coordinate of screen display area where obtaining pixel, the screen display area includes left eye display area
With right eye display area;
Division range belonging to the screen lateral coordinates value obtained from the pixel coordinate is determined according to division data,
In, the division range include left eye angular field of view corresponding with the left eye display area and with the right eye display area pair
The right-eye perspectives range answered;
Obtain mapping coordinates corresponding to the frame picture of video source;
Using preset function to from the division data and the mapping coordinates obtain coordinate values calculated with defeated
Single channel abscissa value and single channel ordinate value out;
Corresponding pixel is taken from video source according to the single channel abscissa value and single channel ordinate value determination
Point in the corresponding screen display area of the division range belonging to the screen lateral coordinates value to show.
Preferably, the data for dividing data and being stored including the use of the channel RGBA, wherein the channel R is for storing
First lateral coordinates minimum value, the channel G is for storing the first lateral coordinates maximum value, and the channel B is for storing first
Ordinate minimum value, the A channel is for storing the first ordinate maximum value.
Preferably, the data that divide further include dividing the cut-off rule for dividing range, and the cut-off rule is located at described
The middle part of first lateral coordinates minimum value and the first lateral coordinates maximum value.
Preferably, it is described using preset function to from the divisions data and the mapping coordinates acquisition coordinate values into
Row is calculated to export single channel abscissa value and single channel ordinate value includes:
Using lerp function to the lateral coordinates value and institute taken out from the division data and the mapping coordinates
The lateral coordinates value for stating mapping coordinates is calculated to export the single channel abscissa value;
Using lerp function to the longitudinal coordinate value and institute taken out from the division data and the mapping coordinates
The longitudinal coordinate value for stating mapping coordinates is calculated to export the single channel ordinate value.
Preferably, described determined according to the single channel abscissa value and the single channel ordinate value takes from video source
Corresponding pixel in the corresponding screen display area of the division range belonging to the screen lateral coordinates value to show
Including:
It is taken in the frame picture of video source accordingly according to the single channel abscissa value and the single channel ordinate value
Pixel in the corresponding screen display area of the division range belonging to the screen lateral coordinates value to show.
Preferably, it is determined from video source described according to the single channel abscissa value and the single channel ordinate value
Corresponding pixel is taken to show in the corresponding screen display area of the division range belonging to the screen lateral coordinates value
Before showing, the method also includes:
The single channel abscissa value and the single channel ordinate value are merged into binary channels UV data;
Described determined according to the single channel abscissa value and the single channel ordinate value takes accordingly from video source
Pixel in the corresponding screen display area of the division range belonging to the screen lateral coordinates value show include:
Determined according to the binary channels UV data takes corresponding pixel in the screen from the frame picture of video source
Display in the corresponding screen display area of division range belonging to lateral coordinates value.
Present invention implementation also provides a kind of image processing apparatus, and using with virtual panoramic video, described device includes:
Module is obtained, for the pixel coordinate of screen display area where obtaining pixel, the screen display area packet
Include left eye display area and right eye display area;
Determining module, for being determined belonging to the screen lateral coordinates value obtained from the pixel coordinate according to division data
Division range, wherein the division range include left eye angular field of view corresponding with the left eye display area and with it is described
The corresponding right-eye perspectives range of right eye display area;
The acquisition module is also used to obtain mapping coordinates corresponding to the frame picture of video source;
Computing module, for obtaining coordinate to from the division data and the mapping coordinates using using preset function
Numerical value is calculated to export single channel abscissa value and single channel ordinate value;
Display module, for being determined from video source according to the single channel abscissa value and the single channel ordinate value
Corresponding pixel is taken to show in the corresponding screen display area of the division range belonging to the screen lateral coordinates value
Show.
Preferably, the data for dividing data and being stored including the use of the channel RGBA, wherein the channel the R storage first
Lateral coordinates minimum value, the channel G store the first lateral coordinates maximum value, and the channel B stores the first ordinate minimum value,
The A channel stores the first ordinate maximum value.
Another further aspect of the embodiment of the present invention provides a kind of computer installation, and the computer installation includes processor, described
The step of processor is for realizing image processing method as described above when executing the computer program stored in memory.
The another aspect of the embodiment of the present invention provides a kind of computer readable storage medium, is stored thereon with computer program,
The step of computer program realizes image processing method as described above when being executed by processor.
Image processing method and device provided by the invention, computer installation and computer readable storage medium, utilize figure
The corresponding relationship of the coordinate system of coordinate system and source video frame picture as where display area, while screen is determined using data are divided
Division range belonging to curtain lateral coordinates.Hereafter, by obtain source video frame picture mapping coordinates, and using lerp function into
Row is corresponding to be calculated to obtain single channel abscissa value and single channel ordinate value, according to the single channel abscissa value and single channel
Ordinate value is the corresponding pixel points that can determine in source video frame picture, by content shown by the pixel in the screen
Display in the corresponding left eye display area of division range belonging to lateral coordinates value or right eye display area, to form Binocular vison
Difference can promote the feeling of immersion that user watches panoramic video in reality environment.
Specific embodiment
To better understand the objects, features and advantages of the present invention, with reference to the accompanying drawing and specific real
Applying mode, the present invention will be described in detail.It should be noted that in the absence of conflict, presently filed embodiment and reality
The feature applied in mode can be combined with each other.
In the following description, numerous specific details are set forth in order to facilitate a full understanding of the present invention, described embodiment
Only some embodiments of the invention, rather than whole embodiments.Based on the embodiment in the present invention, this field
Those of ordinary skill's every other embodiment obtained without making creative work, belongs to guarantor of the present invention
The range of shield.
Unless otherwise defined, all technical and scientific terms used herein and belong to technical field of the invention
The normally understood meaning of technical staff is identical.Term as used herein in the specification of the present invention is intended merely to description tool
The purpose of the embodiment of body, it is not intended that in the limitation present invention.
Under a kind of reality environment scene, watched using head-mounted display (Head Mount Display, HMD)
When panoramic video, left eye display area correspond to that left eye carries out the displaying of image, right eye display area corresponds to right eye and carries out figure
The displaying of picture handles image using this programme to form right and left eyes parallax, promotes visual immersion.
This programme mainly by building VR environment sky ball threedimensional model, and by VR virtual perspective camera (for left eye and
Two VR virtual perspective cameras and two VR environment sky ball threedimensional models are arranged in right eye) it is placed in the central point of threedimensional model, from
And make VR virtual perspective camera content seen in the threedimensional model consistent with the visual angle content of real camera shooting.
Herein, the VR virtual perspective camera is used to render the content to human eye viewing.Thereafter, it obtains and utilizes VR virtual perspective camera wash with watercolours
The frame picture (three-dimensional scene images are rendered to two dimensional image) of dye simultaneously calculates.According to screen where each pixel of frame picture
Curtain position be formed in left eye display area, the image that right eye display area show, by reading export picture screen ranges,
Corresponding that picture loaded by VR virtual perspective camera is replaced with left-eye video image, similarly, right and left eyes view is consequently formed in right eye
Difference promotes visual immersion.
Fig. 1 is the flow chart of the image processing method of an embodiment of the present invention, it should be noted that embodiment of the present invention
Described image processing method be not limited to step and sequence in flow chart shown in FIG. 1.Stream shown according to different requirements,
Step in journey figure can increase, remove or change sequence.
As shown in Figure 1, the image processing method of present embodiment, is applied in virtual panoramic video, may include as follows
Step:
Step 101:The pixel coordinate of screen display area where obtaining pixel, the screen display area includes left eye
Display area and right eye display area.
In present embodiment, the screen display area can correspond to head-mounted display above-mentioned, left eye exhibition
Show that region provides image displaying, right eye display area for providing image exhibition for the right eye of user for the left eye for user
Show.
It is understood that any one pixel in the screen display area can have its relative coordinate position
Set, thus obtain pixel where screen display area pixel coordinate can utilize according to coordinate position go Reverse Turning Control with it is described
The display of pixel corresponding to coordinate position.
Step 102:Division model belonging to the extraction screen lateral coordinates from the pixel coordinate is determined according to data are divided
It encloses, wherein the division range includes left eye angular field of view corresponding with the left eye display area and the right eye show area
The corresponding right-eye perspectives range in domain.
In present embodiment, the data for dividing data and being stored including the use of the channel RGBA, wherein use in the channel R
In storing the first lateral coordinates minimum value, the channel G is for storing the first lateral coordinates maximum value, and the channel B is for depositing
The first ordinate minimum value is stored up, the A channel is for storing the first ordinate maximum value.It is to be understood that the channel R, institute
The abscissa range that the data that the channel G is stored are used to embody source video frame picture is stated, the channel G, the A channel are stored
Data be used to embody the ordinate range of source video frame picture.
It is understood that it is logical to be conducive to promotion available data by storing coordinate data using existing color channel
The utilization rate in road.
In present embodiment, the data that divide further include dividing the cut-off rule for dividing range, the cut-off rule position
In the middle part of the first lateral coordinates minimum value and the first lateral coordinates maximum value.
Step 103:Obtain mapping coordinates corresponding to the frame picture of video source.
In present embodiment, the mapping coordinates are the coordinate that image is labelled to three-dimensional sphere model, the mapping coordinates
There is corresponding relationship with the coordinate where the screen display area.
Step 104:Using preset function to the acquisition coordinate values progress from the division data and the mapping coordinates
It calculates to export single channel abscissa value and single channel ordinate value.
It in present embodiment, is calculated using lerp function, wherein using lerp function to from the division data
And the lateral coordinates value for the lateral coordinates value and the mapping coordinates taken out in the mapping coordinates is calculated to export
The single channel abscissa value.
Similarly, using lerp function to the longitudinal coordinate taken out from the division data and the mapping coordinates
Value and the longitudinal coordinate value of the mapping coordinates are calculated to export the single channel ordinate value.
Specifically, when taking abscissa correlation from the division data, the corresponding abscissa for taking the mapping coordinates
Value, calculating exported value through lerp function is the single channel abscissa value;And ordinate is taken from the division data
When correlation, the corresponding ordinate value for taking the mapping coordinates, calculating exported value through lerp function is the single channel
Transverse and longitudinal coordinate value.
It is understood that the corresponding coordinate value above-mentioned stored using the channel RGBA, wherein the channel R and the channel G are deposited
Storage is abscissa correlation, and it is ordinate correlation that the channel G and A channel, which then store,.In certain instances, R may be selected
The abscissa value that any channel in channel and the channel G is stored.
Step 105:Phase is taken from video source according to the single channel abscissa value and single channel ordinate value determination
The pixel answered in the corresponding screen display area of the division range belonging to the screen lateral coordinates value to show.
More particularly, present embodiment is according to the single channel abscissa value and the single channel ordinate value in video source
Frame picture in take corresponding pixel with the corresponding screen exhibition of the division range belonging to the screen lateral coordinates value
Show in region and shows.
It is corresponding with the coordinate system of source video frame picture using the coordinate system where image display area in present embodiment
Relationship, while division range belonging to screen lateral coordinates is determined using data are divided.Hereafter, by obtaining source video frame picture
Mapping coordinates, and carry out corresponding calculate to obtain single channel abscissa value and single channel ordinate value, root using lerp function
It is the corresponding pixel points that can determine in source video frame picture according to the single channel abscissa value and single channel ordinate value, it will be described
The corresponding left eye display area of the division range belonging to the screen lateral coordinates value of content shown by pixel or right eye
Display in display area increases the feeling of immersion of user to form binocular parallax, promotes visual experience.
As shown in Fig. 2, for the frame picture drawing of present invention video source used under an embodiment.The present embodiment
In, the frame picture of video source is the image of 3840*3840 pixel, and the top half of the image is left eye panorama, lower half
It is divided into right eye panoramic picture, and square, thus can be pushed up by coordinate (0,0), bottom right of the left upper apex of the frame picture
Point is that coordinate (1,1) carries out coordinate value, then has:
It carries out indicating left eye panorama when coordinate data is stored as (0,1,0,0.5) using the channel RGBA, wherein R is logical
What road, the channel G stored is the abscissa value range of left eye panorama;What channel B, A channel stored is left eye panorama
Ordinate value range.
It carries out indicating right eye panoramic picture, similar, R when coordinate data is stored as (0,1,0.5,1) using the channel RGBA
What channel, the channel G stored is the abscissa value range of right eye panoramic picture;What channel B, A channel stored is that right eye panoramic is drawn
The ordinate value range in face.
The coordinate value stored using the channel RGBA can take out corresponding panorama from the frame picture of video source,
Corresponding pixel is picked up from panorama (left eye panorama or right eye panoramic picture) specifically to shield for subsequent
Curtain display area is shown.
At the same time, at head-mounted display end, screen display area also carries out matching and determine, i.e., by taking
The abscissa value of the UV coordinate of screen display area since screen display area coordinate pair answers the frame picture coordinate, thus is gone
The U coordinate of screen display area judge that the content of the corresponding display of the screen display area is the figure in left eye panorama
As the picture material in content or right eye panoramic picture.
It after determining above-mentioned corresponding relationship, needs for frame picture to be labelled on three-dimensional sphere model, the three-dimensional ball face mould
The mapping coordinates system (coordinate of the taken correspondence image of UV representative model) of type and the coordinate system of the frame picture have corresponding relationship.
Using above-mentioned corresponding relationship, the coordinate value for taking each channel in the channel RGBA to be stored using mask function, then
In conjunction with the abscissa and ordinate of the mapping coordinates, is calculated using lerp function and export single channel abscissa value and single channel
Ordinate value, so far, i.e., using single channel abscissa value and single channel ordinate value from corresponding left eye panorama or
Picture material corresponding to corresponding pixel is taken in right eye panoramic picture, and then in left eye display area or right eye show area
(display) is shown in domain.
In the present embodiment, the coordinate system of image show area is corresponding with the coordinate system of frame picture where pixel, thus, in benefit
After left eye panorama and right eye panoramic picture with the channel RGBA acquirement, according to the seat of the pixel in image show area
Mark, judges that the coordinate of pixel belongs to the left eye panorama or right eye panoramic picture.
Thereafter, it is calculated using lerp, when the lateral coordinates x of the mapping coordinates is 0, exports the channel RGBA
The numerical value a that is stored of the channel R;When the lateral coordinates x of the mapping coordinates is 1, the channel the G institute in the channel RGBA is exported
The numerical value b of storage;When the lateral coordinates x of the mapping coordinates is between 0 to 1, the value of output meets:Output=a* (1-
X)+b*x,
Such as x=0.25, if a=1, b=0, then there is output valve 0.75;
As x=0.25, if a=0, b=1, then there is output valve 0.25.
It is understood that it is above-mentioned for for lateral coordinates calculating, longitudinal coordinate similarly, thus using lerp function
The single channel abscissa value and the single channel ordinate value is calculated.
In present embodiment, determines using above-mentioned single channel abscissa value and single channel ordinate and taken from video source
Corresponding pixel, and the pixel obtained can be shown in left eye display area or right eye display area, for making
User's is watched by eyes, due to (can also be that left eye above-mentioned is complete from video source again after carrying out the differentiation of right and left eyes range
Scape picture or right eye panoramic picture) in take corresponding pixel to be shown in corresponding display area, due to the pixel of acquisition
Point carries right and left eyes different information, thus can form binocular parallax, to be conducive to promote the feeling of immersion of user.
Further, for the benefit of transmission of data, can also be by the single channel abscissa value and the single channel ordinate
Value carries out channel and merges to form binary channels UV numerical value, then, the frame from video source can be determined according to the binary channels UV data
Take corresponding pixel with the corresponding screen show area of the division range belonging to the screen lateral coordinates value in picture
It is shown in domain.
It is understood that above-described embodiment mentions for the panorama carried out for video source by upper and lower eyes format
Take, and when video source be left and right eyes format when, then have using the channel RGBA carry out coordinate data be stored as (0,0.5,0,1) when
It indicates left eye panorama, carries out indicating right eye panoramic picture when coordinate data is stored as (0.5,1,0,1) using the channel RGBA.
After taking out left eye panorama and right eye panoramic picture, similarly, it can be similarly obtained using each coordinate relationship and correlation function double
Eye parallax effect, in favor of promoting the feeling of immersion of user.
Fig. 3 is the illustrative functional block diagram of the image processing apparatus of an embodiment of the present invention.As shown in figure 3, figure
The corresponding relationship of the coordinate system of coordinate system and source video frame picture as where processing unit 100 can utilize image display area,
Division range belonging to screen lateral coordinates is determined using division data simultaneously.Hereafter, pass through the patch of acquisition source video frame picture
Figure coordinate, and corresponding calculate to obtain single channel abscissa value and single channel ordinate value, according to institute is carried out using lerp function
It states single channel abscissa value and single channel ordinate value can determine corresponding pixel points in source video frame picture, by the pixel
The shown corresponding left eye display area of content division range belonging to the screen lateral coordinates value of point or right eye are shown
It is shown in region, to form binocular parallax, increases the feeling of immersion of user, promote visual experience.
Image processing apparatus 100 of the invention may include one or more modules, and one or more of modules can be with
Be stored in the memory of terminal and may be configured to by one or more processors (present embodiment be one processing
Device) it executes, to complete the present invention.For example, as shown in figure 3, described image processing unit 100 may include obtain module 11, really
Cover half block 12, computing module 13, display module 14 and merging treatment module 15.It is special that the so-called module of the present invention can be completion one
The program segment for determining function, the implementation procedure than program more suitable for description software in the processor.
It should be noted that each embodiment of corresponding above-mentioned image processing method, image processing apparatus 100 may include
Part or all in each functional module shown in Fig. 3, the function of each module will introduced in detail below.At images above
Identical noun related terms and its specific explanation are readily applicable to following to each in each embodiment of reason method
The function introduction of module.For the sake of saving space and avoiding repetition, details are not described herein again.
Fig. 4 is the illustrative structural schematic diagram of the computer installation of an embodiment of the present invention.It is provided in this embodiment
Computer installation 1 includes:It processor 10, memory 20 and is stored in the memory 20 and can be on the processor 10
The computer program of operation, such as image processing program.When the processor 10 executes the computer program, realize above-mentioned each
Step in image processing method embodiment, such as step 101 shown in FIG. 1 is to step 105.Alternatively, the processor 10
The function of each module in above-mentioned apparatus embodiment is realized when executing the computer program, such as such as the acquisition module 11 in Fig. 3
Realize the function of the pixel coordinate of screen display area where obtaining pixel.
The computer installation 1 can be the calculating such as desktop PC, notebook, palm PC and cloud server and set
It is standby.The computer installation 1 may include, but be not limited to, processor 10 and memory 20.It will be understood by those skilled in the art that
The schematic diagram is only example of the present invention for realizing the computer installation 1 of image processing method, is not constituted to calculating
The restriction of machine device 1 may include perhaps combining certain components or different components than illustrating more or fewer components,
Such as the computer installation 1 can also include input-output equipment, network access equipment, bus etc..
Alleged processor 10 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng the processor 10 is the control centre of the computer installation 1, utilizes various interfaces and connection entire computer dress
Set 1 various pieces.
The memory 20 can be used for storing the computer program and/or module, the processor 10 by operation or
The computer program and/or module being stored in the memory 20 are executed, and calls the data being stored in memory 20,
Realize the various functions of the computer installation 1.The memory 20 can mainly include storing program area and storage data area,
In, storing program area can application program needed for storage program area, at least one function (such as sound-playing function, image
Playing function etc.) etc..In addition, memory 20 may include high-speed random access memory, it can also include non-volatile memories
Device, such as hard disk, memory, plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure
Digital, SD) card, flash card (Flash Card), at least one disk memory, flush memory device or other volatibility are solid
State memory device.
The computer installation 1 may also include image processing apparatus 100 as shown in Figure 3, described image processing unit 100
It can store in the memory 20.
Illustratively, the computer program can be divided into one or more modules, one or more of moulds
Block is stored in the memory 20, and is executed by the processor 10, to complete the present invention.One or more of modules
It can be the series of computation machine program instruction section that can complete specific function, the instruction segment is for describing the computer program
Implementation procedure in the computer installation 1.For example, the computer program can be divided into image as shown in Figure 3
Processing unit 100, including module 11, determining module 12, computing module 13, display module 14 and merging treatment module 15 are obtained,
Each module concrete function is as follows:
Module 11 is obtained, the pixel coordinate of screen display area, the screen show area where can be used for obtaining pixel
Domain includes left eye display area and right eye display area.Furthermore, it may also be used for obtain textures corresponding to the frame picture of video source
Coordinate.
Determining module 12 can be used for determining the screen lateral coordinates value obtained from the pixel coordinate according to division data
Affiliated division range, wherein the division range include left eye angular field of view corresponding with the left eye display area and with
The corresponding right-eye perspectives range of the right eye display area.
Computing module 13 can be used for obtaining using using preset function to from the division data and the mapping coordinates
Coordinate values are calculated to export single channel abscissa value and single channel ordinate value.Specifically, using lerp function pair
The lateral coordinates value of the lateral coordinates value and the mapping coordinates taken out from the division data and the mapping coordinates
It is calculated to export the single channel abscissa value;And using lerp function to from the division data and the mapping coordinates
The longitudinal coordinate value of middle taking-up and the longitudinal coordinate value of the mapping coordinates are calculated, and to export, the single channel is vertical to be sat
Scale value.
Display module 14 can be used for being determined according to the single channel abscissa value and the single channel ordinate value from video
Take corresponding pixel with the corresponding screen display area of the division range belonging to the screen lateral coordinates value in source
Interior display.
Merging treatment module 15 can be used for merging the single channel abscissa value and the single channel ordinate value in pairs
Channel UV data.
If the integrated module of computer installation 1 of the present invention is realized in the form of SFU software functional unit and as independence
Product when selling or using, can store in a computer readable storage medium.Based on this understanding, of the invention
It realizes all or part of the process in above-described embodiment method, can also instruct relevant hardware come complete by computer program
At the computer program can be stored in a computer readable storage medium, which is being executed by processor
When, it can be achieved that the step of above-mentioned each embodiment of the method.Wherein, the computer program includes computer program code, described
Computer program code can be source code form, object identification code form, executable file or certain intermediate forms etc..The meter
Calculation machine readable medium may include:Can carry the computer program code any entity or device, recording medium, USB flash disk,
Mobile hard disk, magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory
Device (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It needs to illustrate
It is that the content that the computer-readable medium includes can be fitted according to the requirement made laws in jurisdiction with patent practice
When increase and decrease, such as in certain jurisdictions, according to legislation and patent practice, computer-readable medium does not include electric carrier wave letter
Number and telecommunication signal.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case where without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Benefit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent elements of the claims
Variation is included in the present invention.Any reference signs in the claims should not be construed as limiting the involved claims.This
Outside, it is clear that one word of " comprising " does not exclude other units or steps, and odd number is not excluded for plural number.System, device or terminal installation right
Multiple units, module or the device stated in it is required that can also pass through software or hardware by the same unit, module or device
To realize.The first, the second equal words are used to indicate names, and are not indicated any particular order.
Embodiment of above is only used to illustrate the technical scheme of the present invention and not to limit it, although referring to the above preferable embodiment party
Formula describes the invention in detail, those skilled in the art should understand that, it can be to technical solution of the present invention
It modifies or equivalent replacement should not all be detached from the spirit and scope of technical solution of the present invention.