CN110262763A - Display methods and device and storage medium and electronic equipment based on augmented reality - Google Patents
Display methods and device and storage medium and electronic equipment based on augmented reality Download PDFInfo
- Publication number
- CN110262763A CN110262763A CN201810235460.0A CN201810235460A CN110262763A CN 110262763 A CN110262763 A CN 110262763A CN 201810235460 A CN201810235460 A CN 201810235460A CN 110262763 A CN110262763 A CN 110262763A
- Authority
- CN
- China
- Prior art keywords
- curved surface
- acquisition image
- display
- webpage view
- rendering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
Present invention discloses a kind of display methods based on augmented reality, comprising: obtains the curved surface recognition result to acquisition image, determines feature object corresponding with the acquisition image;Pre-generated material is obtained according to the determining feature object;And based on the first surface obtained from the curved surface recognition result, on the acquisition image in the form of the second curved surface material described in Overlapping display.Scheme based on the present embodiment so as to improve the application range and processing capacity of augmented reality equipment, while improving the immersion experience of user by pre-generated with the corresponding position Overlapping display of curved surface is identified material in acquisition image.
Description
Technical field
It is the present invention relates to field of computer technology, in particular to a kind of based on AR (Augmented Reality, enhancing
Reality) display methods and device, computer readable storage medium and electronic equipment.
Background technique
AR technology is the position and angle by calculating acquisition image in real time, and the perception information generated using computer is to existing
After the environmental element in the real world carries out " enhancing ", the technology that is directly or indirectly shown.Here perception information can be crossed over
A variety of sensory forms, common includes vision and the sense of hearing, or even is also possible that tactile, body-sensing and smell.In this way, AR
It changes current perception of the people to real world environments, realizes and virtual world is superimposed on real world on the screen, and with
User interacts.
It is different from simple data investigation to show, AR is by incorporating sense of the people to real world for the component of digital world
Know, mixed reality experience on the spot in person is provided for user.The characteristics of exactly being experienced for this immersion, at present AR technology
Business application focuses primarily upon amusement and game service.However, with AR and its relevant technologies (such as image recognition, cloud service)
Development, more and more industrial fields start to recognize the value in conjunction with brought by AR technology, such as medical research and dissection
Training, precision instrument manufacture and maintenance, navigation of electronic map, teleconference and long-range control and business promotion etc..
On the other hand, AR technology is limited by the development of its relevant technologies again simultaneously, the former will often lag behind the hair of the latter
Exhibition progress.By taking image recognition technology as an example, current two-dimensional image identification reached its maturity and accuracy rate and stability all
It is higher, but for the image recognition of the three-dimension curved surfaces such as face also in the stage continuously improved.Correspondingly, AR technology is for acquisition
The identification mainstream of image is still known otherwise using two dimension, curved surface identification and corresponding identification post-processing is lacked relevant
Application achievements, at present still in research.
Summary of the invention
In order to solve the problems, such as to lack AR technical application for curved surface identification and corresponding identification post-processing in the related technology,
The present invention provides a kind of display methods based on AR and devices, computer readable storage medium and electronic equipment.
According to an embodiment of the invention, providing a kind of display methods based on AR, comprising: obtain the curved surface to acquisition image
Recognition result determines feature object corresponding with the acquisition image;It is obtained according to the determining feature object pre-generated
Material;And based on the first surface obtained from the curved surface recognition result, with the second curved surface on the acquisition image
Material described in form Overlapping display.
According to an embodiment of the invention, providing a kind of display device based on AR, comprising: curved surface identification module is set as
The curved surface recognition result to acquisition image is obtained, determines feature object corresponding with the acquisition image;Material obtaining module, if
It is set to and obtains pre-generated material according to the determining feature object;And Overlapping display module, it is set as being based on from institute
State the first surface that curved surface recognition result obtains, on the acquisition image in the form of the second curved surface element described in Overlapping display
Material.
According to an embodiment of the invention, providing a kind of computer readable storage medium, it is stored thereon with computer program, institute
It states and realizes when computer program is executed by processor as described above based on the display methods of AR.
According to an embodiment of the invention, providing a kind of electronic equipment, comprising: processor;And memory, the memory
On be stored with computer-readable instruction, when the computer-readable instruction is executed by the processor realize be based on as described above
The display methods of AR.
The technical solution that the embodiment of the present invention provides can include the following benefits:
The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface
The pre-generated material of corresponding position Overlapping display changes simultaneously so as to enhance the application range and processing capacity of AR equipment
The immersion experience of kind user.
It should be understood that the above general description and the following detailed description are merely exemplary, this can not be limited
Invention.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and in specification together principle for explaining the present invention.
Fig. 1 is shown can be using the exemplary system architecture of display methods or device of the embodiment of the present invention based on AR
Schematic diagram.
Fig. 2 shows the structural schematic diagrams of the computer system of the electronic equipment suitable for being used to realize the embodiment of the present invention.
What Fig. 3 showed AR is normally applied scene.
Fig. 4 is a kind of flow chart of display methods based on AR shown according to an exemplary embodiment.
Fig. 5 shows an example flow diagram of step 430 in embodiment illustrated in fig. 4.
Fig. 6 shows the schematic diagram that point mapping relations are established between cylindrical surface and rectangle.
Fig. 7 shows the schematic illustration for calculating arc side Along ent coordinate.
Fig. 8 shows the AR effect picture using embodiment illustrated in fig. 5 method.
Fig. 9 shows another example flow diagram of step 430 in embodiment illustrated in fig. 4.
Figure 10 shows the depth occlusion exemplary relationship between three-dimensional scene models and occupy-place model.
Figure 11 shows the AR effect picture using embodiment illustrated in fig. 9 method.
Figure 12 is a kind of flow chart of the display methods based on AR shown according to another exemplary embodiment.
Figure 13 shows an example flow diagram of step 1240 in embodiment illustrated in fig. 12.
Figure 14 is a kind of block diagram of display device based on AR shown according to an exemplary embodiment.
Figure 15 is a kind of block diagram of the display device based on AR shown according to another exemplary embodiment.
Figure 16 shows an example block diagram of Overlapping display module 1430 in embodiment illustrated in fig. 15.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein;On the contrary, thesing embodiments are provided so that the present invention will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.
In addition, described feature, structure or characteristic can be incorporated in one or more implementations in any suitable manner
In example.In the following description, many details are provided to provide and fully understand to the embodiment of the present invention.However,
It will be appreciated by persons skilled in the art that technical solution of the present invention can be practiced without one or more in specific detail,
Or it can be using other methods, constituent element, device, step etc..In other cases, it is not shown in detail or describes known side
Method, device, realization or operation are to avoid fuzzy each aspect of the present invention.
Block diagram shown in the drawings is only functional entity, not necessarily must be corresponding with physically separate entity.
I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuit
These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.
Flow chart shown in the drawings is merely illustrative, it is not necessary to including all content and operation/step,
It is not required to execute by described sequence.For example, some operation/steps can also decompose, and some operation/steps can close
And or part merge, therefore the sequence actually executed is possible to change according to the actual situation.
Fig. 1 is shown can be using the exemplary system architecture of the display methods or device based on AR of the embodiment of the present invention
100 schematic diagram.
As shown in Figure 1, system architecture 100 may include one of terminal device 101,102,103 or a variety of, network
104 and server 105.Network 104 between terminal device 101,102,103 and server 105 to provide communication link
Medium.Network 104 may include various connection types, such as wired, wireless communication link or fiber optic cables etc..
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.For example server 105 can be multiple server compositions
Server cluster etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out
Send message etc..Terminal device 101,102,103 can be the various electronic equipments with display screen, including but not limited to intelligent hand
Machine, tablet computer, portable computer and desktop computer etc..Server 105 can be to provide the server of various services.
For example, user, which can use terminal device 103 (being also possible to terminal device 101 or 102), obtains acquisition image,
It is local to carry out curved surface identification and obtain curved surface recognition result to obtain curved surface recognition result, or from server 105, so as to determine with
The corresponding feature object of the acquisition image;And then pre-generated material is obtained according to the determining feature object, here
Material can to prestore or be buffered in terminal device 103 local, be also possible to be obtained from server 105 or other remotely set
It is standby;Finally, terminal device 103 is based on the first surface obtained from the curved surface recognition result, on the acquisition image
Material described in the form Overlapping display of two curved surfaces.
Fig. 2 shows the structural schematic diagrams of the computer system of the electronic equipment suitable for being used to realize the embodiment of the present invention.
It should be noted that Fig. 2 shows the computer system 200 of electronic equipment be only an example, should not be to this hair
The function and use scope of bright embodiment bring any restrictions.
As shown in Fig. 2, computer system 200 includes central processing unit (CPU) 201, it can be read-only according to being stored in
Program in memory (ROM) 202 or be loaded into the program in random access storage device (RAM) 203 from storage section 208 and
Execute various movements appropriate and processing.In RAM 203, it is also stored with various programs and data needed for system operatio.CPU
201, ROM 202 and RAM 203 is connected with each other by bus 204.Input/output (I/O) interface 205 is also connected to bus
204。
I/O interface 205 is connected to lower component: the importation 206 including keyboard, mouse etc.;It is penetrated including such as cathode
The output par, c 207 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.;Storage section 208 including hard disk etc.;
And the communications portion 209 of the network interface card including LAN card, modem etc..Communications portion 209 via such as because
The network of spy's net executes communication process.Driver 210 is also connected to I/O interface 205 as needed.Detachable media 211, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 210, in order to read from thereon
Computer program be mounted into storage section 208 as needed.
Particularly, according to an embodiment of the invention, may be implemented as computer below with reference to the process of flow chart description
Software program.For example, the embodiment of the present invention includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communications portion 209, and/or from detachable media
211 are mounted.When the computer program is executed by central processing unit (CPU) 201, executes and limited in the system of the application
Various functions.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in unit involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part realizes that described unit also can be set in the processor.Wherein, the title of these units is in certain situation
Under do not constitute restriction to the unit itself.
As on the other hand, present invention also provides a kind of computer-readable medium, which be can be
Included in electronic equipment described in above-described embodiment;It is also possible to individualism, and without in the supplying electronic equipment.
Above-mentioned computer-readable medium carries one or more program, when the electronics is set by one for said one or multiple programs
When standby execution, so that method described in electronic equipment realization as the following examples.For example, the electronic equipment can be real
Now each step as shown in Fig. 4, Fig. 5, Fig. 9, Figure 12 and Figure 13.
Before elaborating the technical solution of the embodiment of the present invention, some relevant technical solutions and original introduced below
Reason.
Plane identification and post-processing in AR
AR product common at present is mostly based on the plane identification of picture, and after identifying special object, broadcasting
Related content (such as two-dimensional video or threedimensional model) can only be shown in front of picture.It for example appears in picture around bottle
In the case where body, not only recognition accuracy is low, but also even if identifying that the video successfully, played or model can also be detached from picture,
The immersion of user is caused to be experienced very poor.
What Fig. 3 showed AR is normally applied scene.As shown, using the terminal of integrated AR function to collect certain bottled by user
The image 31 of product, after the characteristic pattern 32 for identifying body, determine the corresponding product of present image 31 exist can AR show
Content;In turn, it is threedimensional model that terminal, which gets displaying content corresponding with the product, to show acquisition in image 31
Threedimensional model 33.Although some elements in threedimensional model 33 can show three-dimensional effect by zoom, such as
Shown in figure, threedimensional model 33 is whole or the front of subject in image 31 is shown in the form of patch, therefore causes to use
The immersion experience at family is very poor.
Curved surface identification
By the research to curved surface features, curved surface can generally be divided into following a few classes: linear stretch face, the surface of revolution,
Bezier (Bei Zi) curve and surface, B-spline curves curved surface, NURBS (Non-Uniform Rational B-Splines, it is non-
Even Rational B-splines) curve and surface, ruled surface, the face of sweeping into, hollows, equidistant surface etc..For the curved surface of different classifications, exist
A variety of curved surface features identification technologies.At present common curved surface features identification technology include principal curvatures detection method, template matching method with
And Point set matching method etc..
Although these methods achieve certain achievement for the research of curved surface features, due to the complexity of curved surface features
Property, the curved surface type and feature type that can be handled at present are still limited.Nevertheless, this has no effect on implementation of the invention.It changes
Yan Zhi, the present invention pay close attention to using curved surface identification result is post-processed accordingly, and and be not concerned with curved surface identify tool
Body realizes that the technology for being able to carry out curved surface identification that is existing and occurring in the future can be applied to the displaying scheme the present invention is based on AR
In.
Based on above-mentioned the relevant technologies, the embodiment of the present invention provides a kind of display methods based on AR and device, computer can
Read storage medium and electronic equipment.
The principle of the technical solution of the embodiment of the present invention and realization details are described in detail below.
Fig. 4 is a kind of flow chart of display methods based on AR shown according to an exemplary embodiment.As shown in figure 4,
This method can be executed by arbitrarily calculating equipment, it may include following steps 410-430.
In step 410, the curved surface recognition result to acquisition image is obtained, determines feature pair corresponding with acquisition image
As.
In one embodiment, further include the steps that obtaining acquisition image before step 410.Acquisition figure based on acquisition
Picture, can current device locally carry out curved surface identification obtain recognition result, can also will acquisition image be uploaded to server into
The identification of row curved surface, and then recognition result is obtained from server.In one embodiment, curved surface identification can also locally carried out
Acquisition image is uploaded simultaneously, subsequent processing is directly carried out if locally identifying successfully, otherwise waits for the curved surface identification of server
As a result.
In one embodiment, the curved surface recognition result of acquisition may include the data of two parts.A part is used for and spy
Sign library is compared, and to determine in feature database with the presence or absence of feature object corresponding with current acquired image, is had, is shown to identify
Success can carry out subsequent AR material and show;Otherwise it indicates that AR material corresponding with current acquired image is not present, so as to
It is handled with suspension.Another part data in curved surface recognition result may include the form parameter of acquired image top-surface camber part,
So as to the subsequent Overlapping display for carrying out material on acquisition image based on the form parameter.
Feature database described here may include the set of characteristic pattern.The method of the present embodiment is applied to business promotion
For, the feature picture library of commodity to be promoted can be pre-created.Once finding and acquiring in feature picture library by curved surface identification
The characteristic pattern of images match indicates the displaying that AR material can be carried out on acquisition image, otherwise can be with termination.At one
In embodiment, it is local that feature database can store the execution equipment in method, can also be remotely stored on server, or both it is simultaneous
Have.
In one embodiment, the determination of feature object can be completed independently in the execution equipment of method, or can also be with
Complete and definitive result returned into method here to execute equipment, the embodiment of the present invention by the server of carry out curved surface identification
To this, also there is no restriction.
At step 420, pre-generated material is obtained according to determining feature object.
Here material include can acquisition image on Overlapping display content, for example including but be not limited in video
Appearance, three-dimensional scene models and webpage view etc..It, can be preparatory still by taking the method for the present embodiment is applied to business promotion as an example
Creation is directed to the material database of commodity to be promoted, and safeguards material database and wait promote the mapping relations between commodity.Implement at one
In example, material database here be can store in any combination of following position: the execution equipment of method is local, executes curved surface identification
Server and above-mentioned execution equipment or the addressable configuration end equipment of server.
In one embodiment, the execution equipment of method is locally carrying out after curved surface is identified as function by step 410, in turn
Material corresponding with feature object can be obtained in local material database.
In another embodiment, the execution equipment of method obtains curved surface recognition result and spy from server by step 410
When levying the definitive result of object, and then corresponding with feature object material can be obtained from configuration end equipment, or can also equally from
Server obtains material corresponding with feature object.In the later case, the material that server obtains can come from itself and deposit
Self-configuring end equipment is stored up or comes, and server can be by material with the definitive result of curved surface recognition result and feature object
It feeds back to method and executes equipment.
In step 430, based on the first surface obtained from curved surface recognition result, with the second curved surface on acquisition image
Form Overlapping display material.
AR in the related technology is different to be only capable of showing planar video or patch on acquisition image based on plane recognition result
When three-dimensional scenic can be on acquisition image with the based on the first surface obtained from curved surface recognition result in the present embodiment
The material of the aforementioned acquisition of form Overlapping display of two curved surfaces.Here first surface for example can be in acquisition image as feature
The curvature portion of object matching foundation, the second curved surface are then the calculated display curved surfaces of form parameter institute according to first surface.
For different display demands, for example, be directed to different materials, first surface and the second curved surface here can it is identical can also be with
It is different.
In one embodiment, the second curved surface here can fit in first surface.It include that video content is with material
Example, step 430 may include: the form parameter based on first surface, and acquisition is when expanding into plane for first surface, the plane with
Mapping relations between first surface;And the frame data of video content in material are converted into rendering texture, it is reflected according to above-mentioned
Texture mapping will be rendered to first surface by penetrating relationship.
In another embodiment, above-mentioned second curved surface can be spaced apart with first surface and around first surface.With
For material includes three-dimensional scene models, at this moment step 430 may include: to correspond to the position of the second curved surface on acquisition image
Show the three-dimensional scene models in material.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge
Not Chu curved surface the pre-generated material of corresponding position Overlapping display, so as to enhance the application range and processing energy of AR equipment
Power, while improving the immersion experience of user.
Fig. 5 shows an example flow diagram of step 430 in embodiment illustrated in fig. 4.In this example, it identifies and ties from curved surface
The curved surface that fruit obtains includes cylindrical surface, and the position that step 430 further corresponds to the cylindrical surface on acquisition image carries out video
The application of a surface of content is shown.As shown in figure 5, for example may include step 531-534.
In step 531, n parts are divided by the top surface arc side on cylindrical surface is corresponding with bottom surface arc side, it is a that 2* (n+1) is obtained
Arc side Along ent.
In order to realize the display of curved surface video, the present embodiment is re-mapped by the way that the video frame of rectangle is converted into rendering texture
Onto cylindrical surface.In one embodiment, that cylindrical surface is interpreted as rectangle straight line (i.e. the central axis on cylindrical surface) in space is curved
Song is formed by face, and stretching to cylindrical surface can be unfolded to obtain corresponding rectangle.It should be noted that cylindrical surface here
It can be complete cylindrical surface, can also be one section in complete cylindrical surface.
Fig. 6 shows the schematic diagram that point mapping relations are established between cylindrical surface and rectangle.As shown in fig. 6, by cylindrical surface 61
Top surface arc side 611 it is corresponding with bottom surface arc side 612 be divided into n parts, including endpoint, a arc side equal part of 2* (n+1) is obtained
Point.
In step 532, obtained in rectangular coordinate system a arc side Along ent of 2* (n+1) on cylindrical surface with by cylindrical surface exhibition
Open the mapping relations between a straight flange Along ent of 2* (n+1) accordingly obtained for rectangle.
It connects as shown in fig. 6, being operated based on equal part identical with cylindrical surface 61, the top margin 621 of rectangle 62 and bottom edge 622 are same
Sample, which can correspond to, is divided into n parts, and a straight flange Along ent of 2* (n+1) can be obtained.Next, can continue to acquire 2* (n+1) a arc side etc.
Mapping relations between branch and a straight flange Along ent of 2* (n+1).
Fig. 7 shows the schematic illustration for calculating arc side Along ent coordinate.As shown in fig. 7, the top surface arc based on cylindrical surface 61
Auxiliary circle 71 is added on side 611, and the inscribed equilateral polygon 72 that number of edges is m is added in auxiliary circle 71, levels off to infinity in m
When, which also approaches the shape for auxiliary circle 71.Directly make line with center of circle O on each vertex of polygon 72, it is assumed that
The angle of any two adjacent vertex and center of circle O line is θ, and using center of circle O as the origin of rectangular coordinate system, it is each can to acquire polygon
The coordinate expressions on vertex (namely arc side Along ent) are as follows.
(xi, yi)=(r*cos i θ, r*sin i θ) ... (1)
Wherein, with point P shown in Fig. 7 for first vertex, anticlockwise vertex serial number is prolonged in i expression, and r indicates top surface
The radius of arc side 611.
Based on the above principles, it can acquire in top surface arc side 611 and class is pushed into a arc side of total 2* (n+1) in bottom surface arc side 612
The coordinate expressions of Along ent.
A ', B ' shown in Fig. 6, for 3 points of C ', it is assumed that A ' and B ' is respectively top surface arc side 611 and bottom surface arc side 612
First vertex on corresponding circle, then 3 points of coordinate expressions in spatial triaxial coordinate system are as follows.
A′(xA′, yA′, zA′)=(r*cos θ, r*sin θ, 0) ... (2)
B′(xB′, yB′, zB′)=(r*cos θ, r*sin θ, h) ... (3)
C′(xC′, yC′, zc′)=(r*cos (θ+360/n), r* (θ+360/n), h) ... (4)
Wherein, h indicates the height on cylindrical surface.
In step 533, the frame data of video content are converted into rendering texture, by the straight flange Along ent successively phase of rectangle
Adjacent is divided into 2n triangle for rectangle by zigzag connection, and the minimum unit of above-mentioned rendering texture corresponds to 2n triangle
One of.
In step 534, according to the mapping relations between arc side Along ent and straight flange Along ent, extremely by rendering texture mapping
Cylindrical surface.
It returns to shown in Fig. 6, is connected by the straight flange Along ent of rectangle 62 is successively adjacent by zigzag, it can be by 62 equal part of rectangle
For 2n triangle, one of them is, for example, △ ABC.Based on similar operations, 2n triangle (its can be obtained on cylindrical surface 61
Middle be arc side on one side, but is still referred to as triangle for brevity), wherein triangle corresponding with △ ABC is △ A ' B ' C '.
The frame data of video content are converted into rendering texture, based on the Along ent mapping relations that step 532 obtains, can be incited somebody to action
The data for being originally intended to rendering △ ABC are mapped in △ A ' B ' C ' and are rendered, and so on can obtain conforming to cylindrical surface 61
Curved surface video effect.
As shown in Figure 6, it is assumed that the texture coordinate on 62 4 vertex of rectangle is respectively (0,0), (0,1), (1,0) and (1,1);
Corresponding, taking the texture coordinate on vertex in top surface arc side 611 is (∑nI* θ/360,0), take the vertex line in bottom surface arc side 612
Reason coordinate is (∑nI* θ/360, h), and then can be obtained between a arc side Along ent of 2* (n+1) and a straight flange Along ent of 2* (n+1)
Mapping relations.It should be noted that 360 the case where being corresponding complete cylindrical surface here, be partial cylindrical surface on cylindrical surface
When, 360 in texture coordinate accordingly may be modified such that top surface arc side 611 and the corresponding central angle size of bottom surface arc side 612.
Fig. 8 shows the AR effect picture using the present embodiment method.As shown in figure 8, terminal of the user using integrated AR function
The image 81 for collecting certain cup-shaped product determines that the corresponding product of present image 81 is deposited after the characteristic pattern 82 for identifying cup body
Can AR show material;In turn, it is video content that terminal, which gets displaying material corresponding with the product, by the place of Fig. 5
Process is managed, the cylindrical surface that cup body can be finally fitted in image 81 shows the effect of curved surface video 83.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge
Not Chu the corresponding position fitting of curved surface show preset video content, so as to enhance the application range and processing energy of AR equipment
Power, while improving the immersion experience of user.
Fig. 9 shows another example flow diagram of step 430 in embodiment illustrated in fig. 4.In this example, step 430 base
In the curved surface obtained from curved surface recognition result, it is spaced apart and surround on the cylindrical surface of the curved surface with the curved surface on acquisition image,
Carry out the Overlapping display of three-dimensional scene models.As shown in figure 9, for example may include step 931-934.
In step 931, the position of occupy-place model is determined according to acquisition image.
Patch effect as similar Fig. 3, this reality can only be embodied by being different from the three-dimensional scenic that AR is shown in the related technology
Applying example may be implemented in around subject around display three-dimensional scene models.
In one embodiment, can be stored in advance can after curved surface identifies successfully for material for three-dimensional scene models here
Pre-stored three-dimensional scene models are obtained, and are shown on the cylindrical surface in acquisition image by rendering.The position on cylindrical surface
It can then be determined according to position of the subject in acquisition image.
In order to embody the hiding relation with subject in rendering three-dimensional scenes model, may also include in material and quilt
The corresponding occupy-place model of subject.For example, it is special that product to be promoted can be pre-configured with when the present embodiment is applied to business promotion
The three-dimensional scene models that sign figure and correspondence can be shown, while can also store occupy-place mould corresponding with product true form parameter
Type;In this way, terminal device is successfully matched to the characteristic pattern of product by curved surface identification, while can be accounted for according to acquisition image determination
The position of bit model, rendering strategy by subsequent sequence can embody and practical consistent hiding relation.
In step 932, the position of the second curved surface is determined according to pre-set depth occlusion relationship.
In order to embody the circular display effect of three-dimensional scene models, the second curved surface where the three-dimensional scene models with adopt
First surface corresponding with characteristic pattern can be spaced apart in collection image.The distance can be according to pre-set three-dimensional scenic mould
Depth occlusion relationship between type and occupy-place model determines.
Figure 10 shows the depth occlusion exemplary relationship between three-dimensional scene models and occupy-place model.As shown in Figure 10, occupy-place
The size and location of model 1010 is corresponding with size and location of the subject in acquisition image, three-dimensional scene models 1020
Position is corresponding with position of second curved surface in acquisition image.
In step 933, occupy-place model is rendered based on transparent texture on acquisition image, to appear object where first surface
Body.
After getting material corresponding with subject, occupy-place model and are determined based on step 931-932 respectively
Position of two curved surfaces in acquisition image, step 933 can start the rendering of occupy-place model and three-dimensional scene models.In order to acquire
Subject itself is shown in image, and above-mentioned occupy-place model can be rendered based on transparent texture.
In step 934, in acquisition image on the second determining curved surface, based on default texture rendering three-dimensional scenes mould
Type, and make in the second curved surface compared with occupy-place model in the big region of picture depth, it is preferential with the rendering of occupy-place model.
The transparent rendering of occupy-place model is connected, step 934 is on the second curved surface based on default texture come rendering three-dimensional scenes
Model.Since occupy-place model is rendered with transparent texture, shown in Fig. 10 pass is blocked to embody in finally acquisition image
System can make in the second curved surface in the rendering of step 934 compared with occupy-place model in the big region of picture depth, with occupy-place mould
The rendering of type is preferential.In other words, in the second curved surface situation more farther than occupy-place modal distance camera (or human eye), preferential wash with watercolours
The transparent texture for contaminating occupy-place model, being equivalent in the region blocked in acquisition image by subject in this way not will do it three-dimensional
The rendering of model of place.
In one embodiment, it for example can be translucent texture for the default texture of rendering three-dimensional scenes model, also
Gradient color can be set.In one embodiment, around occupy-place model settable multiple and different sizes the second curved surface, lead to
It crosses and renders the different stair-stepping effects of texture realization, further improve the immersion experience of user.
Figure 11 shows the AR effect picture using the present embodiment method.As shown in figure 11, user uses the end for integrating AR function
The image 111 that end collects certain cup-shaped product determines the corresponding production of present image 111 after the characteristic pattern 112 for identifying cup body
Product exist can AR show material;In turn, terminal get displaying material corresponding with the product include three-dimensional scene models and
Placeholder model, by the process flow of Fig. 9, multiple cylindrical surfaces that cup body can be finally surround in image 111 show three-dimensional
Around the effect of water curtain.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge
The preset three-dimensional scene models of display Chu not be surround around the corresponding position of curved surface, so as to enhance the application range of AR equipment
And processing capacity, while improving the immersion experience of user.
Figure 12 is a kind of flow chart of the display methods based on AR shown according to another exemplary embodiment.Such as Figure 12 institute
Show, this method can be executed by integrating the terminal device of AR function, it may include following steps 1210-1270.
In step 1210, the image of terminal acquisition subject, and be uploaded to server and carry out curved surface identification.
In one embodiment, the picture that terminal can acquire camera selects, such as enters static shape in terminal
After state, comparison front and back number frame simultaneously selects an optimal frame to be uploaded to server progress curved surface identification, and the curved surface of waiting for server
Recognition result.
In step 1220, terminal is locally carrying out curved surface identification, and subsequent step 1230 is carried out if identifying successfully, no
The then curved surface recognition result of waiting for server.
In one embodiment, terminal can integrate curved surface identification function, at the same be locally stored for acquisition image
The feature picture library being compared and the material database for carrying out AR displaying.For example, the song that terminal can transmit previous server
Face recognition result and corresponding material are all buffered in local, in this way, once subsequent same subject occur, terminal is being identified
The result being locally stored can be called directly after success.
On the other hand, if the curved surface recognition failures of terminal local, it may wait for the curved surface recognition result of server feedback,
Subsequent step 1230 is carried out if server feedback identifies successfully, the image that otherwise return step 1210 carries out next time is adopted
Collection.
In step 1230, terminal obtains the corresponding material of feature object and configuration information.
Based on the curved surface recognition result that local or server are transmitted, terminal can determine feature pair corresponding with acquisition image
As.Here, by taking the present embodiment is applied to business promotion as an example, feature object for example can be specific products to be promoted, above-mentioned
Characteristic pattern for example can be the specific pattern in the product packaging, by being pre-configured between feature object, characteristic pattern and material
Corresponding relationship, terminal can obtain corresponding material when identifying characteristic pattern.
In one embodiment, terminal can also obtain corresponding configuration information while obtaining material.Here material
It can include but is not limited to any combination of video content, three-dimensional scene models and webpage view.Correspondingly, configuration here
Information may include the display order of material, jump relationship etc., such as show three automatically after the broadcasting for completing video content
Model of place is tieed up, or the interactive operation based on user jumps to webpage view after the display for completing three-dimensional scene models, etc.
Deng.
Here material and configuration information both can store on the server for executing curved surface identification, can also be stored in advance
Or terminal device local is cached in after downloading, it can be with separate storage in configuration end equipment.It is applied to business with the present embodiment to push away
Widely example for businessman can provide access interface in configuration end equipment, by businessman's upload characteristic pattern corresponding with product to be promoted,
Material and configuration information etc.;Server, can be from the case where identifying successfully after the curved surface identification request for receiving terminal
It configures end equipment and obtains corresponding material and configuration information, and return to terminal together with or as curved surface recognition result.
In step 1240, terminal carries out the display of material according to configuration information in acquisition image.
Different type based on material, terminal may be selected march plane video according to configuration information and play 1250, three dimensional field
Scape model display 1260 and webpage view show any combination of 1270 equal multiple display modes.
In one embodiment, webpage view here is set as background transparent, only shows the behaviour for interacting with user
Make component (such as button), in response to user operation can trigger jump to the subsequent display page.
In one embodiment, can curved surface video playing 1250 and three-dimensional scene models show 1260 the two at least its
One of after the completion of, the webpage view for automatic jumping to background transparent shows 1270, shows the operational group for interacting with user
Part.In another embodiment, webpage view can be also jumped directly to after step 1230 and show 1270, and then in webpage view
It is middle complete curved surface video for example shown in fig. 5 show and, for example, three-dimensional scene models shown in Fig. 9 render at least within
One of, while operating assembly of the display for being interacted with user in webpage view.
In one embodiment, also settable aobvious after step 1240 when method of the invention being applied to business promotion
Show the step (not shown) of the business promotion page.It in other words, can be in curved surface according to the difference of material and configuration information
After the completion of video playing 1250 or three-dimensional scene models show 1260, the display of the business promotion page is automatic jumped to, it can also be with
The operation in 1270 is shown in webpage view in response to user, jumps to the display of the business promotion page.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, utilizes service in curved surface identification
The recognition result of device can not only improve the accuracy rate of identification, and can save the calculation processing resource of terminal;Another party
Face, the present embodiment carry out the flexible displaying of AR material according to configuration information, can enhance AR equipment after curved surface identifies successfully
Application range and processing capacity, while improving the immersion experience of user.
Figure 13 shows an example flow diagram of step 1240 in embodiment illustrated in fig. 12.As shown in figure 13, step 1240
Such as it is implementable be include step 1241-1249.
In step 1241, whether terminal judges in material to include otherwise webpage view is tied if then carrying out step 1242
Line journey.
The display of webpage view will generally load resource in advance, if until preparing from video content or three-dimensional scene models
Display when jumping to webpage view, just start the resource from server load webpage view, will cause system processing time with
And the unnecessary waste of period of reservation of number.In one embodiment, terminal preferentially obtains in previous step 1230 matches confidence
Breath, and whether determined in material to be downloaded according to configuration information includes webpage view.If including carrying out the present embodiment
Subsequent step loads the resource of webpage view using downloading or showing the time of other materials;If not including, terminate this
The process of embodiment, then according to the content of material, accordingly according to the display of process march plane video for example shown in fig. 5,
Or according to process for example shown in Fig. 9, carry out the rendering of three-dimensional scene models.
In step 1242, whether terminal judges in material to include video content or threedimensional model scene, if then carrying out
Step 1243, otherwise start the preloading of webpage view resource and carry out step 1246.
In one embodiment, according to the configuration information preferentially obtained in step 1230, terminal may further determine that be downloaded
It whether include video content or threedimensional model scene in material.If including the subsequent step of the present embodiment being carried out, under utilization
The time of other materials is carried or shown to load the resource of webpage view;If not including, webpage view resource can be started
It preloads with the rendering for webpage view.
In one embodiment, terminal can show instruction user when starting the preloading of webpage view resource in this step
Picture of waiting, such as display download progress item etc..
In step 1243, terminal judges whether above-mentioned video content or threedimensional model scene come from local, if then into
Otherwise row step 1244 carries out step 1245.
In step 1244, terminal carries out the display of above-mentioned video content or threedimensional model scene, and starts webpage view
The preloading of resource.
In step 1245, terminal starts the preloading of webpage view resource, and carries out above-mentioned video content or three-dimensional mould
The downloading and display of type scene.
When subsequent steps 1242 are determined in material including video content or threedimensional model scene, step 1243 continues to judge this
Whether a little materials are stored in terminal local.If the judging result of step 1243 be it is yes, indicate that the aobvious of local material can be started
Show, while can use and show that the period of local material (corresponds to step to carry out the preloading of webpage view resource
1244).If the judging result of step 1243 be it is no, indicate that the downloading of local material can be started, while carrying out webpage view
The preloading of resource (corresponds to step 1245).
In one embodiment, webpage view can only provide subscriber interaction component (such as button), at this moment in step 1245
The downloading that other materials can first be carried out just carries out the preloading of webpage view resource when starting the display of other materials, thus
The process resource of terminal can be made full use of, while the subsequent time for opening webpage view can be greatly shortened.
In another embodiment, other than subscriber interaction component is provided, webpage view can also be used in video content or
The display of threedimensional model scene.For example, passing through the WebView (webpage for being superimposed a background transparent on the acquisition image of terminal
View), the texture of curved surface video is completed in WebView using webGL (Web Graphics Library, network graphic library)
The rendering of rendering and three-dimensional scene models, while subscriber interaction component being provided.At this moment webpage view can be carried out in step 1245 simultaneously
The preloading of image resource and the downloading of other materials can start the webpage view for rendering background transparent after being fully completed,
The displaying of curved surface video and three-dimensional scene models is provided based on webpage view simultaneously.
In step 1246, judge whether the preloading of webpage view resource is completed, if then carrying out step 1248, otherwise
Carry out step 1247.
In one embodiment, whether terminal can be issued based on web component has been completed the readjustment of resource load and has determined
Whether the preloading of webpage view resource is completed.If completing to preload, the rendering of webpage view can be started, and based on pre-
First it is arranged in webpage view and subscriber interaction component is provided, or march plane video while subscriber interaction component is provided
The rendering of texture rendering and three-dimensional scene models.
In step 1247, start the preloading for retrying webpage view resource, and triggering timing device, if timing be less than it is pre-
If threshold value then continues the judgement of step 1246, otherwise terminate process.
If step 1246 judges the preloading failure of webpage view resource, the timer for triggering step 1247 retries machine
System can determine whether to retry failure, to terminate the stream of the present embodiment when retrying the time more than preset threshold (such as 10 seconds)
Journey.
In step 1248, start the webpage view for rendering background transparent.
Described in subsequent steps 1242, in material only include webpage view when, step 1248 can have been preloaded in resource
At the rear rendering for directly starting webpage view.
It, can be in the local element of display when other materials except webpage view are from local described in subsequent steps 1244
The preloading of webpage view resource is carried out while material, is completed if step 1246 judges that resource preloads, step 1248 can
With after the display that step 1244 completes other materials, in response to user's operation or the rendering of webpage view is automatically begun to.
Described in subsequent steps 1245, in one embodiment, the downloading of other materials except webpage view can be first carried out,
The preloading that webpage view resource is just carried out when starting the display of other materials, if step 1246 judges that resource has preloaded
At then step 1248 in response to user's operation or can automatically begin to webpage after the display that step 1245 completes other materials
The rendering of view.In another embodiment, can be carried out simultaneously in step 1245 webpage view resource preloading and other
The downloading of material can jump to step 1248 after being fully completed and start to render the webpage view of background transparent, be based on simultaneously
Webpage view provides the displaying of curved surface video and three-dimensional scene models.
In step 1249, judge whether the interactive operation for receiving user, if then carrying out according to operation content subsequent
Processing, otherwise terminates process.
As it was noted above, can provide the component for interacting with user in the webpage view of background transparent.This step
The interactive operation that then may be in response to user in rapid carries out subsequent processing.In the example that the present embodiment is applied to business promotion, use
After operating assembly (such as button) on the webpage clicking view of family, such as the official website homepage for promoting object, product page can be jumped to
Face specifically publicizes the page.On the other hand, after the rendering that step 1248 completes webpage view is shown, if in certain time
The interactive operation of user is not all received inside, or acquisition image has occurred that (such as user moves the position of camera for variation
Set), then it can terminate the processing that process continues next acquisition image.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, plain using showing or downloading other
The time of material carries out the preloading of webpage view resource simultaneously, avoids preparing display webpage after the display for completing other materials
The preloading that resource is just carried out when view can be improved treatment effeciency, save the processing time, while the net to be loaded such as shorten user
The time of page view, improve user experience;In addition, by the webpage view of the Overlapping display background transparent on acquisition image,
For providing subscriber interaction component, or the rendering of also achievable curved surface video and three-dimensional scene models simultaneously, end can be saved
The process resource at end, while allowing the presence of the imperceptible webpage of user, further improve user experience.
Following is the device of the invention embodiment, can be used for executing the above-mentioned display methods based on AR of the present invention and implements
Example.For undisclosed details in apparatus of the present invention embodiment, display methods embodiment of the aforementioned present invention based on AR is please referred to.
Figure 14 is a kind of block diagram of display device based on AR shown according to an exemplary embodiment.As shown, this
Display device of the embodiment based on AR includes curved surface identification module 1410, material obtaining module 1420 and Overlapping display module
1430。
Curved surface identification module 1410 is set as obtaining the curved surface recognition result to acquisition image, the determining and acquisition image
Corresponding feature object.
Material obtaining module 1420 is set as obtaining pre-generated material according to the determining feature object.
Overlapping display module 1430 is set as based on the first surface obtained from the curved surface recognition result, in the acquisition
On image in the form of the second curved surface material described in Overlapping display.
The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface
The pre-generated material of corresponding position Overlapping display changes simultaneously so as to enhance the application range and processing capacity of AR equipment
The immersion experience of kind user.
Connect, in one embodiment, curved surface identification module 1410 from the curved surface that curved surface recognition result obtains include cylinder
Face, the position that Overlapping display module 1430 is further arranged to correspond on acquisition image the cylindrical surface carry out video content
Application of a surface is shown.
For example, Overlapping display module 1430 may be configured as: the top surface arc side on the cylindrical surface is corresponding with bottom surface arc side etc.
It is divided into n parts, a arc side Along ent of 2* (n+1) is obtained;2* (n+1) a arc side on the cylindrical surface is obtained in rectangular coordinate system
Along ent to be mapping relations between the corresponding obtained a straight flange Along ent of 2* (n+1) of rectangle by the cylinder face expansion;It will
The frame data of the video content are converted into rendering texture, are connected by the straight flange Along ent of the rectangle is successively adjacent by zigzag
It connects and the rectangle is divided into 2n triangle, the minimum unit of the rendering texture corresponds to the 2n triangle wherein
One of;According to the mapping relations between the arc side Along ent and the straight flange Along ent, by the rendering texture mapping to institute
State cylindrical surface.
The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface
Corresponding position fitting shows that preset video content changes simultaneously so as to enhance the application range and processing capacity of AR equipment
The immersion experience of kind user.
It connects, in another embodiment, above-mentioned second curved surface is obtained with curved surface identification module 1410 from curved surface recognition result
The first surface arrived is spaced apart and around institute's first surface, and Overlapping display module 1430 is further arranged in the acquisition image
The upper position corresponding to second curved surface shows the three-dimensional scene models in the material.
For example, Overlapping display module 1430 may be configured as: determining the position of occupy-place model according to the acquisition image;According to
Pre-set depth occlusion relationship determines the position of second curved surface;It is rendered on the acquisition image based on transparent texture
The occupy-place model, to appear object where the first surface;And on the second curved surface determined in the acquisition image,
The three-dimensional scene models are rendered based on default texture, and make in second curved surface image depth with the occupy-place model compared with
It spends in big region, it is preferential with the rendering of the occupy-place model.
The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface
Around preset three-dimensional scene models are shown around corresponding position, so as to enhance the application range and processing energy of AR equipment
Power, while improving the immersion experience of user.
Figure 15 is a kind of block diagram of the display device based on AR shown according to another exemplary embodiment.As shown,
On the basis of embodiment illustrated in fig. 14, display device of the present embodiment based on AR further includes image capture module 1440 and image
Uploading module 1450;Overlapping display module 1430 further comprise video playback unit 1431, scene display unit 1432 and
Webpage rendering unit 1433.
Image capture module 1440 is set as the image of acquisition subject, and is respectively sent to curved surface identification module 1410
With image uploading module 1450.
Image uploading module 1450 is set as the image that image capture module 1440 acquires being uploaded to server march
Face identification.
Material obtaining module 1420 is notified to carry out material and configuration information if curved surface identification module 1410 identifies successfully
Acquisition, waited if recognition failures image uploading module 1450 receive server recognition result.If subsequent according to figure
As the received recognition result of uploading module 1450, confirmation server curved surface is identified successfully, then notify material obtaining module 1420 into
Otherwise the acquisition of row material and configuration information notifies image capture module 1440 to continue to acquire and transmit image.
In one embodiment, material obtaining module 1420 can also obtain corresponding with confidence while obtaining material
Breath.Here material can include but is not limited to any combination of video content, three-dimensional scene models and webpage view.Accordingly
, configuration information here may include the display order of material, jump relationship etc., such as in the broadcasting for completing video content
Show three-dimensional scene models automatically afterwards, or the interactive operation based on user jumps to after the display for completing three-dimensional scene models
Webpage view, etc..
Here material and configuration information both can store on the server for executing curved surface identification, can also be stored in advance
Or terminal device local is cached in after downloading, it can be with separate storage in configuration end equipment.It is applied to business with the present embodiment to push away
Widely example for businessman can provide access interface in configuration end equipment, by businessman's upload characteristic pattern corresponding with product to be promoted,
Material and configuration information etc.;Server, can be from the case where identifying successfully after the curved surface identification request for receiving terminal
It configures end equipment and obtains corresponding material and configuration information, and return to terminal together with or as curved surface recognition result.
Overlapping display module 1430 notifies video playback unit according to the configuration information that material obtaining module 1420 obtains
1431, scene display unit 1432 and webpage rendering unit 1433 distinguish the broadcasting of march plane video, three-dimensional scene models exhibition
Show and any combination for the multiple display modes such as webpage view is shown.
In one embodiment, webpage view here is set as background transparent, and webpage rendering unit 1433 is only set as
Operating assembly (such as button) of the display for interact with user in webpage view, the operation in response to user triggerable jump
To the subsequent display page.
In one embodiment, can video playback unit 1431 and the two of scene display unit 1432 at least within it
After one completes corresponding material displaying, the webpage view that notice webpage rendering unit 1433 jumps to background transparent shows that display is used
In the operating assembly interacted with user.In another embodiment, also material can be got in material obtaining module 1420 and match
After confidence breath, notice webpage rendering unit 1433 jumps to the display of webpage view, and then the complete curved surface video in webpage view
At least one in display and three-dimensional scene models rendering, while showing in webpage view for interacting with user
Operating assembly.
In one embodiment, when method of the invention being applied to business promotion, video playback unit 1431, scene exhibition
Show unit 1432 and webpage rendering unit 1433, three can also be respectively connected to the module of the display business promotion page (in figure
It is not shown).It in other words, can be in video playback unit 1431, scene display unit according to the difference of material and configuration information
1432 and 1433 three of webpage rendering unit material show after the completion of, notify the module to jump to the aobvious of the business promotion page
Show;Or the module can also jump to the display of the business promotion page in response to operation of the user in webpage view.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, utilizes service in curved surface identification
The recognition result of device can not only improve the accuracy rate of identification, and can save the calculation processing resource of terminal;Another party
Face, the present embodiment carry out the flexible displaying of AR material according to configuration information, can enhance AR equipment after curved surface identifies successfully
Application range and processing capacity, while improving the immersion experience of user.
Figure 16 shows an example block diagram of Overlapping display module 1430 in embodiment illustrated in fig. 15.As shown, scheming
On the basis of 15 illustrated embodiments, the Overlapping display module 1430 of the present embodiment further comprises material determination unit 1434, pre-
Loading unit 1435 and operation monitoring unit 1436.
Material determination unit 1434 is set as judging the story types that material obtaining module 1420 obtains.
Pre-add carrier unit 1435 is arranged for the preloading of webpage view resource.
Operation monitoring unit 1436 is set as the interactive operation of monitoring user.
In one embodiment, determine to include webpage view in material in material determination unit 1434, and including from this
When the video content or threedimensional model scene on ground, notify respectively video playback unit 1431 or scene display unit 1432 accordingly into
The displaying of row video content or threedimensional model scene, and pre-add carrier unit 1435 is notified to carry out the preloading of webpage view resource.
In one embodiment, determine to include webpage view in material in material determination unit 1434, and including under needs
When the video content or threedimensional model scene of load, notice pre-add carrier unit 1435 carries out the preloading of webpage view resource, and leads to
Know that material obtaining module 1420 carries out the downloading of video content or threedimensional model scene.
In one embodiment, determine only include webpage view in material in material determination unit 1434, without including view
Frequently when interior perhaps threedimensional model scene, notice pre-add carrier unit 1435 carries out the preloading of webpage view resource.
In one embodiment, pre-add carrier unit 1435 notifies net when the preloading for determining webpage view resource is completed
Page rendering unit 1433 starts to render the webpage view of background transparent.
When only including webpage view in material, webpage rendering unit 1433 can directly be opened after the completion of resource preloads
The rendering of beginning webpage view.
It, can be in video playback unit 1431 or scene display list when other materials except webpage view are from local
While member 1432 shows local material, the preloading of webpage view resource is carried out by pre-add carrier unit 1435, if preloaded
Unit 1435, which judges that resource preloads, to be completed, then can complete other in video playback unit 1431 or scene display unit 1432
After the display of material, in response to user's operation or the rendering of webpage view is automatically begun to by webpage rendering unit 1433.
In one embodiment, when other materials except webpage view need to download, material obtaining module 1420 can
The downloading for first carrying out other materials, when starting the display of other materials, pre-add carrier unit 1435 just carries out webpage view resource
Preloading.It is completed if pre-add carrier unit 1435 judges that resource preloads, it can be in video playback unit 1431 or scene
After display unit 1432 completes the display of other materials, in response to user's operation or net is automatically begun to by webpage rendering unit 1433
The rendering of page view.In another embodiment, pre-add carrier unit 1435 and material obtaining module 1420 can carry out webpage simultaneously
The preloading of view resource and the downloading of other materials can notify webpage rendering unit 1433 to start wash with watercolours after being fully completed
The webpage view of background transparent is contaminated, while the displaying of curved surface video and three-dimensional scene models being provided based on webpage view.
The webpage view that operation monitoring unit 1436 shows webpage rendering unit 1433 is monitored, and judges whether to receive
To the interactive operation of user, if then carrying out subsequent processing according to operation content.
For example, can provide the component for interacting with user in the webpage view of background transparent.Operation monitoring is single
The interactive operation that member 1436 then may be in response to user carries out subsequent processing.In the example that the present embodiment is applied to business promotion,
After operating assembly (such as button) on user's webpage clicking view, operation monitoring unit 1436 can notify webpage rendering unit
1433 jump to the official website homepage for promoting object, the product page or specifically publicize the page.On the other hand, in webpage rendering unit
After the rendering of 1433 completion webpage views is shown, if operation monitoring unit 1436 does not all receive the friendship of user within a certain period of time
Interoperability, or acquisition image have occurred that variation (such as user move camera position), then operate monitoring unit
1436 can notify image capture module 1440 to continue the processing of next acquisition image.
In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, plain using showing or downloading other
The time of material carries out the preloading of webpage view resource simultaneously, avoids preparing display webpage after the display for completing other materials
The preloading that resource is just carried out when view can be improved treatment effeciency, save the processing time, while the net to be loaded such as shorten user
The time of page view, improve user experience;In addition, by the webpage view of the Overlapping display background transparent on acquisition image,
For providing subscriber interaction component, or the rendering of also achievable curved surface video and three-dimensional scene models simultaneously, end can be saved
The process resource at end, while allowing the presence of the imperceptible webpage of user, further improve user experience.
About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method
Embodiment in be described in detail, no detailed explanation will be given here.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to embodiment of the present disclosure, it is above-described two or more
Module or the feature and function of unit can embody in a module or unit.Conversely, an above-described mould
The feature and function of block or unit can be to be embodied by multiple modules or unit with further division.As module or list
The component of member display may or may not be physical unit, it can and it is in one place, or may be distributed over
In multiple network units.Some or all of the modules therein can be selected to realize disclosure scheme according to the actual needs
Purpose.
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the present invention
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, touch control terminal or network equipment etc.) executes embodiment according to the present invention
Method.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention
Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is limited only by the attached claims.
Claims (14)
1. a kind of display methods based on augmented reality, which is characterized in that the described method includes:
The curved surface recognition result to acquisition image is obtained, determines feature object corresponding with the acquisition image;
Pre-generated material is obtained according to the determining feature object;And
Based on the first surface obtained from the curved surface recognition result, it is superimposed in the form of the second curved surface on the acquisition image
Show the material.
2. the method as described in claim 1, which is characterized in that second curved surface is bonded with the first surface, described
On the acquisition image in the form of the second curved surface material described in Overlapping display, comprising:
The position fitting for corresponding to the first surface on the acquisition image shows the material.
3. method according to claim 2, which is characterized in that the material includes video content, described in the acquisition
Position fitting on image corresponding to the first surface shows the material, comprising:
Based on the form parameter of the first surface, when the first surface is expanded into plane by acquisition, the plane with it is described
Mapping relations between first surface;
By the frame data of the video content be converted into rendering texture, according to the mapping relations by the rendering texture mapping extremely
The first surface.
4. method according to claim 2, which is characterized in that the first surface includes cylindrical surface, and the material includes view
Frequency content, the position fitting that the first surface is corresponded on the acquisition image show the material, comprising:
N parts are divided by the top surface arc side on the cylindrical surface is corresponding with bottom surface arc side, a arc side Along ent of 2* (n+1) is obtained;
A arc side Along ent of 2* (n+1) on the cylindrical surface is obtained in rectangular coordinate system and be rectangle by the cylinder face expansion
The mapping relations between a straight flange Along ent of 2* (n+1) accordingly obtained;
The frame data of the video content are converted into rendering texture, press Z by the straight flange Along ent of the rectangle is successively adjacent
The rectangle is divided into 2n triangle by font connection, and the minimum unit of the rendering texture corresponds to the 2n triangle
One of;
According to the mapping relations between the arc side Along ent and the straight flange Along ent, by the rendering texture mapping to described
Cylindrical surface.
5. the method as described in claim 1, which is characterized in that second curved surface is spaced apart and circular with the first surface
The first surface, it is described on the acquisition image in the form of the second curved surface material described in Overlapping display, comprising:
The position for corresponding to second curved surface on the acquisition image shows the three-dimensional scene models in the material.
6. method as claimed in claim 5, which is characterized in that second curved surface includes cylindrical surface, and the material further includes
Occupy-place model corresponding with object where the first surface, it is described to correspond to second curved surface on the acquisition image
Position show the three-dimensional scene models in the material, comprising:
The position of the occupy-place model is determined according to the acquisition image;
The position of second curved surface is determined according to pre-set depth occlusion relationship;
The occupy-place model is rendered based on transparent texture on the acquisition image, to appear object where the first surface;
And
On the second curved surface determined in the acquisition image, the three-dimensional scene models are rendered based on default texture, and make
It is preferential with the rendering of the occupy-place model in second curved surface compared with the occupy-place model in the big region of picture depth.
7. as the method according to claim 1 to 6, which is characterized in that the method also includes:
Local curved surface identification is carried out to obtain the curved surface recognition result to the acquisition image;
In the local curved surface recognition failures, uploads the acquisition image and carry out long-range curved surface identification;And
From remotely obtaining the curved surface recognition result and the material.
8. as the method according to claim 1 to 6, which is characterized in that the method also includes:
Obtain configuration information corresponding with the feature object;And
The display of the material is carried out according to the configuration information,
Wherein, the material include in video content, three-dimensional scene models and webpage view at least any one, institute
State the display order that configuration information includes at least the material.
9. method according to claim 8, which is characterized in that described to carry out the aobvious of the material according to the configuration information
Show, comprising:
When determining feature object corresponding with the acquisition image, the webpage view is shown;Or
On the acquisition image in the form of the second curved surface video content described in Overlapping display or the three-dimensional scene models it
Afterwards, the webpage view is shown.
10. method as claimed in claim 9, which is characterized in that
Before the display webpage view, the method also includes: when obtaining the material preload with it is described
The corresponding resource of webpage view;
The display webpage view, comprising: the notice of load is completed in response to the resource, carries out the webpage view
The rendering of figure.
11. as the method according to claim 1 to 6, which is characterized in that it is described on the acquisition image with second
Material described in the form Overlapping display of curved surface, comprising:
Overlapping display background is transparent webpage view on the acquisition image;And
The material based on acquisition carried out in the webpage view video content and three-dimensional scene models at least one
Rendering, with video content and three dimensional field described in position Overlapping display corresponding with second curved surface in the acquisition image
At least one of scape model.
12. a kind of display device based on augmented reality, which is characterized in that described device includes:
Curved surface identification module is set as obtaining the curved surface recognition result to acquisition image, and determination is corresponding with the acquisition image
Feature object;
Material obtaining module is set as obtaining pre-generated material according to the determining feature object;And
Overlapping display module is set as based on the first surface obtained from the curved surface recognition result, on the acquisition image
The material described in Overlapping display in the form of the second curved surface.
13. a kind of computer readable storage medium, is stored thereon with computer program, the computer program is executed by processor
The Shi Shixian display methods according to any one of claim 1 to 11 based on augmented reality.
14. a kind of electronic equipment characterized by comprising
Processor;And
Memory is stored with computer-readable instruction on the memory, and the computer-readable instruction is held by the processor
The display methods according to any one of claim 1 to 11 based on augmented reality is realized when row.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810235460.0A CN110262763B (en) | 2018-03-21 | 2018-03-21 | Augmented reality-based display method and apparatus, storage medium, and electronic device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810235460.0A CN110262763B (en) | 2018-03-21 | 2018-03-21 | Augmented reality-based display method and apparatus, storage medium, and electronic device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110262763A true CN110262763A (en) | 2019-09-20 |
CN110262763B CN110262763B (en) | 2021-10-15 |
Family
ID=67911565
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810235460.0A Active CN110262763B (en) | 2018-03-21 | 2018-03-21 | Augmented reality-based display method and apparatus, storage medium, and electronic device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110262763B (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110853143A (en) * | 2019-10-12 | 2020-02-28 | 广州亚美信息科技有限公司 | Scene implementation method and device, computer equipment and storage medium |
CN111599011A (en) * | 2020-07-15 | 2020-08-28 | 杭州电力设备制造有限公司 | WebGL technology-based rapid construction method and system for power system scene |
CN113031781A (en) * | 2021-04-16 | 2021-06-25 | 深圳市慧鲤科技有限公司 | Augmented reality resource display method and device, electronic equipment and storage medium |
CN113763566A (en) * | 2020-06-05 | 2021-12-07 | 光宝电子(广州)有限公司 | Image generation system and image generation method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106373182A (en) * | 2016-08-18 | 2017-02-01 | 苏州丽多数字科技有限公司 | Augmented reality-based human face interaction entertainment method |
CN106780703A (en) * | 2016-12-26 | 2017-05-31 | 上海今尚数码科技有限公司 | A kind of method that virtual furnishings finishing reality dynamic changes threedimensional model textures |
CN106875461A (en) * | 2015-12-10 | 2017-06-20 | 青岛华正信息技术有限公司 | One kind is tinted plane picture 3D model transformation systems and method |
CN107393017A (en) * | 2017-08-11 | 2017-11-24 | 北京铂石空间科技有限公司 | Image processing method, device, electronic equipment and storage medium |
CN107452073A (en) * | 2017-07-31 | 2017-12-08 | 成都九维云联科技有限公司 | Self-help shopping method based on augmented reality |
-
2018
- 2018-03-21 CN CN201810235460.0A patent/CN110262763B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106875461A (en) * | 2015-12-10 | 2017-06-20 | 青岛华正信息技术有限公司 | One kind is tinted plane picture 3D model transformation systems and method |
CN106373182A (en) * | 2016-08-18 | 2017-02-01 | 苏州丽多数字科技有限公司 | Augmented reality-based human face interaction entertainment method |
CN106780703A (en) * | 2016-12-26 | 2017-05-31 | 上海今尚数码科技有限公司 | A kind of method that virtual furnishings finishing reality dynamic changes threedimensional model textures |
CN107452073A (en) * | 2017-07-31 | 2017-12-08 | 成都九维云联科技有限公司 | Self-help shopping method based on augmented reality |
CN107393017A (en) * | 2017-08-11 | 2017-11-24 | 北京铂石空间科技有限公司 | Image processing method, device, electronic equipment and storage medium |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110853143A (en) * | 2019-10-12 | 2020-02-28 | 广州亚美信息科技有限公司 | Scene implementation method and device, computer equipment and storage medium |
CN113763566A (en) * | 2020-06-05 | 2021-12-07 | 光宝电子(广州)有限公司 | Image generation system and image generation method |
CN111599011A (en) * | 2020-07-15 | 2020-08-28 | 杭州电力设备制造有限公司 | WebGL technology-based rapid construction method and system for power system scene |
CN111599011B (en) * | 2020-07-15 | 2023-08-04 | 杭州电力设备制造有限公司 | Power system scene rapid construction method and system based on WebGL technology |
CN113031781A (en) * | 2021-04-16 | 2021-06-25 | 深圳市慧鲤科技有限公司 | Augmented reality resource display method and device, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN110262763B (en) | 2021-10-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105637564B (en) | Generate the Augmented Reality content of unknown object | |
CN110944727B (en) | System and method for controlling virtual camera | |
CN105981076B (en) | Synthesize the construction of augmented reality environment | |
CN110262763A (en) | Display methods and device and storage medium and electronic equipment based on augmented reality | |
CN106254848B (en) | A kind of learning method and terminal based on augmented reality | |
CN110020633A (en) | Training method, image-recognizing method and the device of gesture recognition model | |
CN111862333B (en) | Content processing method and device based on augmented reality, terminal equipment and storage medium | |
CN108027653A (en) | haptic interaction in virtual environment | |
CN108446698A (en) | Method, apparatus, medium and the electronic equipment of text are detected in the picture | |
WO2015102854A1 (en) | Assigning virtual user interface to physical object | |
KR102433857B1 (en) | Device and method for creating dynamic virtual content in mixed reality | |
CN111292427B (en) | Bone displacement information acquisition method, device, equipment and storage medium | |
US20230162452A1 (en) | Implementation and display of augmented reality | |
CN113721804A (en) | Display method, display device, electronic equipment and computer readable storage medium | |
CN109189302B (en) | Control method and device of AR virtual model | |
CN109410299A (en) | A kind of information processing method, device and computer storage medium | |
CN108459704A (en) | Stroke for 3-dimensional digital content operates prediction | |
US11561651B2 (en) | Virtual paintbrush implementing method and apparatus, and computer readable storage medium | |
CN104982090A (en) | Personal information communicator | |
CN111643899A (en) | Virtual article display method and device, electronic equipment and storage medium | |
CN111815780A (en) | Display method, display device, equipment and computer readable storage medium | |
Rumiński et al. | Creation of interactive AR content on mobile devices | |
CN107084740A (en) | A kind of air navigation aid and device | |
CN111971714A (en) | Load indicator in augmented reality environment | |
CN108776544A (en) | Exchange method and device, storage medium, electronic equipment in augmented reality |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |