CN110262763A - Display methods and device and storage medium and electronic equipment based on augmented reality - Google Patents

Abstract

Present invention discloses a kind of display methods based on augmented reality, comprising: obtains the curved surface recognition result to acquisition image, determines feature object corresponding with the acquisition image；Pre-generated material is obtained according to the determining feature object；And based on the first surface obtained from the curved surface recognition result, on the acquisition image in the form of the second curved surface material described in Overlapping display.Scheme based on the present embodiment so as to improve the application range and processing capacity of augmented reality equipment, while improving the immersion experience of user by pre-generated with the corresponding position Overlapping display of curved surface is identified material in acquisition image.

Description

Display methods and device and storage medium and electronic equipment based on augmented reality

Technical field

It is the present invention relates to field of computer technology, in particular to a kind of based on AR (Augmented Reality, enhancing Reality) display methods and device, computer readable storage medium and electronic equipment.

Background technique

AR technology is the position and angle by calculating acquisition image in real time, and the perception information generated using computer is to existing After the environmental element in the real world carries out " enhancing ", the technology that is directly or indirectly shown.Here perception information can be crossed over A variety of sensory forms, common includes vision and the sense of hearing, or even is also possible that tactile, body-sensing and smell.In this way, AR It changes current perception of the people to real world environments, realizes and virtual world is superimposed on real world on the screen, and with User interacts.

It is different from simple data investigation to show, AR is by incorporating sense of the people to real world for the component of digital world Know, mixed reality experience on the spot in person is provided for user.The characteristics of exactly being experienced for this immersion, at present AR technology Business application focuses primarily upon amusement and game service.However, with AR and its relevant technologies (such as image recognition, cloud service) Development, more and more industrial fields start to recognize the value in conjunction with brought by AR technology, such as medical research and dissection Training, precision instrument manufacture and maintenance, navigation of electronic map, teleconference and long-range control and business promotion etc..

On the other hand, AR technology is limited by the development of its relevant technologies again simultaneously, the former will often lag behind the hair of the latter Exhibition progress.By taking image recognition technology as an example, current two-dimensional image identification reached its maturity and accuracy rate and stability all It is higher, but for the image recognition of the three-dimension curved surfaces such as face also in the stage continuously improved.Correspondingly, AR technology is for acquisition The identification mainstream of image is still known otherwise using two dimension, curved surface identification and corresponding identification post-processing is lacked relevant Application achievements, at present still in research.

Summary of the invention

In order to solve the problems, such as to lack AR technical application for curved surface identification and corresponding identification post-processing in the related technology, The present invention provides a kind of display methods based on AR and devices, computer readable storage medium and electronic equipment.

According to an embodiment of the invention, providing a kind of display methods based on AR, comprising: obtain the curved surface to acquisition image Recognition result determines feature object corresponding with the acquisition image；It is obtained according to the determining feature object pre-generated Material；And based on the first surface obtained from the curved surface recognition result, with the second curved surface on the acquisition image Material described in form Overlapping display.

According to an embodiment of the invention, providing a kind of display device based on AR, comprising: curved surface identification module is set as The curved surface recognition result to acquisition image is obtained, determines feature object corresponding with the acquisition image；Material obtaining module, if It is set to and obtains pre-generated material according to the determining feature object；And Overlapping display module, it is set as being based on from institute State the first surface that curved surface recognition result obtains, on the acquisition image in the form of the second curved surface element described in Overlapping display Material.

According to an embodiment of the invention, providing a kind of computer readable storage medium, it is stored thereon with computer program, institute It states and realizes when computer program is executed by processor as described above based on the display methods of AR.

According to an embodiment of the invention, providing a kind of electronic equipment, comprising: processor；And memory, the memory On be stored with computer-readable instruction, when the computer-readable instruction is executed by the processor realize be based on as described above The display methods of AR.

The technical solution that the embodiment of the present invention provides can include the following benefits:

The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface The pre-generated material of corresponding position Overlapping display changes simultaneously so as to enhance the application range and processing capacity of AR equipment The immersion experience of kind user.

It should be understood that the above general description and the following detailed description are merely exemplary, this can not be limited Invention.

Detailed description of the invention

The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention Example, and in specification together principle for explaining the present invention.

Fig. 1 is shown can be using the exemplary system architecture of display methods or device of the embodiment of the present invention based on AR Schematic diagram.

Fig. 2 shows the structural schematic diagrams of the computer system of the electronic equipment suitable for being used to realize the embodiment of the present invention.

What Fig. 3 showed AR is normally applied scene.

Fig. 4 is a kind of flow chart of display methods based on AR shown according to an exemplary embodiment.

Fig. 5 shows an example flow diagram of step 430 in embodiment illustrated in fig. 4.

Fig. 6 shows the schematic diagram that point mapping relations are established between cylindrical surface and rectangle.

Fig. 7 shows the schematic illustration for calculating arc side Along ent coordinate.

Fig. 8 shows the AR effect picture using embodiment illustrated in fig. 5 method.

Fig. 9 shows another example flow diagram of step 430 in embodiment illustrated in fig. 4.

Figure 10 shows the depth occlusion exemplary relationship between three-dimensional scene models and occupy-place model.

Figure 11 shows the AR effect picture using embodiment illustrated in fig. 9 method.

Figure 12 is a kind of flow chart of the display methods based on AR shown according to another exemplary embodiment.

Figure 13 shows an example flow diagram of step 1240 in embodiment illustrated in fig. 12.

Figure 14 is a kind of block diagram of display device based on AR shown according to an exemplary embodiment.

Figure 15 is a kind of block diagram of the display device based on AR shown according to another exemplary embodiment.

Figure 16 shows an example block diagram of Overlapping display module 1430 in embodiment illustrated in fig. 15.

Specific embodiment

Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the present invention will more Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.

In addition, described feature, structure or characteristic can be incorporated in one or more implementations in any suitable manner In example.In the following description, many details are provided to provide and fully understand to the embodiment of the present invention.However, It will be appreciated by persons skilled in the art that technical solution of the present invention can be practiced without one or more in specific detail, Or it can be using other methods, constituent element, device, step etc..In other cases, it is not shown in detail or describes known side Method, device, realization or operation are to avoid fuzzy each aspect of the present invention.

Block diagram shown in the drawings is only functional entity, not necessarily must be corresponding with physically separate entity. I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuit These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.

Flow chart shown in the drawings is merely illustrative, it is not necessary to including all content and operation/step, It is not required to execute by described sequence.For example, some operation/steps can also decompose, and some operation/steps can close And or part merge, therefore the sequence actually executed is possible to change according to the actual situation.

Fig. 1 is shown can be using the exemplary system architecture of the display methods or device based on AR of the embodiment of the present invention 100 schematic diagram.

As shown in Figure 1, system architecture 100 may include one of terminal device 101,102,103 or a variety of, network 104 and server 105.Network 104 between terminal device 101,102,103 and server 105 to provide communication link Medium.Network 104 may include various connection types, such as wired, wireless communication link or fiber optic cables etc..

It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.For example server 105 can be multiple server compositions Server cluster etc..

User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out Send message etc..Terminal device 101,102,103 can be the various electronic equipments with display screen, including but not limited to intelligent hand Machine, tablet computer, portable computer and desktop computer etc..Server 105 can be to provide the server of various services.

For example, user, which can use terminal device 103 (being also possible to terminal device 101 or 102), obtains acquisition image, It is local to carry out curved surface identification and obtain curved surface recognition result to obtain curved surface recognition result, or from server 105, so as to determine with The corresponding feature object of the acquisition image；And then pre-generated material is obtained according to the determining feature object, here Material can to prestore or be buffered in terminal device 103 local, be also possible to be obtained from server 105 or other remotely set It is standby；Finally, terminal device 103 is based on the first surface obtained from the curved surface recognition result, on the acquisition image Material described in the form Overlapping display of two curved surfaces.

Fig. 2 shows the structural schematic diagrams of the computer system of the electronic equipment suitable for being used to realize the embodiment of the present invention.

It should be noted that Fig. 2 shows the computer system 200 of electronic equipment be only an example, should not be to this hair The function and use scope of bright embodiment bring any restrictions.

As shown in Fig. 2, computer system 200 includes central processing unit (CPU) 201, it can be read-only according to being stored in Program in memory (ROM) 202 or be loaded into the program in random access storage device (RAM) 203 from storage section 208 and Execute various movements appropriate and processing.In RAM 203, it is also stored with various programs and data needed for system operatio.CPU 201, ROM 202 and RAM 203 is connected with each other by bus 204.Input/output (I/O) interface 205 is also connected to bus 204。

I/O interface 205 is connected to lower component: the importation 206 including keyboard, mouse etc.；It is penetrated including such as cathode The output par, c 207 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 208 including hard disk etc.； And the communications portion 209 of the network interface card including LAN card, modem etc..Communications portion 209 via such as because The network of spy's net executes communication process.Driver 210 is also connected to I/O interface 205 as needed.Detachable media 211, such as Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 210, in order to read from thereon Computer program be mounted into storage section 208 as needed.

Particularly, according to an embodiment of the invention, may be implemented as computer below with reference to the process of flow chart description Software program.For example, the embodiment of the present invention includes a kind of computer program product comprising be carried on computer-readable medium On computer program, which includes the program code for method shown in execution flow chart.In such reality It applies in example, which can be downloaded and installed from network by communications portion 209, and/or from detachable media 211 are mounted.When the computer program is executed by central processing unit (CPU) 201, executes and limited in the system of the application Various functions.

It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device, Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned Any appropriate combination.

Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction It closes to realize.

Being described in unit involved in the embodiment of the present invention can be realized by way of software, can also be by hard The mode of part realizes that described unit also can be set in the processor.Wherein, the title of these units is in certain situation Under do not constitute restriction to the unit itself.

As on the other hand, present invention also provides a kind of computer-readable medium, which be can be Included in electronic equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying electronic equipment. Above-mentioned computer-readable medium carries one or more program, when the electronics is set by one for said one or multiple programs When standby execution, so that method described in electronic equipment realization as the following examples.For example, the electronic equipment can be real Now each step as shown in Fig. 4, Fig. 5, Fig. 9, Figure 12 and Figure 13.

Before elaborating the technical solution of the embodiment of the present invention, some relevant technical solutions and original introduced below Reason.

Plane identification and post-processing in AR

AR product common at present is mostly based on the plane identification of picture, and after identifying special object, broadcasting Related content (such as two-dimensional video or threedimensional model) can only be shown in front of picture.It for example appears in picture around bottle In the case where body, not only recognition accuracy is low, but also even if identifying that the video successfully, played or model can also be detached from picture, The immersion of user is caused to be experienced very poor.

What Fig. 3 showed AR is normally applied scene.As shown, using the terminal of integrated AR function to collect certain bottled by user The image 31 of product, after the characteristic pattern 32 for identifying body, determine the corresponding product of present image 31 exist can AR show Content；In turn, it is threedimensional model that terminal, which gets displaying content corresponding with the product, to show acquisition in image 31 Threedimensional model 33.Although some elements in threedimensional model 33 can show three-dimensional effect by zoom, such as Shown in figure, threedimensional model 33 is whole or the front of subject in image 31 is shown in the form of patch, therefore causes to use The immersion experience at family is very poor.

Curved surface identification

By the research to curved surface features, curved surface can generally be divided into following a few classes: linear stretch face, the surface of revolution, Bezier (Bei Zi) curve and surface, B-spline curves curved surface, NURBS (Non-Uniform Rational B-Splines, it is non- Even Rational B-splines) curve and surface, ruled surface, the face of sweeping into, hollows, equidistant surface etc..For the curved surface of different classifications, exist A variety of curved surface features identification technologies.At present common curved surface features identification technology include principal curvatures detection method, template matching method with And Point set matching method etc..

Although these methods achieve certain achievement for the research of curved surface features, due to the complexity of curved surface features Property, the curved surface type and feature type that can be handled at present are still limited.Nevertheless, this has no effect on implementation of the invention.It changes Yan Zhi, the present invention pay close attention to using curved surface identification result is post-processed accordingly, and and be not concerned with curved surface identify tool Body realizes that the technology for being able to carry out curved surface identification that is existing and occurring in the future can be applied to the displaying scheme the present invention is based on AR In.

Based on above-mentioned the relevant technologies, the embodiment of the present invention provides a kind of display methods based on AR and device, computer can Read storage medium and electronic equipment.

The principle of the technical solution of the embodiment of the present invention and realization details are described in detail below.

Fig. 4 is a kind of flow chart of display methods based on AR shown according to an exemplary embodiment.As shown in figure 4, This method can be executed by arbitrarily calculating equipment, it may include following steps 410-430.

In step 410, the curved surface recognition result to acquisition image is obtained, determines feature pair corresponding with acquisition image As.

In one embodiment, further include the steps that obtaining acquisition image before step 410.Acquisition figure based on acquisition Picture, can current device locally carry out curved surface identification obtain recognition result, can also will acquisition image be uploaded to server into The identification of row curved surface, and then recognition result is obtained from server.In one embodiment, curved surface identification can also locally carried out Acquisition image is uploaded simultaneously, subsequent processing is directly carried out if locally identifying successfully, otherwise waits for the curved surface identification of server As a result.

In one embodiment, the curved surface recognition result of acquisition may include the data of two parts.A part is used for and spy Sign library is compared, and to determine in feature database with the presence or absence of feature object corresponding with current acquired image, is had, is shown to identify Success can carry out subsequent AR material and show；Otherwise it indicates that AR material corresponding with current acquired image is not present, so as to It is handled with suspension.Another part data in curved surface recognition result may include the form parameter of acquired image top-surface camber part, So as to the subsequent Overlapping display for carrying out material on acquisition image based on the form parameter.

Feature database described here may include the set of characteristic pattern.The method of the present embodiment is applied to business promotion For, the feature picture library of commodity to be promoted can be pre-created.Once finding and acquiring in feature picture library by curved surface identification The characteristic pattern of images match indicates the displaying that AR material can be carried out on acquisition image, otherwise can be with termination.At one In embodiment, it is local that feature database can store the execution equipment in method, can also be remotely stored on server, or both it is simultaneous Have.

In one embodiment, the determination of feature object can be completed independently in the execution equipment of method, or can also be with Complete and definitive result returned into method here to execute equipment, the embodiment of the present invention by the server of carry out curved surface identification To this, also there is no restriction.

At step 420, pre-generated material is obtained according to determining feature object.

Here material include can acquisition image on Overlapping display content, for example including but be not limited in video Appearance, three-dimensional scene models and webpage view etc..It, can be preparatory still by taking the method for the present embodiment is applied to business promotion as an example Creation is directed to the material database of commodity to be promoted, and safeguards material database and wait promote the mapping relations between commodity.Implement at one In example, material database here be can store in any combination of following position: the execution equipment of method is local, executes curved surface identification Server and above-mentioned execution equipment or the addressable configuration end equipment of server.

In one embodiment, the execution equipment of method is locally carrying out after curved surface is identified as function by step 410, in turn Material corresponding with feature object can be obtained in local material database.

In another embodiment, the execution equipment of method obtains curved surface recognition result and spy from server by step 410 When levying the definitive result of object, and then corresponding with feature object material can be obtained from configuration end equipment, or can also equally from Server obtains material corresponding with feature object.In the later case, the material that server obtains can come from itself and deposit Self-configuring end equipment is stored up or comes, and server can be by material with the definitive result of curved surface recognition result and feature object It feeds back to method and executes equipment.

In step 430, based on the first surface obtained from curved surface recognition result, with the second curved surface on acquisition image Form Overlapping display material.

AR in the related technology is different to be only capable of showing planar video or patch on acquisition image based on plane recognition result When three-dimensional scenic can be on acquisition image with the based on the first surface obtained from curved surface recognition result in the present embodiment The material of the aforementioned acquisition of form Overlapping display of two curved surfaces.Here first surface for example can be in acquisition image as feature The curvature portion of object matching foundation, the second curved surface are then the calculated display curved surfaces of form parameter institute according to first surface. For different display demands, for example, be directed to different materials, first surface and the second curved surface here can it is identical can also be with It is different.

In one embodiment, the second curved surface here can fit in first surface.It include that video content is with material Example, step 430 may include: the form parameter based on first surface, and acquisition is when expanding into plane for first surface, the plane with Mapping relations between first surface；And the frame data of video content in material are converted into rendering texture, it is reflected according to above-mentioned Texture mapping will be rendered to first surface by penetrating relationship.

In another embodiment, above-mentioned second curved surface can be spaced apart with first surface and around first surface.With For material includes three-dimensional scene models, at this moment step 430 may include: to correspond to the position of the second curved surface on acquisition image Show the three-dimensional scene models in material.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge Not Chu curved surface the pre-generated material of corresponding position Overlapping display, so as to enhance the application range and processing energy of AR equipment Power, while improving the immersion experience of user.

Fig. 5 shows an example flow diagram of step 430 in embodiment illustrated in fig. 4.In this example, it identifies and ties from curved surface The curved surface that fruit obtains includes cylindrical surface, and the position that step 430 further corresponds to the cylindrical surface on acquisition image carries out video The application of a surface of content is shown.As shown in figure 5, for example may include step 531-534.

In step 531, n parts are divided by the top surface arc side on cylindrical surface is corresponding with bottom surface arc side, it is a that 2* (n+1) is obtained Arc side Along ent.

In order to realize the display of curved surface video, the present embodiment is re-mapped by the way that the video frame of rectangle is converted into rendering texture Onto cylindrical surface.In one embodiment, that cylindrical surface is interpreted as rectangle straight line (i.e. the central axis on cylindrical surface) in space is curved Song is formed by face, and stretching to cylindrical surface can be unfolded to obtain corresponding rectangle.It should be noted that cylindrical surface here It can be complete cylindrical surface, can also be one section in complete cylindrical surface.

Fig. 6 shows the schematic diagram that point mapping relations are established between cylindrical surface and rectangle.As shown in fig. 6, by cylindrical surface 61 Top surface arc side 611 it is corresponding with bottom surface arc side 612 be divided into n parts, including endpoint, a arc side equal part of 2* (n+1) is obtained Point.

In step 532, obtained in rectangular coordinate system a arc side Along ent of 2* (n+1) on cylindrical surface with by cylindrical surface exhibition Open the mapping relations between a straight flange Along ent of 2* (n+1) accordingly obtained for rectangle.

It connects as shown in fig. 6, being operated based on equal part identical with cylindrical surface 61, the top margin 621 of rectangle 62 and bottom edge 622 are same Sample, which can correspond to, is divided into n parts, and a straight flange Along ent of 2* (n+1) can be obtained.Next, can continue to acquire 2* (n+1) a arc side etc. Mapping relations between branch and a straight flange Along ent of 2* (n+1).

Fig. 7 shows the schematic illustration for calculating arc side Along ent coordinate.As shown in fig. 7, the top surface arc based on cylindrical surface 61 Auxiliary circle 71 is added on side 611, and the inscribed equilateral polygon 72 that number of edges is m is added in auxiliary circle 71, levels off to infinity in m When, which also approaches the shape for auxiliary circle 71.Directly make line with center of circle O on each vertex of polygon 72, it is assumed that The angle of any two adjacent vertex and center of circle O line is θ, and using center of circle O as the origin of rectangular coordinate system, it is each can to acquire polygon The coordinate expressions on vertex (namely arc side Along ent) are as follows.

(x_i, y_i)=(r*cos i θ, r*sin i θ) ... (1)

Wherein, with point P shown in Fig. 7 for first vertex, anticlockwise vertex serial number is prolonged in i expression, and r indicates top surface The radius of arc side 611.

Based on the above principles, it can acquire in top surface arc side 611 and class is pushed into a arc side of total 2* (n+1) in bottom surface arc side 612 The coordinate expressions of Along ent.

A ', B ' shown in Fig. 6, for 3 points of C ', it is assumed that A ' and B ' is respectively top surface arc side 611 and bottom surface arc side 612 First vertex on corresponding circle, then 3 points of coordinate expressions in spatial triaxial coordinate system are as follows.

A′(x_A′, y_A′, z_A′)=(r*cos θ, r*sin θ, 0) ... (2)

B′(x_B′, y_B′, z_B′)=(r*cos θ, r*sin θ, h) ... (3)

C′(x_C′, y_C′, z_c′)=(r*cos (θ+360/n), r* (θ+360/n), h) ... (4)

Wherein, h indicates the height on cylindrical surface.

In step 533, the frame data of video content are converted into rendering texture, by the straight flange Along ent successively phase of rectangle Adjacent is divided into 2n triangle for rectangle by zigzag connection, and the minimum unit of above-mentioned rendering texture corresponds to 2n triangle One of.

In step 534, according to the mapping relations between arc side Along ent and straight flange Along ent, extremely by rendering texture mapping Cylindrical surface.

It returns to shown in Fig. 6, is connected by the straight flange Along ent of rectangle 62 is successively adjacent by zigzag, it can be by 62 equal part of rectangle For 2n triangle, one of them is, for example, △ ABC.Based on similar operations, 2n triangle (its can be obtained on cylindrical surface 61 Middle be arc side on one side, but is still referred to as triangle for brevity), wherein triangle corresponding with △ ABC is △ A ' B ' C '.

The frame data of video content are converted into rendering texture, based on the Along ent mapping relations that step 532 obtains, can be incited somebody to action The data for being originally intended to rendering △ ABC are mapped in △ A ' B ' C ' and are rendered, and so on can obtain conforming to cylindrical surface 61 Curved surface video effect.

As shown in Figure 6, it is assumed that the texture coordinate on 62 4 vertex of rectangle is respectively (0,0), (0,1), (1,0) and (1,1)； Corresponding, taking the texture coordinate on vertex in top surface arc side 611 is (∑ⁿI* θ/360,0), take the vertex line in bottom surface arc side 612 Reason coordinate is (∑ⁿI* θ/360, h), and then can be obtained between a arc side Along ent of 2* (n+1) and a straight flange Along ent of 2* (n+1) Mapping relations.It should be noted that 360 the case where being corresponding complete cylindrical surface here, be partial cylindrical surface on cylindrical surface When, 360 in texture coordinate accordingly may be modified such that top surface arc side 611 and the corresponding central angle size of bottom surface arc side 612.

Fig. 8 shows the AR effect picture using the present embodiment method.As shown in figure 8, terminal of the user using integrated AR function The image 81 for collecting certain cup-shaped product determines that the corresponding product of present image 81 is deposited after the characteristic pattern 82 for identifying cup body Can AR show material；In turn, it is video content that terminal, which gets displaying material corresponding with the product, by the place of Fig. 5 Process is managed, the cylindrical surface that cup body can be finally fitted in image 81 shows the effect of curved surface video 83.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge Not Chu the corresponding position fitting of curved surface show preset video content, so as to enhance the application range and processing energy of AR equipment Power, while improving the immersion experience of user.

Fig. 9 shows another example flow diagram of step 430 in embodiment illustrated in fig. 4.In this example, step 430 base In the curved surface obtained from curved surface recognition result, it is spaced apart and surround on the cylindrical surface of the curved surface with the curved surface on acquisition image, Carry out the Overlapping display of three-dimensional scene models.As shown in figure 9, for example may include step 931-934.

In step 931, the position of occupy-place model is determined according to acquisition image.

Patch effect as similar Fig. 3, this reality can only be embodied by being different from the three-dimensional scenic that AR is shown in the related technology Applying example may be implemented in around subject around display three-dimensional scene models.

In one embodiment, can be stored in advance can after curved surface identifies successfully for material for three-dimensional scene models here Pre-stored three-dimensional scene models are obtained, and are shown on the cylindrical surface in acquisition image by rendering.The position on cylindrical surface It can then be determined according to position of the subject in acquisition image.

In order to embody the hiding relation with subject in rendering three-dimensional scenes model, may also include in material and quilt The corresponding occupy-place model of subject.For example, it is special that product to be promoted can be pre-configured with when the present embodiment is applied to business promotion The three-dimensional scene models that sign figure and correspondence can be shown, while can also store occupy-place mould corresponding with product true form parameter Type；In this way, terminal device is successfully matched to the characteristic pattern of product by curved surface identification, while can be accounted for according to acquisition image determination The position of bit model, rendering strategy by subsequent sequence can embody and practical consistent hiding relation.

In step 932, the position of the second curved surface is determined according to pre-set depth occlusion relationship.

In order to embody the circular display effect of three-dimensional scene models, the second curved surface where the three-dimensional scene models with adopt First surface corresponding with characteristic pattern can be spaced apart in collection image.The distance can be according to pre-set three-dimensional scenic mould Depth occlusion relationship between type and occupy-place model determines.

Figure 10 shows the depth occlusion exemplary relationship between three-dimensional scene models and occupy-place model.As shown in Figure 10, occupy-place The size and location of model 1010 is corresponding with size and location of the subject in acquisition image, three-dimensional scene models 1020 Position is corresponding with position of second curved surface in acquisition image.

In step 933, occupy-place model is rendered based on transparent texture on acquisition image, to appear object where first surface Body.

After getting material corresponding with subject, occupy-place model and are determined based on step 931-932 respectively Position of two curved surfaces in acquisition image, step 933 can start the rendering of occupy-place model and three-dimensional scene models.In order to acquire Subject itself is shown in image, and above-mentioned occupy-place model can be rendered based on transparent texture.

In step 934, in acquisition image on the second determining curved surface, based on default texture rendering three-dimensional scenes mould Type, and make in the second curved surface compared with occupy-place model in the big region of picture depth, it is preferential with the rendering of occupy-place model.

The transparent rendering of occupy-place model is connected, step 934 is on the second curved surface based on default texture come rendering three-dimensional scenes Model.Since occupy-place model is rendered with transparent texture, shown in Fig. 10 pass is blocked to embody in finally acquisition image System can make in the second curved surface in the rendering of step 934 compared with occupy-place model in the big region of picture depth, with occupy-place mould The rendering of type is preferential.In other words, in the second curved surface situation more farther than occupy-place modal distance camera (or human eye), preferential wash with watercolours The transparent texture for contaminating occupy-place model, being equivalent in the region blocked in acquisition image by subject in this way not will do it three-dimensional The rendering of model of place.

In one embodiment, it for example can be translucent texture for the default texture of rendering three-dimensional scenes model, also Gradient color can be set.In one embodiment, around occupy-place model settable multiple and different sizes the second curved surface, lead to It crosses and renders the different stair-stepping effects of texture realization, further improve the immersion experience of user.

Figure 11 shows the AR effect picture using the present embodiment method.As shown in figure 11, user uses the end for integrating AR function The image 111 that end collects certain cup-shaped product determines the corresponding production of present image 111 after the characteristic pattern 112 for identifying cup body Product exist can AR show material；In turn, terminal get displaying material corresponding with the product include three-dimensional scene models and Placeholder model, by the process flow of Fig. 9, multiple cylindrical surfaces that cup body can be finally surround in image 111 show three-dimensional Around the effect of water curtain.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image with knowledge The preset three-dimensional scene models of display Chu not be surround around the corresponding position of curved surface, so as to enhance the application range of AR equipment And processing capacity, while improving the immersion experience of user.

Figure 12 is a kind of flow chart of the display methods based on AR shown according to another exemplary embodiment.Such as Figure 12 institute Show, this method can be executed by integrating the terminal device of AR function, it may include following steps 1210-1270.

In step 1210, the image of terminal acquisition subject, and be uploaded to server and carry out curved surface identification.

In one embodiment, the picture that terminal can acquire camera selects, such as enters static shape in terminal After state, comparison front and back number frame simultaneously selects an optimal frame to be uploaded to server progress curved surface identification, and the curved surface of waiting for server Recognition result.

In step 1220, terminal is locally carrying out curved surface identification, and subsequent step 1230 is carried out if identifying successfully, no The then curved surface recognition result of waiting for server.

In one embodiment, terminal can integrate curved surface identification function, at the same be locally stored for acquisition image The feature picture library being compared and the material database for carrying out AR displaying.For example, the song that terminal can transmit previous server Face recognition result and corresponding material are all buffered in local, in this way, once subsequent same subject occur, terminal is being identified The result being locally stored can be called directly after success.

On the other hand, if the curved surface recognition failures of terminal local, it may wait for the curved surface recognition result of server feedback, Subsequent step 1230 is carried out if server feedback identifies successfully, the image that otherwise return step 1210 carries out next time is adopted Collection.

In step 1230, terminal obtains the corresponding material of feature object and configuration information.

Based on the curved surface recognition result that local or server are transmitted, terminal can determine feature pair corresponding with acquisition image As.Here, by taking the present embodiment is applied to business promotion as an example, feature object for example can be specific products to be promoted, above-mentioned Characteristic pattern for example can be the specific pattern in the product packaging, by being pre-configured between feature object, characteristic pattern and material Corresponding relationship, terminal can obtain corresponding material when identifying characteristic pattern.

In one embodiment, terminal can also obtain corresponding configuration information while obtaining material.Here material It can include but is not limited to any combination of video content, three-dimensional scene models and webpage view.Correspondingly, configuration here Information may include the display order of material, jump relationship etc., such as show three automatically after the broadcasting for completing video content Model of place is tieed up, or the interactive operation based on user jumps to webpage view after the display for completing three-dimensional scene models, etc. Deng.

Here material and configuration information both can store on the server for executing curved surface identification, can also be stored in advance Or terminal device local is cached in after downloading, it can be with separate storage in configuration end equipment.It is applied to business with the present embodiment to push away Widely example for businessman can provide access interface in configuration end equipment, by businessman's upload characteristic pattern corresponding with product to be promoted, Material and configuration information etc.；Server, can be from the case where identifying successfully after the curved surface identification request for receiving terminal It configures end equipment and obtains corresponding material and configuration information, and return to terminal together with or as curved surface recognition result.

In step 1240, terminal carries out the display of material according to configuration information in acquisition image.

Different type based on material, terminal may be selected march plane video according to configuration information and play 1250, three dimensional field Scape model display 1260 and webpage view show any combination of 1270 equal multiple display modes.

In one embodiment, webpage view here is set as background transparent, only shows the behaviour for interacting with user Make component (such as button), in response to user operation can trigger jump to the subsequent display page.

In one embodiment, can curved surface video playing 1250 and three-dimensional scene models show 1260 the two at least its One of after the completion of, the webpage view for automatic jumping to background transparent shows 1270, shows the operational group for interacting with user Part.In another embodiment, webpage view can be also jumped directly to after step 1230 and show 1270, and then in webpage view It is middle complete curved surface video for example shown in fig. 5 show and, for example, three-dimensional scene models shown in Fig. 9 render at least within One of, while operating assembly of the display for being interacted with user in webpage view.

In one embodiment, also settable aobvious after step 1240 when method of the invention being applied to business promotion Show the step (not shown) of the business promotion page.It in other words, can be in curved surface according to the difference of material and configuration information After the completion of video playing 1250 or three-dimensional scene models show 1260, the display of the business promotion page is automatic jumped to, it can also be with The operation in 1270 is shown in webpage view in response to user, jumps to the display of the business promotion page.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, utilizes service in curved surface identification The recognition result of device can not only improve the accuracy rate of identification, and can save the calculation processing resource of terminal；Another party Face, the present embodiment carry out the flexible displaying of AR material according to configuration information, can enhance AR equipment after curved surface identifies successfully Application range and processing capacity, while improving the immersion experience of user.

Figure 13 shows an example flow diagram of step 1240 in embodiment illustrated in fig. 12.As shown in figure 13, step 1240 Such as it is implementable be include step 1241-1249.

In step 1241, whether terminal judges in material to include otherwise webpage view is tied if then carrying out step 1242 Line journey.

The display of webpage view will generally load resource in advance, if until preparing from video content or three-dimensional scene models Display when jumping to webpage view, just start the resource from server load webpage view, will cause system processing time with And the unnecessary waste of period of reservation of number.In one embodiment, terminal preferentially obtains in previous step 1230 matches confidence Breath, and whether determined in material to be downloaded according to configuration information includes webpage view.If including carrying out the present embodiment Subsequent step loads the resource of webpage view using downloading or showing the time of other materials；If not including, terminate this The process of embodiment, then according to the content of material, accordingly according to the display of process march plane video for example shown in fig. 5, Or according to process for example shown in Fig. 9, carry out the rendering of three-dimensional scene models.

In step 1242, whether terminal judges in material to include video content or threedimensional model scene, if then carrying out Step 1243, otherwise start the preloading of webpage view resource and carry out step 1246.

In one embodiment, according to the configuration information preferentially obtained in step 1230, terminal may further determine that be downloaded It whether include video content or threedimensional model scene in material.If including the subsequent step of the present embodiment being carried out, under utilization The time of other materials is carried or shown to load the resource of webpage view；If not including, webpage view resource can be started It preloads with the rendering for webpage view.

In one embodiment, terminal can show instruction user when starting the preloading of webpage view resource in this step Picture of waiting, such as display download progress item etc..

In step 1243, terminal judges whether above-mentioned video content or threedimensional model scene come from local, if then into Otherwise row step 1244 carries out step 1245.

In step 1244, terminal carries out the display of above-mentioned video content or threedimensional model scene, and starts webpage view The preloading of resource.

In step 1245, terminal starts the preloading of webpage view resource, and carries out above-mentioned video content or three-dimensional mould The downloading and display of type scene.

When subsequent steps 1242 are determined in material including video content or threedimensional model scene, step 1243 continues to judge this Whether a little materials are stored in terminal local.If the judging result of step 1243 be it is yes, indicate that the aobvious of local material can be started Show, while can use and show that the period of local material (corresponds to step to carry out the preloading of webpage view resource 1244).If the judging result of step 1243 be it is no, indicate that the downloading of local material can be started, while carrying out webpage view The preloading of resource (corresponds to step 1245).

In one embodiment, webpage view can only provide subscriber interaction component (such as button), at this moment in step 1245 The downloading that other materials can first be carried out just carries out the preloading of webpage view resource when starting the display of other materials, thus The process resource of terminal can be made full use of, while the subsequent time for opening webpage view can be greatly shortened.

In another embodiment, other than subscriber interaction component is provided, webpage view can also be used in video content or The display of threedimensional model scene.For example, passing through the WebView (webpage for being superimposed a background transparent on the acquisition image of terminal View), the texture of curved surface video is completed in WebView using webGL (Web Graphics Library, network graphic library) The rendering of rendering and three-dimensional scene models, while subscriber interaction component being provided.At this moment webpage view can be carried out in step 1245 simultaneously The preloading of image resource and the downloading of other materials can start the webpage view for rendering background transparent after being fully completed, The displaying of curved surface video and three-dimensional scene models is provided based on webpage view simultaneously.

In step 1246, judge whether the preloading of webpage view resource is completed, if then carrying out step 1248, otherwise Carry out step 1247.

In one embodiment, whether terminal can be issued based on web component has been completed the readjustment of resource load and has determined Whether the preloading of webpage view resource is completed.If completing to preload, the rendering of webpage view can be started, and based on pre- First it is arranged in webpage view and subscriber interaction component is provided, or march plane video while subscriber interaction component is provided The rendering of texture rendering and three-dimensional scene models.

In step 1247, start the preloading for retrying webpage view resource, and triggering timing device, if timing be less than it is pre- If threshold value then continues the judgement of step 1246, otherwise terminate process.

If step 1246 judges the preloading failure of webpage view resource, the timer for triggering step 1247 retries machine System can determine whether to retry failure, to terminate the stream of the present embodiment when retrying the time more than preset threshold (such as 10 seconds) Journey.

In step 1248, start the webpage view for rendering background transparent.

Described in subsequent steps 1242, in material only include webpage view when, step 1248 can have been preloaded in resource At the rear rendering for directly starting webpage view.

It, can be in the local element of display when other materials except webpage view are from local described in subsequent steps 1244 The preloading of webpage view resource is carried out while material, is completed if step 1246 judges that resource preloads, step 1248 can With after the display that step 1244 completes other materials, in response to user's operation or the rendering of webpage view is automatically begun to.

Described in subsequent steps 1245, in one embodiment, the downloading of other materials except webpage view can be first carried out, The preloading that webpage view resource is just carried out when starting the display of other materials, if step 1246 judges that resource has preloaded At then step 1248 in response to user's operation or can automatically begin to webpage after the display that step 1245 completes other materials The rendering of view.In another embodiment, can be carried out simultaneously in step 1245 webpage view resource preloading and other The downloading of material can jump to step 1248 after being fully completed and start to render the webpage view of background transparent, be based on simultaneously Webpage view provides the displaying of curved surface video and three-dimensional scene models.

In step 1249, judge whether the interactive operation for receiving user, if then carrying out according to operation content subsequent Processing, otherwise terminates process.

As it was noted above, can provide the component for interacting with user in the webpage view of background transparent.This step The interactive operation that then may be in response to user in rapid carries out subsequent processing.In the example that the present embodiment is applied to business promotion, use After operating assembly (such as button) on the webpage clicking view of family, such as the official website homepage for promoting object, product page can be jumped to Face specifically publicizes the page.On the other hand, after the rendering that step 1248 completes webpage view is shown, if in certain time The interactive operation of user is not all received inside, or acquisition image has occurred that (such as user moves the position of camera for variation Set), then it can terminate the processing that process continues next acquisition image.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, plain using showing or downloading other The time of material carries out the preloading of webpage view resource simultaneously, avoids preparing display webpage after the display for completing other materials The preloading that resource is just carried out when view can be improved treatment effeciency, save the processing time, while the net to be loaded such as shorten user The time of page view, improve user experience；In addition, by the webpage view of the Overlapping display background transparent on acquisition image, For providing subscriber interaction component, or the rendering of also achievable curved surface video and three-dimensional scene models simultaneously, end can be saved The process resource at end, while allowing the presence of the imperceptible webpage of user, further improve user experience.

Following is the device of the invention embodiment, can be used for executing the above-mentioned display methods based on AR of the present invention and implements Example.For undisclosed details in apparatus of the present invention embodiment, display methods embodiment of the aforementioned present invention based on AR is please referred to.

Figure 14 is a kind of block diagram of display device based on AR shown according to an exemplary embodiment.As shown, this Display device of the embodiment based on AR includes curved surface identification module 1410, material obtaining module 1420 and Overlapping display module 1430。

Curved surface identification module 1410 is set as obtaining the curved surface recognition result to acquisition image, the determining and acquisition image Corresponding feature object.

Material obtaining module 1420 is set as obtaining pre-generated material according to the determining feature object.

Overlapping display module 1430 is set as based on the first surface obtained from the curved surface recognition result, in the acquisition On image in the form of the second curved surface material described in Overlapping display.

The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface The pre-generated material of corresponding position Overlapping display changes simultaneously so as to enhance the application range and processing capacity of AR equipment The immersion experience of kind user.

Connect, in one embodiment, curved surface identification module 1410 from the curved surface that curved surface recognition result obtains include cylinder Face, the position that Overlapping display module 1430 is further arranged to correspond on acquisition image the cylindrical surface carry out video content Application of a surface is shown.

For example, Overlapping display module 1430 may be configured as: the top surface arc side on the cylindrical surface is corresponding with bottom surface arc side etc. It is divided into n parts, a arc side Along ent of 2* (n+1) is obtained；2* (n+1) a arc side on the cylindrical surface is obtained in rectangular coordinate system Along ent to be mapping relations between the corresponding obtained a straight flange Along ent of 2* (n+1) of rectangle by the cylinder face expansion；It will The frame data of the video content are converted into rendering texture, are connected by the straight flange Along ent of the rectangle is successively adjacent by zigzag It connects and the rectangle is divided into 2n triangle, the minimum unit of the rendering texture corresponds to the 2n triangle wherein One of；According to the mapping relations between the arc side Along ent and the straight flange Along ent, by the rendering texture mapping to institute State cylindrical surface.

The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface Corresponding position fitting shows that preset video content changes simultaneously so as to enhance the application range and processing capacity of AR equipment The immersion experience of kind user.

It connects, in another embodiment, above-mentioned second curved surface is obtained with curved surface identification module 1410 from curved surface recognition result The first surface arrived is spaced apart and around institute's first surface, and Overlapping display module 1430 is further arranged in the acquisition image The upper position corresponding to second curved surface shows the three-dimensional scene models in the material.

For example, Overlapping display module 1430 may be configured as: determining the position of occupy-place model according to the acquisition image；According to Pre-set depth occlusion relationship determines the position of second curved surface；It is rendered on the acquisition image based on transparent texture The occupy-place model, to appear object where the first surface；And on the second curved surface determined in the acquisition image, The three-dimensional scene models are rendered based on default texture, and make in second curved surface image depth with the occupy-place model compared with It spends in big region, it is preferential with the rendering of the occupy-place model.

The displaying scheme based on AR provided according to embodiments of the present invention, by acquisition image in identify curved surface Around preset three-dimensional scene models are shown around corresponding position, so as to enhance the application range and processing energy of AR equipment Power, while improving the immersion experience of user.

Figure 15 is a kind of block diagram of the display device based on AR shown according to another exemplary embodiment.As shown, On the basis of embodiment illustrated in fig. 14, display device of the present embodiment based on AR further includes image capture module 1440 and image Uploading module 1450；Overlapping display module 1430 further comprise video playback unit 1431, scene display unit 1432 and Webpage rendering unit 1433.

Image capture module 1440 is set as the image of acquisition subject, and is respectively sent to curved surface identification module 1410 With image uploading module 1450.

Image uploading module 1450 is set as the image that image capture module 1440 acquires being uploaded to server march Face identification.

Material obtaining module 1420 is notified to carry out material and configuration information if curved surface identification module 1410 identifies successfully Acquisition, waited if recognition failures image uploading module 1450 receive server recognition result.If subsequent according to figure As the received recognition result of uploading module 1450, confirmation server curved surface is identified successfully, then notify material obtaining module 1420 into Otherwise the acquisition of row material and configuration information notifies image capture module 1440 to continue to acquire and transmit image.

In one embodiment, material obtaining module 1420 can also obtain corresponding with confidence while obtaining material Breath.Here material can include but is not limited to any combination of video content, three-dimensional scene models and webpage view.Accordingly , configuration information here may include the display order of material, jump relationship etc., such as in the broadcasting for completing video content Show three-dimensional scene models automatically afterwards, or the interactive operation based on user jumps to after the display for completing three-dimensional scene models Webpage view, etc..

Here material and configuration information both can store on the server for executing curved surface identification, can also be stored in advance Or terminal device local is cached in after downloading, it can be with separate storage in configuration end equipment.It is applied to business with the present embodiment to push away Widely example for businessman can provide access interface in configuration end equipment, by businessman's upload characteristic pattern corresponding with product to be promoted, Material and configuration information etc.；Server, can be from the case where identifying successfully after the curved surface identification request for receiving terminal It configures end equipment and obtains corresponding material and configuration information, and return to terminal together with or as curved surface recognition result.

Overlapping display module 1430 notifies video playback unit according to the configuration information that material obtaining module 1420 obtains 1431, scene display unit 1432 and webpage rendering unit 1433 distinguish the broadcasting of march plane video, three-dimensional scene models exhibition Show and any combination for the multiple display modes such as webpage view is shown.

In one embodiment, webpage view here is set as background transparent, and webpage rendering unit 1433 is only set as Operating assembly (such as button) of the display for interact with user in webpage view, the operation in response to user triggerable jump To the subsequent display page.

In one embodiment, can video playback unit 1431 and the two of scene display unit 1432 at least within it After one completes corresponding material displaying, the webpage view that notice webpage rendering unit 1433 jumps to background transparent shows that display is used In the operating assembly interacted with user.In another embodiment, also material can be got in material obtaining module 1420 and match After confidence breath, notice webpage rendering unit 1433 jumps to the display of webpage view, and then the complete curved surface video in webpage view At least one in display and three-dimensional scene models rendering, while showing in webpage view for interacting with user Operating assembly.

In one embodiment, when method of the invention being applied to business promotion, video playback unit 1431, scene exhibition Show unit 1432 and webpage rendering unit 1433, three can also be respectively connected to the module of the display business promotion page (in figure It is not shown).It in other words, can be in video playback unit 1431, scene display unit according to the difference of material and configuration information 1432 and 1433 three of webpage rendering unit material show after the completion of, notify the module to jump to the aobvious of the business promotion page Show；Or the module can also jump to the display of the business promotion page in response to operation of the user in webpage view.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, utilizes service in curved surface identification The recognition result of device can not only improve the accuracy rate of identification, and can save the calculation processing resource of terminal；Another party Face, the present embodiment carry out the flexible displaying of AR material according to configuration information, can enhance AR equipment after curved surface identifies successfully Application range and processing capacity, while improving the immersion experience of user.

Figure 16 shows an example block diagram of Overlapping display module 1430 in embodiment illustrated in fig. 15.As shown, scheming On the basis of 15 illustrated embodiments, the Overlapping display module 1430 of the present embodiment further comprises material determination unit 1434, pre- Loading unit 1435 and operation monitoring unit 1436.

Material determination unit 1434 is set as judging the story types that material obtaining module 1420 obtains.

Pre-add carrier unit 1435 is arranged for the preloading of webpage view resource.

Operation monitoring unit 1436 is set as the interactive operation of monitoring user.

In one embodiment, determine to include webpage view in material in material determination unit 1434, and including from this When the video content or threedimensional model scene on ground, notify respectively video playback unit 1431 or scene display unit 1432 accordingly into The displaying of row video content or threedimensional model scene, and pre-add carrier unit 1435 is notified to carry out the preloading of webpage view resource.

In one embodiment, determine to include webpage view in material in material determination unit 1434, and including under needs When the video content or threedimensional model scene of load, notice pre-add carrier unit 1435 carries out the preloading of webpage view resource, and leads to Know that material obtaining module 1420 carries out the downloading of video content or threedimensional model scene.

In one embodiment, determine only include webpage view in material in material determination unit 1434, without including view Frequently when interior perhaps threedimensional model scene, notice pre-add carrier unit 1435 carries out the preloading of webpage view resource.

In one embodiment, pre-add carrier unit 1435 notifies net when the preloading for determining webpage view resource is completed Page rendering unit 1433 starts to render the webpage view of background transparent.

When only including webpage view in material, webpage rendering unit 1433 can directly be opened after the completion of resource preloads The rendering of beginning webpage view.

It, can be in video playback unit 1431 or scene display list when other materials except webpage view are from local While member 1432 shows local material, the preloading of webpage view resource is carried out by pre-add carrier unit 1435, if preloaded Unit 1435, which judges that resource preloads, to be completed, then can complete other in video playback unit 1431 or scene display unit 1432 After the display of material, in response to user's operation or the rendering of webpage view is automatically begun to by webpage rendering unit 1433.

In one embodiment, when other materials except webpage view need to download, material obtaining module 1420 can The downloading for first carrying out other materials, when starting the display of other materials, pre-add carrier unit 1435 just carries out webpage view resource Preloading.It is completed if pre-add carrier unit 1435 judges that resource preloads, it can be in video playback unit 1431 or scene After display unit 1432 completes the display of other materials, in response to user's operation or net is automatically begun to by webpage rendering unit 1433 The rendering of page view.In another embodiment, pre-add carrier unit 1435 and material obtaining module 1420 can carry out webpage simultaneously The preloading of view resource and the downloading of other materials can notify webpage rendering unit 1433 to start wash with watercolours after being fully completed The webpage view of background transparent is contaminated, while the displaying of curved surface video and three-dimensional scene models being provided based on webpage view.

The webpage view that operation monitoring unit 1436 shows webpage rendering unit 1433 is monitored, and judges whether to receive To the interactive operation of user, if then carrying out subsequent processing according to operation content.

For example, can provide the component for interacting with user in the webpage view of background transparent.Operation monitoring is single The interactive operation that member 1436 then may be in response to user carries out subsequent processing.In the example that the present embodiment is applied to business promotion, After operating assembly (such as button) on user's webpage clicking view, operation monitoring unit 1436 can notify webpage rendering unit 1433 jump to the official website homepage for promoting object, the product page or specifically publicize the page.On the other hand, in webpage rendering unit After the rendering of 1433 completion webpage views is shown, if operation monitoring unit 1436 does not all receive the friendship of user within a certain period of time Interoperability, or acquisition image have occurred that variation (such as user move camera position), then operate monitoring unit 1436 can notify image capture module 1440 to continue the processing of next acquisition image.

In conclusion the displaying scheme based on AR provided according to embodiments of the present invention, plain using showing or downloading other The time of material carries out the preloading of webpage view resource simultaneously, avoids preparing display webpage after the display for completing other materials The preloading that resource is just carried out when view can be improved treatment effeciency, save the processing time, while the net to be loaded such as shorten user The time of page view, improve user experience；In addition, by the webpage view of the Overlapping display background transparent on acquisition image, For providing subscriber interaction component, or the rendering of also achievable curved surface video and three-dimensional scene models simultaneously, end can be saved The process resource at end, while allowing the presence of the imperceptible webpage of user, further improve user experience.

About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method Embodiment in be described in detail, no detailed explanation will be given here.

It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description Member, but this division is not enforceable.In fact, according to embodiment of the present disclosure, it is above-described two or more Module or the feature and function of unit can embody in a module or unit.Conversely, an above-described mould The feature and function of block or unit can be to be embodied by multiple modules or unit with further division.As module or list The component of member display may or may not be physical unit, it can and it is in one place, or may be distributed over In multiple network units.Some or all of the modules therein can be selected to realize disclosure scheme according to the actual needs Purpose.

Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the present invention The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating Equipment (can be personal computer, server, touch control terminal or network equipment etc.) executes embodiment according to the present invention Method.

Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its Its embodiment.This application is intended to cover any variations, uses, or adaptations of the invention, these modifications, purposes or Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following Claim is pointed out.

It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is limited only by the attached claims.

Claims (14)

1. a kind of display methods based on augmented reality, which is characterized in that the described method includes:

The curved surface recognition result to acquisition image is obtained, determines feature object corresponding with the acquisition image；

Pre-generated material is obtained according to the determining feature object；And

Based on the first surface obtained from the curved surface recognition result, it is superimposed in the form of the second curved surface on the acquisition image Show the material.

2. the method as described in claim 1, which is characterized in that second curved surface is bonded with the first surface, described On the acquisition image in the form of the second curved surface material described in Overlapping display, comprising:

The position fitting for corresponding to the first surface on the acquisition image shows the material.

3. method according to claim 2, which is characterized in that the material includes video content, described in the acquisition Position fitting on image corresponding to the first surface shows the material, comprising:

Based on the form parameter of the first surface, when the first surface is expanded into plane by acquisition, the plane with it is described Mapping relations between first surface；

By the frame data of the video content be converted into rendering texture, according to the mapping relations by the rendering texture mapping extremely The first surface.

4. method according to claim 2, which is characterized in that the first surface includes cylindrical surface, and the material includes view Frequency content, the position fitting that the first surface is corresponded on the acquisition image show the material, comprising:

N parts are divided by the top surface arc side on the cylindrical surface is corresponding with bottom surface arc side, a arc side Along ent of 2* (n+1) is obtained；

A arc side Along ent of 2* (n+1) on the cylindrical surface is obtained in rectangular coordinate system and be rectangle by the cylinder face expansion The mapping relations between a straight flange Along ent of 2* (n+1) accordingly obtained；

The frame data of the video content are converted into rendering texture, press Z by the straight flange Along ent of the rectangle is successively adjacent The rectangle is divided into 2n triangle by font connection, and the minimum unit of the rendering texture corresponds to the 2n triangle One of；

According to the mapping relations between the arc side Along ent and the straight flange Along ent, by the rendering texture mapping to described Cylindrical surface.

5. the method as described in claim 1, which is characterized in that second curved surface is spaced apart and circular with the first surface The first surface, it is described on the acquisition image in the form of the second curved surface material described in Overlapping display, comprising:

The position for corresponding to second curved surface on the acquisition image shows the three-dimensional scene models in the material.

6. method as claimed in claim 5, which is characterized in that second curved surface includes cylindrical surface, and the material further includes Occupy-place model corresponding with object where the first surface, it is described to correspond to second curved surface on the acquisition image Position show the three-dimensional scene models in the material, comprising:

The position of the occupy-place model is determined according to the acquisition image；

The position of second curved surface is determined according to pre-set depth occlusion relationship；

The occupy-place model is rendered based on transparent texture on the acquisition image, to appear object where the first surface； And

On the second curved surface determined in the acquisition image, the three-dimensional scene models are rendered based on default texture, and make It is preferential with the rendering of the occupy-place model in second curved surface compared with the occupy-place model in the big region of picture depth.

7. as the method according to claim 1 to 6, which is characterized in that the method also includes:

Local curved surface identification is carried out to obtain the curved surface recognition result to the acquisition image；

In the local curved surface recognition failures, uploads the acquisition image and carry out long-range curved surface identification；And

From remotely obtaining the curved surface recognition result and the material.

8. as the method according to claim 1 to 6, which is characterized in that the method also includes:

Obtain configuration information corresponding with the feature object；And

The display of the material is carried out according to the configuration information,

Wherein, the material include in video content, three-dimensional scene models and webpage view at least any one, institute State the display order that configuration information includes at least the material.

9. method according to claim 8, which is characterized in that described to carry out the aobvious of the material according to the configuration information Show, comprising:

When determining feature object corresponding with the acquisition image, the webpage view is shown；Or

On the acquisition image in the form of the second curved surface video content described in Overlapping display or the three-dimensional scene models it Afterwards, the webpage view is shown.

10. method as claimed in claim 9, which is characterized in that

Before the display webpage view, the method also includes: when obtaining the material preload with it is described The corresponding resource of webpage view；

The display webpage view, comprising: the notice of load is completed in response to the resource, carries out the webpage view The rendering of figure.

11. as the method according to claim 1 to 6, which is characterized in that it is described on the acquisition image with second Material described in the form Overlapping display of curved surface, comprising:

Overlapping display background is transparent webpage view on the acquisition image；And

The material based on acquisition carried out in the webpage view video content and three-dimensional scene models at least one Rendering, with video content and three dimensional field described in position Overlapping display corresponding with second curved surface in the acquisition image At least one of scape model.

12. a kind of display device based on augmented reality, which is characterized in that described device includes:

Curved surface identification module is set as obtaining the curved surface recognition result to acquisition image, and determination is corresponding with the acquisition image Feature object；

Material obtaining module is set as obtaining pre-generated material according to the determining feature object；And

Overlapping display module is set as based on the first surface obtained from the curved surface recognition result, on the acquisition image The material described in Overlapping display in the form of the second curved surface.

13. a kind of computer readable storage medium, is stored thereon with computer program, the computer program is executed by processor The Shi Shixian display methods according to any one of claim 1 to 11 based on augmented reality.

14. a kind of electronic equipment characterized by comprising

Processor；And

Memory is stored with computer-readable instruction on the memory, and the computer-readable instruction is held by the processor The display methods according to any one of claim 1 to 11 based on augmented reality is realized when row.