CN109656363A

CN109656363A - It is a kind of for be arranged enhancing interaction content method and apparatus

Info

Publication number: CN109656363A
Application number: CN201811548792.0A
Authority: CN
Inventors: 施扬恩; 廖春元; 吴仑; 周俊颖; 韩天思; 侯晓辉
Original assignee: Bright Wind Taiwan (shanghai) Mdt Infotech Ltd
Current assignee: Bright Wind Taiwan (shanghai) Mdt Infotech Ltd
Priority date: 2018-09-04
Filing date: 2018-12-18
Publication date: 2019-04-19
Anticipated expiration: 2038-12-18
Also published as: CN109656363B

Abstract

The purpose of the application is to provide a kind of for the method and apparatus of enhancing interaction content to be arranged.This method includes determining the corresponding target signature information of the target image information based on the corresponding target image information of destination apparatus；Based on the target signature information, the location information of corresponding target area is determined；Determine enhancing interaction content corresponding to the target area.The enhancing interaction content for utilizing user equipment online editing specific region in actual scene, the intuitive effect that " What You See Is What You Get " must be presented are realized, and there is high efficiency.In some embodiments, specific region and corresponding enhancing interaction content are uploaded to the network equipment by user equipment, to support batch addition or the corresponding enhancing interaction content of more new commodity, without having extremely strong scalability to every a commodity repetitive operation.

Description

It is a kind of for be arranged enhancing interaction content method and apparatus

This application claims the excellent of CN 2018110266527 (a kind of for the method and apparatus of enhancing interaction content to be arranged) First weigh.

Technical field

This application involves computer field more particularly to it is a kind of for be arranged enhancing interaction content technology.

Background technique

With the development of the times, the speed that commodity alternate is getting faster.Under normal circumstances, when commodity generate or update Afterwards, the related description of user's commodity is provided for user to every a commodity customized development or use operating instruction workload Greatly, and it is at high cost, if it find that mistake is modified, need by processes such as " offline-online-offline-online ", it is cumbersome and bring Bigger workload.Sometimes the corresponding application program of more new commodity is even needed, so that user experience is poor.

Summary of the invention

The purpose of the application is to provide a kind of for the method and apparatus of enhancing interaction content to be arranged.

According to the one aspect of the application, provide it is a kind of for the method for enhancing interaction content to be arranged, this method comprises: The corresponding target signature information of the target image information is determined based on the corresponding target image information of destination apparatus；Based on described Target signature information determines the location information of corresponding target area；It determines in the interaction of enhancing corresponding to the target area Hold.

In some embodiments, the above method is executed by a user equipment.

According to further aspect of the application, a kind of side for being used to be arranged enhancing interaction content at network equipment end is provided Method, this method comprises: receiving the target signature information about destination apparatus that corresponding user equipment is sent；Match the target The corresponding default characteristic information of characteristic information, the default characteristic information are determined based on pre-set image information；To the use Family equipment sends the default characteristic information.

According to the one aspect of the application, provide a kind of for the equipment of enhancing interaction content to be arranged, wherein the equipment Include:

First module, for determining that the target image information is corresponding based on the corresponding target image information of destination apparatus Target signature information；

Second module determines the location information of corresponding target area for being based on the target signature information；

Third module, for determining enhancing interaction content corresponding to the target area.

According to further aspect of the application, provide a kind of for the network equipment of enhancing interaction content to be arranged, wherein The network equipment includes:

12nd module, the target signature information about destination apparatus sent for receiving corresponding user equipment；

13rd module, for matching the corresponding default characteristic information of the target signature information, the default feature letter Breath is determined based on pre-set image information；

14th module, for sending the default characteristic information to the user equipment.

According to further aspect of the application, provide a kind of for the method for enhancing interaction content, this method packet to be arranged It includes:

User equipment is based on the corresponding target image information of destination apparatus, determines corresponding target signature information, will be described Target signature information is sent to the corresponding network equipment；

The network equipment receives the target signature information about destination apparatus that corresponding user equipment is sent, and matches institute The corresponding default characteristic information of target signature information is stated, and sends the default characteristic information to the user equipment, wherein institute Stating default characteristic information is determined based on pre-set image information；

The user equipment is based on the target signature information and the default characteristic information, determines the user equipment phase For the posture information of the destination apparatus, the location information of corresponding target area is determined based on the target signature information, And determine enhancing interaction content corresponding to the target area.

User equipment determines corresponding target signature information based on the corresponding target image information of destination apparatus, based on described Target signature information matches the corresponding default characteristic information of the destination apparatus, based on the target signature information and described pre- If characteristic information determines posture information of the user equipment relative to the destination apparatus, and the target signature information is sent out It send to the corresponding network equipment, wherein the default characteristic information is obtained based on pre-set image information；

The network equipment determines corresponding enhancing interaction configuration information based on the target signature information, and to the use Family equipment sends the enhancing interaction configuration information；

The user equipment determines the location information of corresponding target area based on the enhancing configuration information, and is based on institute It states enhancing interaction configuration information and determines enhancing interaction content corresponding to the target area.

According to further aspect of the application, provide a kind of for the method for enhancing interaction content to be arranged, wherein the party Method includes:

User equipment determines the corresponding target of the target image information based on the corresponding target image information of destination apparatus Characteristic information is determined the location information of corresponding target area based on the target signature information, and determines the target area Corresponding enhancing interaction content；

The user equipment sends the enhancing interaction configuration information about the destination apparatus to the corresponding network equipment；

The network equipment is based on enhancing interaction configuration information and executes data update operation；

Wherein, the enhancing interaction configuration information includes following at least any one:

The location information of at least one target area；

Enhancing interaction content corresponding at least one target area.

Processor；And

It is arranged to the memory of storage computer executable instructions, the executable instruction makes the place when executed It manages device and executes following operation:

The corresponding target signature information of the target image information is determined based on the corresponding target image information of destination apparatus； Based on the target signature information, the location information of corresponding target area is determined；Determine increasing corresponding to the target area Strong interaction content.

According to further aspect of the application, provide a kind of for the network equipment of enhancing interaction content to be arranged, wherein The equipment includes:

Processor；And

Receive the target signature information about destination apparatus that corresponding user equipment is sent；Match the target signature letter Corresponding default characteristic information is ceased, the default characteristic information is determined based on pre-set image information；To the user equipment Send the default characteristic information.

According to further aspect of the application, a kind of computer-readable medium including instruction is provided, described instruction exists It is performed so that system performs the following operation:

Compared with prior art, the application obtains the image information of destination apparatus, base using the photographic device of user equipment The characteristic information of the destination apparatus is determined in image information, and then determines the position of specific region in image information, user equipment By presetting or the enhancing interaction content superposition of newly-increased, edit operation setting the correspondence specific region based on user is in It is existing.The enhancing interaction content that user equipment online editing specific region is utilized in actual scene is realized, it is very so intuitive that be in The effect of existing " What You See Is What You Get ", and there is high efficiency.In some embodiments, user equipment is by specific region and corresponding increasing Strong interaction content is uploaded to the network equipment, to support batch addition or the corresponding enhancing interaction content of more new commodity, without to every A commodity repetitive operation has extremely strong scalability.

Detailed description of the invention

By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other Feature, objects and advantages will become more apparent upon:

Fig. 1 shows the application scenarios according to some embodiments of the application；

Fig. 2 shows the method flow diagrams for being used to be arranged enhancing interaction content according to some embodiments of the application；

Fig. 3, which is shown, is used to be arranged the method stream of enhancing interaction content at network equipment end according to some embodiments of the application Cheng Tu；

Fig. 4 is shown according to some embodiments of the application for the method flow diagram of enhancing interaction content to be arranged；

Fig. 5 shows the method flow diagram for being used to be arranged enhancing interaction content according to other embodiments of the application；

Fig. 6 shows the functional module signal of the equipment for enhancing interaction content to be arranged according to some embodiments of the application Figure；

Fig. 7 shows the functional module of the network equipment for enhancing interaction content to be arranged according to some embodiments of the application Schematic diagram；

Fig. 8 shows the exemplary system that can be used for implementing each embodiment in the application.

The same or similar appended drawing reference represents the same or similar component in attached drawing.

Specific embodiment

The application is described in further detail with reference to the accompanying drawing.

In a typical configuration of this application, terminal, the equipment of service network and trusted party include one or more Processor (CPU), input/output interface, network interface and memory.

Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium Example.

Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), Digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or Any other non-transmission medium, can be used for storage can be accessed by a computing device information.

The application meaning equipment includes but is not limited to that user equipment, the network equipment or user equipment and the network equipment pass through Network is integrated constituted equipment.The user equipment includes but is not limited to that any one can carry out human-computer interaction with user The mobile electronic product, such as smart phone, tablet computer etc. of (such as human-computer interaction is carried out by touch tablet), the mobile electricity Sub- product can use any operating system, such as android operating system, iOS operating system.Wherein, the network equipment The electronic equipment of numerical value calculating and information processing can be carried out automatically according to the instruction for being previously set or storing including a kind of, Hardware includes but is not limited to microprocessor, specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate Array (FPGA), digital signal processor (DSP), embedded device etc..The network equipment includes but is not limited to computer, net The cloud that network host, single network server, multiple network server collection or multiple servers are constituted；Here, cloud is by based on cloud The a large number of computers or network servers for calculating (Cloud Computing) is constituted, wherein cloud computing is the one of distributed computing Kind, a virtual supercomputer consisting of a loosely coupled set of computers.The network includes but is not limited to interconnect Net, wide area network, Metropolitan Area Network (MAN), local area network, VPN network, wireless self-organization network (Ad Hoc network) etc..Preferably, the equipment Can also be run on the user equipment, the network equipment or user equipment and the network equipment, the network equipment, touch terminal or The network equipment and touch terminal are integrated the program in constituted equipment by network.

Certainly, those skilled in the art will be understood that above equipment is only for example, other are existing or are likely to occur from now on Equipment be such as applicable to the application, should also be included within the application protection scope, and be incorporated herein by reference.

In the description of the present application, the meaning of " plurality " is two or more, unless otherwise specifically defined.

The user equipment of the application meaning includes but is not limited to the meter such as smart phone, tablet computer, intelligent glasses or helmet Calculate equipment.In some embodiments, which further includes the photographic device for acquiring image information, the photographic device one As include photosensitive element for converting optical signals to electric signal, as needed also may include the biography for adjusting incident ray Broadcast the light folding/reflection component (such as camera lens or lens assembly) in path.To be operated convenient for user, in some embodiments In, the user equipment further includes display device, for presenting to user and/or for enhancing interaction content to be arranged, wherein In some embodiments, enhancing interaction content superposition is presented on destination apparatus, such as passes through augmented reality (Augmented Reality, AR) form present and provide human-computer interaction for user, and destination apparatus passes through user equipment (such as transmission-type eye Mirror or with display screen other users equipment) the enhancing interaction content is presented；Wherein, the display device is in some embodiments In be Touch Screen, which can be used not only for output pattern picture, also acts as the input unit of user equipment to connect Receive the operational order (such as the operational order interacted with aforementioned enhancing interaction content) of user.Certainly, those skilled in the art answer It can understand, the input unit of user equipment is not limited only to Touch Screen, other existing input technologies such as can be suitably used for the application, It is also contained in the protection scope of the application, and is incorporated herein by reference.For example, in some embodiments, for receiving The input technology of the operational order of user is realized based on voice control, gesture control and/or eyeball tracking.

Below based on a kind of user equipment with photographic device and Touch Screen, the application is retouched in detail It states.But the application is not limited to the user equipment, other user equipmenies that are existing or being likely to occur from now on such as can be suitably used for The application is also contained in the protection scope of the application, and is incorporated herein by reference.

Fig. 1 is shown according to application scenarios schematic diagram in some embodiments of the application.There is camera shooting dress in one kind as shown in Figure 1 The image information that a kind of electric appliance is presented in the user device screen with Touch Screen is set, it should based on the image information determination of the electric appliance The characteristic information of electric appliance determines the position of specific region in image information, and user equipment will be pre-set or based on user The enhancing interaction content of newly-increased, edit operation setting correspondence specific region, which is superimposed, to be presented.It realizes and is utilized in actual scene The effect of " What You See Is What You Get " is intuitively presented in the enhancing interaction content of user equipment online editing specific region, has efficient Property, corresponding enhancing interaction content can be added or be updated without changing the original client-side program of the electric appliance, can be made User, which obtains, to be supported new appliance or changes the corresponding enhancing interaction content of existing electric appliance (for example, providing saying for electric appliance for user The contents such as bright content or usage mode), it can also support multi-level enhancing interaction content.In some embodiments, user Specific region and corresponding enhancing interaction content are uploaded to the network equipment by equipment, to support batch addition or more new commodity corresponding Enhancing interaction content, it is not necessary that every a commodity repetitive operation, there is extremely strong scalability.

Fig. 2 shows a kind of sides for being used to be arranged enhancing interaction content at user equipment end according to the application one embodiment Method flow chart.The method comprising the steps of S101, step S102 and step S103.Wherein, in step s101, user equipment is based on The corresponding target image information of destination apparatus determines the corresponding target signature information of the target image information；In step S102 In, user equipment is based on the target signature information, determines the location information of corresponding target area；In step s 103, it uses Family equipment determines enhancing interaction content corresponding to the target area.

Here, destination apparatus includes the real-world object in actual scene, such as electric cooker, air-conditioning etc.；Destination apparatus includes But be not limited to household electrical appliance, electronic equipment, mechanical device etc..Target image include present on a user device about target fill The image information set, in some embodiments, target image information are acquired by the photographic device on user equipment, the wherein camera shooting Device is fixedly installed on user equipment, can be with external acquisition equipment.Target signature information is including but not limited to from target One or more target feature point extracted in the corresponding target image of device, and these target feature points reflect the target figure The feature of picture, such as being capable of destination apparatus in recognition target image based on these target feature point information.Target area includes Pre-set region corresponding with destination apparatus, or determining region is selected by user.For example, target area is user Pass through click or the region chosen of drawing a circle on the screen of user equipment.In some embodiments, the location information of target area For three-dimensional position of the target area under the world coordinate system where destination apparatus.Enhancing interaction content includes in user equipment Corresponding with target area label is presented on screen, image, video, links, 3D model or area etc. of leaving a message be discussed, for The interaction content of family editor's setting.

For example, user equipment obtains the target image of destination apparatus (for example, electric cooker of xx model) presentation on the screen Information, and the corresponding target signature information of the target image information, such as user equipment base are determined according to the target image information Feature extraction is carried out to obtain target signature information, wherein extracting target signature to the target image information in feature extraction algorithm The operation of information can locally be completed in user equipment, can also be completed on the corresponding network equipment (for example, cloud).User equipment base It is matched in pre-stored characteristics library in above-mentioned target signature information, match cognization goes out the destination apparatus, such as identifies target Xx model electric cooker in image, wherein feature database may be provided at user equipment, may also be arranged on the network equipment.In some implementations In example, target area and corresponding enhancing interaction content have been preset, and user equipment is according to the mesh extracted from target image Characteristic information is marked, calculates the spatial positional information and posture information for obtaining user equipment, and by the target area and corresponding Enhancing interaction content is presented on the corresponding position of the screen of user equipment.In further embodiments, the target area is by user Increased by way of clicking or drawing a circle on the screen of user equipment, or modifies its position by way of the others such as dragging It sets, for example, user equipment calculates the space for obtaining user equipment first according to the target signature information extracted from target image Location information and posture information；Then the operation according to user in user device screen determines the target area based on target Three-dimensional position under the world coordinate system of device.Then, user equipment determines the corresponding enhancing interaction content in the target area, Wherein, the corresponding enhancing interaction content in the target area is determined according to operations such as the newly-increased of user, editors.For example, user equipment It determines the corresponding enhancing interaction content video in target area, is the newly-increased operation by user, in the corresponding position in the target area Addition is completed on confidence breath.In some embodiments, after the pose of user equipment relative target device changes, user Three-dimensional position of the equipment according to target area under the world coordinate system based on destination apparatus obtains the target area and sets in user Position in standby screen, then the corresponding enhancing interaction content in the target area is presented in superposition.It realizes in actual scene Using the enhancing interaction content of user equipment online editing specific region, the intuitive effect that " What You See Is What You Get " must be presented has High efficiency.

In some embodiments, the enhancing interaction content is used to provide the relevant information of the destination apparatus for user, Such as the text or multimedia presentations of the destination apparatus, the explanation of all parts of the destination apparatus, the destination apparatus Operation instruction etc., and provide integration of user interaction functionality in some embodiments.Those skilled in the art will be understood that above-mentioned enhancing The form of interaction content is only for example, without constituting any restriction to the application；Other are existing or are likely to occur from now on The form of enhancing interaction content such as can be suitably used for the application, be also contained in the protection scope of the application, and with the side of reference Formula is incorporated herein.In addition, those skilled in the art also will be understood that the mode of above-mentioned determining enhancing interaction content is only for example, Other are existing or what is be likely to occur from now on determines that the mode of enhancing interaction content such as can be suitably used for the application, should also be included in this Within the protection scope of application, and it is incorporated herein by reference herein.

In some embodiments, in step s101, it is true to be based on the corresponding target image information of destination apparatus for user equipment Determine the corresponding target signature information of the target image information, and the user equipment phase is determined based on the target signature information For the posture information of the destination apparatus；The method also includes step S104 (not shown), in step S104, Yong Hushe Standby to be based on the posture information, the enhancing interaction content is presented in superposition.Here, posture information includes that user equipment is currently opposite In the spatial positional information and posture information of the destination apparatus.

For example, user equipment obtains the target image of destination apparatus (for example, electric cooker of xx model) presentation on the screen Information, and based on feature extraction algorithm to the target image information carry out feature extraction to obtain target signature information, further according to The target signature information calculates user equipment currently relative to the posture information of destination apparatus.In some embodiments, target Region is increased by way of clicking or drawing a circle on the screen of user equipment by user, based on the target area in user equipment The posture information of corresponding two-dimensional position information and user equipment relative to destination apparatus, determines the target area on screen Three-dimensional position under the world coordinate system based on destination apparatus.And then user increases newly in the corresponding enhancing interaction in the target area Hold, and is superimposed by user equipment and target area and corresponding enhancing interaction content is presented.

In some embodiments, the method also includes step S105 (not shown), user equipment base in step s105 In the target image information of update, the posture information is updated；In step s 102, user equipment is believed based on updated pose The enhancing interaction content is presented in breath, superposition.

For example, user equipment from different perspectives, position obtain destination apparatus and target image information on the screen be presented, In, in some embodiments, which both can correspond to the part of the destination apparatus, may correspond to the mesh Device for mark is whole.User equipment is based on updated target image information, using feature extraction algorithm to the target image information Feature extraction is carried out to obtain target signature information.Under the world coordinate system where destination apparatus, user equipment utilizes determination Target signature information, calculate and obtain spatial positional information and posture information relative to destination apparatus after user equipment updates, And then the position according to track algorithm real-time update target area in user device screen, it is superimposed on the screen of user equipment The corresponding enhancing interaction content in target area is presented.Wherein, track algorithm can be three-dimensional track algorithm, two-dimensional tracking algorithm or Other track algorithms.For three-dimensional track algorithm, SLAM (Simultaneous can be based on to the tracking of target image information Localization And Mapping is positioned immediately and is built figure) algorithm realization.There are many SLAM algorithms, such as SIFT (Scale-Invariant Feature Transform, Scale invariant features transform) SLAM algorithm, SURF (Speeded Up Robust Features accelerates robust features) SLAM algorithm, VINS (Visual-inertial System, vision inertial system System) SLAM algorithm, ORB (Oriented FAST and Rotated BRIEF) SLAM algorithm etc..

Those skilled in the art will be understood that the mode of above-mentioned update posture information is only for example, other are existing or modern The mode for the update posture information being likely to occur afterwards such as can be suitably used for the application, should also be included in the application protection scope it It is interior, and be incorporated herein by reference herein.

In some embodiments, in step s105, user equipment is tracked based on the target image information of update, with Update the posture information.

For example, user equipment transformation shooting angle, position obtain the target image information of destination apparatus presentation on the screen. User equipment is based on updated target image information, carries out feature extraction to the target image information using feature extraction algorithm To obtain target signature information, and under the world coordinate system based on destination apparatus, user equipment utilizes determining target signature Information calculates the spatial positional information and posture information obtained after user equipment updates relative to destination apparatus.For example, in user The information that equipment obtains recognizer is sent to track algorithm, initializes and determines the initial of enhancing interaction content Position of appearing and presentation mode after, user equipment still is based on above-mentioned track algorithm according to the target image information of update, with The variation of track target image information and the posture information of real-time update user equipment.

Wherein, above-mentioned track algorithm can be three-dimensional track algorithm, two-dimensional tracking algorithm or other track algorithms.With regard to three-dimensional For track algorithm, SLAM (Simultaneous Localization And can be based on to the tracking of target image information Mapping is positioned immediately and is built figure) algorithm realization.There are many SLAM algorithms, such as SIFT (Scale-Invariant Feature Transform, Scale invariant features transform) SLAM algorithm, SURF (Speeded Up Robust Features, Accelerate robust features) SLAM algorithm, VINS (Visual-inertial System, vision inertia system) SLAM algorithm, ORB (Oriented FAST and Rotated BRIEF) SLAM algorithm etc..

The tracking frame of existing SLAM algorithm substantially can be divided into following several parts:

1) camera or other sensor informations are obtained；

2) pose of observer is estimated by camera or other sensor informations；

3) optimize the information of pose, point map and the winding detection of different moments observer；

4) winding detection judges whether observer reached previous position；

5) map is established.

And two-dimensional tracking algorithm is tracked according only to two-dimensional image information without regard to three-dimensional information, input with Output and three-dimensional tracking are almost the same.For example, the two-dimensional tracking algorithm is that KCF nuclear phase closes filter tracking algorithm.It completes to mesh After the identification of device for mark, track algorithm constantly calculates the new pose of user equipment, to update destination apparatus in user equipment Screen in position.In one embodiment, the pose for updating user equipment based on two-dimensional tracking algorithm generally comprises following Several steps:

1) posture information of the user equipment obtained after selection identification destination apparatus, sends it to track algorithm, with Track algorithm picks target device characteristics point and corresponding matching template；

2) according to the posture information of previous frame, the point in 1) in characteristic point and Feature Points Matching template is calculated in present frame Position, find around these positions with 1) in the most matched point of characteristic point, obtain a matching double points；

3) according to the matching double points in 2), the posture information of the corresponding user equipment of present frame is calculated.

Certainly, those skilled in the art will be understood that above-described three-dimensional or two-dimensional tracking algorithm is only for example, not Any restriction is carried out to the specific embodiment of the application, other existing or track algorithms for being likely to occur from now on are as can be applicable in It in the application, is also contained in the protection scope of the application, and is incorporated herein by reference.

In some embodiments, in step s 103, user equipment is operated based on the content addition of user, increases the mesh newly Mark enhancing interaction content corresponding to region.

It is determined for example, the corresponding enhancing interaction content in target area is operated according to the content addition of user, user exists first Increase target area on the screen of user equipment by way of clicking or drawing a circle, user equipment is extracted according to from target image Target signature information obtain user equipment relative target device pose so that it is determined that the target area based on destination apparatus World coordinate system under three-dimensional position.Then, user equipment determines the corresponding enhancing interaction content in the target area, wherein The target area increase newly corresponding enhancing interaction content include but is not limited to label, image, video, link, 3D model or The interaction contents such as message area are discussed.In turn, after the pose of user equipment relative target device changes, the mesh is calculated Region corresponding position in user device screen is marked, then the corresponding enhancing interaction content in the target area is presented in superposition.

In some embodiments, in step s 103, user equipment is based on the location information, determines the target area Corresponding enhancing interaction content.

For example, in some embodiments, the corresponding enhancing interaction content in target area has been preset, user equipment is utilized Target image determines target signature information, calculates and obtains user equipment currently relative to the posture information of destination apparatus, and is shielding The corresponding enhancing interaction content in the target area is presented on curtain.Then, in the corresponding enhancing interaction in the user setting target area Hold, such as enhancing interaction content picture is revised as enhancing interaction content video.In turn, when user equipment scans the target again After device, the enhancing interaction content of target area superposition is modified enhancing interactive video.In further embodiments, user The position of target area is modified by way of dragging on the screen of user equipment, user equipment is mentioned according to from target image The target signature information taken determines three-dimensional position of the target area under the world coordinate system based on destination apparatus.In turn, when After the pose of user equipment relative target device changes, it is corresponding in user device screen that the target area is calculated Position, then the corresponding enhancing interaction content in the target area is presented in superposition.New target area can also further be modified Corresponding augmented reality content, method is same as described above, and which is not described herein again.Wherein in the corresponding enhancing in the target area Interaction content includes but is not limited to label, image, video, link, 3D model or the interaction contents such as message area is discussed.

In some embodiments, this method further includes step S106 (not shown).In step s 106, user equipment according to Operation is arranged in the contents attribute of user, and the attribute information of the enhancing interaction content is arranged.

For example, target area corresponds to a label, it is arranged according to the contents attribute of user and is operated, by the content of the label " beginning " is set as " beginning/closing ".In another example target area corresponds to a label, it is arranged according to the contents attribute of user and is grasped Make, changes size, the color of the label.For another example target area corresponds to a label, it is arranged according to the contents attribute of user and is grasped Make, the direction of the position or rotation change of the mobile label label.After user equipment scans the destination apparatus again, use The corresponding enhancing interaction content in the target area is presented in family equipment on the screen.In some embodiments, the contents attribute of user Other opposite enhancings interaction content (such as picture, video or 3D model) of setting operation are applicable in.

Those skilled in the art will be understood that the attribute information of above-mentioned enhancing interaction content is only for example, other are existing Or the attribute information for the enhancing interaction content being likely to occur from now on such as can be suitably used for the application, should also be included in the protection of the application Within the scope of, and be incorporated herein by reference herein.

In some embodiments, this method further includes step S107 (not shown), in step s 107, user equipment according to Operation is arranged in the triggering command of user, and triggering command corresponding to the enhancing interaction content is arranged.

For example, in some embodiments, user equipment is based on the pre-set corresponding enhancing of target area superposition presentation and hands over Mutual content, for example, user equipment is the enhancing interaction content label that pop-up is presented based on target area.In other embodiments In, user equipment is arranged according to the triggering command of user to be operated, the corresponding triggering command of setting enhancing interaction content.For example, setting It sets by clicking the target area and triggers the link of pop-up superposition；In another example setting is aobvious to trigger by double-clicking target area Diagram piece or broadcasting video etc..

In some embodiments, in step s 107, user equipment is arranged according to the triggering command of user and operates, and sets respectively Set it is multiple enhancing interaction contents respectively corresponding to triggering command.

For example, in some embodiments, user equipment is arranged according to the triggering command of user to be operated, multiple mesh are respectively set Mark the triggering command of the corresponding enhancing interaction content in region, such as the triggering of the corresponding enhancing interaction content video of setting target area a Instruction, setting are the enhancing interaction content video for triggering pop-up superposition by double-clicking target area a；B pairs of target area is set The triggering command of interaction content label should be enhanced, setting triggers the highlighted Overlapping display of the label by clicking target area b. In further embodiments, user equipment is arranged according to the triggering command of user and operates, and it is corresponding more that a target area is arranged The triggering command of a enhancing interaction content.For example, the triggering command of the corresponding enhancing interaction content picture of setting target area c, Setting controls enhancing interaction content picture amplification display by voice " opening ", and then is arranged and is by the picture for double-clicking amplification Relevant enhancing interactive video is triggered to present.The superposition in user device screen is realized to present in multi-level enhancing interaction Hold.In another example the triggering command of the corresponding enhancing interaction content 3D model of setting target area d, video, setting " are broadcast by voice Put " the enhancing interaction content video playing is controlled, setting is mobile by gesture control enhancing interaction content 3D model or revolves Turn.

In some embodiments, this method further includes step S108 (not shown), in step S108, user equipment according to The sequence of operation of user creates operation, creates the corresponding sequence of operation, and the sequence of operation is used in timing sequentially trigger more A enhancing interaction content.

For example, user creates the corresponding sequence of operation, Yong Hushe to multiple target areas and corresponding enhancing interaction content Standby target area a, target area b and the target area c determined based on user, and create the operation sequence of three target areas Column, such as the sequence of operation are successively to trigger in target area a, target area b and the corresponding enhancing interaction of target area c Hold and presents.Such as it is highlighted target area a, target area b and target area c successively.In another example triggering target area first Domain a corresponding label " beginning " highlights Overlapping display, then triggers the corresponding picture amplification display of target area b, last triggering target C corresponding video playing in region is presented.

Those skilled in the art will be understood that the mode of the above-mentioned creation sequence of operation is only for example, other are existing or modern The mode for the creation sequence of operation being likely to occur afterwards such as can be suitably used for the application, should also be included in the application protection scope it It is interior, and be incorporated herein by reference herein.

Step S101 includes that sub-step S101a, sub-step S101b and sub-step S101c (do not show in some embodiments Out).In sub-step S101a, user equipment is based on the corresponding target image information of destination apparatus, determines corresponding target signature Information；In sub-step S101b, user equipment is based on the target signature information, matches the corresponding pre- of the destination apparatus If characteristic information, wherein the default characteristic information is obtained based on pre-set image information；In sub-step S101c, user Equipment is based on the target signature information and the default characteristic information, determines the user equipment relative to the destination apparatus Posture information.Here, posture information includes the spatial positional information and posture information of the user equipment.Default characteristic information It is to be obtained based on pre-set image information, is stored in the feature database of user equipment local or the network equipment, wherein feature database Pre-set image comprising the destination apparatus that one or more is produced in advance, also may include the feature extracted from each image respectively Information (for example, characteristic point)；In some embodiments, default characteristic information (for example, characteristic point) can also based on pre-production, The threedimensional model of destination apparatus determines, such as default characteristic information is to extract three dimensional point cloud or geometry according to the threedimensional model What data determined.

For example, user equipment obtains the target image information of destination apparatus presentation on the screen, user equipment is based on feature Extraction algorithm carries out feature extraction to the target image information to obtain target signature information, wherein extracting target signature information Operation can locally be completed in user equipment, can also be completed on the corresponding network equipment (for example, cloud).User equipment is based on described Target signature information, matching obtains corresponding default special in being stored in user equipment local or network equipment end feature database Reference breath.User equipment be based on the target signature information and the default characteristic information, determine the user equipment relative to The posture information of the destination apparatus.Such as user equipment it is local or on the network equipment corresponding with user equipment (such as Cloud server) target image information and preset image or model etc. (but not limited to this) are matched and compared, i.e., It can get the user equipment currently relative to the spatial positional information and posture information of the destination apparatus, wherein described pre- If image or model correspond to identical (for example, same model) destination apparatus.The posture information of user equipment can It is carried out based on a specified coordinate system, such as under the world coordinate system where specified destination apparatus, determines the space of user equipment Location information and posture information.

In some embodiments, the target signature information includes the multiple targets determined according to the target image information Characteristic point；Wherein, the method also includes step S109 (not shown) and step S110 (not shown), in step S109, use Distributed intelligence of the family equipment based on the target feature point determines effective identification region of the target image information；In step In S110, user equipment is based on effective identification region, updates the pre-set image information.

For example, user equipment obtains the target image information about destination apparatus, based on feature extraction algorithm to the target Image information carries out feature extraction to obtain multiple target feature points, distributed intelligence of the user equipment based on target feature point, really Effective identification region of the fixed target image information.For example, presetting amount threshold, the quantity for counting target feature point is greater than When equal to preset amount threshold, it is determined that the target image information comprising these target feature points is effective identification region.Again Such as the distributed intelligence based on multiple target feature points, made a reservation for using K-Means scheduling algorithm when the quantity of target feature point meets Amount threshold when, it is determined that the target image information comprising these target feature points be effective identification region.It should be comprising effective The target image information of identification region more perfectly reacts the feature of destination apparatus, and user equipment is based on the target image information Effective identification region updates storage original pre-set image information in feature database.Wherein original pre-set image information with comprising The target image information of effective identification region matches, and one or more pre-set image informations are stored in feature database.In some realities It applies in example, user equipment is based on the corresponding target feature point information of target image, the posture information of user equipment can be calculated, thus Obtain the shooting angle of user equipment, the information such as target image picture accounting presented in user device screen.

In some embodiments, this method further includes step S111 (not shown), and in step S111, user equipment is based on The target image information updates the pre-set image information.

For example, there are the related pre-set image informations of at least one destination apparatus in feature database, wherein the pre-set image is believed Breath matches the target image of destination apparatus there is no enough characteristic points.In some embodiments, when user obtains target When the related target image information of device, user rule of thumb judges user equipment and destination apparatus relative position and angle more It is suitble to, visually observes that the target image picture accounting presented in user device screen is appropriate, i.e., the target image information is completeer The feature of the reflection destination apparatus of beauty.User equipment updates original pre-set image information using the target image information.Another In some embodiments, after user equipment photographic subjects image frame, target figure at this time is extracted using default feature extracting method The characteristic point of destination apparatus as in is then used when the characteristic point quantity of extraction is greater than or equal to preset characteristic point threshold value The family device prompts user target image information more perfectly reflects the feature of destination apparatus；And more using the target image information Original pre-set image information in new feature library.

It should be noted that the application does not limit variant method and step under the premise of will not cause contradictory Sequencing, and the same method and step may also be performed a plurality of times.Such as in some embodiments, in pre-set image information After being updated, user equipment matches the corresponding of the destination apparatus based on the target signature information for extracting from target image information Default characteristic information, wherein the default characteristic information is obtained based on updated pre-set image information；Then, Yong Hushe It is standby to be based on the target signature information and the default characteristic information, determine the user equipment relative to the destination apparatus Posture information.

In some embodiments, in sub-step S101b, the target signature information is sent to corresponding by user equipment The network equipment, and receive the default characteristic information that the network equipment returns.

For example, user equipment by destination apparatus correspond to target image information determine target signature information be sent to it is corresponding The network equipment.The network equipment receives the target signature information about destination apparatus that user equipment is sent, and is being stored in network It is matched in the feature database of equipment, obtains the default characteristic information to match with the target signature information, the default spy Reference breath is determined based on pre-set image information；The network equipment sends the default characteristic information to user equipment.And then user Equipment receives the default characteristic information returned.

In some embodiments, in step s 102, the target signature information is sent to corresponding net by user equipment Network equipment receives the enhancing interaction configuration information about the destination apparatus that the network equipment returns, and is based on the increasing Strong configuration information determines the location information of corresponding target area；In step s 103, user equipment is interacted based on the enhancing Configuration information determines enhancing interaction content corresponding to the target area.Herein, enhancing interaction configuration information includes at least one Enhancing interaction content corresponding to the location information of a target area or at least one target area.

For example, destination apparatus is corresponded to the target signature information that target image determines by user equipment is sent to the network equipment. The network equipment matches according to target signature information and obtains default characteristic information, and wherein default characteristic information is based on being stored in network and setting Standby pre-set image information obtains, and is based further on the target signature information and the default characteristic information, determines target spy Reference ceases corresponding enhancing interaction configuration information.User equipment receives the enhancing interaction about destination apparatus that the network equipment returns Configuration information, and the location information for corresponding to target area is determined based on enhancing interaction configuration information.And then user equipment is based on Enhance interaction configuration information, determines the corresponding enhancing interaction content in target area.Wherein, enhancing interaction configuration information includes extremely Enhancing interaction content corresponding to the location information of a few target area or at least one target area.

In some embodiments, this method further includes step S112 (not shown), and in step S112, user equipment is to right The network equipment answered sends the enhancing interaction configuration information about the destination apparatus；Wherein, the enhancing interaction configuration information Including following at least any one: the location information of at least one target area；Enhancing corresponding at least one target area is handed over Mutual content.

For example, target image information and corresponding enhancing interaction configuration information are sent to the network equipment by user equipment, In, it is described to enhance location information that interaction configuration information includes at least one target area (such as target area a is based on target Three dimensional local information under device world coordinate system) or at least one target area corresponding to enhancing interaction content (such as The contents such as the corresponding enhancing interaction picture of target area b).In some embodiments, realize multiple users to target area and Asynchronous, the cooperative operation of corresponding enhancing interaction content, are further sent by respective user equipment to the corresponding network equipment Enhancing interaction configuration information about the destination apparatus.

It should be noted that, although be described in the way of enhancing interaction content to setting by taking user equipment as an example herein , but enhancing interaction content is set and is not limited only to based on user equipment.In fact, in some embodiments, being interacted to enhancing The setting of content can also carry out on other equipment (for example, network server).

It should be noted that, the pre-set image information for extracting default characteristic information is not limited only to the mesh to entity simultaneously Obtained by device for mark is taken pictures.Such as in some embodiments, for the three-dimensional digital model of destination apparatus, user utilizes in the scene Script generates virtual camera, and camera is placed into the front position of identification region, and rendering generates corresponding digital picture and makees For pre-set image information.

In addition, the location information in order to determine corresponding target area, in some embodiments, user in editing machine three It ties up and is imported in scene after the corresponding two dimension target image information of destination apparatus, the positioning to the enhancing interaction content to be arranged is It is carried out based on the target image information in the three-dimensional scenic；Wherein, in some cases, the pixel in target image information In a plane, which is parallel to a coordinate plane of three-dimensional scenic.The operation such as dragging, rotation based on user, is used It is also accordingly determined in the target area of setting enhancing interaction content relative to the position of target image information in space.

Fig. 3 shows a kind of side for being used to be arranged enhancing interaction content at network equipment end according to the application one embodiment Method flow chart.The method comprising the steps of S201, step S202 and step S203, wherein in step s 201, the network equipment receives The target signature information about destination apparatus that corresponding user equipment is sent；In step S202, described in network equipment matching The corresponding default characteristic information of target signature information, wherein the default characteristic information is determined based on pre-set image information； In step S203, the network equipment sends the default characteristic information to the user equipment.

For example, the network equipment receives the target signature information about destination apparatus that user equipment is sent, it is based on the target Characteristic information matches the corresponding default characteristic information of the destination apparatus, such as matching obtains accordingly in network equipment feature database Default characteristic information, the default characteristic information based on pre-set image information determine.And then the network equipment is sent to user equipment The corresponding default characteristic information of the target signature information of the destination apparatus.In some embodiments, default characteristic information (example Such as, characteristic point) can based on pre-production, destination apparatus threedimensional model determine, such as default characteristic information be according to this three What dimension module extraction three dimensional point cloud or geometric data determined.

In some embodiments, wherein this method further includes step S204 (not shown), step S205 (not shown) and step Rapid S206 (not shown), in step S204, the network equipment receives the target about destination apparatus that the user equipment is sent Characteristic information；In step S205, the network equipment determines corresponding enhancing interaction configuration information based on the target signature information； In step S206, the network equipment sends the enhancing interaction configuration information to the user equipment.

For example, the network equipment receives the target signature information about destination apparatus that user equipment is sent, it is based on the mesh It marks characteristic information matching and determines corresponding enhancing interaction configuration information；In some embodiments, the network equipment is according to target signature Information matches obtain default characteristic information, and wherein default characteristic information is obtained based on the pre-set image information for being stored in the network equipment It takes, is based further on the target signature information and the default characteristic information, determine that the corresponding enhancing of target signature information is handed over Mutual configuration information.The network equipment sends corresponding enhancing interaction configuration information to the user equipment.

In some embodiments, this method further includes step S207 (not shown) and step S208 (not shown), in step In S207, the network equipment receives the enhancing interaction configuration information about destination apparatus that the user equipment is sent；In step In S208, the network equipment is based on enhancing interaction configuration information and executes data update operation.

For example, the network equipment, which receives the enhancing interaction about destination apparatus that one or more user equipmenies are sent, matches confidence Breath.In some embodiments, the network equipment, which is received and created, stores the destination apparatus and corresponding enhancing interaction configuration information；? In other embodiments, the network equipment receive and update storage user equipment transmission about destination apparatus and corresponding enhancing The destination apparatus of original storage and corresponding enhancing interaction configuration information update are newest received target by interaction configuration information Device and corresponding enhancing interaction configuration information.

According to the one aspect of the application, provide a kind of for the method for enhancing interaction content to be arranged.It, should with reference to Fig. 4 Method includes:

According to further aspect of the application, provide a kind of for the method for enhancing interaction content to be arranged.With reference to Fig. 5, This method comprises:

The user equipment determines the location information of corresponding target area based on the enhancing configuration information, and is based on institute It states enhancing interaction configuration information and determines enhancing interaction content corresponding to the target area.According to another side of the application Face provides a kind of for the method for enhancing interaction content to be arranged.This method comprises:

The location information of at least one target area；

Enhancing interaction content corresponding at least one target area.

Fig. 6 is shown to be shown according to a kind of user device architecture for enhancing interaction content to be arranged of the application one embodiment It is intended to.The equipment includes the first module 101, the second module 102 and third module 103.Wherein, the first module 101 is for being based on The corresponding target image information of destination apparatus determines the corresponding target signature information of the target image information；Second module 102 For being based on the target signature information, the location information of corresponding target area is determined；Described in third module 103 is used to determine Enhancing interaction content corresponding to target area.

For example, user equipment obtains the target image of destination apparatus (for example, electric cooker of xx model) presentation on the screen Information, and the corresponding target signature information of the target image information, such as user equipment base are determined according to the target image information Feature extraction is carried out to obtain target signature information, wherein extracting target signature to the target image information in feature extraction algorithm The operation of information can locally be completed in user equipment, can also be completed on the corresponding network equipment (for example, cloud).User equipment base It is matched in pre-stored characteristics library in above-mentioned target signature information, match cognization goes out the destination apparatus, such as identifies target Xx model electric cooker in image, wherein feature database may be provided at user equipment, may also be arranged on the network equipment.In some implementations In example, target area and corresponding enhancing interaction content have been preset, and user equipment is according to the mesh extracted from target image Characteristic information is marked, calculates the spatial positional information and posture information for obtaining user equipment, and by the target area and corresponding Enhancing interaction content is presented on the corresponding position of the screen of user equipment.In further embodiments, the target area is by user Increased by way of clicking or drawing a circle on the screen of user equipment, or modifies its position by way of the others such as dragging It sets, for example, user equipment calculates the space for obtaining user equipment first according to the target signature information extracted from target image Location information and posture information, then the operation according to user in user device screen determines the target area based on target Three-dimensional position under the world coordinate system of device.Then, user equipment determines the corresponding enhancing interaction content in the target area, Wherein, the corresponding enhancing interaction content in the target area is determined according to operations such as the newly-increased of user, editors, for example, user equipment It determines the corresponding enhancing interaction content video in target area, is the newly-increased operation by user, in the corresponding position in the target area Addition is completed on confidence breath.In some embodiments, after the pose of user equipment relative target device changes, user Three-dimensional position of the equipment according to target area under the world coordinate system based on destination apparatus obtains the target area and sets in user Position in standby screen, then the corresponding enhancing interaction content in the target area is presented in superposition.It realizes in actual scene Using the enhancing interaction content of user equipment online editing specific region, the intuitive effect that " What You See Is What You Get " must be presented has High efficiency.

In some embodiments, described in the first module 101 is used to determine based on the corresponding target image information of destination apparatus The corresponding target signature information of target image information, and determine the user equipment relative to institute based on the target signature information State the posture information of destination apparatus；The equipment further includes 104 (not shown) of the 4th module, and the 4th module 104 is used to be based on institute Posture information is stated, the enhancing interaction content is presented in superposition.Here, posture information includes user equipment currently relative to the mesh The spatial position of device for mark and posture.

In some embodiments, the equipment further includes 105 (not shown) of the 5th module, and the 5th module 105 is for being based on The target image information of update updates the posture information；Second module 102 is used to be based on updated posture information, superposition The enhancing interaction content is presented.

In some embodiments, the 5th module 105 is tracked for the target image information based on update, to update State posture information.

1) camera or other sensor informations are obtained；

2) pose of observer is estimated by camera or other sensor informations；

4) winding detection judges whether observer reached previous position；

5) map is established.

In some embodiments, third module 103 increases the target area newly for the content addition operation based on user Corresponding enhancing interaction content.

In some embodiments, third module 103 is used to be based on the location information, determines corresponding to the target area Enhancing interaction content.

In some embodiments, which further includes 106 (not shown) of the 6th module, the 6th module 106 be used for according to Operation is arranged in the contents attribute at family, and the attribute information of the enhancing interaction content is arranged.

In some embodiments, equipment further includes 107 (not shown) of the 7th module, and the 7th module 107 is used for according to user Triggering command operation is set, triggering command corresponding to the enhancing interaction content is set.

In some embodiments, the 7th module 107 is used to be arranged according to the triggering command of user and operate, and is respectively set multiple Enhance interaction content respectively corresponding to triggering command.

In some embodiments, which further includes 108 (not shown) of the 8th module, the 8th module 108 be used for according to The sequence of operation at family creates operation, creates the corresponding sequence of operation, and the sequence of operation is multiple for sequentially triggering in timing Enhance interaction content.

In some embodiments, the first module 101 includes first unit 101a, second unit 101b and third unit 101c.First unit 101a is used to be based on the corresponding target image information of destination apparatus, determines corresponding target signature information；The Two unit 101b are used to be based on the target signature information, match the corresponding default characteristic information of the destination apparatus, wherein The default characteristic information is obtained based on pre-set image information；Third unit 101c is used to be based on the target signature information With the default characteristic information, posture information of the user equipment relative to the destination apparatus is determined.Here, posture information Spatial positional information and posture information including the user equipment.Default characteristic information is obtained based on pre-set image information , it is stored in the feature database of user equipment local or the network equipment, wherein feature database includes one or more productions in advance Destination apparatus pre-set image, also may include the characteristic information (for example, characteristic point) extracted from each image respectively；One In a little embodiments, default characteristic information (for example, characteristic point) can also be true based on pre-production, destination apparatus threedimensional model It is fixed, such as default characteristic information is determined according to threedimensional model extraction three dimensional point cloud or geometric data.

In some embodiments, the target signature information includes the multiple targets determined according to the target image information Characteristic point；Wherein, the equipment further includes 109 (not shown) of the 9th module and the tenth module 110, and the 9th module 109 is used for base In the distributed intelligence of the target feature point, effective identification region of the target image information is determined；Tenth module 110 is used for Based on effective identification region, the pre-set image information is updated.

In some embodiments, which further includes 111 (not shown) of the 11st module, and the 11st module 111 is used for base In the target image information, the pre-set image information is updated.

In some embodiments, second unit 101b is set for the target signature information to be sent to corresponding network It is standby, and receive the default characteristic information that the network equipment returns.

In some embodiments, the second module 102 is used to the target signature information being sent to the corresponding network equipment, Receive the interactive configuration information of the enhancing about the destination apparatus that the network equipment returns and based on the enhancing with confidence Breath determines the location information of corresponding target area；Third module 103 is used to determine institute based on enhancing interaction configuration information State enhancing interaction content corresponding to target area.Herein, enhance the position that interaction configuration information includes at least one target area Enhancing interaction content corresponding to confidence breath or at least one target area.

In some embodiments, which further includes 112 (not shown) of the 12nd module, the 12nd module 112 be used for The corresponding network equipment sends the enhancing interaction configuration information about the destination apparatus；Wherein, confidence is matched in the enhancing interaction Breath includes following at least any one: the location information of at least one target area；Enhancing corresponding at least one target area Interaction content.

Fig. 7 is shown to be shown according to a kind of network equipment infrastructure for enhancing interaction content to be arranged of the application one embodiment It is intended to.The equipment includes the 13rd module 201, the 14th module 202 and the 15th module 203, wherein the 13rd module 201 is used In the target signature information about destination apparatus for receiving corresponding user equipment transmission；14th module 202 is for matching institute The corresponding default characteristic information of target signature information is stated, wherein the default characteristic information is determined based on pre-set image information 's；15th module 205 is used to send the default characteristic information to the user equipment.

In some embodiments, wherein the equipment further includes 204 (not shown) of the 16th module, the 17th module 205 206 (not shown) of (not shown) and the 18th module, the 16th module 204 are used to receive that the user equipment to send about mesh The target signature information of device for mark；17th module 205 is used to determine that corresponding enhancing is interacted based on the target signature information Configuration information；18th module 206 is used to send the enhancing interaction configuration information to the user equipment.

In some embodiments, which further includes that 207 (not shown) of the 19th module and the 20th module 208 (are not shown Out).19th module 207 is used to receive the enhancing interaction configuration information about destination apparatus that the user equipment is sent；The 20 modules 208, which are used to execute data based on enhancing interaction configuration information, updates operation.

Present invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage has calculating Machine code, when the computer code is performed, such as preceding described in any item methods are performed.

Present invention also provides a kind of computer program products, when the computer program product is executed by computer equipment When, such as preceding described in any item methods are performed.

Present invention also provides a kind of computer equipment, the computer equipment includes:

One or more processors；

Memory, for storing one or more computer programs；

When one or more of computer programs are executed by one or more of processors so that it is one or Multiple processors realize such as preceding described in any item methods.

Fig. 8 shows the exemplary system that can be used for implementing each embodiment described herein.

As shown in figure 8, in some embodiments, system 800 can be as any one user in each embodiment Equipment or the network equipment.In some embodiments, system 800 may include computer-readable Jie of one or more with instruction Matter (for example, system storage or NVM/ store equipment 820) and simultaneously quilt is coupled with the one or more computer-readable medium Be configured to execute instruction with realize module thereby executing movement described herein one or more processors (for example, (one It is a or multiple) processor 805).

For one embodiment, system control module 810 may include any suitable interface controller, with to (one or It is multiple) at least one of processor 805 and/or any suitable equipment or component that communicate with system control module 810 mentions For any suitable interface.

System control module 810 may include Memory Controller module 830, to provide interface to system storage 815.It deposits Memory controller module 830 can be hardware module, software module and/or firmware module.

System storage 815 can be used for for example, load of system 800 and storing data and/or instruction.For a reality Example is applied, system storage 815 may include any suitable volatile memory, for example, DRAM appropriate.In some embodiments In, system storage 815 may include four Synchronous Dynamic Random Access Memory of Double Data Rate type (DDR4SDRAM).

For one embodiment, system control module 810 may include one or more input/output (I/O) controller, with Equipment 820 is stored to NVM/ and (one or more) communication interface 825 provides interface.

For example, NVM/ storage equipment 820 can be used for storing data and/or instruction.NVM/ storage equipment 820 may include appointing It anticipates nonvolatile memory appropriate (for example, flash memory) and/or to may include that any suitable (one or more) is non-volatile deposit Equipment is stored up (for example, one or more hard disk drives (HDD), one or more CD (CD) drivers and/or one or more Digital versatile disc (DVD) driver).

NVM/ storage equipment 820 may include a part for the equipment being physically mounted on as system 800 Storage resource or its can by the equipment access without a part as the equipment.For example, NVM/ storage equipment 820 can It is accessed by network via (one or more) communication interface 825.

(one or more) communication interface 825 can be provided for system 800 interface with by one or more networks and/or with Other any equipment communications appropriate.System 800 can be according to any mark in one or more wireless network standards and/or agreement Quasi- and/or agreement is carried out wireless communication with the one or more components of wireless network.

For one embodiment, at least one of (one or more) processor 805 can be with system control module 810 The logic of one or more controllers (for example, Memory Controller module 830) is packaged together.For one embodiment, (one It is a or multiple) at least one of processor 805 can encapsulate with the logic of one or more controllers of system control module 810 Together to form system in package (SiP).For one embodiment, at least one of (one or more) processor 805 It can be integrated on same mold with the logic of one or more controllers of system control module 810.For one embodiment, At least one of (one or more) processor 805 can be with the logic of one or more controllers of system control module 810 It is integrated on same mold to form system on chip (SoC).

In various embodiments, system 800 can be, but not limited to be: server, work station, desk-top calculating equipment or movement It calculates equipment (for example, lap-top computing devices, handheld computing device, tablet computer, net book etc.).In various embodiments, System 800 can have more or fewer components and/or different frameworks.For example, in some embodiments, system 800 includes One or more video cameras, keyboard, liquid crystal display (LCD) screen (including touch screen displays), nonvolatile memory port, Mutiple antennas, graphic chips, specific integrated circuit (ASIC) and loudspeaker.

It should be noted that the application can be carried out in the assembly of software and/or software and hardware, for example, can adopt With specific integrated circuit (ASIC), general purpose computer or any other realized similar to hardware device.In one embodiment In, the software program of the application can be executed to implement the above steps or functions by processor.Similarly, the application Software program (including relevant data structure) can be stored in computer readable recording medium, for example, RAM memory, Magnetic or optical driver or floppy disc and similar devices.In addition, hardware can be used to realize in some steps or function of the application, example Such as, as the circuit cooperated with processor thereby executing each step or function.

In addition, a part of the application can be applied to computer program product, such as computer program instructions, when its quilt When computer executes, by the operation of the computer, it can call or provide according to the present processes and/or technical solution. Those skilled in the art will be understood that the existence form of computer program instructions in computer-readable medium includes but is not limited to Source file, executable file, installation package file etc., correspondingly, the mode that computer program instructions are computer-executed include but Be not limited to: the computer directly execute the instruction or the computer compile the instruction after execute program after corresponding compiling again, Perhaps the computer reads and executes the instruction or after the computer reads and install and execute corresponding installation again after the instruction Program.Here, computer-readable medium can be for computer access any available computer readable storage medium or Communication media.

Communication media includes whereby including, for example, computer readable instructions, data structure, program module or other data Signal of communication is transmitted to the medium of another system from a system.Communication media may include having the transmission medium led (such as electric Cable and line (for example, optical fiber, coaxial etc.)) and can propagate wireless (not having the transmission the led) medium of energy wave, such as sound, electricity Magnetic, RF, microwave and infrared.Computer readable instructions, data structure, program module or other data can be embodied as example wireless Medium (such as carrier wave or be such as embodied as spread spectrum technique a part similar mechanism) in modulated message signal. Term " modulated message signal " refers to that one or more feature is modified or is set in a manner of encoded information in the signal Fixed signal.Modulation can be simulation, digital or Hybrid Modulation Technology.

As an example, not a limit, computer readable storage medium may include such as computer-readable finger for storage Enable, the volatile and non-volatile that any method or technique of the information of data structure, program module or other data is realized, can Mobile and immovable medium.For example, computer readable storage medium includes, but are not limited to volatile memory, such as with Machine memory (RAM, DRAM, SRAM)；And nonvolatile memory, such as flash memory, various read-only memory (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memory (MRAM, FeRAM)；And magnetic and optical storage apparatus (hard disk, Tape, CD, DVD)；Or other currently known media or Future Development can store the computer used for computer system Readable information/data.

Here, including a device according to one embodiment of the application, which includes for storing computer program The memory of instruction and processor for executing program instructions, wherein when the computer program instructions are executed by the processor When, trigger method and/or technology scheme of the device operation based on aforementioned multiple embodiments according to the application.

It is obvious to a person skilled in the art that the application is not limited to the details of above-mentioned exemplary embodiment, Er Qie In the case where without departing substantially from spirit herein or essential characteristic, the application can be realized in other specific forms.Therefore, no matter From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and scope of the present application is by appended power Benefit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent elements of the claims Variation is included in the application.Any reference signs in the claims should not be construed as limiting the involved claims.This Outside, it is clear that one word of " comprising " does not exclude other units or steps, and odd number is not excluded for plural number.That states in device claim is multiple Unit or device can also be implemented through software or hardware by a unit or device.The first, the second equal words are used to table Show title, and does not indicate any particular order.

Claims

1. a kind of for the method for enhancing interaction content to be arranged, wherein this method comprises:

The corresponding target signature information of the target image information is determined based on the corresponding target image information of destination apparatus；

Based on the target signature information, the location information of corresponding target area is determined；

Determine enhancing interaction content corresponding to the target area.

2. according to the method described in claim 1, wherein, it is described determined based on the corresponding target image information of destination apparatus described in The corresponding target signature information of target image information, comprising:

The corresponding target signature information of the target image information, and base are determined based on the corresponding target image information of destination apparatus Posture information of the user equipment relative to the destination apparatus is determined in the target signature information；

The method also includes:

Based on the posture information, the enhancing interaction content is presented in superposition.

3. according to the method described in claim 2, wherein, the method also includes:

Target image information based on update updates the posture information；

Described to be based on the posture information, the enhancing interaction content is presented in superposition, comprising:

Based on updated posture information, the enhancing interaction content is presented in superposition.

4. according to the method described in claim 3, wherein, the target image information based on update updates the pose letter Breath, comprising:

Target image information based on update is tracked, to update the posture information.

5. according to the method described in claim 1, wherein, enhancing interaction content corresponding to the determination target area, Include:

Content addition operation based on user, increases enhancing interaction content corresponding to the target area newly.

6. according to the method described in claim 1, wherein, enhancing interaction content corresponding to the determination target area, Include:

Based on the location information, enhancing interaction content corresponding to the target area is determined.

7. method according to claim 5 or 6, wherein the method also includes:

It is arranged according to the contents attribute of user and is operated, the attribute information of the enhancing interaction content is set.

8. method according to any one of claims 5 to 7, wherein the method also includes:

It is arranged according to the triggering command of user and is operated, triggering command corresponding to the enhancing interaction content is set.

9. the increasing is arranged according to the method described in claim 8, wherein, described be arranged according to the triggering command of user operates Triggering command corresponding to strong interaction content, comprising:

According to the triggering command of user be arranged operate, be respectively set it is multiple enhancing interaction contents respectively corresponding to triggering command.

10. method according to claim 8 or claim 9, wherein the method also includes:

Sequence creation operation depending on the user's operation creates the corresponding sequence of operation, and the sequence of operation is for suitable in timing The secondary multiple enhancing interaction contents of triggering.

11. described to determine institute based on the corresponding target image information of destination apparatus according to the method described in claim 1, wherein State the corresponding target signature information of target image information, comprising:

Based on the corresponding target image information of destination apparatus, corresponding target signature information is determined；

Based on the target signature information, the corresponding default characteristic information of the destination apparatus is matched, wherein the default spy Reference breath is obtained based on pre-set image information；

Based on the target signature information and the default characteristic information, determine the user equipment relative to the destination apparatus Posture information.

12. according to the method for claim 11, wherein the target signature information includes according to the target image information Determining multiple target feature points；

The method also includes:

Based on the distributed intelligence of the target feature point, effective identification region of the target image information is determined；

Based on effective identification region, the pre-set image information is updated.

13. according to the method for claim 11, wherein the method also includes:

Based on the target image information, the pre-set image information is updated.

14. according to the method for claim 11, wherein it is described to be based on the target signature information, match the target dress The corresponding default characteristic information set, comprising:

The target signature information is sent to the corresponding network equipment, and receives the default feature letter that the network equipment returns Breath.

15. according to the method for claim 11, wherein it is described to be based on the target signature information, determine corresponding target The location information in region, comprising:

The target signature information is sent to the corresponding network equipment, and receive that the network equipment returns about the mesh The enhancing interaction configuration information of device for mark；

The location information of corresponding target area is determined based on the enhancing configuration information；

Enhancing interaction content corresponding to the determination target area, comprising:

Based on enhancing interaction configuration information, enhancing interaction content corresponding to the target area is determined.

16. according to the method described in claim 1, wherein, the method also includes:

The enhancing interaction configuration information about the destination apparatus is sent to the corresponding network equipment；

The location information of at least one target area；

Enhancing interaction content corresponding at least one target area.

17. a kind of method for being used to be arranged enhancing interaction content at network equipment end, wherein this method comprises:

Receive the target signature information about destination apparatus that corresponding user equipment is sent；

The corresponding default characteristic information of the target signature information is matched, the default characteristic information is based on pre-set image information Determining；

The default characteristic information is sent to the user equipment.

18. according to the method for claim 17, wherein the method also includes:

Receive the target signature information about destination apparatus that the user equipment is sent；

Corresponding enhancing interaction configuration information is determined based on the target signature information；

The enhancing interaction configuration information is sent to the user equipment.

19. method described in 7 or 18 according to claim 1, wherein the method also includes:

Receive the enhancing interaction configuration information about destination apparatus that the user equipment is sent；

Data, which are executed, based on enhancing interaction configuration information updates operation.

20. a kind of be used to be arranged the equipment for enhancing interaction content at user equipment end, wherein the equipment includes:

First module, for determining the corresponding target of the target image information based on the corresponding target image information of destination apparatus Characteristic information；

21. equipment according to claim 20, wherein first module is used for

The equipment further include:

4th module, for being based on the posture information, the enhancing interaction content is presented in superposition.

22. equipment according to claim 21, wherein the equipment further include:

5th module updates the posture information for the target image information based on update；

4th module, is used for

23. equipment according to claim 21, wherein the 5th module is used for

24. equipment according to claim 20, wherein the third module is used for

25. equipment according to claim 20, wherein the third module is used for

26. the equipment according to claim 24 or 25, wherein the equipment further include:

6th module is operated for being arranged according to the contents attribute of user, and the attribute information of the enhancing interaction content is arranged.

27. equipment according to claim 20, wherein first module includes:

First unit determines corresponding target signature information for being based on the corresponding target image information of destination apparatus；

Second unit, for matching the corresponding default characteristic information of the destination apparatus based on the target signature information, Described in default characteristic information be to be obtained based on pre-set image information；

Third unit determines that the user equipment is opposite for being based on the target signature information and the default characteristic information In the posture information of the destination apparatus.

28. a kind of for the network equipment of enhancing interaction content to be arranged, wherein the equipment includes:

13rd module, the target signature information about destination apparatus sent for receiving corresponding user equipment；

14th module, for matching the corresponding default characteristic information of the target signature information, the default characteristic information is It is determined based on pre-set image information；

15th module, for sending the default characteristic information to the user equipment.

29. a kind of for the method for enhancing interaction content to be arranged, wherein this method comprises:

User equipment is based on the corresponding target image information of destination apparatus, corresponding target signature information is determined, by the target Characteristic information is sent to the corresponding network equipment；

The network equipment receives the target signature information about destination apparatus that corresponding user equipment is sent, and matches the mesh The corresponding default characteristic information of characteristic information is marked, and sends the default characteristic information to the user equipment, wherein described pre- If characteristic information is determined based on pre-set image information；

The user equipment be based on the target signature information and the default characteristic information, determine the user equipment relative to The posture information of the destination apparatus determines the location information of corresponding target area based on the target signature information, and really Enhancing interaction content corresponding to the fixed target area.

30. a kind of for the method for enhancing interaction content to be arranged, wherein this method comprises:

User equipment determines corresponding target signature information based on the corresponding target image information of destination apparatus, is based on the target Characteristic information matches the corresponding default characteristic information of the destination apparatus, is based on the target signature information and the default spy Reference breath determines posture information of the user equipment relative to the destination apparatus, and the target signature information is sent to The corresponding network equipment, wherein the default characteristic information is obtained based on pre-set image information；

The network equipment determines corresponding enhancing interaction configuration information based on the target signature information, and sets to the user Preparation send the enhancing interaction configuration information；

The user equipment determines the location information of corresponding target area based on the enhancing configuration information, and is based on the increasing Strong interaction configuration information determines enhancing interaction content corresponding to the target area.

31. a kind of for the method for enhancing interaction content to be arranged, wherein this method comprises:

User equipment determines the corresponding target signature of the target image information based on the corresponding target image information of destination apparatus Information determines the location information of corresponding target area based on the target signature information, and determines that the target area institute is right The enhancing interaction content answered；

The location information of at least one target area；

Enhancing interaction content corresponding at least one target area.

32. a kind of for the equipment of enhancing interaction content to be arranged, wherein the equipment includes:

Processor；And

It is arranged to the memory of storage computer executable instructions, the executable instruction makes the processor when executed It executes according to claim 1 to the operation of any one of 19 the methods.

33. a kind of computer-readable medium including instruction, described instruction wants system progress according to right Ask the operation of any one of 1 to 19 the method.