CN104181884A - Device and method for controlling intelligent home based on panoramic view - Google Patents
Device and method for controlling intelligent home based on panoramic view Download PDFInfo
- Publication number
- CN104181884A CN104181884A CN201410392711.8A CN201410392711A CN104181884A CN 104181884 A CN104181884 A CN 104181884A CN 201410392711 A CN201410392711 A CN 201410392711A CN 104181884 A CN104181884 A CN 104181884A
- Authority
- CN
- China
- Prior art keywords
- panorama sketch
- coordinate
- controlled plant
- panoramic view
- touch screen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a device and a method for controlling an intelligent home based on a panoramic view. The method comprises the steps of photographing and editing a panoramic view of the internal part of a residence room; generating a coordinate set of controlled equipment image areas in the panoramic view; establishing a control link between actually controlled equipment and the controlled equipment image areas in the panoramic view; loading the panoramic view to a touch screen so as to realize seamless splicing circulation display; clicking controlled equipment images in the panoramic view so as to realize visualized control for the actually controlled equipment; and setting suspension type shortcut operating buttons and a pull-down type slide-out operating window, thereby being capable of establishing a vivid and visual dynamic live-action operating interface quickly, simply and conveniently, extremely easy to use by users, and capable of greatly reducing the cost of engineering construction.
Description
Technical field
The present invention relates to a kind of intelligent home control device and method based on panoramic view user interface.
Background technology
Smart Home not only has traditional inhabitation function, and comfortable safety, high-quality and pleasant family life space are provided, and also changes the instrument with active wisdom into by original passive static structures, and omnibearing information exchange functions is provided.Help family to keep information interchange unimpeded with outside, optimize people's life style, help people effectively to arrange the time, strengthen the security of life staying idle at home, even economize on the use of funds for various energy expenditures.
Some are applied to the user interface of intelligent home control device at present, adopt one dimension interface display, on this operation interface, be provided with some buttons, and be furnished with respectively pattern or word, represent the controlled plant in the families such as air-conditioning, light fixture, as used " 1 " to represent air-conditioning, " 2 " represent light fixture etc., user set up when user interface can only be by interface on intrinsic button set.When setting up button and link with the control of air-conditioning as wish, make this air-conditioning enter login state, press interface simultaneously and be marked with the button of air-conditioning pattern or word, now outside broadcast air-conditioning code automatically, air-conditioning in login state in residence can receive this code, thereby forms corresponding relation.But, because current Smart Home operation interface adopts text formatting mostly, in practical operation, often occur that the icon on interface does not exist in finishing household.In addition, be mostly equiped with now many air-conditionings in family, control when link setting up, operator must remember controlled plant in the air-conditioning equipment, particularly manager of the actual correspondence of each air-conditioning button institute when more, obscures unavoidably.Even if user marks on button side voluntarily, also exist troublesome poeration, the deficiencies such as the aesthetic property of destruction operation interface.When particularly controlled plant control, often, because controlling too much impalpable controlled plant accurately of button, cause puzzlement.
The user interface that also has some to be applied to intelligent home control device has adopted the method for setting up three-dimensional operating interface, with the corresponding controlled plant of icon representation, and according to the fitting structure of actual household, the icon of controlled plant is placed on to the relevant position of house tomograph, thus the establishment at complete operation interface.The operation interface of setting up according to this kind of method, although performing content is close with actual household structure, but these tomograph existing problems: 1, tomograph is made comparatively complicated, need to make for the shelter of each different decorated styles the tomograph matching, take time and effort, cost of manufacture is high; 2, be subject to the restriction of terminal display size, the image size of tomograph is restricted, if have multiple different controlled plants for a larger shelter, in tomograph, cannot show one by one, even if show one by one, also will cause the picture size of controlled plant too small, man-machine interaction be poor, without actual application value; 3, tomograph can only inwardly project from an angle, just cannot show all controlled plants with this incident angle vertical direction, analogy enters doorway angle from gate inwardly projects tomograph, but the controlled plant at porch, gate vertical direction metope just cannot show, therefore certainly will cause part controlled plant in tomograph, to embody.
Summary of the invention
The technical problem to be solved in the present invention, be to provide a kind of intelligent home control device and method based on panoramic view, can set up quickly and easily vividly intuitively dynamically outdoor scene operation interface, and be not subject to the restriction of terminal screen size can easily carry more opertaing device, be extremely easy to user's use and greatly saved the cost of engineering construction.
The present invention is achieved in that a kind of intelligent home furnishing control method based on panoramic view, comprises the steps:
Step 10, the shelter house interior that sequentially shooting comprises some controlled plants are looked around photo, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and this panoramic pictures equal proportion is zoomed to it and highly meets the requirement of the vertical display resolution of Touch Screen, panorama sketch size is more than or equal to screen display size all the time, then at its summit, lower left corner place, one true origin A (0,0) is set; For benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 with true origin A (0,0) ... dn};
Step 20, set up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Step 30, panorama sketch is written into Touch Screen, summit, lower left corner place in Touch Screen viewing area arranges a true origin B (0,0),, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area, when using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen, in the time that panorama sketch moves to the end, its left and right end image seamless is connected, continue circulation and show;
Step 40, click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0, 0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restores under original state with initial point A (0 with side-play amount L1, 0) be the coordinate d (x of benchmark, y), and by this coordinate d (x, y) with the corresponding stationary coordinate collection D{d1 in controlled plant graphics field, d2, d3 ... dn} mates mutually, then triggering this panorama sketch locates controlled plant image-region and links with the control of actual controlled equipment, realize the visual control to actual controlled equipment.
Further, described step 10 is further specially,
Using photographic equipment to take one group in shelter house interior and comprise some controlled plants at indoor photo of looking around a week, then on PC, use distant view photograph synthetics software, is a rectangle panoramic pictures by this group photomosaic.Left and right end to this panoramic pictures accurately cuts, and be up to the standard 360 degree field angle and left and right end also can seamless spliced flow displays together.
Further, described step 10 is further specially,
On PC, use configuration tool software to edit some controlled plant image-regions in panorama sketch, obtain each end points coordinate of controlled plant image-region, described end points set of coordinates is become to some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn}.
Further, described step 40 is further specially,
Click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0,0) be coordinate d ' (the x ' of benchmark, y), restore under original state with initial point A (0 by following logic judging method, 0) be the coordinate d (x, y) of benchmark:
y=y
In above formula, x ' is the screen taps operation coordinate that touch screen obtains; Panorama sketch is in vertical direction without mobile, therefore y is constant; L1 is panorama sketch mobile side-play amount in the horizontal direction; L is the total length of panorama sketch, when | when L1|=L, L1 zero clearing is calculated to carry out next time.
Further, also comprise step 50, on operation interface, be provided for activating the floated button of prompt operation, prompt operation window at ordinary times in hidden state with reduce blocking and disturbing interface.
Further, also comprise step 60, be provided for selecting the drop-down window that skids off in floor, room etc. on operation interface, this drop-down window that skids off does not occupy operation interface in the time not using.
The invention also discloses a kind of intelligent home control device based on panoramic view, it comprises:
Panorama sketch pretreatment module, look around photo for sequentially taking the shelter house interior that comprises some controlled plants, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and by this panoramic pictures equal proportion zoom to its requirement that highly meets the vertical display resolution of Touch Screen panorama sketch size be more than or equal to all the time screen display size, then at its summit, lower left corner place, one true origin A (0 is set, 0), with true origin A (0, 0) be benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn},
Controlled plant communication link module, for setting up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Panorama sketch display module, for panorama sketch is written into Touch Screen, summit, lower left corner place in Touch Screen viewing area arranges a true origin B (0,0),, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area, when using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen.In the time that panorama sketch moves to the end, its left and right end image seamless is connected, continues circulation and show.
Controlled plant control module, for clicking the controlled plant image of panorama sketch, obtain on Touch Screen with initial point B (0, 0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restores under original state with initial point A (0 with side-play amount L1, 0) be the coordinate d (x of benchmark, y), and by this coordinate d (x, y) with the corresponding stationary coordinate collection D{d1 in controlled plant graphics field, d2, d3 ... dn} mates mutually, then triggering this panorama sketch locates controlled plant graphics field and links with the control of actual controlled equipment, realize the visual control to actual controlled equipment.
Further, described panorama sketch pretreatment module is further specially:
By use configuration tool software on PC, some controlled plant image-regions in panorama sketch are edited, obtain each end points coordinate of controlled plant image-region, described end points set of coordinates is become to some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn}.
Further, described controlled plant control module is further specially:
Click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0,0) be coordinate d ' (the x ' of benchmark, y), restore under original state with initial point A (0 by following logic judging method, 0) be the coordinate d (x, y) of benchmark:
In above formula, x ' is the screen taps operation coordinate that touch screen obtains; Panorama sketch is in vertical direction without mobile, therefore y is constant; L1 is panorama sketch mobile side-play amount in the horizontal direction; L is the total length of panorama sketch, when | when L1|=L, L1 zero clearing is calculated to carry out next time.
Further, described control device also comprises:
Prompt operation module is provided for activating the floated button of prompt operation on operation interface.
Further, described control device also comprises:
Drop-down window module is provided for selecting the drop-down window that skids off in floor, room on operation interface.
Tool of the present invention has the following advantages: the intelligent home control device and the method that the present invention is based on panoramic view, with the inside panoramic view in actual room as graphical operation interface, man-machine interaction is simple, directly perceived, and interface view and actual room scene are consistent, and are extremely easy to user and understand operation; Interface view can be done in the horizontal direction infinite loop and slide, and formation can rotate freely the dynamic outdoor scene visual effect of 360 degree scopes, has operating experience on the spot in person; On panoramic view, can easily carry more opertaing device quantity, floated prompt operation button and pull-down skid off window and have reduced the impact on outdoor scene picture, make simple interface lucid and lively, clear directly perceived, have hommization.
Brief description of the drawings
The present invention is further illustrated in conjunction with the embodiments with reference to the accompanying drawings.
Fig. 1 is the inventive method flowchart;
Fig. 2 is operation interface panorama sketch original state coordinate schematic diagram in the present invention;
Fig. 3 is operation interface panorama sketch moving process coordinate schematic diagram in the present invention;
Fig. 4 is the shortcut window schematic diagram of operation interface in the present invention;
Fig. 5 is the status bar schematic diagram of operation interface in the present invention;
Fig. 6 is the schematic diagram of the drop-down window of operation interface in the present invention;
Fig. 7 is that schematic diagram is selected in the floor room of operation interface in the present invention;
Fig. 8 is the working envelope figure of control device of the present invention.
Embodiment
As shown in Figure 1, the present invention is based on the intelligent home furnishing control method of panoramic view, comprise the steps:
Step 10, sequentially take the shelter house interior that comprises some controlled plants and look around photo, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and this panoramic pictures equal proportion is zoomed to its requirement that highly meets the vertical display resolution of Touch Screen (the height of this panoramic pictures and the height of Touch Screen are consistent), panorama sketch size is more than or equal to screen display size all the time, then at its summit, lower left corner place, one true origin A (0 is set, 0) (as shown in Figure 2), with true origin A (0, 0) be benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn},
Step 20, set up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Step 30, panorama sketch is written into Touch Screen, at summit, the lower left corner place of Touch Screen viewing area, one true origin B (0,0) is set,, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area.When using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen.In the time that panorama sketch moves to the end, its left and right end image seamless is connected, continues circulation and show.As shown in Figure 2 and Figure 3;
Controlled plant image in step 40, click panorama sketch, obtain on Touch Screen with initial point B (0,0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restore the coordinate d (x, y) that is benchmark with initial point A (0,0) under original state with side-play amount L1.Coupling coordinate d (x, y) and coordinate set D{d1, d2, d3 ... the relation of inclusion of dn}, realizes the visual control to controlled plant;
Step 50 is provided for activating the floated button of prompt operation on operation interface, prompt operation window at ordinary times in hidden state to reduce the blocking and disturbing of interface, prompt operation pop-up window is as shown in Figure 4;
Step 60 is provided for selecting the drop-down window that skids off in floor, room etc. on operation interface, and this drop-down window that skids off does not occupy operation interface in the time not using, and pull-down state hurdle, window are as shown in Fig. 5, Fig. 6, Fig. 7.
Be illustrated with a specific embodiment below:
1, take indoor distant view photograph and generate the coordinate set of controlled plant regional location on panoramic view: first taking one group of photo that house interior comprises controlled plant and looks around a week, on PC, use distant view photograph synthetics software to be compiled as the indoor panoramic pictures that a seamless connection from beginning to end shows, and then use the configuration tool software on PC to carry out equipment region editor to panorama sketch.For example: air-conditioning is the object of a rectangle profile on panorama sketch, sketch out so the image-region of air-conditioning, the position in this region is just decided by four end points in panorama sketch.The coordinate of supposing these four end points is respectively: (0,0), (0,1), (2,1), (2,0), so can be by d1={ (0,0), (0,1), (2,1), (2,0) } coordinate set represent the image-region position of this rectangle air-conditioning in panorama sketch.Accordingly, can obtain other various difformities, the polytype controlled plant position coordinates collection in panorama sketch: d2, d3 ... dn, has finally formed the image-region position coordinates set D{d1 of all controlled plants in panorama sketch, d2, d3 ... dn}.
2, setting up controlled plant image-region links with the control between actual controlled equipment: first make the controlled plant in residence enter login state, and these equipment are carried out to code setting.Then on panorama sketch, find the image-region of controlled plant directly to click, outwards broadcasting equipment control routine, sets up control link thereby aforementioned controlled equipment receives this device code.
3, be written into panorama sketch: Touch Screen viewing area (is illustrated as to example with 800*480 resolution, lower same) as a coordinate axis system, its four end points are respectively: lower-left point (0,0), upper left point (0,480), lower-right most point (800,0), upper right point (800,480), determine that its lower-left point is true origin B (0,0); Panorama sketch is loaded into the original state of on-screen display (osd) area coordinate system, its four end points are respectively: lower-left point (0,0), upper left point (0,480), lower-right most point (L, 0), upper right point (L, 480), determine that its lower-left point is true origin A (0,0).Under original state, the true origin A of panorama sketch overlaps with the true origin B of Touch Screen viewing area, has determined accordingly the absolute coordinates of all controlled plant image-regions position in coordinate system in panorama sketch, as shown in Figure 2.
4, panorama sketch left and right end seamless connection circulation Display Realization: panorama sketch only limits to the demonstration of sliding in the horizontal direction and circulate, its principle is the seamless connection flow display between the end image of panorama sketch left and right.If the length of panorama sketch is L, the length L >800 of actual panorama sketch, the visible part of panorama sketch on on-screen display (osd) area is 800*480 to the maximum so, and the sightless length of panorama sketch is L-800.Slide and all record panorama sketch lower left corner coordinate (L1 each time, 0), in the time that panorama sketch is moved to the left and-L1>=L-800 illustrates that panorama sketch has moved to the right-hand member end, now the left end Image Mosaics of panorama sketch is continued to show to its right, as shown in Figure 3.Otherwise in the time that panorama sketch moves right and L1>=0 illustrate that panorama sketch has moved to the left end end, now the right-hand member Image Mosaics of panorama sketch is continued to its left side to show.
5, on panorama sketch, equipment drawing is identified as region clicking operation: on Touch Screen, carrying out clicking operation is to realize by obtaining the coordinate figure of screen taps position correspondence in panorama sketch.By Touch Screen, as a coordinate system, panorama sketch region is as another coordinate system.Under original state, the true origin A of panorama sketch overlaps with the true origin B of Touch Screen viewing area.Now suppose that panorama sketch displacement is in the horizontal direction L1 (initial value is 0), in the time clicking Touch Screen, obtained the coordinate (x ', y) under Touch Screen coordinate system, this coordinate is in the scope of 800*480.When L1<0 and-L1<=L-800, can be by calculating the coordinate of this click location correspondence under panorama sketch coordinate system for (x '-L1, y), when-L1>L-800 and-when L1<L, now screen display be the picture that panorama sketch left and right end image splices mutually, so in the time of x ' >L+L1, illustrate that click location coordinate has dropped on the right-hand component of this spliced map, can calculate the coordinate of this click location correspondence under panorama sketch coordinate system is (x '-L-L1, y), otherwise click location is to drop on the part on the left side of spliced map, can calculate equally the coordinate of this click location correspondence under panorama sketch coordinate system is (x '-L1, y), as shown in Figure 3, as L1>0 and L1<800, now screen also will show the picture of holding image to splice mutually about panorama sketch, so in the time of x ' >L1, illustrate that click location coordinate has dropped on the right-hand component of this spliced map, can calculate the coordinate of this click location correspondence under panorama sketch coordinate system is (x '-L1, y), otherwise click location is to drop on the part on the left side of spliced map, can calculate equally the coordinate of this click location correspondence under panorama sketch coordinate system is (x '+L-L1, y), also as shown in Figure 3, as L1>800 and L1<=L, can calculate the coordinate of this click location correspondence under panorama sketch coordinate system for (x '+L-L1, y), when | when L1|=L, L1 zero clearing is calculated to carry out next time.To sum up arrange out and under panorama sketch coordinate system, obtain that to click the computing formula of operating position absolute coordinates d (x, y) as follows:
y=y
Obtain the absolute coordinates d (x of clicking operation position under panorama sketch coordinate system, y) after, coordinate set D{d1 by this coordinate and aforementioned controlled equipment drawing as regional location, d2, d3 ... dn} mates the controlled plant that can find actual association one by one, thereby realize the controlled plant image of clicking in panorama sketch on Touch Screen, completed the function to actual controlled equipment sending controling instruction.On panoramic image interface, be also provided with one for activating the floated button of prompt operation, this button can freely dragging to any position that does not affect normal running.Click this button and will activate contextual model prompt operation pop-up window, as shown in Figure 4.This window comprises that conventional scene and user edit scene, make interface operation more flexible and user-friendly.
Also be provided with a status bar in upper end, panoramic image interface, down-drawable skids off window for selecting the operation such as floor, room.The action of a downslide can start the drop-down window that skids off, and returns and only needs the upper button operation of sliding or return.Wherein status bar as shown in Figure 5, drop-downly skids off window as shown in Figure 6, and floor, room are selected as shown in Figure 7.The drop-down window that skids off does not occupy operation interface at ordinary times, makes simple interface lucid and lively, and outdoor scene viewing area is larger, and easy operating uses more.
Complete after above-mentioned steps, a kind of intelligent home furnishing control method based on panoramic view has just been set up.
The true panoramic view that the main interface of control operation of this Smart Home is house interior, it need to take out all controlled plants that just can control the inside, room true room being seen scene.In the time of interface operation, the indoor panoramic view sliding on main interface, do in the horizontal direction infinite loop mobile display, simulate room inner horizontal direction 360 and spend the dynamic outdoor scene visual effect of scope, make operator as placed oneself in the midst of real house interior, there is operating experience on the spot in person.The method has been cancelled the man-machine interaction mode of traditional menu button form on main interface, and user only need to find the material picture of controlled plant on panoramic view, clicks this image and just can realize control operation.And the method has been used floated quick botton and started prompt operation pop-up window, and pull-down sliding out window confession user selects the method such as floor, room, reduced the use of non-outdoor scene graphic element on interface.Such man-machine interaction mode has advantages of simple, directly perceived, convenient, easy-to-use.For example user need to turn on light, and the panoramic view touching on operation interface by slip so finds the position of light fixture, and then the image of clicking this light fixture just can directly be opened light.
The principal feature of the method is:
First take one group of photo that house interior comprises controlled plant and looks around a week, be compiled as the indoor panorama sketch of a seamless connection from beginning to end, then obtain the coordinate set of controlled plant image-region position in panorama sketch, finally set up the control between controlled plant image-region in actual controlled equipment and panorama sketch and link.On panoramic image interface, adopt Pop-up action pane and concealed drop-down window to reduce the impact on outdoor scene picture.
Taking the inside panoramic view in actual room as the graphical interface of controlling, man-machine interaction is simple, directly perceived, and interface operation is convenient, easy-to-use.The household electric appliances operating process of simulation in true environment, has operating experience on the spot in person.
True picture with controlled plant in panoramic view substitutes traditional form such as icon, button, and man-machine interface true nature reduces non-element in kind and uses.Interface view and actual room scene are consistent, and are extremely easy to user and understand operation.
The panoramic view at interface can do in the horizontal direction infinite loop and slide, and formation can rotate freely the dynamic outdoor scene visual effect of 360 degree scopes, can easily hold more opertaing device quantity on panoramic view simultaneously.
In the time of the logic control operations such as contextual model, adopt translucent pop-up window, the drop-down mode of operation that skids off window, reduce blocking and disturbing outdoor scene interface.
Based on said method, the invention also discloses a kind of intelligent home control device based on panoramic view, as shown in Figure 8, it comprises:
Panorama sketch pretreatment module, look around photo for sequentially taking the shelter house interior that comprises some controlled plants, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and by this panoramic pictures equal proportion zoom to its requirement that highly meets the vertical display resolution of Touch Screen panorama sketch size be more than or equal to all the time screen display size, then at its summit, lower left corner place, one true origin A (0 is set, 0), with true origin A (0, 0) be benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn},
Controlled plant communication link module, for setting up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Panorama sketch display module, for panorama sketch is written into Touch Screen, summit, lower left corner place in Touch Screen viewing area arranges a true origin B (0,0),, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area, when using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen.In the time that panorama sketch moves to the end, its left and right end image seamless is connected, continues circulation and show;
Controlled plant control module, for clicking the controlled plant image of panorama sketch, obtain on Touch Screen with initial point B (0, 0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restores under original state with initial point A (0 with side-play amount L1, 0) be the coordinate d (x of benchmark, y), and by this coordinate d (x, y) with the corresponding original coordinates collection D{d1 in controlled plant graphics field, d2, d3 ... dn} mates mutually, then triggering this panorama sketch locates controlled plant graphics field and links with the control of actual controlled equipment, realize the visual control to actual controlled equipment.
Particularly, described controlled plant control module can be reduced under selected controlled plant image-region original state the coordinate that is benchmark with initial point A (0,0) as follows:
Controlled plant image in step 1, click panorama sketch, obtains the coordinate d ' that is benchmark with initial point B (0,0) on Touch Screen (x ', y);
Array function under step 2, horizontal offset L1 substitution by panorama sketch on Touch Screen, the coordinate that to calculate under selected controlled plant image-region original state be benchmark with initial point A (0,0);
y=y
In above formula, x ' is the screen taps operation coordinate that touch screen obtains; Panorama sketch is in vertical direction without mobile, therefore y is constant; L1 is panorama sketch mobile side-play amount in the horizontal direction; L is the total length of panorama sketch, when | when L1|=L, L1 zero clearing is calculated to carry out next time.
Further, described controlled plant control module also comprises a prompt operation module, for carry out quick control operation on operation interface, prompt operation window at ordinary times in hidden state with reduce blocking and disturbing interface; Or on operation interface for the quick drop-down window that skids off of selecting floor, room etc., this drop-down window that skids off does not occupy operation interface in the time not using.By above-mentioned setting, can make man-machine interaction more convenient.
Tool of the present invention has the following advantages: the intelligent home control device and the method that the present invention is based on panoramic view, with the inside panoramic view in actual room as graphical operation interface, man-machine interaction is simple, directly perceived, and interface view and actual room scene are consistent, and are extremely easy to user and understand operation; Interface view can be done in the horizontal direction infinite loop and slide, and formation can rotate freely the dynamic outdoor scene visual effect of 360 degree scopes, has operating experience on the spot in person; On panoramic view, can easily carry more opertaing device quantity, floated prompt operation button and pull-down skid off window and have reduced the impact on outdoor scene picture, make simple interface lucid and lively, clear directly perceived, have hommization.
Although more than described the specific embodiment of the present invention; but being familiar with those skilled in the art is to be understood that; our described specific embodiment is illustrative; instead of for the restriction to scope of the present invention; those of ordinary skill in the art are in equivalent modification and the variation done according to spirit of the present invention, all should be encompassed in the scope that claim of the present invention protects.
Claims (10)
1. the intelligent home furnishing control method based on panoramic view, is characterized in that: comprise the steps:
Step 10, sequentially take the shelter house interior that comprises some controlled plants and look around photo, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and this panoramic pictures equal proportion is zoomed to it and highly meets the requirement of the vertical display resolution of Touch Screen, panorama sketch size is more than or equal to screen display size all the time, then at its summit, lower left corner place, one true origin A (0 is set, 0), with this true origin A (0, 0) be benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn},
Step 20, set up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Step 30, panorama sketch is written into Touch Screen, summit, lower left corner place in Touch Screen viewing area arranges a true origin B (0,0),, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area, when using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen, in the time that panorama sketch moves to the end, its left and right end image seamless is connected, continue circulation and show;
Step 40, click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0, 0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restores under original state with initial point A (0 with side-play amount L1, 0) be the coordinate d (x of benchmark, y), by this coordinate d (x, y) with the corresponding stationary coordinate collection D{d1 in controlled plant graphics field, d2, d3 ... dn} mates mutually, then triggering this panorama sketch locates controlled plant image-region and links with the control of actual controlled equipment, realize the visual control to actual controlled equipment.
2. a kind of intelligent home furnishing control method based on panoramic view according to claim 1, is characterized in that: described step 10 is further specially,
On PC, use configuration tool software to edit some controlled plant image-regions in panorama sketch, obtain each end points coordinate of controlled plant image-region, described end points set of coordinates is become to some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn}.
3. a kind of intelligent home furnishing control method based on panoramic view according to claim 1, is characterized in that: described step 40 is further specially,
Click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0,0) be coordinate d ' (the x ' of benchmark, y), restore under original state with initial point A (0 by following logic judging method, 0) be the coordinate d (x, y) of benchmark:
In above formula, x ' is the screen taps operation coordinate that touch screen obtains; Panorama sketch is in vertical direction without mobile, therefore y is constant; L1 is panorama sketch mobile side-play amount in the horizontal direction; L is the total length of panorama sketch, when | when L1|=L, L1 zero clearing is calculated to carry out next time.
4. a kind of intelligent home furnishing control method based on panoramic view according to claim 1, is characterized in that: also comprise step 50, be provided for activating the floated button of prompt operation on operation interface.
5. a kind of intelligent home furnishing control method based on panoramic view according to claim 1, is characterized in that: also comprise step 60, be provided for selecting the drop-down window that skids off in floor, room on operation interface.
6. the intelligent home control device based on panoramic view, is characterized in that: it comprises:
Panorama sketch pretreatment module, look around photo for sequentially taking the shelter house interior that comprises some controlled plants, this group photomosaic is compiled as to the rectangle panoramic pictures of a horizontal 360-degree field angle, and by this panoramic pictures equal proportion zoom to its requirement that highly meets the vertical display resolution of Touch Screen panorama sketch size be more than or equal to all the time screen display size, then at its summit, lower left corner place, one true origin A (0,0) is set; For benchmark, generate some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 with true origin A (0,0) ... dn};
Controlled plant communication link module, for setting up on actual controlled equipment and panorama sketch by stationary coordinate collection D{d1, d2, d3 ... control link between the controlled plant image-region that dn} locates;
Panorama sketch display module, for panorama sketch is written into Touch Screen, summit, lower left corner place in Touch Screen viewing area arranges a true origin B (0,0),, under original state, the true origin A of described panorama sketch overlaps with the true origin B of on-screen display (osd) area, when using finger sliding panorama sketch, while choosing controlled plant image, calculate panorama sketch and move horizontally side-play amount L1 on screen.In the time that panorama sketch moves to the end, its left and right end image seamless is connected, continues circulation and show.
Controlled plant control module, for clicking the controlled plant image of panorama sketch, obtain on Touch Screen with initial point B (0, 0) be coordinate d ' (the x ' of benchmark, y), and panorama sketch true origin A is compared to the horizontal offset L1 of initial point B, this coordinate d ' calculates mutually and restores under original state with initial point A (0 with side-play amount L1, 0) be the coordinate d (x of benchmark, y), and by this coordinate d (x, y) with the corresponding stationary coordinate collection D{d1 in controlled plant graphics field, d2, d3 ... dn} mates mutually, then triggering this panorama sketch locates controlled plant graphics field and links with the control of actual controlled equipment, realize the visual control to actual controlled equipment.
7. a kind of intelligent home control device based on panoramic view according to claim 6, is characterized in that: described panorama sketch pretreatment module is further specially:
By use configuration tool software on PC, some controlled plant image-regions in panorama sketch are edited, obtain each end points coordinate of controlled plant image-region, described end points set of coordinates is become to some controlled plant image-regions position corresponding stationary coordinate collection D{d1 on panorama sketch, d2, d3 ... dn}.
8. a kind of intelligent home control device based on panoramic view according to claim 6, is characterized in that: described controlled plant control module is further specially:
Click the controlled plant image in panorama sketch, obtain on Touch Screen with initial point B (0,0) be coordinate d ' (the x ' of benchmark, y), restore under original state with initial point A (0 by following logic judging method, 0) be the coordinate d (x, y) of benchmark:
y=y
In above formula, x ' is the screen taps operation coordinate that touch screen obtains; Panorama sketch is in vertical direction without mobile, therefore y is constant; L1 is panorama sketch mobile side-play amount in the horizontal direction; L is the total length of panorama sketch, when | when L1|=L, L1 zero clearing is calculated to carry out next time.
9. a kind of intelligent home control device based on panoramic view according to claim 6, is characterized in that: described control device also comprises:
Prompt operation module is provided for activating the floated button of prompt operation on operation interface.
10. a kind of intelligent home control device based on panoramic view according to claim 6, is characterized in that: described control device also comprises:
Drop-down window module is provided for selecting the drop-down window that skids off in floor, room on operation interface.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410392711.8A CN104181884B (en) | 2014-08-11 | 2014-08-11 | A kind of intelligent home control device and method based on panoramic view |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410392711.8A CN104181884B (en) | 2014-08-11 | 2014-08-11 | A kind of intelligent home control device and method based on panoramic view |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104181884A true CN104181884A (en) | 2014-12-03 |
CN104181884B CN104181884B (en) | 2017-06-27 |
Family
ID=51963021
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201410392711.8A Active CN104181884B (en) | 2014-08-11 | 2014-08-11 | A kind of intelligent home control device and method based on panoramic view |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104181884B (en) |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104597759A (en) * | 2014-12-26 | 2015-05-06 | 深圳市兰丁科技有限公司 | Network video based household control method and system and intelligent household management system |
CN105259765A (en) * | 2015-09-18 | 2016-01-20 | 小米科技有限责任公司 | Method and device for generating control interface |
CN105376125A (en) * | 2015-12-08 | 2016-03-02 | 深圳众乐智府科技有限公司 | Control method and device for intelligent home system |
CN105487393A (en) * | 2015-11-26 | 2016-04-13 | 英业达科技有限公司 | Control device and operating method thereof |
CN106325715A (en) * | 2016-09-28 | 2017-01-11 | 北京奥阁门科技有限公司 | Method and system for display picture and intelligent equipment |
CN106597860A (en) * | 2015-10-19 | 2017-04-26 | 上海澜腾智能科技有限公司 | Household electrical appliance control system, and control device, construction method and control method thereof |
CN107123092A (en) * | 2017-04-28 | 2017-09-01 | 广东远峰电子科技股份有限公司 | A kind of method and apparatus at real-time switching panoramic picture visual angle |
WO2017147909A1 (en) * | 2016-03-04 | 2017-09-08 | 华为技术有限公司 | Target device control method and apparatus |
CN107203144A (en) * | 2016-03-18 | 2017-09-26 | 美的集团股份有限公司 | The control method and system of smart home, Centralized Controller |
CN107678329A (en) * | 2017-09-08 | 2018-02-09 | 广州市迪士普智慧科技有限公司 | A kind of panorama interconnected control systems and control method |
CN107786717A (en) * | 2016-08-24 | 2018-03-09 | 深圳市鼎芯无限科技有限公司 | A kind of control method of intelligent lamp |
CN108008870A (en) * | 2016-11-01 | 2018-05-08 | 阿里巴巴集团控股有限公司 | A kind of image presentation method, device, electronic equipment |
CN108268434A (en) * | 2016-12-30 | 2018-07-10 | 粉迷科技股份有限公司 | Hyperlink edit methods and system in stereo scene |
CN108319151A (en) * | 2018-02-09 | 2018-07-24 | 广东美的制冷设备有限公司 | Control method, device, system, mobile terminal and the storage medium of household appliance |
CN108605052A (en) * | 2016-02-09 | 2018-09-28 | 高通股份有限公司 | IoT control interfaces are expanded into user equipment from IoT controllers |
CN110673505A (en) * | 2019-11-19 | 2020-01-10 | 珠海格力电器股份有限公司 | Smart home control method and device and storage medium |
CN110989914A (en) * | 2019-11-25 | 2020-04-10 | 北京城市网邻信息技术有限公司 | Multimedia information acquisition method and device |
WO2021000803A1 (en) * | 2019-06-29 | 2021-01-07 | 华为技术有限公司 | Method for controlling small window of screen and related device |
CN114518860A (en) * | 2020-11-18 | 2022-05-20 | 深圳Tcl新技术有限公司 | Method and device for creating panoramic picture based on large screen, intelligent terminal and medium |
CN115529201A (en) * | 2022-05-31 | 2022-12-27 | 青岛海尔智能家电科技有限公司 | Method, system, device, server and storage medium for generating family environment panorama based on digital twinning |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1046650A (en) * | 1989-04-20 | 1990-10-31 | Rca许可公司 | The TV image that is used for self-centering is amplified and the method and apparatus of panorama display system |
CN1367892A (en) * | 1999-10-20 | 2002-09-04 | 皇家菲利浦电子有限公司 | Information processing device |
WO2007100853A2 (en) * | 2006-02-28 | 2007-09-07 | Mark Cowtan | Internet-based, dual-paned virtual tour presentation system with versatile tabbed menu-driven areas |
CN101051223A (en) * | 2007-04-29 | 2007-10-10 | 浙江工业大学 | Air conditioner energy saving controller based on omnibearing computer vision |
CN101208723A (en) * | 2005-02-23 | 2008-06-25 | 克雷格·萨默斯 | Automatic scene modeling for the 3D camera and 3D video |
CN101404726A (en) * | 2008-10-20 | 2009-04-08 | 深圳华为通信技术有限公司 | Control method, system and apparatus for far-end camera |
CN101763607A (en) * | 2008-12-25 | 2010-06-30 | 上海杰图软件技术有限公司 | Online exhibition platform system constructed by using panoramic electronic map and construction method thereof |
CN101995828A (en) * | 2009-08-24 | 2011-03-30 | 厦门立林科技有限公司 | Building method of user operating interface of intelligent home control device |
CN103294024A (en) * | 2013-04-09 | 2013-09-11 | 宁波杜亚机电技术有限公司 | Intelligent home system control method |
CN103389653A (en) * | 2013-06-26 | 2013-11-13 | 程勇 | Implementation method of palm control of devices of house bathroom |
WO2014066858A1 (en) * | 2012-10-25 | 2014-05-01 | Microsoft Corporation | Providing images with zoomspots |
-
2014
- 2014-08-11 CN CN201410392711.8A patent/CN104181884B/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1046650A (en) * | 1989-04-20 | 1990-10-31 | Rca许可公司 | The TV image that is used for self-centering is amplified and the method and apparatus of panorama display system |
CN1367892A (en) * | 1999-10-20 | 2002-09-04 | 皇家菲利浦电子有限公司 | Information processing device |
CN101208723A (en) * | 2005-02-23 | 2008-06-25 | 克雷格·萨默斯 | Automatic scene modeling for the 3D camera and 3D video |
WO2007100853A2 (en) * | 2006-02-28 | 2007-09-07 | Mark Cowtan | Internet-based, dual-paned virtual tour presentation system with versatile tabbed menu-driven areas |
CN101051223A (en) * | 2007-04-29 | 2007-10-10 | 浙江工业大学 | Air conditioner energy saving controller based on omnibearing computer vision |
CN101404726A (en) * | 2008-10-20 | 2009-04-08 | 深圳华为通信技术有限公司 | Control method, system and apparatus for far-end camera |
CN101763607A (en) * | 2008-12-25 | 2010-06-30 | 上海杰图软件技术有限公司 | Online exhibition platform system constructed by using panoramic electronic map and construction method thereof |
CN101995828A (en) * | 2009-08-24 | 2011-03-30 | 厦门立林科技有限公司 | Building method of user operating interface of intelligent home control device |
WO2014066858A1 (en) * | 2012-10-25 | 2014-05-01 | Microsoft Corporation | Providing images with zoomspots |
CN103294024A (en) * | 2013-04-09 | 2013-09-11 | 宁波杜亚机电技术有限公司 | Intelligent home system control method |
CN103389653A (en) * | 2013-06-26 | 2013-11-13 | 程勇 | Implementation method of palm control of devices of house bathroom |
Non-Patent Citations (1)
Title |
---|
唐菲菲等: "智能家居***全景界面可控点自动识别的研究", 《现代电子技术》 * |
Cited By (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104597759A (en) * | 2014-12-26 | 2015-05-06 | 深圳市兰丁科技有限公司 | Network video based household control method and system and intelligent household management system |
CN105259765B (en) * | 2015-09-18 | 2018-06-05 | 小米科技有限责任公司 | Generate the method and device of control interface |
CN105259765A (en) * | 2015-09-18 | 2016-01-20 | 小米科技有限责任公司 | Method and device for generating control interface |
CN106597860A (en) * | 2015-10-19 | 2017-04-26 | 上海澜腾智能科技有限公司 | Household electrical appliance control system, and control device, construction method and control method thereof |
CN105487393A (en) * | 2015-11-26 | 2016-04-13 | 英业达科技有限公司 | Control device and operating method thereof |
CN105376125A (en) * | 2015-12-08 | 2016-03-02 | 深圳众乐智府科技有限公司 | Control method and device for intelligent home system |
CN105376125B (en) * | 2015-12-08 | 2018-12-18 | 深圳众乐智府科技有限公司 | A kind of smart home system control method and device |
CN108605052A (en) * | 2016-02-09 | 2018-09-28 | 高通股份有限公司 | IoT control interfaces are expanded into user equipment from IoT controllers |
WO2017147909A1 (en) * | 2016-03-04 | 2017-09-08 | 华为技术有限公司 | Target device control method and apparatus |
CN108353151A (en) * | 2016-03-04 | 2018-07-31 | 华为技术有限公司 | The control method and device of target device |
CN107203144A (en) * | 2016-03-18 | 2017-09-26 | 美的集团股份有限公司 | The control method and system of smart home, Centralized Controller |
CN107786717A (en) * | 2016-08-24 | 2018-03-09 | 深圳市鼎芯无限科技有限公司 | A kind of control method of intelligent lamp |
CN106325715B (en) * | 2016-09-28 | 2020-04-03 | 王海磊 | Method and system for displaying picture and intelligent device |
CN106325715A (en) * | 2016-09-28 | 2017-01-11 | 北京奥阁门科技有限公司 | Method and system for display picture and intelligent equipment |
CN108008870A (en) * | 2016-11-01 | 2018-05-08 | 阿里巴巴集团控股有限公司 | A kind of image presentation method, device, electronic equipment |
CN108268434A (en) * | 2016-12-30 | 2018-07-10 | 粉迷科技股份有限公司 | Hyperlink edit methods and system in stereo scene |
CN107123092A (en) * | 2017-04-28 | 2017-09-01 | 广东远峰电子科技股份有限公司 | A kind of method and apparatus at real-time switching panoramic picture visual angle |
CN107123092B (en) * | 2017-04-28 | 2021-10-12 | 远峰科技股份有限公司 | Method and device for switching panoramic image view angle in real time |
CN107678329A (en) * | 2017-09-08 | 2018-02-09 | 广州市迪士普智慧科技有限公司 | A kind of panorama interconnected control systems and control method |
CN108319151A (en) * | 2018-02-09 | 2018-07-24 | 广东美的制冷设备有限公司 | Control method, device, system, mobile terminal and the storage medium of household appliance |
WO2021000803A1 (en) * | 2019-06-29 | 2021-01-07 | 华为技术有限公司 | Method for controlling small window of screen and related device |
US11797143B2 (en) | 2019-06-29 | 2023-10-24 | Huawei Technologies Co., Ltd. | Method for controlling small screen window and related device |
CN110673505A (en) * | 2019-11-19 | 2020-01-10 | 珠海格力电器股份有限公司 | Smart home control method and device and storage medium |
CN110989914A (en) * | 2019-11-25 | 2020-04-10 | 北京城市网邻信息技术有限公司 | Multimedia information acquisition method and device |
CN110989914B (en) * | 2019-11-25 | 2021-01-29 | 北京城市网邻信息技术有限公司 | Multimedia information acquisition method and device |
CN114518860A (en) * | 2020-11-18 | 2022-05-20 | 深圳Tcl新技术有限公司 | Method and device for creating panoramic picture based on large screen, intelligent terminal and medium |
WO2022105584A1 (en) * | 2020-11-18 | 2022-05-27 | 深圳Tcl新技术有限公司 | Method and apparatus for creating panoramic picture on basis of large screen, and intelligent terminal and medium |
CN114518860B (en) * | 2020-11-18 | 2023-08-01 | 深圳Tcl新技术有限公司 | Method and device for creating panoramic picture based on large screen, intelligent terminal and medium |
GB2616188A (en) * | 2020-11-18 | 2023-08-30 | Shenzhen Tcl New Tech Co Ltd | Method and apparatus for creating panoramic picture on basis of large screen, and intelligent terminal and medium |
CN115529201A (en) * | 2022-05-31 | 2022-12-27 | 青岛海尔智能家电科技有限公司 | Method, system, device, server and storage medium for generating family environment panorama based on digital twinning |
Also Published As
Publication number | Publication date |
---|---|
CN104181884B (en) | 2017-06-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104181884A (en) | Device and method for controlling intelligent home based on panoramic view | |
AU2018324085B2 (en) | Systems, methods, and graphical user interfaces for interacting with augmented and virtual reality environments | |
RU2528016C2 (en) | Method and device for controlling lighting device in virtual room | |
KR102071575B1 (en) | Moving robot, user terminal apparatus, and control method thereof | |
JP6525617B2 (en) | Image processing apparatus and control method thereof | |
US11417365B1 (en) | Methods, systems and apparatuses for multi-directional still pictures and/or multi-directional motion pictures | |
US20190212901A1 (en) | Manipulation of content on display surfaces via augmented reality | |
JP7121811B2 (en) | Method, apparatus, and storage medium for displaying three-dimensional spatial views | |
US8594980B2 (en) | Method and apparatus of template model view generation for home monitoring and control | |
WO2019046597A1 (en) | Systems, methods, and graphical user interfaces for interacting with augmented and virtual reality environments | |
KR20220062031A (en) | 3D Virtual Room-Based User Interface for Home Automation Systems | |
CN109460177A (en) | A kind of image processing method and terminal device | |
US10775190B2 (en) | Interactive virtual reality tour | |
CN112232900A (en) | Information display method and device | |
CN110675505A (en) | Indoor and outdoor house watching system based on panoramic virtual and actual seamless fusion | |
CN107506457A (en) | A kind of method and apparatus of data processing | |
CN103648043B (en) | Search control method and control device to intelligent television | |
CN107590337A (en) | A kind of house ornamentation displaying interactive approach and device | |
CN106030488A (en) | Portals for visual interfaces | |
US20120098967A1 (en) | 3d image monitoring system and method implemented by portable electronic device | |
WO2021042622A1 (en) | Mind map presentation method and apparatus, storage medium, and electronic device | |
US20190371068A1 (en) | Tap to Add Photo to Object in Augmented Reality | |
CN106716501A (en) | Visual decoration design method, apparatus therefor, and robot | |
CN106406430B (en) | A kind of the scene Collaborative modeling system and its method of double touch screens based on wide-angle fold | |
CN103795925A (en) | Interactive main-and-auxiliary-picture real-time rendering photographing method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |