CN103023970B - Method and system for storing mass data of Internet of Things (IoT) - Google Patents
Method and system for storing mass data of Internet of Things (IoT) Download PDFInfo
- Publication number
- CN103023970B CN103023970B CN201210461075.0A CN201210461075A CN103023970B CN 103023970 B CN103023970 B CN 103023970B CN 201210461075 A CN201210461075 A CN 201210461075A CN 103023970 B CN103023970 B CN 103023970B
- Authority
- CN
- China
- Prior art keywords
- data
- value
- internet
- things
- lightweight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method and a system for storing mass data of the Internet of Things (IoT). The system comprises a plurality of data receiving nodes, a master node server and a database cluster. The method comprises the following steps of: (1) carrying out preprocessing on IoT data, and putting the preprocessed data in the database cluster consisting of a master node, slave nodes and the data receiving nodes; (2) creating sample records, which take sample elements as storage units, on the master node according to the static and dynamic information of the data in the database cluster; (3) after the sample records are encapsulated, sending the encapsulated sample records to all the slave nodes by the master node so as to be subjected to fragmentation processing and/or separated storage; and (4) after the slave nodes complete storage, uploading results to the master node, and updating the data in the database cluster by the master node, thereby completing storage. According to the method and the system, the cluster is extended by fully utilizing the existing storage technologies so as to store the mass data, the IoT data are divided into lightweight data and multimedia data, and particularly, a fragmentation strategy is adopted aiming at a large-scale amount of data, so that the time expenditure caused by the extension of a storage space is avoided.
Description
Technical field
The present invention relates to a kind of Internet of Things storage means and system, particularly based on Internet of Things date storage method and the system of NoSQL.
Background technology
Internet of Things (Internet of Things, IoT) by various information awareness apparatus, article and the Internet are connected, allow all General Physics objects that can be independently addressable realize information exchange, finally reach the object of Weigh sensor, location, tracking, monitor and managment.
Data are aspects for Internet of Things most worthy, under environment of internet of things, data from different sensing equipments, and represent billions of objects, and in general, Internet of Things data present following characteristics:
1, multi-source heterogeneous.Internet of Things data from different awareness apparatus, as RFID(Radio FrequencyIdentification, radio frequency identification) recognizer, video equipment, temperature sensor, humidity sensor etc.The data deriving from distinct device have different semantemes and structure.
2, ultra-large.Internet of Things contains a large amount of awareness apparatus, and awareness apparatus is image data constantly, result in the sharply expansion of data scale, defines mass data.
3, temporal and spatial correlations.In Internet of things system, each sampled data possesses Time and place attribute, in order to describe object state dynamic change over time and space.
4, multidimensional scalar.Current, Internet of Things application is integrated with all kinds of awareness apparatus of multiple difference usually, can the simultaneously multiple index amount of perception (i.e. multidimensional event), thus sampled data normally multidimensional or even higher-dimension.
Current, the storage of Internet of Things data mainly contains three kinds of forms: local type, distributed and centralized.Local type refers to that sampled data is stored in the local memory cell of equipment; Distributed index, according to being stored in some node selected in network, realizes the access to data by intermediate mechanism; In centralized finger network, the data of each node are put together, and are sent to concentrated in long-range data center storage.The maximum defect of first two mode be exactly node resource as: memory space, disposal ability, electricity etc. are all limited, therefore can not support that the Internet of Things needing large-scale data and intensive inquiry is applied.Meanwhile, these two kinds of methods are also not easy to data and share between different applications.Therefore, centralized scheme is absorbed in current increasing research.
The centralized solution of process mass data is mainly divided into two classes: parallel database and cloud database.Parallel database forms primarily of multiple relevant database, support the storage of the structural data of magnanimity, but this kind of database performance is far below NoSQL(Not Only SQL, non-relational database), due to the distributed lock mechanism adopted, concurrent degree is extremely restricted; In addition, this type of database data form is fixed, cannot additions and deletions data field neatly, and effectively tackle isomeric data, therefore, parallel database is also not suitable for storage administration magnanimity Internet of Things data.The principal mode of cloud database is NoSQL database, NoSQL follows BASE model (Basically Available(is substantially available), Soft-state(soft state/flexible affairs), EventualConsistency(final consistency)), therefore possess high-performance and have enhanced scalability, NoSQL is without the need to fixed table structure, usually there is not attended operation, large data access possesses the performance advantage that relevant database is incomparable.
Summary of the invention
The present invention proposes a kind of Internet of Things mass data storage means to solve the problems of the technologies described above, its step comprises:
1) preliminary treatment is carried out to Internet of Things data, pretreated data are put into the data-base cluster be made up of host node, partial node and data reception node;
Described preliminary treatment is:
Internet of Things Data classification 1-1) sampling obtained is lightweight data and multi-medium data;
1-2) described lightweight data are carried out data deduplication process, particular value extraction and data deduplication process are carried out to described multi-medium data;
2) according to static information and the multidate information of data in described data-base cluster, setting up on the primary node with SampleElement is the SampleRecord record of memory cell; SampleElement is most basic unit of storage, and the data in cluster are made up of some SampleRecord, and a SampleRecord is made up of some SampleElement.
3) burst process and/or Separate Storage is carried out to being issued to each partial node by host node after the encapsulation of described SampleRecord record;
4) result is uploaded to host node after completing storage by described partial node, and host node upgrades the data in this data-base cluster, completes storage.
Described static information comprises: the ID of data, affiliated field and data type; Described multidate information comprises: lightweight data and multi-medium data; Described lightweight data comprise: value type and character type; Described lightweight multi-medium data comprises: lightweight multimedia data type (video, image, audio frequency ...), data format (if type is image, then can be jpep, gif, png etc. here), interest value, digest value and point to a pointer of multi-medium data original value.
In described preliminary treatment, multi-medium data particular value extracts according to interest value and digest value; Described interest value is set by the user, and described digest value uses MD5 or SHA algorithm.Essence for digest value is extraction one " data fingerprint ", reaches the object of unique identification data with a shorter value.
Preferably, described data deduplication process uses setting threshold values or block level duplicate removal.
Preferably, described preliminary treatment also comprises data scrubbing process, and described data scrubbing is treated to Missing Data Filling and level and smooth noise data.
Preferably, described memory cell SampleElement=<key:value> sequence key-value pair represents, wherein key represents data name, and value is data samples.
Preferably, described multi-medium data and light weight store through data separating; Interest value and/or the digest value of described SampleRecord data recording multimedia data do not store multi-medium data original value; Described SampleRecord records all lightweight data.
Also propose a kind of Internet of Things mass data storage system based on said method the present invention, comprising: the data-base cluster of the data reception node that multiple data reception node server forms, a host node server and multiple partial node server composition; It is characterized in that,
In described data-base cluster, data comprise: static information and multidate information;
Described static information comprises: the ID of article, affiliated field and object type; Described multidate information comprises: lightweight data and lightweight multi-medium data;
Described host node is for receiving client-requested and management cluster; Described partial node is for storing data; Described host node being set up with SampleElement is the SampleRecord record of memory cell;
After described data reception node carries out preliminary treatment to data after being used for receiver networking sampled data, stored in data-base cluster;
Adopt block level De-weight method to information duplicate removal in described multidate information, in described multidate information, multi-medium data separates with lightweight data and stores.
Described data-base cluster also comprises: standby host node, and described secondary node is used for avoiding single principal point to lose efficacy; This data-base cluster is made up of NoSQL or RDBMS database.
The present invention seeks to design a kind of Internet of Things storage system, meet following demand:
System designed by the present invention is arranged in the data Layer of Internet of Things four levels (sensing layer, network layer, data Layer and application layer), for the storage of Internet of Things data.Storage system framework designed by the present invention as shown in Figure 2, comprises two parts in this storage architecture: data reception node and data-base cluster.Data reception node is mainly used in receiver networking sampled data, after preliminary treatment, stored in data-base cluster.Data-base cluster mainly comprises three class roles: host node, partial node and standby host node.Host node is mainly used in receiving client-requested, and management cluster, as the mapping relations between: service data block and partial node, and monitoring write request whether data of assessment write exceed maximum storage capacity etc.; Partial node is also indifferent to the management work of whole cluster, only for storing applied data; Standby host node, is mainly used in avoiding single point failure, and user does not recognize the existence of this node, and when host node lost efficacy, this node will replace whole work of host node.Standby host node, just for the situation preventing single point failure, in fact, does not have standby host node whole system can normally work yet, therefore as simple from systemic-function, standby host node is dispensable.
This system adopts without sharing (Shared-Nothing) framework, compared with shared drive and shared disk framework, possesses stronger extensibility.
Be mainly refer to that each CPU has privately owned region of memory and privately owned disk space without share framework, and 2 CPU can not access same magnetic disk space, the communication between CPU is connected by network.Illustrate A-shared drive, B-shared disk and C-without the difference (wherein M represents internal memory (Memory), P represents processor (Processor), D represent disk (Disk)) between share framework as shown in Figure 1.
Under system designed by the present invention, the present invention proposes Data Share System.In current storage scheme, only pay close attention to and describe object data own, do not pay close attention to and how to store more effectively to realize data sharing between the application of each Internet of Things and to cooperate.Shared mechanism in the present invention is theoretical based on body (ontology), so-called body, the most famous and proposed " the clear and definite normalized illustration that body is conceptual model " by Gruber by the definition of extensively quoting.Body is used to describe the relation between the concept of certain field even in wider scope and concept, make these concepts and close the clear and definite unique definition tying up in shared scope and there is everybody and jointly approve, like this, between man-machine and just can exchange between machine..At present, body has been widely used in the fields such as Semantic Web, intelligent information retrieval, information integerated, digital library.
The present invention adopts ontology theory, is because current Internet of Things application is all comparatively closed, defines application closed loop, in fact current Internet of Things is not also real Internet of Things, the just things local area network of, also lack between each local area network (LAN) and interconnect, data are difficult to share.What therefore the present invention wished by ontology theory authority data is abstract, makes different application under Chinese noun and term, can reach common understanding, is more of value to sharing of data.For actualizing technology means, the Ontology Language OWL (Ontology Web Language) of current existing standard, it has supplied powerful primitive.Can see the associated description in such as " 2011-Ontology Based Service Discovery Method for Internet of Things " and " based on ontological Internet of Things application service research " these two sections of articles.
According to data abstraction standard, data are conceptualized as four kinds:
Entity (Entity) class: for describing entity, as people, automobile, booth etc., for booth, the essential information of this kind of entity can be described below: <Shed:temperature, light, humidity>.Subclass can be expanded on this basis.
Field (Field) class: the Internet of Things application that user profile is different, as traffic, logistics and agricultural etc.Same entity plays different roles in different fields, and therefore, the be concerned information of same entity in different field is also different.With artificial example, definable base attribute is as follows: <People:name, height, weight>.At medical field, what pay close attention to is the physiological data of human body, therefore in the field, People class is inherited as Patient, Patient class is expanded in People class: <Patient:name, height, weight, heartbeat, pulse, blood_pressure>; And in logistics field, people may as courier, People class is inherited as Messenger, more pays close attention to the contact method of people in the case, therefore in this field, Messenger class is expanded in People class: <People:name, height, weight, mobile, fix_phone, fax>.
Movable (Activity) class: it is specific movable to describe in order to family, as " reservation ", " (for booth) mends (light) ", " capturing (traffic violations vehicle) ".
Application (Application) class: user completes specific application, applies and is completed by multiple different field entity and different activities.
Data are abstract according to this standard row, and the information of necessity is stored in database, based on the consistent information understood and describe accurately, entity class comprised each other can be understood between different application, field belonging to entity, which field what describe this entity has, the concrete meaning of respective activity ... finally can realize data sharing, and the inter-related task that cooperated exactly.For intelligent inventory management system, system every day all can by the sampled data statistics goods quantity in stock in database, one amount finds certain goods, as apple (entity in the fruit field) shortage of stock, it will carry out retrieving (activity) in supplier, then the row rate of exchange (activity) etc. time, and the apple finally ordering (activity) respective numbers in selected supplier.Based on consistent understanding, all links in application can be completed automatically by machine, and there will not be semantic conflict, for this system above, under the constraint of field conditions, it is finally retrieved, what order is apple in fruit, but not the apple in digital product.
Swap data needs medium to carry between different systems, such as passage, by word document as carrier, and can read by word program, also can be presented among browser with the html page.Specific program can only understand specific data format, and word document can not be understood by viewed device naturally, also just correctly cannot present word wherein.Because XML language is all readable for human and computer, and be widely used in every field, therefore in the system of the present invention, the exchanges data between different application uses XML to complete for medium.Time data are stored in database, still deposit according to the form of database, when digital independent out will transfer to other system or application, just carried by XML and data are transmitted in the past.
Beneficial effect of the present invention
1, lightweight data and multi-medium data Separate Storage in the present invention, conveniently manages these two classes data respectively.Save the information such as the ID of article, field, type in the static information of object, expand basis for data sharing in the future provides.These information are of value to the retrieval of data, simultaneously have article mark resolution system etc. auxiliary under, also help carry out data sharing between different information service system, other system or application according to the information such as field, type, can understand the data obtained after obtaining data exactly.
2. pair mass data increases pre-treatment step, and for increasing the flexibility of system, these steps are configurable, and that is user can by optimum configurations the need of carrying out part or all of step.
3. can make full use of existing memory technology expansion cluster to store mass data, under the concurrent condition of height, provide good readwrite performance.Internet of Things data are entered to be categorized as lightweight data and multi-medium data, and in preliminary treatment, multi-medium data is carried out light-weight technologg, extract information and the summary info of user's care, the all data of data are all converted into lightweight data and store, original multimedia data is stored in another set, though overall data memory space expense slightly increases, decrease the volume of core data, be of value to efficiency when improving market demand.
4. Distribution Strategy setting does not take stripping strategy for the data that scale is less, and the data for large scale quantities can take stripping strategy, bring unnecessary time overhead to avoid the expansion of memory space.
Accompanying drawing explanation
Fig. 1 is the schematic diagram without share framework, shared disk and shared drive, A-shared disk formula (Share Disk), B-and without shared (Share Nothing), C-shared drive formula (Share Memory);
Fig. 2 is Internet of Things mass data storage system construction drawing of the present invention;
Fig. 3 is Internet of Things mass data storage system pretreatment process schematic diagram of the present invention;
Fig. 4 is storage policy duplicate removal process schematic in Internet of Things mass data storage means of the present invention;
Fig. 5 is representation schematic diagram in Internet of Things mass data storage means one embodiment of the present invention;
Fig. 6 is storage policy data fragmentation schematic diagram in Internet of Things mass data storage means of the present invention;
Fig. 7 (a), 7(b) be experimental result picture based on data fragmentation in storage policy in Internet of Things mass data storage means one embodiment of the present invention;
Fig. 8 is the piecemeal schematic diagram based on geographical space index edit code in Internet of Things mass data storage means one embodiment of the present invention.
Embodiment
Under the storage system that the present invention proposes, storage policy of the present invention comprises: pretreatment strategy, data representation form and data distribution strategy.
One. preliminary treatment
Data prediction completes primarily of data reception node, and different Internet of Things application has general character processing demands, also has individual character processing demands simultaneously.As Fig. 3, the present invention proposes a kind of preliminary treatment mechanism to meet the demand of these two aspects.First data reception node carries out the process of common requirement after receiving data, and processing procedure is as follows:
1. Data classification, original sampling data is divided into two classes: lightweight data, comprises numeric type and character type; Multi-medium data, comprises video, image, audio frequency and signal etc.Different types of data handling procedure is different.
2. extract particular value (specific value), this process is only for multi-medium data.The particular value of the present invention's definition comprises two classes: interest value (interest value), refer to the information that the application extracted by respective algorithms is concerned about the most, what usual application was concerned about is not multi-medium data itself, but the interested information of application be implied in wherein or knowledge, such as can analyze platform stream of people density from station platform monitor video, traffic violations quantity etc. in one hour in Traffic Surveillance Video, can be analyzed;
3. digest value (digestvalue), calculated by special algorithm equally, as MD5, digest value is used for the concise and to the point description of unifying multi-medium data, insignificant with the digest value that is not both that interest value is maximum, it is the character string of a string regular length, digest value is of value to many matchmakers data volume duplicate removal and retrieval, such as, if two sampled images are the same, so their digest value also will be the same, when no matter determining whether image repeats or carry out image retrieval, all only need comparison digest value.
4. data scrubbing, the sampled data deriving from physical world is all incomplete, noisy or even wrong usually, and data scrubbing object guarantees the correctness of data.Common means fill missing values and smooth noise.For missing values, if application allows the existence of missing values, then global constant such as " unknown " or "-∞ " is used to represent missing values; Otherwise, then the mean value of nearest N number of value is used to fill missing values (N can be self-defined according to actual needs).For noise data, use branch mailbox (binning) algorithm, around investigating data, neighbor carrys out smooth noise.It should be noted that, in Internet of Things, the sudden change of data is normal, therefore, when smooth noise, as long as this value is in normal range (NR), then can not be regarded as noise.
5. data deduplication, there is the high feature of redundancy in Internet of Things data, normally causes because higher sample frequency or multiple awareness apparatus are sampled to same object simultaneously.As shown in Figure 4, the straightforward procedure of a process redundant data is setting threshold value, if the value be currently received is less than threshold value with the last value difference stored in database, then this value is not stored into database; Otherwise then this value must be stored into database.But said method is only effective to one-dimensional data, Internet of Things data normally multidimensional, this makes data deduplication problem more complicated.In the case, the method that the present invention takes be in a record data of any one dimension be determined can stored in database, then this record must be stored into database.Certainly, this can cause larger space expense, because some dimension may long-time data not change, and need not at every turn all stored in database.For saving space, in a database, the present invention takes another method---and block level duplicate removal method tackles this problem.Block level duplicate removal mainly refers in a database, and data are carried out piecemeal, because data exist redundancy, then may produce much identical block, only deposits a copy for the block that data are identical, and other block uses pointer to quote this block.Block level duplicate removal is that all files are resolved into data block, then hashing algorithm is passed through, for each piece creates a cryptographic Hash, and compare with the cryptographic Hash of other all data blocks, if the cryptographic Hash of two data blocks is completely the same, one of them block will be deleted, and replace with the pointer pointing to another block.Different product checks that data block size is different, and the data block of some Supplier Selection fixed sizes, some then use the data block of different size.Specifically can see (Tan Yujuan; Data deduplication technical research [D] in data backup system; The Central China University of Science and Technology; 2012)
Be more than the general character treatment step that in the present invention, data prediction has to pass through, for increasing the flexibility of system, these steps are configurable, and that is user can by optimum configurations the need of carrying out part or all of step.After step process above, data will import " self-defined process " module into tackle personalized preliminary treatment needs, this module needs user program to realize, if unrealized, this module of default situations is left intact to data, only has a data access interface, be similar in programming and define a do-nothing function, this function is complete, and just any work is not done in its inside, then this module does not carry out any process to the data imported into.A kind of mode realized uses programming mode (the Aspect-Oriented Programming towards tangent plane, AOP), have benefited from this kind of programming mode, after realizing self-defined processing module, user is without the need to compiling existing code, and module dynamic is loaded into existing program.
Two. data representation
In the present invention, memory cell minimum in database is defined as SampleElement, is defined as follows:
SampleElement＝<key：ralue> (1),
SampleElement is an orderly key-value pair, wherein key ∈ String, is equivalent to the title of value; And value ∈ String ∪ Number is for storing actual sample value.The example of some SampleElement is as follows: <temperature:50>, <nPeople:121>or< audioText: " Hello world " >, represent that sample temperature is 50 degree respectively, number be 121 people and from audio frequency by text " Hello world " that speech recognition extracts.
In the present invention, basic memory cell is a record, is referred to as SampleRecord, be equivalent to the ordered set of SampleElement, as shown in Figure 5, a SampleRecord is made up of two parts information: static information, comprise the ID of article, affiliated field, object type etc.; Multidate information, namely about object actual sample value, comprise time, the state information of place and other objects self and environmental information, multidate information is made up of two parts, and a part is lightweight data, and another part is multi-medium data.SampleRecord is defined as follows:
SampleRecord=((rID,sID,tID,field,type…),(t,loc,(v↓i)↓(i＝1)↑n,(v↓j↑m)↓)(j=1)↑x)
(2),
Wherein, rID, sID, tID ∈ String, rID are the ID of this record, and tID, sID are the article code of unique identification object respectively, and the standard belonging to article code; Field ∈ String is used for field belonging to marking articles, and according to the shared mechanism in the present invention, field is necessary; Type etc. are other static descriptors of object; T ∈ Instant is the sampling time; Loc ∈ Point is sampling position, and when not having positioning equipment, loc is equivalent to awareness apparatus present position, Point be 2 dimension values as (x, y), in the present invention, space is not limited in GPS category, allow definition Custom Space; V ∈ String ∪ Number refers to that the light weight of sampling is through data; v
m∈ SampleMedia is for expressing multi-medium data, and multi-medium data SampleMedia is defined as follows:
SampleMedia＝(type，format，v
interest，v
digest，p
original)(3).
Wherein, type, format ∈ String, represents type (as audio frequency, image or video) and the form (as having jpeg for image, forms such as gif, png) of this multi-medium data respectively; v
interest∈ String ∪ Number refers to the interest value extracted from multi-medium data; v
digest∈ String refers to multi-medium data digest value; p
originalit is the pointer pointing to multi-medium data original value, because multi-medium data amount is larger, for convenience of expression and the storage of data,, the original value of not storing multimedia data in SampleRecord, only store associated lightweight data (interest value, digest value etc.), initial data will be stored in another set (Collection).Because concrete memory technology of the present invention is not limited to RDBMS, therefore in the present invention, the table concept in RDBMS is replaced by the concept of set, as shown in Figure 5, a set is made up of some SampeRecord, and correspondingly, SampleRecord can regard as a line in RDMBS table.
The Unified Expression to Internet of Things isomeric data can be realized by SampleRecord, and contribute to sharing of data.
Following table 1 gives the example of SampleRecord
Table 1SampleRecord example
Example 1 represents the sampled data to booth in the application of Internet of Things agriculture field, and the sampling time is t1, and place is gps coordinate (40.1,20.5), and illuminance is 50 luxs, and relative humidity is 56%, and humidity is 23.1 degrees Celsius.Example 2 derives from the overspeed detection of field of traffic, sampling position is (123,1024), this is a Custom Space coordinate, and over-speed vehicles speed is 210km/h, a multi-medium data capturePhoto is comprised in record, refer to the over-speed vehicles photo captured, form is jpeg, and the interest value extracted from photo is license plate number BJ0A435, digest value is a MD5 value b38767a34dd2d764c0d597986010e5b1, and the initial data pointer pointing to this photo is p1.
Three. Data distribution8
3.1 bursts (Sharding)
The storage system that the present invention relates to adopts the logical architecture of distributed memory system,
Data fragmentation is necessary selection, especially when big data quantity.As shown in Figure 6, burst refers to the process stored to different machines by data scatter.When allocation methods makes cluster can use powerful server, realize the storage of mass data.A cluster can be divided into some logic bursts, and a machine is divided into one or more burst according to actual needs.
Burst contributes to promoting storage capacity, and server performance, but not can both contribute to the lifting of performance in any case.When burst, the time of implementation T of an inquiry
qshown in following formula:
Wherein, T
trefer to the propagation delay time of client and server; T
prefer to resolve query statement and time instruction being distributed to partial node; T
srefer to the time that query statement performs when single machine; T
arefer to the time collecting Query Result from multiple node, with the machine data N of parallel execution of instructions
pbe directly proportional.Usual Tt and Tp will much smaller than Ts and Ta, and therefore Tq approximates
if the time of parallel execution of instructions saving is less than the overhead collecting result and cause, the so overall time of implementation will increase thereupon.For detecting the performance impact brought at many bursts, invention has been related experiment, result is as Fig. 7 (a), 7(b) and table 2 shown in:
Table 2 burst and different pieces of information scale renewal rewards theory time ratio (w representation unit: ten thousand) in not burst situation
Experimentally result, the Distribution Strategy setting about data in the present invention does not take stripping strategy for the data that scale is less, and the data for large scale quantities can take stripping strategy, bring unnecessary time overhead to avoid the expansion of memory space.
3.2 multi-medium data Separate Storages
Multi-medium data has following two notable features: 1) comparatively lightweight data, need a large amount of memory spaces; 2) transmitting multimedia data needs more bandwidth resources.For these two features, mean that multi-medium data needs to consume ample resources and comprises computational resource, storage resources and bandwidth resources, very high I/O load can be caused to the access of multi-medium data simultaneously.
For improving cluster performance, multi-medium data and light weight store through data separating by the present invention.Multi-medium data original value will be stored in another set, and user gathers unaware to this, does not namely recognize the existence of this set from the visual angle of user, and the cluster storing lightweight data is called as main cluster (main-cluster).As mentioned before, if need the original value obtaining multi-medium data, the initial data pointer in SampleMedia can be used.The benefit of this strategy is: 1) the more resource of main cluster can be saved for the treatment of user's request instead of be transmitted large multimedia data file; 2) operation can be optimized respectively to the data in two clusters pointedly, as compressed targetedly two class different pieces of informations, use different file system and index etc.
The present invention does not limit concrete database technology, if a certain database technology meets data storage requirement, and the system that can realize designed by the present invention based on this technology.High-performance, high concurrency and data format is flexibly provided in view of current NoSQL can be provided as mass data, in reply Internet of Things magnanimity isomeric data, there is obvious advantage compared to conventional RD BMS database technology, will the specific implementation based on NoSQL be set forth below.
Current database technology, is mainly divided into two classes: relevant database (RDBMS) and non-relational database (NoSQL), therefore, if do not adopt RDBMS, so NoSQL database can adopt.
Described in Data Share System and storage policy are followed above, in addition, due to the existing inquiry of NoSQL and non-fully holder networking data demand, therefore need to design and Implement a set of query grammar being applicable to Internet of Things application demand; Meanwhile, also need to realize for the index designed by Internet of Things data characteristics in NoSQL.The present invention, for NoSQL, is described in detail technology of the present invention:
One. query grammar
Internet of Things inquiry is mainly divided three classes: 1) historical query, obtains the historical data of some timing nodes or certain a period of time; 2) track inquiry, obtains the movement locus of object, as inquired about the transportation route of goods in logistics system; 3) preference inquiry, obtain interested value by a certain condition, as patient be in emergency time, inquire about nearest hospital site.
For historical query, query grammar is as follows:
getByTimeStamp:
getByTimeRange:
Input parameter is a set (Collection), with a time point or time interval, output is the set (Set of a SampleRecord, note, this two classes " set " concept being denoted as Collection and Set is different, and Collection refers to the organizational form of data in database, and Set refers to the collective concept of universality, hereafter for avoiding semantic conflict, will replace with English respectively), if corresponding time point or interval interior no record, return sky (null).The historic state of a certain object can be obtained by such inquiry.
For track inquiry, query grammar is as follows:
getTrackingPath:
Input parameter is a Collection and article ID, is input as the orderly Set of a Point.Due to position and the time correlation of object, this Set is given tacit consent to temporally sequence and is sorted.Grammer below helps getTrackingPath to operate the information of the concern that projects out:
getRecordSet:
getElement:SampleRecord×key|SampleElement
GetRecordSet operation is used for the Set obtaining a SampleRecord according to a key assignments, and in getTrackingPath operation, query key is article ID.GetElement operation obtains a SampleElement from a SampleRecord, and when its query key is loc, can project out geography information from a SampleRecord.
For preference inquiry, usually wish the object geographic location that acquisition objects perimeter is contiguous or concurrent.For the latter, can complete by the operation of historical query, and for the inquiry in geographical position, grammer is as follows:
getObjsNearPoint:
getObjDistance:Point×Point|Distance
GetObjsNearPoint operation is for obtaining the some objects nearest from some geographical position, and be input as a Collection and location point, output is that the orderly Set of a SampleRecord, this Set sort according to the distance of position.GetObjDistance operation is in order to obtain the distance between 2, and the present invention devises a kind of coding and indexed mode in order to improve distance calculated performance.
The present invention devises a special construction---and SampleMedia is in order to express multi-medium data, similar to SampleRecord, and it is in fact also a Set comprising some SampleElement.SampeMedia is equivalent to an Embedded Set and is included in SampleRecord.SampleMedia uses a key assignments to obtain equally, and grammer is as follows:
getMultiMedia:SampleRecord×key|SampleMedia
After acquisition SampleMedia, getElement can be used to operate and to obtain corresponding SampleElemen.
It should be noted that an operation is routinely as created, upgrading, insert, delete, and follows database itself and realizes, repeat no more.
Based in the realization of NoSQL, its query language and traditional RDBMS use the SQL of standard completely different.Although some databases support class SQL query language, most NoSQL system does not have unified language, for avoiding confusion, hereafter the operation using false code to combine above being provided some examples, avoiding the puzzlement that different N oSQL query language causes.
Example 1: to inquire about between 2012-07-25 to 2012-07-28 all speed of a motor vehicle more than the violation license plate number of 200km/h.
for each SampleRecord in
set(getByTimeRange(Collection,“2012-07-25”,”2012-07-28”)AND
getElement(speed,SampleRecord).value>200):
return getElement(plateNumber,getMultiMedia(key，SampleRecord)).value
Example 2: inquiry ID is the transport track of " ID123 ".
for each SampleRecord in
set(sort(getRecordSet(Collection,”ID123”),t))
return getElement(SampleRecord,loc).value
Example 3: 10 mobile telephone positions that inquiry is nearest apart from (112.02,111.01)
for each SampleRecord in Collection:
Point=getElement(loc,SampleRecord).value
sort(set(getObjDistance(Point,(112.02,111.01))))
return set.limit(10)
Two, index
In Internet of Things, the topmost feature of sampled data is space time correlation.A good index of design contributes to the recall precision improving temporal and spatial correlations.In the storage system designed by the present invention, space is not limited in GPS space, and user can the space of self-defined any size, such as, can define the space of one 30 kilometers × 30 kilometers of sizes, in order to represent the geographical space within five rings, Beijing City.Also can expanding space according to actual needs simultaneously, when all including in into as needed the space within by Beijing City six ring, the space of about 60 kilometers × 55 kilometers can be expanded to.For reaching such demand, the present invention devises a twin-stage B+ and sets index.
As shown in Figure 8, in the first order, space is split into medium sized space block (sapce block), for the ability making space have infinite expanding, coding for space block adopts spiral strategy, when spatial expansion, its identification number also shows growth thereupon, the index record of this one-level mapping relations of space block ID and position range; In the second level, space block is split into atomic unit, in the example of fig. 8, space block is by 4 segmentations, and be finally made up of 16 × 16 atomic units, the size of atomic unit can customize, the segmentation of more lean can be carried out, such as 10 segmentations, finally form 1024 × 1024 atomic units composition, the location point mapping of encoding interior with block of the index record of this one-level.
Pass through said method, a point can be converted into binary Hash (hash) value, to facilitate index, because simple as (x, y) such point cannot carry out index, needs to make to become possibility to the index of location point by certain coded system.As shown in Figure 8, with point (52,23), for example, cryptographic Hash is made up of two parts: the binary value of space block ID, and (52,23) corresponding blocks ID is 10, and being converted into binary system is 1010; Coding in block, point (52,23) is 00110101 by the value that encryption algorithm draws.Because contiguous point all can be assigned in identical block or the block that closes on, so close some cryptographic Hash prefix is the same or close.By block ID tentatively judge 2 whether close, if 2 block ID are identical or close, then 2 close; Further, in same, if 2 close, in its block, coding prefix is identical.
For Fig. 8, in block, encryption algorithm is as follows:
1) for the first time, be 4 sub-blocks, encode the block comminute being numbered 10 as follows to each sub-block, point (52,23) is divided into the lower left corner, this time divides and obtains coding 00:
2) N is carried out by first step method
d-1 iteration, wherein, N
dfor the segmentation times defined in vain, N in example in fig. 8
d=4.After three iteration the 2nd time, the 3rd time, the 4th obtains coding 11,01,01 respectively.
3) final, obtain according to the order of sequence being encoded to 00110101 in complete block.
Therefore, the cryptographic Hash obtaining point (52,23) is 101000110101.
The benefit of index building is as stated above: 1) when mass data, and all will not be loaded into internal memory by index, dynamic is loaded into the index of the block needed; 2) make space neatly infinite expanding become possibility; 3) the cryptographic Hash coded system of two-part is named the distance between 2 is more directly perceived, retrieves also more efficient, even can by observation learn 2 whether close.
Claims (10)
1. an Internet of Things mass data storage means, its step comprises:
1) preliminary treatment is carried out to Internet of Things data, pretreated data are put into the data-base cluster be made up of host node, partial node and data reception node;
Described preliminary treatment is:
Internet of Things Data classification 1-1) sampling obtained is lightweight data and multi-medium data;
1-2) described lightweight data are carried out data deduplication process, particular value extraction and data deduplication process are carried out to described multi-medium data;
2) according to static information and the multidate information of data in described data-base cluster, setting up on the primary node with SampleElement is the SampleRecord record of memory cell;
3) burst process and/or Separate Storage is carried out to being issued to each partial node by host node after the encapsulation of described SampleRecord record;
4) result is uploaded to host node after completing storage by described partial node, and host node upgrades the data in this data-base cluster, completes storage.
2. mass data storage means as claimed in claim 1, it is characterized in that, described static information comprises: article ID, affiliated field and object type; Described multidate information comprises: lightweight data and lightweight multi-medium data; Described lightweight data comprise: value type and character type; Described lightweight multi-medium data comprises: the pointer of multimedia data type, data format, interest value, digest value and sensing multi-medium data original value.
3. mass data storage means as claimed in claim 1 or 2, is characterized in that, in described preliminary treatment, multi-medium data particular value extracts according to interest value and digest value; Described interest value is set by the user, and described digest value uses MD5 or SHA algorithm to calculate.
4. mass data storage means as claimed in claim 1, is characterized in that, described data deduplication process uses setting threshold values or block level duplicate removal.
5. mass data storage means as claimed in claim 1, it is characterized in that, described preliminary treatment also comprises data scrubbing process, and described data scrubbing is treated to fills missing values and smooth noise.
6. mass data storage means as claimed in claim 1, it is characterized in that, described memory cell SampleElement sequence key-value pair <key:value> represents, wherein key represents data name, and value is data samples.
7. mass data storage means as claimed in claim 1, it is characterized in that, described multi-medium data and lightweight data separating store; Interest value and/or the digest value of described SampleRecord data recording multimedia data do not store the original value of multi-medium data; Described SampleRecord records all lightweight data.
8. adopt an Internet of Things mass data storage system for method described in claim 1, comprising: the data reception node that multiple data reception node server forms, the data-base cluster be made up of host node server and multiple partial node server; It is characterized in that,
In described data-base cluster, data comprise: static information and multidate information;
Described static information comprises: the ID of article, affiliated field and object type; Described multidate information comprises: lightweight data and lightweight multi-medium data;
Described host node server is for receiving client-requested and management cluster; Described partial node server is for storing data; Described host node server being set up with SampleElement is the SampleRecord record of memory cell;
Described data reception node is used for receiver networking sampled data, carries out preliminary treatment, and will carry out pretreated data stored in data-base cluster to data.
9. Internet of Things mass data storage system as claimed in claim 8, is characterized in that, adopt block level De-weight method to information duplicate removal in described multidate information, and in described multidate information, multi-medium data separates with lightweight data and stores.
10. Internet of Things mass data storage system as claimed in claim 8, it is characterized in that, described data-base cluster also comprises: standby host node, and described standby host node is used for avoiding single host node to lose efficacy; This data-base cluster is made up of NoSQL or RDBMS database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210461075.0A CN103023970B (en) | 2012-11-15 | 2012-11-15 | Method and system for storing mass data of Internet of Things (IoT) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210461075.0A CN103023970B (en) | 2012-11-15 | 2012-11-15 | Method and system for storing mass data of Internet of Things (IoT) |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103023970A CN103023970A (en) | 2013-04-03 |
| CN103023970B true CN103023970B (en) | 2015-07-22 |
Family
ID=47972096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210461075.0A Active CN103023970B (en) | 2012-11-15 | 2012-11-15 | Method and system for storing mass data of Internet of Things (IoT) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103023970B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10171319B2 (en) | 2015-06-05 | 2019-01-01 | Cisco Technology, Inc. | Technologies for annotating process and user information for network flows |
| US10171357B2 (en) | 2016-05-27 | 2019-01-01 | Cisco Technology, Inc. | Techniques for managing software defined networking controller in-band communications in a data center network |
Families Citing this family (70)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9286047B1 (en) | 2013-02-13 | 2016-03-15 | Cisco Technology, Inc. | Deployment and upgrade of network devices in a network environment |
| CN104102455B (en) * | 2013-04-11 | 2018-03-09 | 深圳市腾讯计算机***有限公司 | Method of data transfer, device and storage system |
| CN104318481A (en) * | 2014-05-04 | 2015-01-28 | 国家电网公司 | Power-grid-operation-oriented holographic time scale measurement data extraction conversion method |
| CN104052804A (en) * | 2014-06-09 | 2014-09-17 | 深圳先进技术研究院 | Method, device and cluster for sharing data streams between different task topologies |
| CN104182465A (en) * | 2014-07-21 | 2014-12-03 | 安徽华贞信息科技有限公司 | Network-based big data processing method |
| CN105302838B (en) * | 2014-07-31 | 2019-01-15 | 华为技术有限公司 | Classification method, lookup method and equipment |
| CN104268617B (en) * | 2014-09-04 | 2017-09-05 | 浙江托普仪器有限公司 | A kind of plant data comprehensive management system based on RFID |
| CN106464702B (en) * | 2014-11-27 | 2019-09-10 | 深圳市迈斯云门禁网络科技有限公司 | Management system for internet of things and method |
| CN104504147B (en) | 2015-01-04 | 2018-04-10 | 华为技术有限公司 | A kind of resource coordination method of data-base cluster, apparatus and system |
| CN104850593B (en) * | 2015-04-27 | 2018-10-30 | 张凌 | A kind of storage of emergency materials data and circulation monitoring method based on big data |
| US10374904B2 (en) | 2015-05-15 | 2019-08-06 | Cisco Technology, Inc. | Diagnostic network visualization |
| US9800497B2 (en) | 2015-05-27 | 2017-10-24 | Cisco Technology, Inc. | Operations, administration and management (OAM) in overlay data center environments |
| US10089099B2 (en) | 2015-06-05 | 2018-10-02 | Cisco Technology, Inc. | Automatic software upgrade |
| US10033766B2 (en) | 2015-06-05 | 2018-07-24 | Cisco Technology, Inc. | Policy-driven compliance |
| US9967158B2 (en) | 2015-06-05 | 2018-05-08 | Cisco Technology, Inc. | Interactive hierarchical network chord diagram for application dependency mapping |
| US10142353B2 (en) | 2015-06-05 | 2018-11-27 | Cisco Technology, Inc. | System for monitoring and managing datacenters |
| CN106708815B (en) * | 2015-07-15 | 2021-09-17 | 中兴通讯股份有限公司 | Data processing method, device and system |
| CN104994171A (en) * | 2015-07-15 | 2015-10-21 | 上海斐讯数据通信技术有限公司 | Distributed storage method and system |
| CN105183834B (en) * | 2015-08-31 | 2018-10-30 | 上海电科智能***股份有限公司 | A kind of traffic big data semantic applications method of servicing based on ontology library |
| CN105389683A (en) * | 2015-11-25 | 2016-03-09 | 北京华油信通科技有限公司 | Cloud computing support system |
| CN105608155B (en) * | 2015-12-17 | 2018-09-25 | 北京华油信通科技有限公司 | Mass data distributed memory system |
| US10931629B2 (en) | 2016-05-27 | 2021-02-23 | Cisco Technology, Inc. | Techniques for managing software defined networking controller in-band communications in a data center network |
| CN106055697A (en) * | 2016-06-15 | 2016-10-26 | 安徽天枢信息科技有限公司 | Unstructured event log data classification and storage method and device |
| US10289438B2 (en) | 2016-06-16 | 2019-05-14 | Cisco Technology, Inc. | Techniques for coordination of application components deployed on distributed virtual machines |
| US10708183B2 (en) | 2016-07-21 | 2020-07-07 | Cisco Technology, Inc. | System and method of providing segment routing as a service |
| CN107786365B (en) * | 2016-08-31 | 2020-06-19 | 北京金山云网络技术有限公司 | Cluster capacity expansion method and device |
| US10972388B2 (en) | 2016-11-22 | 2021-04-06 | Cisco Technology, Inc. | Federated microburst detection |
| CN108241639B (en) * | 2016-12-23 | 2019-07-23 | 中科星图股份有限公司 | A kind of data duplicate removal method |
| CN106844480B (en) * | 2016-12-23 | 2019-03-15 | 中科星图股份有限公司 | A kind of cleaning comparison storage method |
| CN106936899B (en) * | 2017-02-25 | 2021-02-05 | 九次方大数据信息集团有限公司 | Configuration method of distributed statistical analysis system and distributed statistical analysis system |
| CN106851735B (en) * | 2017-03-03 | 2020-08-04 | 国动物联网技术（上海）有限公司 | Method for processing repeated transmission of multi-gateway data through L oRaWAN |
| US10708152B2 (en) | 2017-03-23 | 2020-07-07 | Cisco Technology, Inc. | Predicting application and network performance |
| US10523512B2 (en) | 2017-03-24 | 2019-12-31 | Cisco Technology, Inc. | Network agent for generating platform specific network policies |
| US10594560B2 (en) | 2017-03-27 | 2020-03-17 | Cisco Technology, Inc. | Intent driven network policy platform |
| US10250446B2 (en) | 2017-03-27 | 2019-04-02 | Cisco Technology, Inc. | Distributed policy store |
| US10764141B2 (en) | 2017-03-27 | 2020-09-01 | Cisco Technology, Inc. | Network agent for reporting to a network policy system |
| US10873794B2 (en) | 2017-03-28 | 2020-12-22 | Cisco Technology, Inc. | Flowlet resolution for application performance monitoring and management |
| CN106960060B (en) * | 2017-04-10 | 2020-07-31 | 聚好看科技股份有限公司 | Database cluster management method and device |
| CN107170072A (en) * | 2017-04-14 | 2017-09-15 | 怀宁鑫橙信息技术有限公司 | A kind of cargo data long-range control method |
| WO2018218664A1 (en) * | 2017-06-02 | 2018-12-06 | 深圳齐心集团股份有限公司 | Big data stationery searching system |
| CN107273512B (en) * | 2017-06-21 | 2020-06-16 | 深圳市盛路物联通讯技术有限公司 | Method and device for data deduplication based on device type and geographic position |
| US10680887B2 (en) | 2017-07-21 | 2020-06-09 | Cisco Technology, Inc. | Remote device status audit and recovery |
| US10554501B2 (en) | 2017-10-23 | 2020-02-04 | Cisco Technology, Inc. | Network migration assistant |
| US10523541B2 (en) | 2017-10-25 | 2019-12-31 | Cisco Technology, Inc. | Federated network and application data analytics platform |
| US10594542B2 (en) | 2017-10-27 | 2020-03-17 | Cisco Technology, Inc. | System and method for network root cause analysis |
| CN108334557B (en) * | 2017-12-29 | 2022-03-11 | 东软集团（上海）有限公司 | Aggregated data analysis method and device, storage medium and electronic equipment |
| US11233821B2 (en) | 2018-01-04 | 2022-01-25 | Cisco Technology, Inc. | Network intrusion counter-intelligence |
| US11765046B1 (en) | 2018-01-11 | 2023-09-19 | Cisco Technology, Inc. | Endpoint cluster assignment and query generation |
| CN108121828A (en) * | 2018-01-17 | 2018-06-05 | 清华大学 | A kind of multi-source heterogeneous data managing method and system based on key-value pair data storehouse |
| CN108256089B (en) * | 2018-01-24 | 2019-06-18 | 清华大学 | The transform method and device of Internet of Things machine data |
| US10999149B2 (en) | 2018-01-25 | 2021-05-04 | Cisco Technology, Inc. | Automatic configuration discovery based on traffic flow data |
| US10574575B2 (en) | 2018-01-25 | 2020-02-25 | Cisco Technology, Inc. | Network flow stitching using middle box flow stitching |
| US10798015B2 (en) | 2018-01-25 | 2020-10-06 | Cisco Technology, Inc. | Discovery of middleboxes using traffic flow stitching |
| US10873593B2 (en) | 2018-01-25 | 2020-12-22 | Cisco Technology, Inc. | Mechanism for identifying differences between network snapshots |
| US10826803B2 (en) | 2018-01-25 | 2020-11-03 | Cisco Technology, Inc. | Mechanism for facilitating efficient policy updates |
| US10917438B2 (en) | 2018-01-25 | 2021-02-09 | Cisco Technology, Inc. | Secure publishing for policy updates |
| US11128700B2 (en) | 2018-01-26 | 2021-09-21 | Cisco Technology, Inc. | Load balancing configuration based on traffic flow telemetry |
| CN108650292A (en) * | 2018-03-27 | 2018-10-12 | 吉旗（成都）科技有限公司 | A kind of internet of things equipment message can quick locating query storage method |
| CN109144966A (en) * | 2018-07-06 | 2019-01-04 | 航天星图科技（北京）有限公司 | A kind of high-efficiency tissue and management method of massive spatio-temporal data |
| CN109271389A (en) * | 2018-09-29 | 2019-01-25 | 重庆大学 | A kind of Internet of Things storage model based on modified NoSQL technology |
| CN109344161A (en) * | 2018-12-04 | 2019-02-15 | 大唐网络有限公司 | A kind of mass data storage means based on mongodb |
| CN109945919B (en) * | 2019-02-22 | 2020-11-13 | 北京邮电大学 | System and method for monitoring temperature and humidity in whole process of Internet of things for fresh food transportation |
| CN109947743A (en) * | 2019-02-28 | 2019-06-28 | 长沙理工大学 | A kind of the NoSQL big data storage method and system of optimization |
| EP3963456A2 (en) | 2019-04-30 | 2022-03-09 | Clumio, Inc. | Cloud-based data protection service |
| CN110276359A (en) * | 2019-05-24 | 2019-09-24 | 天津亿玛科技有限公司 | A kind of feature extracting method and device |
| CN111104457A (en) * | 2019-10-30 | 2020-05-05 | 武汉大学 | Massive space-time data management method based on distributed database |
| CN111159305B (en) * | 2020-04-02 | 2020-08-04 | 支付宝(杭州)信息技术有限公司 | Vehicle data processing method, device and system based on block chain |
| CN111863226A (en) * | 2020-07-10 | 2020-10-30 | 广州市挖米科技有限责任公司 | Data management method and device for medical data |
| CN113986873B (en) * | 2021-09-26 | 2024-04-26 | 中国人民解放军国防大学军事管理学院 | Method for processing, storing and sharing data modeling of mass Internet of things |
| CN115712684B (en) * | 2022-11-24 | 2023-10-24 | 南京鼎山信息科技有限公司 | Storage method and system for dynamic information of articles in Internet of things |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673288A (en) * | 2009-10-13 | 2010-03-17 | 中兴通讯股份有限公司 | Method and system for reading and writing files in IPTV system |
| CN101795211A (en) * | 2010-01-13 | 2010-08-04 | 北京中创信测科技股份有限公司 | Data storage method and system |
| WO2011088657A1 (en) * | 2010-01-20 | 2011-07-28 | 中国科学院计算机网络信息中心 | Method, device and internet system for processing internet address information |
-
2012
- 2012-11-15 CN CN201210461075.0A patent/CN103023970B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673288A (en) * | 2009-10-13 | 2010-03-17 | 中兴通讯股份有限公司 | Method and system for reading and writing files in IPTV system |
| CN101795211A (en) * | 2010-01-13 | 2010-08-04 | 北京中创信测科技股份有限公司 | Data storage method and system |
| WO2011088657A1 (en) * | 2010-01-20 | 2011-07-28 | 中国科学院计算机网络信息中心 | Method, device and internet system for processing internet address information |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10171319B2 (en) | 2015-06-05 | 2019-01-01 | Cisco Technology, Inc. | Technologies for annotating process and user information for network flows |
| US10171357B2 (en) | 2016-05-27 | 2019-01-01 | Cisco Technology, Inc. | Techniques for managing software defined networking controller in-band communications in a data center network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103023970A (en) | 2013-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103023970B (en) | Method and system for storing mass data of Internet of Things (IoT) | |
| Qin et al. | When things matter: A survey on data-centric internet of things | |
| CN106649656B (en) | Database-oriented space-time trajectory big data storage method | |
| CN103488673B (en) | For performing the method for reconciliation process, controller and data-storage system | |
| CN105183834B (en) | A kind of traffic big data semantic applications method of servicing based on ontology library | |
| Qin et al. | When things matter: A data-centric view of the internet of things | |
| CN107577805A (en) | A kind of business service system towards the analysis of daily record big data | |
| CN106897954A (en) | Smart city space time information cloud platform | |
| CN105069703A (en) | Mass data management method of power grid | |
| CN109144966A (en) | A kind of high-efficiency tissue and management method of massive spatio-temporal data | |
| US20170060622A1 (en) | System and method for a data processing architecture | |
| CN109213752A (en) | A kind of data cleansing conversion method based on CIM | |
| CN113392227A (en) | Metadata knowledge map engine system facing rail transit field | |
| CN108595664A (en) | A kind of agricultural data monitoring method under hadoop environment | |
| CN103412903B (en) | The Internet of Things real-time searching method and system predicted based on object of interest | |
| CN115238015A (en) | Space-time big data platform based on micro-service | |
| Li et al. | TMS-RFID: Temporal management of large-scale RFID applications | |
| Bai et al. | Determining topological relationship of fuzzy spatiotemporal data integrated with XML twig pattern | |
| Moharm et al. | Big data in ITS: Concept, case studies, opportunities, and challenges | |
| Pokorný | Database architectures: Current trends and their relationships to environmental data management | |
| CN112905571B (en) | Train rail transit sensor data management method and device | |
| CN113254517A (en) | Service providing method based on internet big data | |
| Sun et al. | A distributed incremental information acquisition model for large-scale text data | |
| Gubareva et al. | Literature Review on the Smart City Resources Analysis with Big Data Methodologies | |
| CN115689224A (en) | Taxi demand prediction method based on track semantization and graph convolution network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |