CN104506352A - Internet-of-things data preprocessing method and system - Google Patents
Internet-of-things data preprocessing method and system Download PDFInfo
- Publication number
- CN104506352A CN104506352A CN201410808353.4A CN201410808353A CN104506352A CN 104506352 A CN104506352 A CN 104506352A CN 201410808353 A CN201410808353 A CN 201410808353A CN 104506352 A CN104506352 A CN 104506352A
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- things
- event
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention relates to an internet-of-things data preprocessing method and system. The system comprises a user authority check module, a data and event filter module and a data and event validity check module, wherein the user authority check module is used for checking access legality, giving an access sub-network or a user corresponding authority according to an authority configuration file and refusing illegal access requests to guarantee the access safety of an internet-of-things server; the data and event filter module is used for only remaining sense data and event notices meeting requirements and discarding waste data according to a data filter configuration file; the data and event validity check module is used for checking data validity item by item, performing specified preprocessing operation according to a validity configuration file and saving legal data to a database after data preprocessing. The method is simple, and the error rate of the system can be reduced.
Description
Technical field
The present invention relates to a kind of method and system of Internet of Things data prediction.
Background technology
Internet of Things is that equipment interior is among a small circle formed self-organizing network by specific procotol, and the center of Internet of Things is an aggregation node, and this node generally has the concatenation ability with outside public network (Internet etc.).Internet of Things mainly solves the networking issue of stand-alone device, and networking does not rely on global network, the proprietary network of general composition self-organizing.If Internet of Things needs any wide-area deployment, the mode generally taked is, use Internet of Things networking among a small circle, then aggregation node and the central server of each Internet of Things subnet networks, the network of a finally formation trans-regional deployment.
Internet of Things is fragile network, and Internet of Things subnet may break down, and Internet of Things sub-network data may occur transmission abnormality, and subnet monitoring difficulty, network configuration may change at any time.Central server will process this data, needs the data prediction mechanism of complete set, to reduce the probability of system fault.
Summary of the invention
The object of this invention is to provide a kind of method of Internet of Things data prediction, the error rate of Internet of things system can be reduced.
The present invention adopts following scheme to realize: a kind of method of Internet of Things data prediction, is characterized in that comprising the following steps:
Step S11: to access validity checking, and according to competence profile, give access subnet or the corresponding authority of user, refuse illegal access request, ensure the access security of Internet of Things server;
Step S12: adopt Internet of Things data filtering rule, filter sensing data illegal in sensing data; Adopt Internet of Things event filtering rule, filter out illegal event message; Adopt user right and the business rule of correspondence, filter illegal user's inquiry request;
Step S13: according to Internet of Things data and event type, the form of checking data or event, for the form that can not normally identify, reporting system keeper writing system Log, data abandon or back-up processing; Can identify for form, but the organizational form of data and Internet of Things server require different, need the format adaptation work doing Internet of Things data and event; Finally the Internet of Things valid data after verification and event are submitted to database.
In an embodiment of the present invention, described different Data Source comprise Internet of Things subnet sensing data, for the event notice of Internet of Things subnet and booking reader's inquiry request; For the sensing data of Internet of Things subnet, need to check whether sub-network authorizes access, and there is the authority uploading sensing data, give authorisation network sensing data upload permissions; For the event notice of Internet of Things subnet, need to check whether sub-network authorizes access, and there is event notice report authority, give authorisation network event notice and report authority; For booking reader's inquiry request, need subscriber authentication, rightness of business is verified, only accepts the inquiry request in the mandate business of validated user, refuses unauthorized service inquiry request.
In an embodiment of the present invention, described illegal sensing data comprises transfer of data distortion, data formatting error and value data distortion.
Another object of the present invention is to provide a kind of Internet of Things data pretreatment, it is characterized in that comprising: user right correction verification module, to access validity checking, and according to competence profile, give access subnet or the corresponding authority of user, refuse illegal access request, ensure the access security of Internet of Things server; Data and event filtering module, according to data filtering configuration file, only retain satisfactory sensing data and event notice, abandon junk data; And data and event validity checking module, according to validity configuration file, check that data validity also makes the pretreatment operation specified item by item; After data prediction, legal data are saved in database.
The present invention conceives simply, and the compounding practice of three modules just can achieve a butt joint into data submission etc. to the process of misdata, valid data from verification, farthest can reduce the error rate of system.
Accompanying drawing explanation
Fig. 1 is the general frame of data prediction of the present invention.
Fig. 2 is user right correction verification module block diagram of the present invention.
Fig. 3 is data and event filtering module frame chart.
Fig. 4 is data and event validity correction verification module block diagram.
Embodiment
For enabling above-mentioned purpose of the present invention, feature and advantage more become apparent, and are described in detail the specific embodiment of the present invention below in conjunction with accompanying drawing.
Set forth detail in the following description so that fully understand the present invention.But the present invention can be different from alternate manner described here to implement with multiple, those skilled in the art can when without prejudice to doing similar popularization when intension of the present invention.Therefore the present invention is not by the restriction of following public embodiment.
Internet of Things data prediction engine can be deployed in Internet of Things data center server, may be deployed in things-internet gateway node (correspondence with foreign country node).The object of Internet of Things data prediction comprises three kinds: the sensing data of Internet of Things subnet; The event notice of Internet of Things subnet and the query manipulation (query event) of booking reader.For above-mentioned three kinds of data types, data prediction engine carries out preliminary treatment through following process:
Step S11: to access validity checking, and according to competence profile, give access subnet or the corresponding authority of user, refuse illegal access request, ensure the access security of Internet of Things server;
Step S12: adopt Internet of Things data filtering rule, filter sensing data illegal in sensing data; Adopt Internet of Things event filtering rule, filter out illegal event message; Adopt user right and the business rule of correspondence, filter illegal user's inquiry request;
Step S13: according to Internet of Things data and event type, the form of checking data or event, for the form that can not normally identify, reporting system keeper writing system Log, data abandon or back-up processing; Can identify for form, but the organizational form of data and Internet of Things server require different, need the format adaptation work doing Internet of Things data and event; Finally the Internet of Things valid data after verification and event are submitted to database.
Concrete, refer to Fig. 1, the present embodiment provides a kind of Internet of Things data pretreatment, and this system comprises: (1) user right correction verification module, and effect user validation and rightness of business, refuse illegal request of data; (2) data and event filtering module, according to data filtering configuration file, only retains satisfactory sensing data and event notice, abandons junk data; (3) data and event validity checking module, according to validity configuration file, checks that data validity also makes the pretreatment operation (span standardization etc.) specified item by item.After data prediction, legal data are saved in database.
User right correction verification module major function is access validity checking, and according to competence profile, gives access subnet or the corresponding authority of user, refuse illegal access request, ensures the access security of Internet of Things server.For different Data Sources, authorization check takes different strategies.For the sensing data of Internet of Things subnet, need to check whether sub-network authorizes access, and there is the authority uploading sensing data, give authorisation network sensing data upload permissions; For the event notice of Internet of Things subnet, need to check whether sub-network authorizes access, and there is event notice report authority, give authorisation network event notice and report authority; For booking reader's inquiry request, need subscriber authentication, rightness of business is verified, only accepts the inquiry request in the mandate business of validated user, refuses unauthorized service inquiry request.The block diagram of user right correction verification module as shown in Figure 2.
Data and event filtering module, for tentatively filtering invalid data (transfer of data distortion, data type mistake, data formatting error, event type mistake, event format mistake, value data distortion, false data), reduce follow-up data processing difficulty and error probability, improve the stability of Internet of Things server.Adopt Internet of Things data filtering rule, filter sensing data (transfer of data distortion, data formatting error, value data distortion etc.) illegal in sensing data; Adopt Internet of Things event filtering rule, filter out illegal event message; Adopt user right and the business rule of correspondence, filter illegal user's inquiry request.Internet of Things sensing data data and event filtering module frame chart are as shown in Figure 3.
Internet of Things authorization check module and data filtering module can stop most unauthorized access, but cannot stop the data of distortion in Lawful access.Data can be caused to store for the data of distortion or incomplete data and data analysis makes mistakes.Internet of Things data and the effect of event validity correction verification module are exactly correct the distortion data in Lawful access.Concrete steps are as follows: according to Internet of Things data and event type, the form of checking data or event, and for the form that can not normally identify, reporting system keeper writing system Log, data abandon or back-up processing.Can identify for form, but the organizational form of data and Internet of Things server require different, need the format adaptation work doing Internet of Things data and event.External module is submitted in the final step of data validation.Data and event validity correction verification module, as shown in Figure 4.
Although the present invention with preferred embodiment openly as above; but it is not for limiting the present invention; any those skilled in the art without departing from the spirit and scope of the present invention; the Method and Technology content of above-mentioned announcement can be utilized to make possible variation and amendment to technical solution of the present invention; therefore; every content not departing from technical solution of the present invention; the any simple modification done above embodiment according to technical spirit of the present invention, equivalent variations and modification, all belong to the protection range of technical solution of the present invention.The foregoing is only preferred embodiment of the present invention, all equalizations done according to the present patent application the scope of the claims change and modify, and all should belong to covering scope of the present invention.
Claims (8)
1. a method for Internet of Things data prediction, is characterized in that comprising the following steps:
Step S11: to access validity checking, and according to competence profile, give access subnet or the corresponding authority of user, refuse illegal access request, ensure the access security of Internet of Things server;
Step S12: adopt Internet of Things data filtering rule, filter sensing data illegal in sensing data; Adopt Internet of Things event filtering rule, filter out illegal event message; Adopt user right and the business rule of correspondence, filter illegal user's inquiry request;
Step S13: according to Internet of Things data and event type, the form of checking data or event, for the form that can not normally identify, reporting system keeper writing system Log, data abandon or back-up processing; Can identify for form, but the organizational form of data and Internet of Things server require different, need the format adaptation work doing Internet of Things data and event; Finally the Internet of Things valid data after verification and event are submitted to database.
2. the method for Internet of Things data prediction according to claim 1, is characterized in that: described different Data Source comprise Internet of Things subnet sensing data, for the event notice of Internet of Things subnet and booking reader's inquiry request; For the sensing data of Internet of Things subnet, need to check whether sub-network authorizes access, and there is the authority uploading sensing data, give authorisation network sensing data upload permissions; For the event notice of Internet of Things subnet, need to check whether sub-network authorizes access, and there is event notice report authority, give authorisation network event notice and report authority; For booking reader's inquiry request, need subscriber authentication, rightness of business is verified, only accepts the inquiry request in the mandate business of validated user, refuses unauthorized service inquiry request.
3. the method for Internet of Things data prediction according to claim 1, is characterized in that: described illegal sensing data comprises transfer of data distortion, data formatting error and value data distortion.
4. an Internet of Things data pretreatment, is characterized in that comprising:
User right correction verification module, to access validity checking, and according to competence profile, gives access subnet or the corresponding authority of user, refuses illegal access request, ensure the access security of Internet of Things server;
Data and event filtering module, according to data filtering configuration file, only retain satisfactory sensing data and event notice, abandon junk data; And
Data and event validity checking module, according to validity configuration file, check that data validity also makes the pretreatment operation specified item by item; After data prediction, legal data are saved in database.
5. Internet of Things data pretreatment according to claim 4, is characterized in that: described different Data Source comprise Internet of Things subnet sensing data, for the event notice of Internet of Things subnet and booking reader's inquiry request; For the sensing data of Internet of Things subnet, need to check whether sub-network authorizes access, and there is the authority uploading sensing data, give authorisation network sensing data upload permissions; For the event notice of Internet of Things subnet, need to check whether sub-network authorizes access, and there is event notice report authority, give authorisation network event notice and report authority; For booking reader's inquiry request, need subscriber authentication, rightness of business is verified, only accepts the inquiry request in the mandate business of validated user, refuses unauthorized service inquiry request.
6. Internet of Things data pretreatment according to claim 4, is characterized in that: described data and event filtering module adopt Internet of Things data filtering rule, filter sensing data illegal in sensing data; Adopt Internet of Things event filtering rule, filter out illegal event message; Adopt user right and the business rule of correspondence, filter illegal user's inquiry request.
7. Internet of Things data pretreatment according to claim 4, is characterized in that: described illegal sensing data comprises transfer of data distortion, data formatting error and value data distortion.
8. Internet of Things data pretreatment according to claim 4, it is characterized in that: described data and event validity checking module are according to Internet of Things data and event type, the form of checking data or event, for the form that can not normally identify, reporting system keeper writing system Log, data abandon or back-up processing; Can identify for form, but the organizational form of data and Internet of Things server require different, need the format adaptation work doing Internet of Things data and event; Finally the Internet of Things valid data after verification and event are submitted to database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410808353.4A CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410808353.4A CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104506352A true CN104506352A (en) | 2015-04-08 |
| CN104506352B CN104506352B (en) | 2018-04-20 |
Family
ID=52948067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410808353.4A Expired - Fee Related CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104506352B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106454836A (en) * | 2015-08-06 | 2017-02-22 | 中兴通讯股份有限公司 | Method and device for enhancing use security of equipment certificate |
| CN107360240A (en) * | 2017-07-20 | 2017-11-17 | 广东小天才科技有限公司 | A kind of data-updating method and system |
| CN109224999A (en) * | 2018-09-29 | 2019-01-18 | 湖北航鹏化学动力科技有限责任公司 | Vibration mixing apparatus control method and system based on Internet of Things |
| CN110113421A (en) * | 2019-05-08 | 2019-08-09 | 西南民族大学 | A kind of big data information processing system based on Internet of Things |
| CN110800261A (en) * | 2017-06-27 | 2020-02-14 | 亚马逊科技公司 | Model and filter deployment across IOT networks |
| CN111142493A (en) * | 2020-01-10 | 2020-05-12 | 汉威科技集团股份有限公司 | Heterogeneous data acquisition device and method for industrial safety supervision |
| CN112565233A (en) * | 2020-11-30 | 2021-03-26 | 上海瑞眼科技有限公司 | Data verification method and system for fire-fighting Internet of things access platform |
| US11088820B2 (en) | 2017-06-27 | 2021-08-10 | Amazon Technologies, Inc. | Secure models for IoT devices |
| US11350360B2 (en) | 2017-06-27 | 2022-05-31 | Amazon Technologies, Inc. | Generating adaptive models for IoT networks |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102195991A (en) * | 2011-06-28 | 2011-09-21 | 辽宁国兴科技有限公司 | Terminal security management and authentication method and system |
| CN103078947A (en) * | 2013-01-11 | 2013-05-01 | 江苏物联网研究发展中心 | Mass data exchange and transaction architecture system of internet of things |
| CN103139058A (en) * | 2013-01-28 | 2013-06-05 | 公安部第一研究所 | Internet of things security access gateway |
| WO2013181980A1 (en) * | 2012-06-06 | 2013-12-12 | 深圳光启创新技术有限公司 | Handshake synchronization method and system based on visible light communication |
-
2014
- 2014-12-24 CN CN201410808353.4A patent/CN104506352B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102195991A (en) * | 2011-06-28 | 2011-09-21 | 辽宁国兴科技有限公司 | Terminal security management and authentication method and system |
| WO2013181980A1 (en) * | 2012-06-06 | 2013-12-12 | 深圳光启创新技术有限公司 | Handshake synchronization method and system based on visible light communication |
| CN103078947A (en) * | 2013-01-11 | 2013-05-01 | 江苏物联网研究发展中心 | Mass data exchange and transaction architecture system of internet of things |
| CN103139058A (en) * | 2013-01-28 | 2013-06-05 | 公安部第一研究所 | Internet of things security access gateway |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106454836A (en) * | 2015-08-06 | 2017-02-22 | 中兴通讯股份有限公司 | Method and device for enhancing use security of equipment certificate |
| CN110800261A (en) * | 2017-06-27 | 2020-02-14 | 亚马逊科技公司 | Model and filter deployment across IOT networks |
| US11088820B2 (en) | 2017-06-27 | 2021-08-10 | Amazon Technologies, Inc. | Secure models for IoT devices |
| CN110800261B (en) * | 2017-06-27 | 2022-01-11 | 亚马逊科技公司 | Model and filter deployment across IOT networks |
| US11350360B2 (en) | 2017-06-27 | 2022-05-31 | Amazon Technologies, Inc. | Generating adaptive models for IoT networks |
| CN107360240A (en) * | 2017-07-20 | 2017-11-17 | 广东小天才科技有限公司 | A kind of data-updating method and system |
| CN109224999A (en) * | 2018-09-29 | 2019-01-18 | 湖北航鹏化学动力科技有限责任公司 | Vibration mixing apparatus control method and system based on Internet of Things |
| CN110113421A (en) * | 2019-05-08 | 2019-08-09 | 西南民族大学 | A kind of big data information processing system based on Internet of Things |
| CN111142493A (en) * | 2020-01-10 | 2020-05-12 | 汉威科技集团股份有限公司 | Heterogeneous data acquisition device and method for industrial safety supervision |
| CN112565233A (en) * | 2020-11-30 | 2021-03-26 | 上海瑞眼科技有限公司 | Data verification method and system for fire-fighting Internet of things access platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104506352B (en) | 2018-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104506352A (en) | Internet-of-things data preprocessing method and system | |
| CN109714174B (en) | Internet of things equipment digital identity management system and method based on block chain | |
| CN109729180B (en) | Whole system intelligent community platform | |
| CN103179130B (en) | A kind of information system intranet security management platform and management method | |
| WO2021008560A1 (en) | Mobile application security analysis method based on blockchain technology | |
| CN111930723B (en) | Scientific and technological achievement data fusion method based on big data | |
| KR20190136011A (en) | Core network access provider | |
| CN103441926B (en) | Security gateway system of numerically-controllmachine machine tool network | |
| CN104796969A (en) | Network accessing method and system for equipment of internet of things on basis of recognition codes of internet of things | |
| US20180211462A1 (en) | System and method for entry access control using radio frequency communication | |
| CN102255924A (en) | Multi-stage security interconnection platform based on trusted computing and processing flow thereof | |
| CN101587576A (en) | Public inquiring and supervising system of public security cases | |
| CN102868702B (en) | System login device and system login method | |
| CN102244664A (en) | Multistage interconnection safety management centre subsystem of multistage safety interconnection platform | |
| CN107832602B (en) | Unified electronic seal system based on identification | |
| CN102307197A (en) | Trusted enhancement subsystem of multilevel security intercommunication platform | |
| CN113630409B (en) | Abnormal flow identification method based on DNS analysis flow and IP flow fusion analysis | |
| CN106302498B (en) | A kind of database access firewall system based on login parameters | |
| CN104468818A (en) | Service processing system and method for internet of things | |
| CN111327602B (en) | Equipment access processing method, equipment and storage medium | |
| CN102299926A (en) | Data exchange prepositioning subsystem of multistage safe interconnection platform | |
| CN113033175A (en) | Standard effectiveness evaluation method and system | |
| CN104104526A (en) | Network logging-on behavior monitoring method, device and system | |
| CN110855602B (en) | Internet of things cloud platform event identification method and system | |
| CN111478773A (en) | Registration method of distributed assets of Internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180420 Termination date: 20201224 |