CN104507107A - Method for preprocessing signaling data - Google Patents
Method for preprocessing signaling data Download PDFInfo
- Publication number
- CN104507107A CN104507107A CN201410791579.8A CN201410791579A CN104507107A CN 104507107 A CN104507107 A CN 104507107A CN 201410791579 A CN201410791579 A CN 201410791579A CN 104507107 A CN104507107 A CN 104507107A
- Authority
- CN
- China
- Prior art keywords
- signaling
- data
- buffer
- file
- signaling data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011664 signaling Effects 0.000 title claims abstract description 125
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000007781 pre-processing Methods 0.000 title abstract description 8
- 230000008569 process Effects 0.000 claims abstract description 21
- 238000004140 cleaning Methods 0.000 claims abstract description 17
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 4
- 230000008878 coupling Effects 0.000 claims description 3
- 238000010168 coupling process Methods 0.000 claims description 3
- 238000005859 coupling reaction Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002224 dissection Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000012010 growth Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Communication Control (AREA)
Abstract
The invention discloses a method for preprocessing signaling data, which comprises the following steps: monitoring newly added signaling files of the FTP interface machine of the signaling acquisition layer, transmitting signaling data to the Buffer of the signaling sharing layer, and cleaning and converting the signaling data in the Buffer. Compared with the prior art, the method for preprocessing the signaling data has the advantages that the data preprocessing operation is carried out before the signaling data are transmitted from the signaling acquisition layer to the signaling sharing layer and are written into the file, the secondary reading of the signaling file in the preprocessing process is avoided, the time for processing the whole signaling data is effectively saved, and the practicability is high.
Description
Technical field
The present invention relates to mobile communications industry monitoring signaling technology, specifically a kind of preprocess method of practical, signaling data.
Background technology
The continuous speedup of development of mobile communications network, various kinds of equipment mixed networking in network, network configuration is increasingly sophisticated, in net, class of business is also constantly updated, along with network size constantly expand, number of users increase, per minutely all can produce a large amount of signaling datas, higher requirement is proposed to the collection of signaling information, warehousing and treatment effeciency etc.
In a communication signaling monitoring platform, signal collecting layer, mainly from mobile radio communication, obtain original signaling data, and the data needed for transmitting to convergence device, by modes such as shunting, light splitting, shunt, realize the collection to the original signalling coding transmitted in existing network, and adopt the form of regulation to carry out the transmission of encapsulation backward signalling inclusion layer, in order to dissection process, possesses certain storage capacity.Signaling inclusion layer, major function receives to send from acquisition layer convergence device the signaling data arrived, and carries out data cleansing and put in storage according to specific format, and resolve full dose signaling data.
Technical Architecture traditionally, signaling data is mainly aggregated on FTP (File Transfer Protocol) interface message processor (IMP) at signal collecting layer, in time having being generated of new signaling data file, all signaling file to be sent to data sharing layer by FTP interface message processor (IMP) by FTP host-host protocol, and then again read the signaling file transmitted and carry out the preliminary treatment such as data cleansing.Due to the growth of mobile subscriber and the expansion of communication service, signaling data can be increasing, conventional method is difficult to the demand meeting real-time process, based on this, a kind of preprocess method that effectively can solve the signaling data of an above-mentioned difficult problem is now provided, the method avoids the secondary of signaling file in preprocessing process to read, and effectively saves the time of whole signaling data process.
Summary of the invention
Technical assignment of the present invention is for above weak point, provides a kind of preprocess method of practical, signaling data.
A preprocess method for signaling data, its specific implementation process is:
1) the newly-increased signaling file of the FTP interface message processor (IMP) of signal collecting layer is monitored;
2) signaling data is transferred to the Buffer of signaling inclusion layer: first signaling data is written in Buffer temporary before being transferred to signaling inclusion layer, and then signaling data adopts File Transfer Protocol transmission;
3) signaling data cleaning and conversion in Buffer: the newline of signaling data temporary in coupling Buffer, obtain data line, field segmentation is carried out to each row of data, then obtain field cleaning transformation rule and data cleansing and conversion operations are carried out to each row of data, then the data after cleaning and conversion are written to signaling inclusion layer local file;
4) releasing memory Buffer, jumps to step 1), continues the monitoring FTP interface message processor (IMP) of signal collecting layer being carried out to newly-increased file.
The detailed process of described step 1) is: whether have new signaling file generated in timing monitoring signal collecting layer FTP interface message processor (IMP), if had, obtain newly-increased file name, newly-built signaling file of the same name in the storage of signaling inclusion layer, then log-on data extracts task.
Described step 2) detailed process be:
One, one piece of internal memory Buffer is opened up at signaling inclusion layer;
Two, by FTP host-host protocol, the newly-increased file in signal collecting layer FTP interface message processor (IMP) is transferred in internal memory Buffer by the form of data flow, fills up Buffer, then perform step 3) and the data in Buffer are cleaned and changed; If Buffer does not fill up, illustrate that the signaling file in signal collecting layer has read complete, then perform step 3) and step 4);
Three, when signal collecting layer signaling file does not read complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer are discontented with a line, empty Buffer, the data of these discontented a line are stored in Buffer again, then perform step one, otherwise perform step 4;
Signal collecting layer signaling file reads complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer meet a line, perform step 4, otherwise the signaling data less than a line is written in error log, then jumps to step 4);
Four, the data line of acquisition is split field according to field delimiter.
The detailed process of described step 3) is: obtain signaling data cleaning transformation rule, field cleaning and conversion are carried out to the data after field segmentation, change and successfully then the form of this line number document flow according to this write in the newly-built signaling file of step 1), error reason is then write in error log by convert failed; When signal collecting layer signaling file does not read complete, finally jump to step 3, otherwise jump to step 4).The preprocess method of a kind of signaling data of the present invention, has the following advantages:
The preprocess method of a kind of signaling data of this invention can shorten the pretreated flow process of whole signaling, improves treatment effeciency; Signaling data is transferred to signaling inclusion layer from signal collecting layer and namely carried out data prediction operation before writing in files, avoid the secondary of signaling file in preprocessing process to read, effectively save the time of whole signaling data process, practical, applied widely, be easy to promote.
Accompanying drawing explanation
Accompanying drawing 1 is preliminary treatment realization flow figure of the present invention.
Embodiment
Below in conjunction with the drawings and specific embodiments, the invention will be further described.
After traditional signaling file is transferred to signaling inclusion layer by the FTP interface message processor (IMP) of signal collecting layer, again read the signaling file transmitted, the operation of signaling being cleaned and changing, the present invention proposes a kind of preprocess method of signaling data, the method considers that signaling is transferred to signaling inclusion layer from FTP interface message processor (IMP) and new files stream the data transmitted can be written to the feature of file by document flow, first be written to before data writing in files in internal memory Buffer, data in internal memory Buffer are carried out to the preliminary treatment such as cleaning, then the data in Buffer are written to the file of data sharing layer by document flow, the method can shorten the pretreated flow process of whole signaling, improve treatment effeciency.Namely signaling data is transferred to signaling inclusion layer from signal collecting layer namely carried out data prediction operation before writing in files, avoids the secondary of signaling file in preprocessing process to read, effectively saves the time of whole signaling data process.As shown in Figure 1, its specific implementation process is:
1) the newly-increased signaling file of the FTP interface message processor (IMP) of signal collecting layer is monitored;
2) signaling data is transferred to the Buffer of signaling inclusion layer: first signaling data is written in Buffer temporary before being transferred to signaling inclusion layer, and then signaling data adopts File Transfer Protocol transmission;
3) signaling data cleaning and conversion in Buffer: the newline of signaling data temporary in coupling Buffer, obtain data line, field segmentation is carried out to each row of data, then obtain field cleaning transformation rule and data cleansing and conversion operations are carried out to each row of data, then the data after cleaning and conversion are written to signaling inclusion layer local file;
4) releasing memory Buffer, jumps to step 1), continues the monitoring FTP interface message processor (IMP) of signal collecting layer being carried out to newly-increased file.
Further, the detailed process of above-mentioned steps is:
1, whether have new signaling file generated in timing monitoring signal collecting layer FTP interface message processor (IMP), if had, obtain newly-increased file name, newly-built signaling file of the same name in the storage of signaling inclusion layer, then log-on data extracts task;
2, one piece of internal memory Buffer is opened up at signaling inclusion layer;
3, by FTP host-host protocol, the newly-increased file in signal collecting layer FTP interface message processor (IMP) is transferred in internal memory Buffer by the form of data flow, fills up Buffer, then perform step 4-6 and the data in Buffer are cleaned and changed; If Buffer does not fill up, illustrate that the signaling file in signal collecting layer has read complete, then perform step 4-7;
4, signal collecting layer signaling file does not read complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer are discontented with a line, empty Buffer, the data of these discontented a line are stored in Buffer again, then perform step 3, otherwise perform step 5;
Signal collecting layer signaling file reads complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer meet a line, perform step 5, otherwise the signaling data less than a line is written in error log, then jumps to step 7;
5, the data line of acquisition is split field according to field delimiter;
6, signaling data cleaning transformation rule is obtained, field cleaning and conversion are carried out to the data after field segmentation, change and successfully then the form of this line number document flow according to this write in the newly-built signaling file of step 1, error reason is then write in error log by convert failed;
If signal collecting layer signaling file does not read complete, finally jump to step 4, otherwise jump to step 7;
7, releasing memory Buffer, jumps to step 1, continues the monitoring FTP interface message processor (IMP) of signal collecting layer being carried out to newly-increased file.
Above-mentioned embodiment is only concrete case of the present invention; scope of patent protection of the present invention includes but not limited to above-mentioned embodiment; claims of the preprocess method of any a kind of signaling data according to the invention and the those of ordinary skill of any described technical field to its suitable change done or replacement, all should fall into scope of patent protection of the present invention.
Claims (4)
1. a preprocess method for signaling data, is characterized in that, its specific implementation process is:
1) the newly-increased signaling file of the FTP interface message processor (IMP) of signal collecting layer is monitored;
2) signaling data is transferred to the Buffer of signaling inclusion layer: first signaling data is written in Buffer temporary before being transferred to signaling inclusion layer, and then signaling data adopts File Transfer Protocol transmission;
3) signaling data cleaning and conversion in Buffer: the newline of signaling data temporary in coupling Buffer, obtain data line, field segmentation is carried out to each row of data, then obtain field cleaning transformation rule and data cleansing and conversion operations are carried out to each row of data, then the data after cleaning and conversion are written to signaling inclusion layer local file;
4) releasing memory Buffer, jumps to step 1), continues the monitoring FTP interface message processor (IMP) of signal collecting layer being carried out to newly-increased file.
2. the preprocess method of a kind of signaling data according to claim 1, it is characterized in that, the detailed process of described step 1) is: whether have new signaling file generated in timing monitoring signal collecting layer FTP interface message processor (IMP), if had, obtain newly-increased file name, newly-built signaling file of the same name in the storage of signaling inclusion layer, then log-on data extracts task.
3. the preprocess method of a kind of signaling data according to claim 2, is characterized in that, described step 2) detailed process be:
One, one piece of internal memory Buffer is opened up at signaling inclusion layer;
Two, by FTP host-host protocol, the newly-increased file in signal collecting layer FTP interface message processor (IMP) is transferred in internal memory Buffer by the form of data flow, fills up Buffer, then perform step 3) and the data in Buffer are cleaned and changed; If Buffer does not fill up, illustrate that the signaling file in signal collecting layer has read complete, then perform step 3) and step 4);
Three, when signal collecting layer signaling file does not read complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer are discontented with a line, empty Buffer, the data of these discontented a line are stored in Buffer again, then perform step one, otherwise perform step 4;
Signal collecting layer signaling file reads complete: obtain by the mode finding newline a line signaling data be stored in internal memory Buffer, if the data in Buffer meet a line, perform step 4, otherwise the signaling data less than a line is written in error log, then jumps to step 4);
Four, the data line of acquisition is split field according to field delimiter.
4. the preprocess method of a kind of signaling data according to claim 3, it is characterized in that, the detailed process of described step 3) is: obtain signaling data cleaning transformation rule, field cleaning and conversion are carried out to the data after field segmentation, change and successfully then the form of this line number document flow according to this write in the newly-built signaling file of step 1), error reason is then write in error log by convert failed; When signal collecting layer signaling file does not read complete, finally jump to step 3, otherwise jump to step 4).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410791579.8A CN104507107B (en) | 2014-12-19 | 2014-12-19 | Method for preprocessing signaling data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410791579.8A CN104507107B (en) | 2014-12-19 | 2014-12-19 | Method for preprocessing signaling data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104507107A true CN104507107A (en) | 2015-04-08 |
| CN104507107B CN104507107B (en) | 2018-06-19 |
Family
ID=52948808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410791579.8A Active CN104507107B (en) | 2014-12-19 | 2014-12-19 | Method for preprocessing signaling data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104507107B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430534A (en) * | 2015-10-29 | 2016-03-23 | Tcl集团股份有限公司 | Intelligent device reporting data method and system |
| CN109561389A (en) * | 2018-12-28 | 2019-04-02 | 海南中智信信息技术有限公司 | A method of based on flow of the people real-time in mobile phone signaling zoning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769547A (en) * | 2011-05-03 | 2012-11-07 | ***通信集团浙江有限公司 | System and method for providing signaling data |
| CN103346905A (en) * | 2013-06-14 | 2013-10-09 | 吴建进 | Method and device for analyzing signaling |
| CN103607329A (en) * | 2013-12-04 | 2014-02-26 | 北京中创信测科技股份有限公司 | IMS network any end-to-end media QoS monitoring method and system |
| CN203466974U (en) * | 2013-09-04 | 2014-03-05 | 河北烽联信息技术有限公司 | Equipment for mobile communication network signaling monitoring system |
-
2014
- 2014-12-19 CN CN201410791579.8A patent/CN104507107B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769547A (en) * | 2011-05-03 | 2012-11-07 | ***通信集团浙江有限公司 | System and method for providing signaling data |
| CN103346905A (en) * | 2013-06-14 | 2013-10-09 | 吴建进 | Method and device for analyzing signaling |
| CN203466974U (en) * | 2013-09-04 | 2014-03-05 | 河北烽联信息技术有限公司 | Equipment for mobile communication network signaling monitoring system |
| CN103607329A (en) * | 2013-12-04 | 2014-02-26 | 北京中创信测科技股份有限公司 | IMS network any end-to-end media QoS monitoring method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430534A (en) * | 2015-10-29 | 2016-03-23 | Tcl集团股份有限公司 | Intelligent device reporting data method and system |
| CN105430534B (en) * | 2015-10-29 | 2019-12-10 | Tcl集团股份有限公司 | method and system for reporting data by intelligent equipment |
| CN109561389A (en) * | 2018-12-28 | 2019-04-02 | 海南中智信信息技术有限公司 | A method of based on flow of the people real-time in mobile phone signaling zoning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104507107B (en) | 2018-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101039212B (en) | Fast data storage method | |
| CN102983884B (en) | A kind of method by magnetic channel transmission data | |
| CN102724122B (en) | Reliable serial port transmission method and device based on hardware system | |
| CN111064726B (en) | Method and system for realizing data conversion between GOOSE protocol and HDLC protocol | |
| CN102480335A (en) | Method and system for transmitting business data | |
| CN108566674A (en) | Automatic networking register method based on the LORA communication technologys | |
| CN101163148A (en) | Ethernet transmission method of supporting non-standard velocity and corresponding device and access equipment | |
| CN104507107A (en) | Method for preprocessing signaling data | |
| CN101964745A (en) | Network forwarding equipment and message forwarding method thereof | |
| CN103581953A (en) | Association method for wireless drive test data and signaling monitoring data | |
| CN103442091A (en) | Data transmission method and device | |
| CN103532758B (en) | Be applicable to the configuration processing method of transmission of future generation, data equipment fusion | |
| CN111858103B (en) | Compatible realization method for communication of high and low versions of modules in software system | |
| CN104735097A (en) | Information collecting method and system | |
| CN101330503B (en) | Method, system and equipment for transmitting data of Ethernet physical layer | |
| CN103701715A (en) | Method and device for sending and receiving Ethernet data packet based on multiple E1 channels | |
| US10979345B2 (en) | Remote medium access control (MAC) based networks | |
| CN103118023B (en) | A kind of method and system of the data of transmission specification in a network | |
| CN101707525A (en) | Charging method and device | |
| CN103631716B (en) | The conversion method and equipment of a kind of debug command | |
| CN103532846A (en) | Route link selection method and device | |
| CN104243347A (en) | Data sending and receiving method and device based on SHDSL | |
| CN103731238A (en) | Data transmitting method and data transmitting device | |
| CN204928857U (en) | Unified communications management system | |
| CN109560894B (en) | Method and equipment for adapting transmission rate of repeater |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |