CN114827126B - IPTVCDN user play log reporting method and system - Google Patents

IPTVCDN user play log reporting method and system Download PDF

Info

Publication number
CN114827126B
CN114827126B CN202210296575.7A CN202210296575A CN114827126B CN 114827126 B CN114827126 B CN 114827126B CN 202210296575 A CN202210296575 A CN 202210296575A CN 114827126 B CN114827126 B CN 114827126B
Authority
CN
China
Prior art keywords
log
module
session
user
reporting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210296575.7A
Other languages
Chinese (zh)
Other versions
CN114827126A (en
Inventor
曾学俊
龚亮
刘艳秋
姚晶晶
杨尘
冯仁伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Comservice Enrising Information Technology Co Ltd
Original Assignee
China Comservice Enrising Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Comservice Enrising Information Technology Co Ltd filed Critical China Comservice Enrising Information Technology Co Ltd
Priority to CN202210296575.7A priority Critical patent/CN114827126B/en
Publication of CN114827126A publication Critical patent/CN114827126A/en
Application granted granted Critical
Publication of CN114827126B publication Critical patent/CN114827126B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/143Termination or inactivation of sessions, e.g. event-controlled end of session
    • H04L67/145Termination or inactivation of sessions, e.g. event-controlled end of session avoiding end of session, e.g. keep-alive, heartbeats, resumption message or wake-up for inactive or interrupted session
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/146Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2181Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses an IPTVCDN user play log reporting method and system, which judge whether the session of the read log file content survives or not, wait and merge the report of a plurality of logs belonging to the same session, prevent the phenomenon that the log generated by CDN nodes does not distinguish whether the current user is directly reporting or not, and cause that the log of the same session ID in a jump play scene is not processed when HTTP protocol is played, thereby causing inaccurate play calculation, effectively ensuring that the user play log reporting is accurate and effective, judging whether to report the log file content or not through the real-time index of the log file, dynamically outputting the report command of whether to report or not according to the real-time index of the log file, effectively generating the phenomena of uneven peak, log number and log size for CDN node logs, and timely and accurately reporting the user play log.

Description

IPTVCDN user play log reporting method and system
Technical Field
The invention relates to the technical field of communication, in particular to an IPTVCDN user play log reporting method and system.
Background
In the IPTV service, CDN nodes are distributed in various regions, after each play request of a user is finished, the CDN nodes immediately generate a log catalog for playing records to push stream nodes, and in the CDN general field, node log reporting mainly has 2 schemes: firstly, after a user requests record generation, the plug flow program stores the log data in a memory, and the log process of the program synchronously performs responsible log analysis, extraction and reporting. And secondly, the program writes the user log into a local disk, and other log processing programs directly report the whole file or report the file after local analysis is completed after reading the file. In the IPTV industry, CDN user play logs are an important basis for revenue settlement between an IPTV operator and a source provider (SP for short), where the number of user plays and the duration of user plays are the most important. The existing log reporting scheme mainly has the following problems:
1. session ID merging cannot be performed for IPTV traffic. The session ID is a core indicator of the access of the user to the CDN in the IPTV service, and the existence of the session ID represents whether the connection between the user and the CDN service still survives. When the HTTP protocol in IPTV plays, in the jump play scenes such as fast forward, fast backward, x times speed and pause, the CDN node generates a plurality of logs due to the fact that the HTTP play cannot keep connection, and after the jump play, the current log reporting method does not distinguish whether the current user plays the logs directly or not, and can not wait and combine the logs belonging to the same session ID to report, so that the log of the same session ID in the jump play scene has no treatment when the HTTP protocol plays. Such reporting would result in inaccurate CDN play single billing costs. Particularly, in the scene of fast forward and fast backward playing, the charging basis in the IPTV industry is that the playing time is longer than 5 minutes, the charging period is month, and the same user can be caused to generate a plurality of short playing records without combination, so that the accurate settlement of CDN playing media resources cannot be performed.
2. The reporting frequency cannot be dynamically adjusted according to real-time loads such as the network condition of CDN nodes, the number of logs, the size of the logs and the like. In the IPTV service, the distribution of the time period of the main playing video of the user is very uneven, the peak time period is generally 11-13 points and 18-21 points, one process is carried out in the same minute, thousands of logs are generated in the peak time period, and the average of the low peak time is tens of logs.
Disclosure of Invention
The invention aims to provide an IPTCDN user play log reporting method and system, which can not dynamically adjust reporting frequency according to real-time loads such as CDN node network conditions, log number, log size and the like by carrying out session ID merging on IPTV services, and can effectively solve the phenomena of peak, log number and log size nonuniformity generated by CDN node logs by judging whether the session of the log survives in real time, waiting and merging the multiple logs belonging to the same session, and reporting the log by judging whether the log file content is reported through real-time indexes of a log file.
The invention is realized by the following technical scheme:
the first aspect of the present invention provides a method for reporting an IPTVCDN user play log, which specifically includes the following steps:
s1, acquiring log file content under a CDN node log directory;
s2, circularly reading the content of the log file according to the row, cutting the log file according to the requirement of a storage format, and storing the cut log file;
s3, judging whether the session of the read log file content survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
s4, storing the log file content of the S3;
s5, judging whether to report the content of the log file according to the real-time index of the log file:
if the judgment result is reported, executing a step S6;
if the judgment result is not reported, executing the step S2;
s6, reporting the content of the log file.
According to the method and the device, whether the session of the read log file content survives or not is judged, a plurality of logs belonging to the same session are waited and reported in a combined mode, the phenomenon that the logs generated by CDN nodes are not distinguished whether the current user is playing and directly reported, and the logs of the same session ID in a jump playing scene are not processed when an HTTP protocol is played is avoided, so that the situation that playing calculation is inaccurate is caused, the fact that CDN service users play logs accurately and effectively report is effectively ensured, whether the log file content is reported is judged through real-time indexes of the log files, and the phenomenon that whether the height peaks, the number of the logs and the log size are uneven is generated for the CDN node by combining the real-time indexes of the log files and dynamically outputting the report command of whether the log file is provided is effectively achieved, so that the CDN user play logs can be timely and accurately reported.
Further, the S1 specifically includes:
s11, initializing a log process and acquiring initial configuration parameters;
s12, loading initial parameter configuration into a memory;
s13, acquiring a CDN node log directory;
s14, judging whether the log catalog has the log or not, and executing the step S2 until the judgment result is that the log catalog has the log.
Further, the step S2 specifically includes:
s21, circularly reading the content of the log file in the log storage directory according to the row, wherein the reading priority of the log file is the current log and the history log;
s22, analyzing the fields of the log file content and cutting the fields;
s23, judging whether the field is abnormal:
if the field is not abnormal, executing the step S3;
if the field is abnormal, the error log is stored in the local directory, and the content of the next log file is analyzed.
S25, storing the analyzed log file.
Further, the CDN session survival determination step specifically includes:
s31, acquiring a session value of a single log;
s32, judging whether the session is in the memory or not:
if the session is determined to be in the memory, executing step S33;
if the session is determined not to be in the memory, executing step S34;
s33, acquiring historical data of the session;
s34, judging whether session survives:
if the session is judged to be alive, merging the log and the existing data in the history memory and then storing the merged log;
if the session is judged to be ended, the log is analyzed, and the playing time and the starting and ending time of the existing log are combined.
Further, after the session is determined to survive, the method further includes:
judging whether to merge log reporting for the surviving session:
if the judgment result is that the logs are reported, the log is saved, the next log of the session is waited for, and the logs of the same session are merged;
and if the judgment result is that the log is not combined, reporting the log.
Further, the step S5 specifically includes:
s51, acquiring real-time indexes according to the content of the log file after analysis is completed;
s52, setting a threshold parameter for the real-time index;
and S53, combining the real-time index with the characteristic of the sign symbol function, and outputting a reporting instruction.
Further, the step S53 specifically includes:
report instruction
Figure BDA0003563615530000031
C-log center connectivity index, number of x-node connected log centers, sys metrics -user log real-time indicator, number of times NoS_Now-memory data has been sent, time difference between TD_Now-time since last report, number of existing log entries in NoL _Now-memory, SLFS_Now-single log file size, upper limit of NoS_set-memory data sending times, TD_set-report timeUpper difference limit, upper limit of number of log in NoL _set-memory, upper limit of size of slfs_set-single log file.
Further, the method further comprises the following steps: step S7, processing the local user log file, wherein the step S7 specifically comprises the following steps:
s71, moving the log which is successfully reported to obtain a moving result:
the log does not move, and the secondary loading is carried out in the next scanning;
moving the user log file to a local backup directory;
s72, scanning a backup catalog to check whether an expired file exists:
if the expired file exists, storing the expired file list, and circularly deleting until the whole expired file list is deleted.
The second aspect of the present invention provides an IPTVCDN user play log reporting system, which includes a system initialization module, a task scheduling module, a user log reading module and a user log reporting module:
the system initialization module includes: a parameter configuration module and a program log module;
the task scheduling module comprises: the timer management module and the report instruction generation module;
the user log reading module comprises: the system comprises a user log scanning module, a user log analyzing module, a session survival inquiring module and a user log storing module;
the user log processing module comprises: a user log reporting module and a local user log recovery module;
s1, a system initialization module obtains log file contents under a CDN node log directory;
s2, the timer management module circularly reads the content of the log file according to the row, cuts the log according to the requirement of a storage format, and stores the cut log in a memory;
s3, the session survival inquiry module judges whether session of the log file content read by the user log analysis module survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
s4, the user log saving module saves log file contents into a system memory;
s5, the user log reporting module reports the log file content in the user log storage module according to the reporting instruction generated by the reporting instruction generating module:
if the instruction is a sending index, executing step S6;
if the instruction is the temporary non-transmission index, executing step S2;
s6, a user log reporting module reports log file contents;
further, the program log module is used for initializing a log process, and the parameter configuration module is used for loading system configuration parameters into a memory;
the user log scanning module scans the log storage catalog according to rows after receiving the task scheduling instruction, and stores the log file names into a list for the subsequent modules to use;
the timer management module is used for acquiring log file contents at regular time;
the user log analysis module is used for cutting and analyzing the log file content acquired by the timer management module at regular time according to a certain rule and storing the log file content into an IPTV log standard format;
the session survival query module is configured to query whether session survives:
if the session is ended, the log is sent;
if session survives, the log record is stored in a memory, and the user finishes playing, namely the feedback is finished when the session ID is inquired, all playing time lengths of the session ID are combined, and the session ID is reported uniformly;
the user log saving module is used for saving the log which is already played in the log analyzing module, and feeding back the number of the saved log, the memory size and the retransmission times in real time;
the report instruction generation module is used for combining the real-time index with the characteristic of the sign symbol function to generate an instruction whether to report the information in the user log storage module;
the log reporting module is used for extracting data in the log record storage module after receiving the instruction sent by the task scheduling module, reporting the content of the log file and feeding back the reporting result to the task scheduling module;
and the local log recycling module is used for moving the log which is successfully reported to the local backup catalog after the successful reporting instruction is received, and cleaning the expired log in the local backup catalog at regular time.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the method solves the problem of log reporting of the skip play scene of the IPTVCDN user through the CDN session real-time judging method. And requesting in real time and judging whether the session is still making CDN data requests. If the user is still in service, the log will not be reported until the user finishes playing, the session end request is collected and the user playing time is reported once, so that the mode can not cause delay of log reporting, because the user is in normal playing practice, and meanwhile, the problem that the CDN user plays the log too short in time and settles inaccurately is solved;
2. by combining with real-time indexes of the log file, dynamically outputting a command whether to report, the phenomena of peak, log number and uneven log size are effectively generated for the CDN node logs. By the method, the log reporting is intelligently performed on the basis of the output result of the reporting instruction algorithm by combining the connectivity of the network, the number of logs in the current reporting memory, the size of the logs and the real-time requirement of the logs, so that the log reporting is stable, on time and accurate.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the drawings that are needed in the examples will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and that other related drawings may be obtained from these drawings without inventive effort for a person skilled in the art. In the drawings:
fig. 1 is a schematic diagram of an IPTVCDN user play log field in an embodiment of the present invention;
fig. 2 is a flowchart of reporting an IPTVCDN user play log in an embodiment of the present invention;
FIG. 3 is a schematic diagram of a system component module according to an embodiment of the present invention;
FIG. 4 is a flow chart of local log consumption in an embodiment of the invention;
FIG. 5 is a flow chart of single log parsing in an embodiment of the present invention;
fig. 6 is a flowchart of CDN session survival determination in an embodiment of the present invention;
FIG. 7 is a flowchart of an intelligent report instruction algorithm in an embodiment of the present invention;
FIG. 8 is a flow chart of the local user log reclamation module in an embodiment of the present invention.
Detailed Description
For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.
Example 1
As shown in fig. 2, a first aspect of this embodiment provides an IPTVCDN user play log reporting method, which specifically includes the following steps:
s1, acquiring log file content under a CDN node log directory;
s2, circularly reading the content of the log file according to the row, cutting the log file according to the requirement of a storage format, and storing the cut log file;
s3, judging whether the session of the read log file content survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
s4, storing the log file content of the S3;
s5, judging whether to report the content of the log file according to the real-time index of the log file:
if the judgment result is reported, executing a step S6;
if the judgment result is not reported, executing the step S2;
s6, reporting the content of the log file.
By judging whether the session of the read log file content survives or not, waiting, merging and reporting a plurality of logs belonging to the same session, the phenomenon that the logs generated by CDN nodes are not distinguished whether the current user is playing and directly reporting, and the logs of the same session ID in a jump playing scene are not processed when HTTP protocol is played is caused, so that inaccurate playing calculation is caused, the situation that CDN service users play logs are accurately and effectively reported is effectively ensured, whether the log file content is reported is judged through real-time indexes of the log files, and the phenomenon that whether the high-low peaks, the number of the logs and the uneven log size are generated for CDN node logs is effectively combined with the real-time indexes of the log files, so that the CDN user play logs can be timely and accurately reported.
In some possible embodiments, S1 specifically includes:
s11, initializing a log process and acquiring initial configuration parameters;
s12, loading initial parameter configuration into a memory;
s13, acquiring a CDN node log directory;
s14, judging whether the log catalog has the log or not, and executing the step S2 until the judgment result is that the log catalog has the log.
As shown in fig. 4, S2 specifically includes:
s21, circularly reading the content of the log file in the log storage directory according to the row, wherein the reading priority of the log file is the current log and the history log;
s22, analyzing the fields of the log file content and cutting the fields;
s23, judging whether the field is abnormal:
if the field is not abnormal, executing the step S3;
if the field is abnormal, the error log is stored in the local directory, and the content of the next log file is analyzed.
S25, storing the analyzed log file.
As shown in fig. 6, the CDN session survival determination step specifically includes:
s31, acquiring a session value of a single log;
s32, judging whether the session is in the memory or not:
if the session is determined to be in the memory, executing step S33;
if the session is determined not to be in the memory, executing step S34;
s33, acquiring historical data of the session;
s34, judging whether session survives:
if the session is judged to be alive, merging the log and the existing data in the history memory and then storing the merged log;
if the session is judged to be ended, the log is analyzed, and the playing time and the starting and ending time of the existing log are combined.
In some possible embodiments, after the session is determined to survive, the method further includes:
judging whether to merge log reporting for the surviving session:
if the judgment result is that the logs are reported, the log is saved, the next log of the session is waited for, and the logs of the same session are merged;
and if the judgment result is that the log is not combined, reporting the log.
As shown in fig. 7, S5 specifically includes:
s51, acquiring real-time indexes according to the content of the log file after analysis is completed;
s52, setting a threshold parameter for the real-time index;
and S53, combining the real-time index with the characteristic of the sign symbol function, and outputting a reporting instruction.
S53 specifically includes:
report instruction
Figure BDA0003563615530000081
C-log center connectivity index, number of x-node connected log centers, sys metrics -user log real-time index, number of times of data transmission in the NoS_Now-memory, time difference between TD_Now-time of last report, number of existing log in the NoL _Now-memory, SLFS_Now-single log file size, upper limit of data transmission in the NoS_set-memory, upper limit of time difference between TD_set-time of report, upper limit of number of log in the NoL _set-memory, upper limit of SLFS_set-single log file size.
As shown in fig. 8, further includes: step S7, processing the local user log file, wherein the step S7 specifically comprises the following steps:
s71, moving the log which is successfully reported to obtain a moving result:
the log does not move, and the secondary loading is carried out in the next scanning;
moving the user log file to a local backup directory;
s72, scanning a backup catalog to check whether an expired file exists:
if the expired file exists, storing the expired file list, and circularly deleting until the whole expired file list is deleted.
Example 2
As shown in fig. 2 and fig. 3, a second aspect of the present embodiment provides an IPTVCDN user play log reporting system, including a system initialization module, a task scheduling module, a user log reading module, and a user log reporting module:
the system initialization module comprises: a parameter configuration module and a program log module;
the task scheduling module comprises: the timer management module and the report instruction generation module;
the user log reading module comprises: the system comprises a user log scanning module, a user log analyzing module, a session survival inquiring module and a user log storing module;
the user log processing module comprises: a user log reporting module and a local user log recovery module;
s1, a system initialization module obtains log file contents under a CDN node log directory;
s2, the timer management module circularly reads the content of the log file according to the row, cuts the log according to the requirement of a storage format, and stores the cut log in a memory;
s3, the session survival inquiry module judges whether session of the log file content read by the user log analysis module survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
s4, the user log saving module saves log file contents into a system memory;
s5, the user log reporting module reports the log file content in the user log storage module according to the reporting instruction generated by the reporting instruction generating module:
if the instruction is a sending index, executing step S6;
if the instruction is the temporary non-transmission index, executing step S2;
s6, a user log reporting module reports log file contents;
the program log module is used for initializing a log process, and the parameter configuration module is used for loading system configuration parameters into the memory;
after receiving the task scheduling instruction, the user log scanning module scans the log storage directory according to rows, saves the log file names into a list and uses the list by the subsequent module;
the timer management module is used for acquiring log file contents at regular time;
the user log analysis module is used for cutting and analyzing the log file content acquired by the timer management module at regular time according to a certain rule and storing the log file content into an IPTV log standard format;
the session survival query module is used for querying whether the session survives:
if the session is ended, the log is sent;
if session survives, the log record is stored in a memory, and the user finishes playing, namely the feedback is finished when the session ID is inquired, all playing time lengths of the session ID are combined, and the session ID is reported uniformly;
the user log saving module is used for saving the log which is already played in the log analyzing module, feeding back the number of the saved log in real time, the memory size and the retransmission times;
the report instruction generation module is used for combining the real-time index with the characteristic of the sign symbol function to generate an instruction whether to report the information in the user log storage module;
the log reporting module is used for extracting data in the log record storage module after receiving the instruction sent by the task scheduling module, reporting the content of the log file and feeding back the reporting result to the task scheduling module;
the local log recovery module is used for moving the log which is successfully reported to the local backup catalog after the successful reporting instruction is received, and cleaning the expired log in the local backup catalog at regular time.
Example 3
As shown in the figure 2 of the drawings,
the system initialization module comprises: parameter configuration module and program log module:
the parameter configuration module is used for loading system configuration parameters into the memory, wherein the configuration parameters comprise: the method comprises the steps of a master-slave log center IP address, a master-slave port, a CDN system engineering name, a CDN service node service IP, a port, a log catalog, whether logs are combined, reporting frequency, single reporting size and the like. The real-time index comprises: the upper limit of the number of times of sending the memory data, namely, noS_set (NumberOfSend_set), the upper limit of the reporting time difference, namely, TD_set (TimeDifference_set), the upper limit of the number of log stripes in the memory, namely, noL _set (NumberOfLog_set), the upper limit of the size of a single log file, namely, SLFS_set (SingleLogFilesize_set), are Set in a program configuration file according to the performance of a CDN push node server, and parameters are configured as default: memory data transmission times upper limit nos_set=3, reporting time difference upper limit td_set=3 (minutes), log number upper limit in memory NoL _set=10000 (stripes), single log file size upper limit slfs_set=5 (Mb);
the program log module is used for initializing a log process, and is mainly used for initializing the program log process, an initialization task comprises basic configuration such as log preservation time, preservation size, record level and the like, after the initialization task is completed, the system automatically detects the condition of a local system log file, and performs file cutting, backup and deletion operations, and meanwhile, the whole system reporting system can call the log module at any position to record the program log;
the task scheduling module comprises: the timer management module and the report instruction generation module:
the timer management module is a trigger for running the whole program, and is determined by a timing interval, and the detection time interval can be set as follows: the log center connectivity check frequency is 10 seconds, the log reading module operation frequency is 1 minute and the history log deleting module operation frequency is 10 minutes;
the report instruction generation module is the core of the whole program and intelligently generates a data report instruction by combining various indexes in real time. The real-time index comprises: the method comprises the steps of a log center connectivity index C, the number of times NoS_Now (number of times of transmission of memory data), the time difference TD_Now (time difference_Now) from last time of reporting, the number of existing log bars NoL _Now (number of times of transmission of memory data) in a memory, a single log file size SLFS_Now (single log file size_Now), and outputting a result 0/1 by combining the characteristics of sign symbol functions according to the real-time index, wherein the instruction 0 means that the log is not reported temporarily, and the instruction 1 means that the log needs to be reported.
The user log reading module comprises: the system comprises a user log scanning module, a user log analyzing module, a session survival inquiring module and a user log storing module:
and the log scanning module scans the log catalog of the CDN plug-flow program after receiving the task scheduling instruction, and stores the names of the log files into a list according to the latest time sequence for the subsequent modules to use.
The user log analysis module cuts and analyzes the CDN ticket log according to a certain rule, stores the CDN ticket log into an IPTV log standard format, cuts the analysis rule by using an 'I' symbol, generates a dictionary corresponding to each field, and if a certain row of log does not accord with the IPTV standard, independently writes the log into a local disk, stores the log into an error log file, and discovers abnormality and alarms by other monitoring programs.
As shown in fig. 6, in the session survival query module, the first field of each ticket log is a session ID, and in order to ensure that the session is ended, the module sends the session to the CDN node service to confirm, if the session is ended, the log will be sent, if the session is still alive, the log record will be stored in the memory, and will not be combined, parsed and reported, and when the user finishes playing and reports again, that is, the session ID queries the feedback 404 (representing that the session is ended), and after all the playing durations and playing start-stop times of the session are combined, the report is unified.
The user log saving module is used for saving the log which is already played in the log analyzing module, feeding back the number of the saved log, the memory size and the retransmission times in real time.
As shown in fig. 8, the user log processing module includes: user log reporting module and local user log recovery module:
the user log reporting module is mainly responsible for log reporting, extracts data in the user log storage module after receiving a sending instruction issued by the task scheduling module, reports the data to the log center, and feeds back a reporting result to the task scheduling module. If the program fails, the number of times of transmission is recorded, and the program is transmitted to the computer together when the scheduling instruction is calculated next time, so that endless retransmission of the program is avoided.
The local log recovery module mainly has two tasks, namely, after a successful report instruction is received, the successfully reported log is moved to a local backup catalog, a failed report log is not moved, and the log is reloaded according to the time sequence when the next user log scanning module inquires; secondly, processing the local expired log, cleaning log files in the backup directory, wherein the threshold value of the cleaning log is the preservation time, and the log files exceeding 7 days in the directory are deleted.
Example 3
As shown in fig. 6:
a1, acquiring initial configuration parameters, analyzing fields, and cutting the fields;
a2, judging whether the field is abnormal:
if the field is not abnormal, executing the step A3;
if the field is abnormal, storing an error log into a local directory, analyzing the next log record, and executing the step A7;
a3, judging whether the session survives:
if the session survives, executing the step A4;
if the session is ended, executing the step A5;
a4, judging whether to merge the log and report:
if the judgment result is that the combined log is reported, the log data are all saved in the memory, the next session log is waited to arrive, ID combination is carried out, and the step A7 is executed;
if the judgment result is that the log is not combined, uploading the record, and executing the step A5;
a5, carrying out field escape aiming at special services, wherein the special services comprise screen size and play type;
a6, storing the parsed log field into a memory, and reporting log information when the real-time index of the log file content reaches a transmission threshold value;
and A7, ending the analysis.
Example 5
The IPTV field includes HTTP and RTSP services. Wherein RTSP play is a long connection, a connection signaling can be sent, and there is a special pause and exit signaling, so the procedure must be generated after the user play accepts. The HTTP cannot transfer the connection state information, so that the user generates a log each time (playing, dragging, pausing for more than 5 minutes) and further generates a plurality of logs, which means that the same session ID plays the same movie multiple times, and the actual playing behavior only plays 1 movie, resulting in deviation between the reporting of the user playing log and the actual playing.
As shown in fig. 1, the IPTVCDN user play log field includes session, user, IP, start-time and End-time, and by the method of determining whether the CDN session survives in real time, the log reporting of the IPTVCDN user skip play scene is solved. The CDN plug-flow provides a session query, interface access example:
http://127.0.0.1:12345/getActiveSessionInfocl_session=1241252
if the session survives, the user still plays, and the push program returns the playing content ID, playing protocol (HTTP/RTSP), playing start time, etc. of the session, and returns JSON results as follows:
Figure BDA0003563615530000121
if the session ends, i.e., the user plays end, the CDN plug-flow returns to 404, i.e., the query session ID does not exist.
And returning a result according to the interface, if the access result is 404, the log of the session can be reported at the present time. If the access result returns to normal JSON data, the log will not be reported until the user playing is completed and the user session is ended, and the reporting program reports once after summarizing the user playing time, so that the fact that the log which is generated locally is still reported when the user session is still in progress is eliminated, and the fact that the CDN service user playing log reporting is accurate and effective is effectively ensured.
2. The intelligent report instruction generation method effectively aims at the phenomena of peak, log number and uneven log size of the CDN node log generation. By the method, the configuration parameters are combined: the method comprises the steps of dynamically and intelligently reporting the log, wherein the upper limit of the transmission times of the memory data is NoS_set (NumberOfSend_set), the upper limit of the reporting time difference is TD_set (TimeDifference_set), the upper limit of the number of log stripes in the memory is NoL _set (NumberOfLog_set), and the upper limit of the size of a single log file is SLFS_set (SingleLogFileSize_set).
The pre-condition is as follows:
condition 1. Configured reporting program threshold parameters, default configuration is: memory data transmission times upper limit nos_set=3, reporting time difference upper limit td_set=3 (minutes), log number upper limit in memory NoL _set=10000 (stripes), single log file size upper limit slfs_set=5 (Mb);
and 2, the main program analyzes the file content of the log catalog, and obtains and loads the file content, the file size and the log number into the memory.
The algorithm is based on the sign function characteristics: when x >0, sign (x) =1; when x=0, sign (x) =0; when x <0, gn (x) = -1; wherein, the real-time parameters of the system comprise:
a. log center connectivity index C
b. Number of times of data transmission NoS_Now (NumberOfSend_Now)
c. Time difference TD_Now from last time of reporting (TimeDifference_Now)
d. Existing log entry number NoL _now (NumberOfLog_now) in memory
e. Single log file size SLFS_Now (SingleLogFileSize_Now)
The calculation mode of the log center connectivity index C is as follows: the program accesses the main log center and the standby log center in sequence, and the index is the number of the node connected log centers, for example: only with the standby node, x=1; if the two log centers are communicated with each other, x=2; if none are, x=0. And inputting a function, wherein the result is 1 or 0, and the calculation formula is as follows:
C=sign(x)
the calculation mode of the user log real-time index Sys_ metrics (Systemmetrics) is as follows: acquiring a current value of a system, including: noS_Now, TD_Now, noL_Now, SLFS_Now, system configuration values are: noS_set, TD_set, noL_set and SLFS_set, taking the maximum value with 1 after the quotient of the real-time index and the system configuration index is calculated, and taking the difference between the result and 1, and finally inputting a sign function sign to obtain the result;
the calculation formula is as follows:
Figure BDA0003563615530000131
wherein, the real-time index of the class 4 user log has a minimum value of 0 and a maximum value of infinity (single statistics condition). The scene met is as follows:
scene one: when the maximum value of the class 4 index is less than or equal to 1, then:
Figure BDA0003563615530000132
further calculate sys_metrics=sign (0) =0;
scene II: when the maximum value of the class 4 index is greater than 1, the value is denoted as Max_metric:
Figure BDA0003563615530000133
further, sys_metrics=sign (max_metrics) =1 was calculated.
Multiplying the log center connectivity index C with the user log real-time index Sys_metrics, reporting when the result is 1, and not reporting when the result is 0.
The calculation formula of the reporting instruction is as follows:
report instruction
Figure BDA0003563615530000134
Figure BDA0003563615530000141
By the method, the class 5 real-time indexes of the system are combined, whether the command is reported is dynamically output, and the play log of the CDN user is intelligently reported timely and accurately.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims (7)

1. The IPTVCDN user play log reporting method is characterized by comprising the following steps:
s1, acquiring log file content under a CDN node log directory;
s2, circularly reading the content of the log file according to the row, cutting the log file according to the requirement of a storage format, and storing the cut log file;
s3, judging whether the session of the read log file content survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
the CDN session survival judging step specifically comprises the following steps:
s31, acquiring a session value of the log;
s32, judging whether the session is in the memory or not;
if the session is determined to be in the memory, executing step S33;
if the session is determined not to be in the memory, executing step S34;
s33, acquiring historical data of the session;
s34, judging whether session survives:
if the session is judged to be alive, merging the log and the existing data in the history memory and then storing the merged log;
if the session is judged to be ended, analyzing the log, merging the playing time and the start-stop time of the existing log, and reporting the merged playing time and the start-stop time;
s4, storing the log file content of the S3;
s5, judging whether to report the content of the log file according to the real-time index of the log file:
if the judgment result is reported, executing a step S6;
if the judgment result is not reported, executing the step S2;
the step S5 specifically comprises the following steps:
s51, acquiring real-time indexes according to the content of the log file after analysis is completed;
s52, setting a threshold parameter for the real-time index;
and S53, combining the real-time index with the characteristic of the sign symbol function, and outputting a reporting instruction.
The step S53 specifically includes:
Figure FDA0004278711480000011
c-log center connectivity index, number of x-node connected log centers, sys metrics -user log real-time index, number of times of data transmission in the NoS_Now-memory, time difference between TD_Now-time of last report, number of existing log in the NoL _Now-memory, SLFS_Now-single log file size, upper limit of data transmission in the NoS_set-memory, upper limit of time difference between TD_set-time of report, upper limit of number of log in the NoL _set-memory, upper limit of SLFS_set-single log file size;
s6, reporting the content of the log file.
2. The IPTVCDN user play log reporting method according to claim 1, wherein S1 specifically includes:
s11, initializing a log process and acquiring initial configuration parameters;
s12, loading initial parameter configuration into a memory;
s13, acquiring a CDN node log catalog;
s14, judging whether the log catalog has the log or not, and executing the step S2 until the judgment result is that the log catalog has the log.
3. The IPTVCDN user play log reporting method according to claim 1, wherein the S2 specifically includes:
s21, circularly reading the content of the log file in the log storage directory according to the row, wherein the reading priority of the log file is the current log and then the history log;
s22, analyzing the fields of the log file content and cutting the fields;
s23, judging whether the field is abnormal:
if the field is not abnormal, executing the step S3;
if the field is abnormal, an error log is stored in a local directory, and the content of the next log file is analyzed;
s25, storing the analyzed log file.
4. The method for reporting an IPTVCDN user play log according to claim 1, wherein after the session is determined to survive, further comprising:
judging whether to merge log reporting for the surviving session:
if the judgment result is that the logs are reported, the log is saved, the next log of the session is waited for, and the logs of the same session are merged;
and if the judgment result is that the log is not combined, reporting the log.
5. The IPTVCDN user play log reporting method of claim 1, further comprising: step S7, processing the local user log file, wherein the step S7 specifically comprises the following steps:
s71, moving the log which is successfully reported to obtain a moving result:
the log file which is successfully reported is moved to a local backup catalog;
s72, scanning a backup catalog to check whether an expired file exists:
if the expired file exists, storing the expired file list, and circularly deleting until the whole expired file list is deleted.
6. The IPTVCDN user play log reporting system is characterized by comprising a system initialization module, a task scheduling module, a user log reading module and a user log reporting module:
the system initialization module includes: a parameter configuration module and a program log module;
the task scheduling module comprises: the timer management module and the report instruction generation module;
the user log reading module comprises: the system comprises a user log scanning module, a user log analyzing module, a session survival inquiring module and a user log storing module;
the user log processing module comprises: a user log reporting module and a local user log recovery module;
s1, a system initialization module obtains log file contents under a CDN node log directory;
s2, the timer management module circularly reads the content of the log file according to the row, cuts the log according to the requirement of a storage format, and stores the cut log in a memory;
s3, the session survival inquiry module judges whether session of the log file content read by the user log analysis module survives or not:
if the session survives, executing step S2;
if the session is ended, executing step S4;
the CDN session survival judging step specifically comprises the following steps:
s31, acquiring a session value of the log;
s32, judging whether the session is in the memory or not;
if the session is determined to be in the memory, executing step S33;
if the session is determined not to be in the memory, executing step S34;
s33, acquiring historical data of the session;
s34, judging whether session survives:
if the session is judged to be alive, merging the log and the existing data in the history memory and then storing the merged log;
if the session is judged to be ended, analyzing the log, merging the playing time and the start-stop time of the existing log, and reporting the merged playing time and the start-stop time;
s4, the user log saving module saves the log file content of the S3 into a system memory;
s5, the user log reporting module reports the log file content in the user log storage module according to the reporting instruction generated by the reporting instruction generating module:
if the instruction is a sending index, executing step S6;
if the instruction is the temporary non-transmission index, executing step S2;
the step S5 specifically comprises the following steps:
s51, acquiring real-time indexes according to the content of the log file after analysis is completed;
s52, setting a threshold parameter for the real-time index;
and S53, combining the real-time index with the characteristic of the sign symbol function, and outputting a reporting instruction.
The step S53 specifically includes:
Figure FDA0004278711480000041
c-log center connectivity index, number of x-node connected log centers, sys metrics -user log real-time index, number of times of data transmission in the NoS_Now-memory, time difference between TD_Now-time of last report, number of existing log in the NoL _Now-memory, SLFS_Now-single log file size, upper limit of data transmission in the NoS_set-memory, upper limit of time difference between TD_set-time of report, upper limit of number of log in the NoL _set-memory, upper limit of SLFS_set-single log file size;
s6, a user log reporting module reports log file contents;
s7, the local user log recovery module processes the local user log file.
7. The IPTVCDN user play log reporting system of claim 6, wherein the program log module is configured to initialize a log process, and the parameter configuration module is configured to load system configuration parameters into a memory;
the user log scanning module scans the log storage catalog according to rows after receiving the task scheduling instruction, and stores the log file names into a list for the subsequent modules to use;
the timer management module is used for acquiring log file contents at regular time;
the user log analysis module is used for cutting and analyzing the log file content acquired by the timer management module at regular time according to a certain rule and storing the log file content into an IPTV log standard format;
the session survival query module is configured to query whether session survives:
if the session is ended, the log is reported;
if session survives, the log record is stored in a memory, and the user finishes playing, namely the feedback is finished when the session ID is inquired, all playing time lengths of the session ID are combined, and the session ID is reported uniformly;
the user log saving module is used for saving the log which is already played in the log analyzing module, and feeding back the number of the saved log, the memory size and the retransmission times in real time;
the report instruction generation module is used for combining the real-time index with the characteristic of the sign symbol function to generate an instruction whether to report the information in the user log storage module;
the log reporting module is used for extracting data in the user log storage module after receiving the instruction sent by the task scheduling module, reporting the log file content and feeding back the reporting result to the task scheduling module;
and the local log recycling module is used for moving the log which is successfully reported to the local backup catalog after the successful reporting instruction is received, and cleaning the expired log in the local backup catalog at regular time.
CN202210296575.7A 2022-03-24 2022-03-24 IPTVCDN user play log reporting method and system Active CN114827126B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210296575.7A CN114827126B (en) 2022-03-24 2022-03-24 IPTVCDN user play log reporting method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210296575.7A CN114827126B (en) 2022-03-24 2022-03-24 IPTVCDN user play log reporting method and system

Publications (2)

Publication Number Publication Date
CN114827126A CN114827126A (en) 2022-07-29
CN114827126B true CN114827126B (en) 2023-07-14

Family

ID=82531313

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210296575.7A Active CN114827126B (en) 2022-03-24 2022-03-24 IPTVCDN user play log reporting method and system

Country Status (1)

Country Link
CN (1) CN114827126B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100112436A (en) * 2009-04-09 2010-10-19 티에스온넷(주) System for tracking and integrating user's session log and db query log and method thereof
CN102799514A (en) * 2011-05-24 2012-11-28 中兴通讯股份有限公司 Method and system for managing log records
CN102891873A (en) * 2011-07-21 2013-01-23 腾讯科技(深圳)有限公司 Method for storing log data and log data storage system
CN103178982A (en) * 2011-12-23 2013-06-26 阿里巴巴集团控股有限公司 Method and device for analyzing log
CN105635329A (en) * 2014-11-03 2016-06-01 中兴通讯股份有限公司 Online log generation method and apparatus
CN106326342A (en) * 2016-08-04 2017-01-11 北京京东尚科信息技术有限公司 Reporting method and device of page flow data and electronic equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100112436A (en) * 2009-04-09 2010-10-19 티에스온넷(주) System for tracking and integrating user's session log and db query log and method thereof
CN102799514A (en) * 2011-05-24 2012-11-28 中兴通讯股份有限公司 Method and system for managing log records
CN102891873A (en) * 2011-07-21 2013-01-23 腾讯科技(深圳)有限公司 Method for storing log data and log data storage system
CN103178982A (en) * 2011-12-23 2013-06-26 阿里巴巴集团控股有限公司 Method and device for analyzing log
CN105635329A (en) * 2014-11-03 2016-06-01 中兴通讯股份有限公司 Online log generation method and apparatus
CN106326342A (en) * 2016-08-04 2017-01-11 北京京东尚科信息技术有限公司 Reporting method and device of page flow data and electronic equipment

Also Published As

Publication number Publication date
CN114827126A (en) 2022-07-29

Similar Documents

Publication Publication Date Title
US7849227B2 (en) Stream data processing method and computer systems
US8135979B2 (en) Collecting network-level packets into a data structure in response to an abnormal condition
CN111966289B (en) Partition optimization method and system based on Kafka cluster
CN110418154B (en) Multimedia data pushing method, device and system
US11032584B2 (en) Picture storage method, apparatus and video monitoring system
CN111030888B (en) Domain name system DNS capacity measuring method, device, equipment and medium
US11102289B2 (en) Method for managing resource state information and system for downloading resource
CN110620699B (en) Message arrival rate determination method, device, equipment and computer readable storage medium
CN113891175B (en) Live broadcast push flow method, device and system
CN105357584A (en) User behavior log classification uploading method and system based on intelligent television
CN110162512A (en) A kind of log searching method, apparatus and storage medium
EP3800894A1 (en) Method and apparatus for detecting video playing quality
US20220353314A1 (en) Network data scheduling method and edge node thereof
CN114189705A (en) Live broadcast card pause processing method and system
CN110795264A (en) Monitoring management method and system and intelligent management terminal
CN113747186A (en) Data processing method, device, terminal and storage medium
CN114301988A (en) Distributed calling method and device, storage medium and electronic equipment
CN113472858B (en) Buried point data processing method and device and electronic equipment
CN114827126B (en) IPTVCDN user play log reporting method and system
CN111479161B (en) Live broadcast quality data reporting method and device
CN114598622B (en) Data monitoring method and device, storage medium and computer equipment
CN112764988B (en) Data segment acquisition method and device
CN111224756B (en) Method and device for determining data transmission abnormity, storage medium and electronic equipment
CN116132698A (en) Content recording method, playing method, CDN system and storage medium
CN110233747B (en) Data reporting method and cloud platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant