CN109558305B - Log data sorting method and device - Google Patents
Log data sorting method and device Download PDFInfo
- Publication number
- CN109558305B CN109558305B CN201710895707.7A CN201710895707A CN109558305B CN 109558305 B CN109558305 B CN 109558305B CN 201710895707 A CN201710895707 A CN 201710895707A CN 109558305 B CN109558305 B CN 109558305B
- Authority
- CN
- China
- Prior art keywords
- access
- log data
- preset
- video
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a log data sorting method and device. Wherein, the method comprises the following steps: acquiring an access IP address of a target user in log data, wherein the log data is used for recording data information of all users in the process of accessing the network video; extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried when a target user triggers to access a network video after initiating an access request; and sequencing the target log data. The invention solves the technical problem that the change condition of the data is difficult to be intuitively understood due to the fact that the log data of the network video cannot be effectively extracted in the related technology.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a log data sorting method and device.
Background
In the related art, when log data such as a webpage or a network video clicked by a target user is processed, the content of a video file linked in a related website or website is large, so that the loading data and the playing data of the website are correspondingly increased, and target users watching various videos may have hundreds of millions of people. At present, a large amount of target user log data cannot be effectively analyzed, data can be screened only according to a single condition, and effective data cannot be extracted. Therefore, a website manager cannot know the data required by the website manager when analyzing the log, and the extracted data cannot be sequenced and the change condition of the data cannot be intuitively known in the current technology because the log data for playing the network video cannot be effectively extracted.
Aiming at the technical problem that the change condition of data is difficult to understand intuitively due to the fact that log data of a network video cannot be extracted effectively in the related technology, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a log data sorting method and a log data sorting device, which at least solve the technical problem that the change condition of data is difficult to understand intuitively because the log data of a network video cannot be effectively extracted in the related technology.
According to an aspect of an embodiment of the present invention, there is provided a log data sorting method, including: acquiring an access IP address of a target user in log data, wherein the log data is used for recording data information of all users in a network video access process; extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried by the target user when triggering access to a network video after initiating an access request; and sequencing the target log data.
Further, the preset access characteristic information includes at least one of: the method comprises the following steps of presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the preset character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the preset character string, extracting target log data according to the target user access IP address and the preset access characteristic information comprises the following steps: acquiring a preset character string in log data; determining the loading starting position of each section of log data according to the preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and extracting target log data according to the loading initial position of each section of log data.
Further, sorting the target log data comprises: according to a preset screening condition, screening the log data of the access IP address; and sequencing the screened log data of the access IP address of the target user according to the loading starting time of each section of log data.
Further, when the preset access characteristic information is an access link webpage, the access link webpage is an initial webpage in each execution of an access behavior, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access link webpage of a target user; and extracting the target log data according to the access link webpage of the target user.
Further, when the preset access characteristic information is an access browser, the accessing browser is a browser used by the target user for accessing the network video, and extracting the target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access browser of a target user; and extracting the target log data according to the access browser of the target user and the access IP address of the target user.
Further, when the preset access characteristic information is a video identifier, the video identifier is used to indicate a target video accessed by a target user, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring a video identifier; and extracting the target log data according to the video identification and the access IP address of the target user.
According to another aspect of the embodiments of the present invention, there is also provided a log data sorting apparatus, including: the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring an access IP address of a target user in log data, and the log data is used for recording data information of all users in a network video access process; the extraction unit is used for extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried by the target user when the target user triggers to access the network video after initiating an access request; and the sequencing unit is used for sequencing the target log data.
Further, the preset access characteristic information includes at least one of: the method comprises the following steps of presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the preset character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the preset character string, the extraction unit comprises: the acquisition module is used for acquiring a preset character string in the log data; the determining module is used for determining the loading starting position of each section of log data according to the preset character string and the preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and the extraction module is used for extracting target log data according to the loading initial position of each section of log data.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium having a program stored thereon, the program executing any one of the log data sorting methods described above.
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, and the program executes the log data sorting method described in any one of the above.
In the embodiment of the invention, the access IP address of the target user in the log data can be acquired, the target log data can be extracted from the log data according to the acquired access IP address and the preset access characteristic information, and then the extracted target log data can be sequenced. In this embodiment, data in log data of the network videos can be extracted through the access IP address and the preset access characteristic information, and the extracted target log data are sorted, so that a website manager can clearly know data of netizens corresponding to each network video, and after the extracted data are sorted, the website manager can know data such as the IP address of a user watching the videos, so as to know the data change condition. The method solves the technical problem that the log data of the network video cannot be effectively extracted in the related technology, so that the change condition of the data is difficult to understand visually.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of a log data ordering method according to an embodiment of the invention;
fig. 2 is a schematic diagram of a log data sorting apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to facilitate the understanding of the present invention by the user, the following explains some terms or nouns appearing in the embodiments of the present invention:
HTTP, hypertext transfer protocol, is the most widely used network protocol on the internet. All www files have to comply with this standard.
And the state code is a digital code representing the HTTP response state of the web server.
And the UA is used for identifying the browser for loading the access request.
Video is blocked, and the video cannot be smoothly watched due to short stop when the video is played.
Video code rate, the number of data bits transmitted per unit time during data transmission.
Range segments the requested content and provides it to http as a loading header, so that the readable content is recorded during loading, and the Range can be used to recover the transmitted content after network interruption and recovery.
In accordance with an embodiment of the present invention, there is provided a method embodiment of log data ordering, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
The following embodiments can be applied to log data of network videos, because network video files are generally large, currently, mainstream video websites can segment videos according to a certain size when video stream pushing is performed, so that after a request of a user for watching the videos is received, the videos are sent in segments, wherein a client held by the user can perform a video segmentation request (for example, an Http range request), a server can load the videos in segments according to the request, and play the loaded data in segments, so that a complete video can be formed finally. When the video file is loaded, the start time and the end time of each video segment can be recorded, and the ID information of the video watched by the netizen, the browser loaded by the netizen, the segmentation sequence of the video and other data can be extracted. In the related art, a large amount of data recorded in a log file can be extracted, and video data extracted in the related art is not sorted, so that a website manager looks disorderly, and the website manager cannot clearly know the video watching state of a user. According to the embodiment of the invention, the required log data can be extracted by using the preset access characteristic information and the access IP address information, and then the log data can be sequenced, so that the log data according to the downloading sequence and the playing sequence can be reasonably obtained.
The present invention is described below with reference to preferred implementation steps, and fig. 1 is a flowchart of a log data sorting method according to an embodiment of the present invention, and as shown in fig. 1, the method includes the following steps:
step S102, obtaining an access IP address of a target user in log data, wherein the log data is used for recording data information in the process of accessing the network video by all users.
The network video in the invention can be a video linked with a webpage in each website, and each video portal website has a plurality of pages, such as a recommendation page, a video page, a real news page, a sports page, a financial page, an animation page and the like. The website in the invention can relate to various websites of a domestic network, such as a Baidu website, a Google website, a Facebook website, an Aiqiyi website and the like, each website comprises a plurality of pages, and each page has a plurality of corresponding connected web page addresses. In the embodiment of the invention, taking a video webpage in a website as an example, a plurality of videos are displayed on the video webpage, when the videos are displayed, a video theme of each video, a poster picture and a video introduction corresponding to the video are displayed, after a user clicks display information on a display screen, a website server receives an instruction, reads a corresponding video from a video database, and loads the corresponding video into a buffer memory of a client, so that the user can watch the video smoothly and without a card, and when the video is loaded, the video is generally loaded in segments according to a video ID, the size of the video and the current network state. When segmenting a video, the video may be segmented by a preset segmentation technique, which may be a range technique.
For the above log data can include various information, the present invention can include but is not limited to: the method comprises the following steps of accessing an IP address, a target video ID, a browser (namely UA) by a user, accessing a webpage address, the segmentation duration of the video, the starting time of loading each video and the ending time of loading each video. The access IP address of the user may be an access IP address of a user client, and the specific access IP address of the user is not limited in the present invention, for example, ip1.1.1.1, ip1.2.2.2, ip2.1.2.1, etc., and the target video ID may be identification information determined after identifying the stored video in each website, for example, it is determined that the video ID of video a is 1000, the video ID of video B is 1001, and the video ID of video C is 1002. The access browser may be a browser initiating a network video access request, and the specific browser in the present invention is not limited, and may include but is not limited to: ***, chrome, Aiqiyi, search fox, you ku, etc. The access webpage address can be a webpage address for initiating a network video access request, and the webpage address can be understood as a webpage address of a user when searching videos, such as ***.com, souhu.com, and the like. In addition, the segmentation duration for the video may be the duration of each video after the video is segmented, for example, the total duration of video a is 130 minutes, and needs to be segmented into 13 segments, and then the duration of each video is 10 minutes. And the start time for loading each video and the end time for loading each video may be at the time of loading the identification of each video, e.g., the start time for loading video a is 11:00 and the end time for loading video a is 11: 02.
In the above log data of the present invention, for each video, a plurality of users can play, and then log data corresponding to one video and a plurality of users can be generated. The method can be used for sequencing the start time and the end time of the loaded videos of the same user of the extracted same video, so that a website manager can see information such as the start time and the end time of watching the network videos of each user in one video, and the website manager can visually know the information.
Optionally, the specific content of the video and the classification of the video are not specifically limited in the present invention, for example, the video may be various types of updated dramas or movies being updated, the video may be various types of dramas, and the name of the video is also not limited, for example, the video is "westerners", "warwolfs", or the like. The video ID of each video is recorded, and each video is identified through the video ID, so that the video content is distinguished.
And step S104, extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried when a target user triggers to access the network video after initiating an access request.
Before extracting the log data, the website administrator may input the video ID corresponding to the extracted video, so as to extract the log data corresponding to the video ID. Under the data corresponding to the video ID, log data extraction can be performed according to the access ID address and each preset access characteristic information, so that data desired by a website manager can be obtained.
Through the above steps, target log data, which may be the target log data obtained according to the access IP address, for example, log data of the viewing video corresponding to the user is extracted through IP1.1.2.2. The log data can also be extracted through preset access characteristic information, wherein the preset access characteristic information comprises at least one of the following information: presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the presetting character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the presetting character string, extracting target log data according to a target user access IP address and the preset access characteristic information comprises the following steps: acquiring a preset character string in log data; determining the loading starting position of each section of log data according to a preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and extracting target log data according to the loading initial position of each section of log data.
The above embodiment takes a preset character string as an example, and the preset character string is extracted from a segmented video, where the specific content of the preset character string is not limited in the present invention, and a network video may be segmented by the preset character string, so as to extract multiple pieces of data of the video according to the preset character string, for example, the preset character string is "RANGE", and extract relevant data of RANGE, for example, the video is divided into three segments, and each segment of the video is 20 minutes, so that a first segment of the video, i.e., RANGE1-20, a second segment of the video, i.e., RANGE20-40, and a third segment of the video, i.e., RANGE40-60, can be extracted, where RANGE1 can be understood that the start position of the first segment of the video is 1, and the end position of the first segment is 20. The preset partition field may include fields of various formats, such as "-", and the preset field may be understood as a cut field to cut a start position and an end position of each piece of video, such as "RANGE 1-20", the start position and the end position of each piece of video are separated by the preset partition field "-" and are respectively referred to as RANGE1 and RANGE20, and the loading start position of each piece of video can be extracted by the preset partition field and the preset character string. The initial positions for extracting the three video sections are respectively RANGE1, RANGE20 and RANGE 40. And obtaining corresponding target log data through the extracted loading initial position.
After the target log data are extracted according to the loading initial position of each section of log data, the target log data can be sorted, and the log data of the access IP address can be screened according to a preset screening condition during sorting; and sequencing the screened log data of the access IP address of the target user according to the loading starting time of each section of log data.
The preset screening condition may be the size of the loading start position of each video segment, or the length of the loading start position. For example, the target logs may be sorted according to the sizes of RANGE1, RANGE20, and RANGE40 in the three pieces of video, and the target log data corresponding to the access IP address of one user in the same video may be sorted.
In addition, when the preset access characteristic information is an access link webpage, the access link webpage is an initial webpage in each execution of an access behavior, and the extracting of the target log data according to the target user access IP address and the preset access characteristic information comprises: acquiring an access link webpage of a target user; and extracting target log data according to the access link webpage of the target user.
Com, the access link webpage may be the above-mentioned access webpage address, for example, ***. By screening the access link webpage, the access address of the same access webpage address can be extracted. When the access link webpage is extracted, the access webpage address can be acquired according to another preset character segment, where the preset character segment can be understood as identification information of the access link webpage, for example, the preset character segment is "referr," and each access link webpage can be stored in log data by extracting the preset character segment, so that in the target log data needing to be extracted, the target log data can be extracted according to the preset character segment and the corresponding access link webpage.
Optionally, when the preset access characteristic information is an access browser, the access browser is a browser used by a target user to access a network video, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access browser of a target user; and extracting target log data according to the access browser of the target user and the access IP address of the target user.
The access browser may be multiple types of browsers, and is determined according to the types of browsers used by different users, for example, a QQ browser, a chrome, and the like, and the access browser may be extracted according to a preset character segment to obtain an access browser, for example, "UA", where a specific type of the corresponding access browser may be stored behind the UA, and the access browser used by the user may be extracted as the chrome, taking the UA chrome as an example. Through the identification information of the access browser and the extraction of the specific content of the browser, the condition for extracting the log data can be obtained correspondingly, and when the data is required to be extracted, the target log data is extracted according to the access browser.
For the above embodiment, when the preset access characteristic information is a video identifier, the video identifier is used to indicate a target video accessed by a target user, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring a video identifier; and extracting target log data according to the video identification and the access IP address of the target user.
In the above embodiment, the video identifier may be understood as data identifying video content or video code, and may also be a video ID. When extracting the target log data, the video identification information that the website administrator wants to extract may be extracted in advance through this embodiment, and when extracting, the video identification information may also be obtained through a preset character string, such as "ID", through which the video is identified, so that when extracting the target log data, the log data is extracted through the video identification and the subsequent video content.
Optionally, when the data is extracted, the multiple pieces of preset access characteristic information and one or more pieces of access IP addresses of the users may be obtained in combination, for example, when any multiple pieces of information, such as the video identifier, the access IP address, the access browser, and the access link address, are the same, the video information is determined to be video information of the same netizen, for example, when the access IP address, the video identifier, and the access browser are the same, it may be determined that the playing users of the several video segments are the same user, and the following takes the content of the extracted multiple pieces of video log data as an example, the log data is extracted, for example:
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1001，IP 2.2.2.2，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 30-40；
ID 1000, IP1.1.1.1, UA chrome, REFERER ***.com, RANGE 10-20. The above ID may be understood as video identification, IP may be understood as an access IP address of a user, UA may be understood as an access browser, referr may be understood as an access link web page, RANGE may be understood as a preset string, and by analyzing and extracting, ID 1000, IP1.1.1.1, UA chrome, referr basic.com, RANGE 1-10 may be obtained;
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 20-30；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 10-20；
the log data of the video segments can be understood as the log data of the video segments of the same netizen. By extracting the conditions of the plurality of log data, the related data of the same netizen in the same video can be obtained, so that the target log data can be obtained.
Step S106, sequencing the target log data.
When the target log data are sorted, sorting can be performed in multiple ways, and each extraction condition can be sorted respectively, for example, sorting according to the size of a video ID, sorting according to the size of an IP address, determining according to the type of a browser, determining according to the length of a character string accessing a link webpage, determining according to the starting position of a preset character string, and determining according to the loading starting time or the loading ending time. For example, the size of the loading start position in the preset character string is sorted, and taking the data as an example, a sorting result can be obtained:
ID 1000，IP 1.1.1.1，UA chrome,REFERER ***.com，RANGE 1-10；
ID 1000，IP 1.1.1.1，UA chrome,REFERER ***.com，RANGE 10-20；
ID 1000，IP 1.1.1.1，UA chrome,REFERER ***.com，RANGE 20-30。
through the steps, the access IP address of the target user in the log data can be obtained, the target log data are extracted from the log data according to the obtained access IP address and the preset access characteristic information, and then the extracted target log data can be sequenced. In this embodiment, data in log data of the network videos can be extracted through the access IP address and the preset access characteristic information, and the extracted target log data is sorted, so that a website manager can clearly know data of netizens corresponding to each network video, and after the extracted data are sorted, the website manager can know data such as the IP address of a user watching the videos. The method solves the technical problem that the log data of the network video cannot be effectively extracted in the related technology, so that the change condition of the data is difficult to understand visually.
In the embodiment, the desired log data can be extracted from huge video log data according to the preset conditions, the playing sequence and the downloading sequence of each user related to the same video in the log data can be extracted according to the preset conditions, log data of loaded videos of different netizens can be obtained, and a website manager can know the loading sequence in the log data more intuitively, so that the website performance is further optimized.
The present invention will be described below with reference to a preferred embodiment.
The network video files are generally large, when video streaming pushing is carried out on a mainstream video website, regular cutting is carried out on videos according to a certain size, http range requests are carried out in a segmentation mode through a video player, and the ranges are continuously played through the video player to form a complete video. In the formed w3c log, when a video loading log is seen, the log is the log of multiple range requests, each request forms a log, all video logs on a website are written in the same log file, the video id and the slicing sequence of the range request are recorded in the logs, and when a certain play stream needs to be extracted, the video id is filtered in the large log to obtain all the request logs. The request log is very huge, and the traditional video id filtering mode can only obtain all requests of one video, but cannot embody the downloading sequence, different playing streams of different netizens and the like to load the stream log more specifically and intuitively.
In the invention, the video stream played by each user in a period of time can be clearly combed out from the disordered big log file, and the method is used for daily performance analysis and other works.
First, in step 11, all log entries having a video id may be selected from a large log file according to the video id provided by a user, and stored in a memory.
Step 12, traversing the log entries obtained in step 11, performing character string combination on four field information of video ID/netizen IP/UA/REFERER of each log, regarding all the same character strings of the three fields as a netizen, and summarizing the requests of the same netizen, for example, the log is:
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1001，IP 2.2.2.2，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 30-40；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 10-20；
after carding and gathering, the following results are obtained:
(netizen 1)10001.1.1.1, chrome, ***.com:
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 20-30；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 10-20。
(netizen 2) ID 1001, IP 2.2.2.2, UA chrome, referr basic.com:
ID 1001，IP 2.2.2.2，UA chrome，REFERER ***.com，RANGE 1-10。
then the sequencing of the video loading stream is carried out,
step 13, processing the log set of all netizens in this step, taking out the range field in the log, taking "-" as the cutting field, the 1 st field after cutting being the range start (loading start position), the 2 nd field being the range end (loading end position), for example: ID 1000, IP1.1.1.1, UA chrome, REFERER baldu.com, RANGE 1-10. In the log, the start position of the load obtained after cutting is 1, and the end position is 10.
After the start and end positions of all video segments are obtained, sorting is performed according to the start positions, such as the log stream of (netizen 1)10001.1.1.1 chroma basic.
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 1-10；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 10-20；
ID 1000，IP 1.1.1.1，UA chrome，REFERER ***.com，RANGE 20-30。
According to the steps, the obtained result is that a netizen can get all the loading stream logs according to the playing sequence when watching a video.
Therefore, the broadcast stream can be accurately and specifically screened according to the netizen characteristics, so that the loading stream is changed from disorder to sequence, and later analysis is visually carried out.
Fig. 2 is a schematic diagram of a log data sorting apparatus according to an embodiment of the present invention, and as shown in fig. 2, the apparatus may include: the acquiring unit 21 is configured to acquire an access IP address of a target user in log data, where the log data is used to record data information of all users in a process of accessing a network video; the extracting unit 23 is configured to extract target log data from the log data according to the access IP address and preset access characteristic information, where the preset access characteristic information is used to record information carried when a target user triggers to access a network video after initiating an access request; and a sorting unit 25 configured to sort the target log data.
With the above embodiment, the access IP address of the target user in the log data can be acquired through the acquisition unit 21, the target log data is extracted from the log data through the extraction unit 23 according to the acquired access IP address and the preset access characteristic information, and then the extracted target log data can be sorted through the sorting unit 25. In this embodiment, data in log data of the network videos can be extracted through the access IP address and the preset access characteristic information, and the extracted target log data is sorted, so that a website manager can clearly know data of netizens corresponding to each network video, and after the extracted data are sorted, the website manager can know data such as the IP address of a user watching the videos. The method solves the technical problem that the log data of the network video cannot be effectively extracted in the related technology, so that the change condition of the data is difficult to understand visually.
Wherein the preset access characteristic information comprises at least one of the following: the method comprises the following steps of presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the preset character string is used for indicating a server to segment a returned loading result according to a target user access request, and when preset access characteristic information is the preset character string, the extraction unit comprises: the acquisition module is used for acquiring a preset character string in the log data; the determining module is used for determining the loading starting position of each section of log data according to a preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and the extraction module is used for extracting target log data according to the loading initial position of each section of log data.
The log data sorting device may further include a processor and a memory, the obtaining unit 21, the extracting unit 23, the sorting unit 25, and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the video data is analyzed by adjusting the kernel parameters so as to sort the target data in the log data and obtain the data desired by the user.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium having a program stored thereon, the program executing any one of the log data sorting methods described above.
According to another aspect of the embodiments of the present invention, there is also provided a processor, where the processor is configured to execute a program, and the program executes the log data sorting method according to any one of the foregoing methods.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps: acquiring an access IP address of a target user in log data, wherein the log data is used for recording data information of all users in the process of accessing the network video; extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried when a target user triggers to access a network video after initiating an access request; and sequencing the target log data.
Optionally, the preset access characteristic information includes at least one of: presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the presetting character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the presetting character string, extracting target log data according to a target user access IP address and the preset access characteristic information comprises the following steps: acquiring a preset character string in log data; determining the loading starting position of each section of log data according to a preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and extracting target log data according to the loading initial position of each section of log data.
Optionally, when the processor executes a program, the processor may further perform screening processing on log data of an access IP address according to a preset screening condition; and sequencing the screened log data of the access IP address of the target user according to the loading starting time of each section of log data.
Optionally, when the processor executes the program, when the preset access characteristic information is an access link webpage, the access link webpage is an initial webpage in each execution access behavior, and extracting the target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access link webpage of a target user; and extracting target log data according to the access link webpage of the target user.
Optionally, when the processor executes the program, when the preset access characteristic information is an access browser, the access browser may be a browser used by a target user to access a network video, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access browser of a target user; and extracting target log data according to the access browser of the target user and the access IP address of the target user.
Optionally, when the processor executes the program, when the preset access characteristic information is a video identifier, the video identifier is used to indicate a target video accessed by a target user, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring a video identifier; and extracting target log data according to the video identification and the access IP address of the target user.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: acquiring an access IP address of a target user in log data, wherein the log data is used for recording data information of all users in the process of accessing the network video; extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried when a target user triggers to access a network video after initiating an access request; and sequencing the target log data.
Optionally, the preset access characteristic information includes at least one of: presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the presetting character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the presetting character string, extracting target log data according to a target user access IP address and the preset access characteristic information comprises the following steps: acquiring a preset character string in log data; determining the loading starting position of each section of log data according to a preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data; and extracting target log data according to the loading initial position of each section of log data.
Optionally, when the computer program product executes a program, the screening processing may be performed on log data of an access IP address according to a preset screening condition; and sequencing the screened log data of the access IP address of the target user according to the loading starting time of each section of log data.
Optionally, when the computer program product executes a program, when the preset access characteristic information is an access link webpage, the access link webpage is an initial webpage in each execution access behavior, and extracting the target log data according to the target user access IP address and the preset access characteristic information includes: acquiring an access link webpage of a target user; and extracting target log data according to the access link webpage of the target user.
Optionally, when the computer program product executes a program, when the preset access characteristic information is an access browser, the access browser may be a browser used by a target user to access a network video, and extracting target log data according to an access IP address of the target user and the preset access characteristic information includes: acquiring an access browser of a target user; and extracting target log data according to the access browser of the target user and the access IP address of the target user.
Optionally, when the computer program product executes a program, when the preset access characteristic information is a video identifier, the video identifier is used to indicate a target video accessed by a target user, and extracting target log data according to the target user access IP address and the preset access characteristic information includes: acquiring a video identifier; and extracting target log data according to the video identification and the access IP address of the target user.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (8)
1. A method for log data ordering, comprising:
acquiring an access IP address of a target user in log data, wherein the log data is used for recording data information of all users in a network video access process;
extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried when the target user triggers to access a network video after initiating an access request;
sorting the target log data;
wherein the preset access characteristic information includes at least one of: the method comprises the following steps of presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the preset character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the preset character string, extracting target log data according to the target user access IP address and the preset access characteristic information comprises the following steps:
acquiring a preset character string in log data;
determining the loading starting position of each section of log data according to the preset character string and a preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data;
and extracting target log data according to the loading initial position of each section of log data.
2. The method of claim 1, wherein sorting the target log data comprises:
according to a preset screening condition, screening the log data of the access IP address;
and sequencing the screened log data of the access IP address of the target user according to the loading starting time of each section of log data.
3. The method of claim 1, wherein when the preset access characteristic information is an access link webpage, the access link webpage is a starting webpage in each execution of an access behavior, and extracting target log data according to the target user access IP address and the preset access characteristic information comprises:
acquiring an access link webpage of a target user;
and extracting the target log data according to the access link webpage of the target user.
4. The method according to claim 1, wherein when the preset access characteristic information is an access browser, the access browser is a browser used by the target user to access the network video, and extracting target log data according to the target user access IP address and the preset access characteristic information comprises:
acquiring an access browser of a target user;
and extracting the target log data according to the access browser of the target user and the access IP address of the target user.
5. The method of claim 1, wherein when the preset access characteristic information is a video identifier, the video identifier is used to indicate a target video accessed by a target user, and extracting target log data according to the target user access IP address and the preset access characteristic information comprises:
acquiring a video identifier;
and extracting the target log data according to the video identification and the access IP address of the target user.
6. An apparatus for sorting log data, comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an access IP address of a target user in log data, and the log data is used for recording data information in the process of accessing network videos by all users;
the extraction unit is used for extracting target log data from the log data according to the access IP address and preset access characteristic information, wherein the preset access characteristic information is used for recording information carried by the target user when the target user triggers to access the network video after initiating an access request;
the sorting unit is used for sorting the target log data;
wherein the preset access characteristic information includes at least one of: the method comprises the following steps of presetting a character string, accessing a link webpage, accessing a browser and a video identifier, wherein the preset character string is used for indicating a server to segment a returned loading result according to a target user access request, and when the preset access characteristic information is the preset character string, the extraction unit comprises:
the acquisition module is used for acquiring a preset character string in the log data;
the determining module is used for determining the loading starting position of each section of log data according to the preset character string and the preset separation field, wherein the preset separation field is used for indicating the starting position and the ending position of loading each section of log data;
and the extraction module is used for extracting target log data according to the loading initial position of each section of log data.
7. A storage medium having a program stored thereon, characterized in that the program executes the log data sorting method according to any one of claims 1 to 5.
8. A processor for running a program, wherein the program when running performs the log data ordering method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710895707.7A CN109558305B (en) | 2017-09-27 | 2017-09-27 | Log data sorting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710895707.7A CN109558305B (en) | 2017-09-27 | 2017-09-27 | Log data sorting method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109558305A CN109558305A (en) | 2019-04-02 |
| CN109558305B true CN109558305B (en) | 2022-05-27 |
Family
ID=65864135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710895707.7A Active CN109558305B (en) | 2017-09-27 | 2017-09-27 | Log data sorting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109558305B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112422337A (en) * | 2020-11-11 | 2021-02-26 | 网宿科技股份有限公司 | Method, system and device for generating log identification |
| CN117118824B (en) * | 2023-10-20 | 2024-02-27 | 成都卓拙科技有限公司 | Log data collection method and device |
| CN117150105B (en) * | 2023-10-27 | 2023-12-26 | 四川银亿科技有限公司 | Data acquisition method and acquisition platform based on webpage |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686094A (en) * | 2014-01-01 | 2014-03-26 | 中安消技术有限公司 | Video monitoring log generating method and video monitoring log generating system |
| CN105872731A (en) * | 2015-11-26 | 2016-08-17 | 乐视云计算有限公司 | Data processing method and device |
| CN106603296A (en) * | 2016-12-20 | 2017-04-26 | 北京奇虎科技有限公司 | Log processing method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8508570B2 (en) * | 2008-08-12 | 2013-08-13 | Polycom, Inc. | Method to populate contact list from recent call log |
| US8990291B2 (en) * | 2010-07-21 | 2015-03-24 | Empire Technology Development Llc | Information processing apparatus, server-client system, and computer program product |
| CN102868936B (en) * | 2012-09-06 | 2015-06-10 | 北京邮电大学 | Method and system for storing video logs |
| US9078027B1 (en) * | 2013-03-15 | 2015-07-07 | Tribune Broadcasting Company, Llc | Systems and methods for enabling functionality of a trigger mechanism based on log entries in a traffic log |
| CN104581218B (en) * | 2014-11-10 | 2017-11-14 | 北京国双科技有限公司 | Web TV rating flows to monitoring method and device |
-
2017
- 2017-09-27 CN CN201710895707.7A patent/CN109558305B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103686094A (en) * | 2014-01-01 | 2014-03-26 | 中安消技术有限公司 | Video monitoring log generating method and video monitoring log generating system |
| CN105872731A (en) * | 2015-11-26 | 2016-08-17 | 乐视云计算有限公司 | Data processing method and device |
| CN106603296A (en) * | 2016-12-20 | 2017-04-26 | 北京奇虎科技有限公司 | Log processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 基于Web的网站访问统计分析***;孟昭鹏等;《计算机工程与设计》;20050528(第05期);203-204、207 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109558305A (en) | 2019-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10089402B1 (en) | Display of videos based on referrers | |
| US11190821B2 (en) | Methods and apparatus for alerting users to media events of interest using social media analysis | |
| CN106331778B (en) | Video recommendation method and device | |
| CN109684513B (en) | Low-quality video identification method and device | |
| KR101999389B1 (en) | Identification and presentation of internet-accessible content associated with currently playing television programs | |
| US20220318257A1 (en) | Methods, systems, and media for providing a media search engine | |
| US20070294265A1 (en) | Identification of content downloaded from the internet and its source location | |
| CN101594241B (en) | Method and system for downloading network video | |
| CN109587521B (en) | Video stuck judgment method and device | |
| US11816172B2 (en) | Data processing method, server, and computer storage medium | |
| CN109558305B (en) | Log data sorting method and device | |
| KR20160104661A (en) | Methods, systems, and media for presenting supplemental information corresponding to on-demand media content | |
| JP5226784B2 (en) | Method and apparatus for providing moving image search service | |
| CN107566906B (en) | Video comment processing method and device | |
| WO2015070807A1 (en) | Program recommendation method and device for smart television | |
| CN107341395A (en) | A kind of method for intercepting reptile | |
| CN111767430B (en) | Video resource pushing method, video resource pushing device and storage medium | |
| US8572073B1 (en) | Spam detection for user-generated multimedia items based on appearance in popular queries | |
| CN109275045B (en) | DFI-based mobile terminal encrypted video advertisement traffic identification method | |
| CN113407886A (en) | Network crime platform identification method, system, device and computer storage medium | |
| CN113038153A (en) | Financial live broadcast violation detection method, device and equipment and readable storage medium | |
| CN112231518B (en) | Method, system, electronic device and storage medium for discovering network propagation behavior of works | |
| CN113453076B (en) | User video service quality evaluation method, device, computing equipment and storage medium | |
| CN112584255B (en) | Method and device for playing streaming media data, computer equipment and storage medium | |
| CN110275998B (en) | Method and device for determining webpage attribute data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |