CN108200471B - A kind of construction method of standard data set that evaluating and testing encrypted video QoE - Google Patents
A kind of construction method of standard data set that evaluating and testing encrypted video QoE Download PDFInfo
- Publication number
- CN108200471B CN108200471B CN201810016344.XA CN201810016344A CN108200471B CN 108200471 B CN108200471 B CN 108200471B CN 201810016344 A CN201810016344 A CN 201810016344A CN 108200471 B CN108200471 B CN 108200471B
- Authority
- CN
- China
- Prior art keywords
- video
- terminal
- network
- data
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/4508—Management of client data or end-user data
- H04N21/4516—Management of client data or end-user data involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
- H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Social Psychology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a kind of construction methods of standard test data set for evaluating and testing encrypted video QoE, it is characterized in constructing universal data collection platform, cooperated by terminal applies and HTTP server, video network data, video QoE parameter and user MOS scoring during HTTP server single-point can acquire mobile terminal viewing video simultaneously, realize standard data set building.The data acquisition modes combined using distributed feedback drive test terminal with VPN centralization due to the present invention and the standard data set construction method based on data flow, therefore the present invention supports the data of crowdsourcing model to acquire in which can be convenient, and it is suitable for encryption and non-encrypted video traffic based on TCP and udp protocol, the true comprehensive and completeness of the data set of building is good, to overcome the shortcomings of available data collection construction method.
Description
Technical field
The invention belongs to mobile communication network technology fields, and in particular to mobile network-oriented operator/equipment vendor carries out
The construction method of the standard data set of encrypted video QoE evaluation and test.
Background technique
Good experience is provided for user and guarantee is the starting point of carrier network optimization, Mobile Network Operator/equipment
Quotient increasingly pays close attention to the assessment of mobile video quality of service.In recent years, popularizing with 3G/4G network, mobile network video obtains
Swift and violent development is arrived.It, will to flow caused by mobile video business in 2021 according to the statistical forecast in 2016 of Cisco, the U.S.
Account for 75% or more of mobile flow.The mobile video business of mainstream is based on HTTP self adaptation stream (HTTP Adaptive at present
Stream, HAS).But the attention with people to privacy of user, more and more content suppliers start to its video content into
Row encrypted transmission.Therefore, the comprehensive data set for obtaining support mobile video evaluation the quality has become mobile network's fortune
The problem of seeking quotient/equipment vendor's urgent need to resolve.
The data set for supporting mobile video evaluation the quality at least should include Objective Video quality of service, user experience
(Quality of Experience, QoE) quality and network data.Objective quality of service refers to the objective of video traffic
Quality index, the common initial buffer including video, Caton, clarity and fluency.Video traffic QoE refers to video industry
The subjective perception of business is experienced, and generallys use the five-grade marking system subjectivity mean opinion score (the Mean Opinion of ITU-T proposition at present
Score, MOS), by the way that QoE is analyzed from difference to and then being quantified as 1~5 point well.For example, the U.S. " Mobile in 2014
" the Prometheus:toward quality-of- that Computing Systems and Applications " meeting is included
Experience estimation for mobile apps from passive network measurements " joint
Terminal testing and core net actively measure, and user experience parameter (QoE Metrics) are obtained from terminal side, at the beginning of mainly including video
Begin buffering and Caton, obtains IP/TCP stream rank statistics network quality (Quality of Service, QoS) ginseng from core net
Number uses the technique study relationship of qos parameter and QoE parameter of machine learning.In addition to this, there are also many other researchs
Person or exploitation metadata acquisition tool, or cooperate to construct on the basis of acquisition data with content supplier/network operator
Experimental data set has carried out similar research and has achieved certain research achievement.For example, being embodied in Europe in 2015
" the YoMoApp:A tool for of " European Conference on Networks and Communications "
Analyzing QoE of YouTube HTTP adaptive streaming in mobile networks ", the tool is only
QoE parameter can be acquired in terminal, include mainly video initial buffer, Caton, the network data and use of video traffic cannot be obtained
Family MOS.Dutch periodical " Computer Communications " " the Network-based video included in 2017
Freeze detection and prediction in HTTP adaptive streaming ", under non-encrypted scene from
The content delivery network service device of content supplier obtains video playing log, passes through decompiling official terminal in the case where encrypting scene
Video playing log is obtained using (Application, APP), main includes the segment information and Caton information of video.But they
Have a common problem at present: experimental data set acquisition still has difficulty, the especially feelings in video-encryption transmission
Under condition, the complete experimental data set of terminal video QoE parameter, network data and user MOS can not be obtained simultaneously.
Summary of the invention
The present invention proposes a kind of construction method of standard data set for evaluating and testing encrypted video QoE, constructs universal data collection
Platform is cooperated by terminal APP and HTTP server, can acquire mobile terminal viewing video process simultaneously on HTTP server
In video network data, video QoE parameter and user MOS scoring, realize standard data set building, to overcome existing reality
Test the deficiency of data set construction method.
The construction method that the present invention evaluates and tests the standard data set of encrypted video QoE is adopted by way of mobile terminal drive test
Video, user and network data under set network in mobile video traffic watching process construct standard data set, feature
It is: is mutually tied by distributed feedback drive test terminal and the centralized data acquisition of VPN (Virtual Private Network)
It closes, video, user and network data needed for data acquisition platform only can acquire building data set in HTTP server side；HTTP
Server uses IP-based data acquisition modes to be suitable for the encryption and non-encrypted shifting of various IP/TCP and IP/UDP agreements
Dynamic data service；Concrete operation step are as follows:
(1) mobile terminal accesses data acquisition platform in such a way that VPN is acted on behalf of, and opens terminal APP selection video and carries out
Viewing；
(2) it includes terminal brand and model, operating system, Termination ID, network that terminal APP, which is obtained used in viewing video,
Simultaneously video playing log is recorded in end message including operator, network access mode, IP address of terminal and signal strength indication；
Wherein network operator includes mobile, connection and telecommunications, and network access mode includes 3G, 4G and WIFI, and IP address of terminal is to see
The IP address of terminal when seeing video, signal strength indication are quantified as 0~4 according to practical received signal strength from weak to strong；
(3) video information is obtained by terminal APP and video playing log is recorded, video information includes video ID, view
The playable duration of frequency title and video；
(4) the user's operation behavior in video display process is recorded by terminal APP and broadcasting log, Yong Hucao is recorded
Making behavior includes suspending, jumping forward, jump backward and stop to play；
(5) pass through the QoE parameter of terminal APP record mobile terminal network video to broadcasting log；The QoE parameter master of video
To be the objective quality index of mobile video business, the first time slack including video prolong, the Caton information of video, video resolution and
Video actual play duration；Wherein the first time slack of video, which prolongs, refers to when playing passed through since user be played to clicking
Between, at the time of Caton information includes video cardton and Caton duration；
(6) the MOS scoring that user is recorded by terminal APP, then uploads to HTTP server for journal file；Video is broadcast
Terminal APP prompts user to score when discharging into, and user is according to the experience in watching process, by video tastes from difference to favorable comment
It is divided into 1~5；
(7) terminal APP provides automatic record screen function, and after the license of user, terminal APP can be to video display process
It realizes record screen, prompts user to upload to HTTP server after completion to be played；
(8) HTTP server crawl video-see during network data and with the terminal log of upload realization be associated with；
When user selects one video of viewing, terminal APP initiates " selection video " message to server, and server is according to movement at this time
The IP address of terminal grabs network packet；When video playing is completed, terminal APP initiates " finishing playing " to server and disappears
Breath, server stops being the mobile terminal packet capturing at this time, and by the network data file of preservation and the terminal log that receives into
Row association；
(9) data analytic construction standard data set: first according to the HAS video of differing transmission protocols carrying from HTTP service
Different qos parameters is extracted in the network data of device crawl, comprising: under extracting to the HAS video network data carried based on TCP
Row real time rate, packet loss and round-trip delay (Round Trip Time, RTT) parameter, only to the network data carried based on UDP
Extract downstream rate parameter；Then the synchronization message of HTTP server is reported to according to terminal APP timing, by terminal log with
Qos parameter realizes time synchronization；It finally will include end message, video information, user's operation behavior, video QoE parameter, user
The standard data set of MOS scoring and network QoS parameter saves as file or deposit database.
Construction method of the present invention evaluation and test suitable for the standard data set of encrypted video QoE, it is distributed due to taking
The data acquisition modes that feedback drive test terminal is combined with VPN centralization, the architecture design of data acquisition platform are supported many
The data acquisition for the crowdsourcing model (Crowdsourcing) that multi-user tests simultaneously；Data acquisition is efficient and convenient, watches video mistake
Terminal APP records screen, recording played state and prompt user automatically and scores to form journal file and be automatically uploaded to HTTP clothes in journey
Business device can obtain automatically in HTTP server single-point acquiring and related network data, video QoE parameter and MOS score；Data
Dimensional information is comprehensive, and other than network data, video QoE parameter and MOS scoring, data set further includes end message, video letter
Breath, user's operation behavior and record screen file；Collecting method scalability is strong, the data set construction method tool based on data flow
There is versatility, mobile network-oriented operator/equipment vendor carries out the building of the standard exercise data set under encrypted video QoE evaluation and test
Method is suitable for encryption and non-encrypted video traffic based on TCP and udp protocol.The method have the advantages that:
(1) compared to the common single-point/multi-point data acquisition of traditional mobile video business, since data of the present invention acquisition is flat
Platform is combined by terminal APP and HTTP server, only can obtain video network data, terminal simultaneously in HTTP server single-point
Information, video information, QoE parameter, user's operation behavior, MOS score data and record screen file, data acquisition are convenient and true
Comprehensively.
(2) present invention develops the independent Android and IOS mobile terminal that HAS video website is supported suitable for mainstream
APP.For the video website of open programming interface, terminal APP directly passes through interface and realizes video playing；For being programmed without open
Realize video playing in the address of the website of interface, the video that APP is obtained by web crawlers.Pass through both modes, terminal APP
Realize the support to more major video website.
(3) authentic and valid QoE parameter and user MOS can be obtained by simple interactive operation to score.It is broadcast in video
Terminal APP automatically records the broadcast state of video and is saved in journal file during putting, and video broadcasting condition includes video QoE
Parameter and user's operation behavior.At the end of video playing, terminal APP automatically prompting user scores to the video of viewing, then will
The video playing journal file of formation is reported to HTTP server.
(4) the record screen file support of terminal APP repeatedly scores to watched video, and then obtains more accurate use
Family MOS scoring.
(5) present invention develops the HTTP server based on C++, can be respectively the video that is played of each access terminal from
Dynamic crawl network data, and the video playing journal file and record screen file that report with terminal are realized and are associated with.Simultaneously go back multiple access,
Mostly concurrent, support carries out extensive data acquisition by way of Crowdsourcing.
(6) HTTP server carries out data acquisition, this IP-based data acquisition modes according to IP address of mobile terminal
It can be widely applicable for the encryption and non-encrypted mobile data services of various IP/TCP and IP/UDP agreements.
Detailed description of the invention
Fig. 1 is the network topology schematic diagram of the data acquisition platform of the embodiment of the present invention 1.
Fig. 2 is that the data acquisition platform terminal APP of the embodiment of the present invention 1 and HTTP server cooperative work principle are illustrated
Figure.
Fig. 3 is 2 data resolving schematic diagram of the embodiment of the present invention.
Fig. 4 is the time synchronization schematic illustration in 2 data resolving of the embodiment of the present invention.
Specific embodiment
The construction method for the standard data set that present invention evaluation and test is suitable for evaluating and testing encrypted video QoE mainly includes data acquisition
Platform and data parsing, further specifically introduce below by drawings and examples.
Embodiment 1:
Present embodiments provide building side of the network-oriented operator evaluation and test suitable for the standard data set of encrypted video QoE
The universal data collection platform of method and data acquisition based on data acquisition platform, Fig. 1 are that the data of the present embodiment are adopted
Collect the network topology schematic diagram of platform.As shown in Figure 1, mobile terminal can access major video website by two ways: (a)
In such a way that VPN is acted on behalf of, using mobile terminal A1 by VPN proxy server A3, using switch A 5 and HTTP service
Device A7 realizes access major video website；(b) in such a way that local area network wireless routes, pass through no route using mobile terminal A2
By A4, access major video website is realized using switch A 5 and HTTP server A7.It watches during video, is taken in HTTP
It is engaged in actively acquiring network data, terminal video broadcast state and user's MOS score data on the network interface card 1 of device A7.The data are adopted
Collecting platform includes with lower module:
Data acquisition local area network A8 as shown in Figure 1 provides access and the reality by way of network share for mobile terminal
Now to the access of major video website, mainly by vpn server A3, wireless routing A4, switch A 5, Windows server A 6
It is formed with HTTP server A7.Data acquisition local area network A8 can be configured to any Class C IP address, be configured as in Fig. 1
192.168.137.0/24 network segment.Wherein vpn server A3 is the double netcard vpn server built based on Linux, switch A 5
For gigabit L2 interchanger, Windows server A 6 is domain and the Dynamic Host Configuration Protocol server of local area network, and HTTP server A7 is double netcard
Windows server.The network interface card 0 of vpn server A3 configures public network IP, provides VPN access, 1 configuration office of network interface card for mobile terminal
Net IP in domain.The network interface card 1 of HTTP server A7 configures LAN IP, and as the gateway of local area network, and network interface card 0 configures public network IP, can
To access major video website.By on vpn server A3 configuration strategy route, the data of network interface card 0 can pass through 1 turn of network interface card
It is dealt into the network interface card 1 of HTTP server A7, then by way of sharing network with HTTP server A7 network interface card 0, HTTP server A7
The accessible major video website of network interface card 1, thereby realize VPN act on behalf of by way of (a) access major video website；
Mobile terminal A2 is directly connected to wireless routing A4, can be realized and accesses major video in such a way that local area network wireless routes (b)
Website.Data are acquired local area network A8 and can be supported so more with expanded configuration more vpn server A3 and HTTP server A7
VPN user's access and higher network bandwidth access major video website is provided, be applicable to larger scale data acquisition.
Fig. 2 is the data acquisition platform terminal APP and HTTP server cooperative work schematic illustration of the present embodiment.Such as figure
The principle of mobile terminal A1 or A2 and HTTP server the A7 cooperative work of data acquisition platform shown in 2: mobile terminal APP
Comprising Android and ios release, the APP of Android version is developed based on Java, and the APP of ios release is developed based on Swift.
HTTP server is the client/server based on Microsoft Foundation class libraries (Microsoft Foundation Classes, MFC)
The HTTP server of (Client/Server, C/S) mode, using completing port technology (I/O Completion Port,
IOCP), it can satisfy the concurrent demand of multi-mobile-terminal, multi-connection.Start HTTP server, monitor terminal APP connection first
C1 is requested, while terminal connects VPN or local area network WIFI B1.When terminal APP initiates the connection B2, while it is fixed to start synchronizing thread
When send synchronizing information B10 and send end message B3, at this time HTTP server record terminal IP C2, and receive terminal letter
Cease C3.Then user selects video source B4 and to send video information B5, at this time HTTP server terminal IP and end based on the received
Client information, starting packet capturing thread C4 crawl address is the network packet C6 of IP, and receives video information C5.Then terminal
APP video enters broadcast state B6, and completion to be played or user terminate terminal APP prompt user's scoring B7 after playing.
Terminal APP automatically records video broadcasting condition, user's scoring, video information, end message to view in whole process
Frequency plays log, uploads log B8 to terminal APP after user's scoring, accordingly HTTP server receives APP log C7 and protects
It is stored to local.If user continues to watch, user continues to select video source B4, otherwise step after repeating terminates B9 and closes
Close synchronizing thread and terminal APP.It should be noted that uploading later no matter user's selection continues to watch video or knot to log
Beam, HTTP server can terminate packet capturing thread C8, and the network packet grabbed is saved as file and and video playing
Log is associated with C8, finally obtains in network data, video playing log and the video display process in a watching process
Record screen file, the standard data set C9 of evaluation and test encrypted video QoE can be obtained using the parsing of further data.
Due to the IOCP technology of use, HTTP server is still during interacting with terminal APP for above-mentioned HTTP server
The connection request of other terminal APP can be so received simultaneously, i.e. HTTP server supports multiple terminals, how concurrent connection request, branch
It holds and carries out extensive data acquisition by way of crowdsourcing.It watches during video, HTTP server is each of each terminal
Video acquires network data respectively, can be accurately by the broadcast state of each video and its network data pair by synchronizing information
It should get up.Due to HTTP server according to IP address acquire network data, so data acquisition platform be widely applicable for it is various
Encryption and non-encrypted mobile data services based on IP/TCP and IP/UDP agreement.
Embodiment 2:
Present embodiments provide the number of the construction method of the standard data set of network-oriented operator evaluation and test encrypted video QoE
Standard data set is constructed according to parsing.Fig. 3 is the data resolving schematic diagram of the present embodiment 2.As shown in Figure 3, in embodiment 1
On the basis of obtaining network data and terminal plays log, to network data parsed to obtain qos parameter and with log is played it is real
Now synchronous building standard test data set.
Network data D2 is the network data based on TCP/UDP agreement obtained by the data acquisition platform of Fig. 1, is saved
For the file of PCAP format.PCAP file format is a kind of network packet storage format that Wireshark is supported, comprising primary
All-network data during video-see.Video playing log D1 is that terminal APP is automatically recorded and uploaded in embodiment 1
Log, your video broadcasting condition, user's scoring, video information, end message.
Data parsing parses network QoS parameter D3 first from PCAP file, the synchronizing information then initiated by terminal
Video playing log D1 and qos parameter D3 time are synchronized.TCP network QoS parameter D3 is time series, including lower scanning frequency
Rate, downlink packet loss and round-trip delay, UDP only include downstream rate time series.Statistical interval △ t may be configured as in embodiment
0.1,0.5 or 1 second.The wherein downstream rate in the time (rate), packet loss (loss) and round-trip delay (rtt) statistical method
It is as follows:
V is the downlink data amount in the △ t time, and lostPkt and N are downlink packet loss and total downlink data packet in the △ t time
Quantity, rttiIt is i-th of downlink data packet in the time from being issued to the time delay being identified.
Due to mobile terminal time and HTTP server time, there may be deviations, so needing to be initiated according to terminal same
Message is walked further by video playing log and above-mentioned time series synchronization, Fig. 4 gives in the present embodiment data resolving
Time synchronization schematic illustration.In Fig. 4, terminal time is indicated with ti, i=1,2...n；Ti, i=1,2...n indicate HTTP
Server time；Synchronization message i, i=1,2...n indicate the synchronization message that terminal APP timing is initiated to HTTP server；Black
Triangle indicates the broadcast state of terminal APP video player, including the use for jumping forward, jump backward, suspending and stopping to play
The QoE parameter of family operation behavior and initial buffer and video cardton.It is as shown in Figure 4: terminal APP terminal time t1,
T3, t4 and tn have initiated to have sent synchronization message 1, synchronization message 2, synchronization message 3, synchronization message n, and HTTP is in server time
T1, T3, T4 and Tn receive corresponding synchronization message.User has carried out " jumping forward " operation after synchronization message 1, according to
It is HTTP server time T2 that synchronization message 1, which " should can jump forward " playing status synchronism, is indicated are as follows: T2=t2+ (T1-
t1)；T5 the and t6 moment has recorded a Caton information of player: Caton, t6 moment video playing knot occur for t5 moment video
Beam Caton starts to play, and t5 can use method same as described above synchronous for HTTP service respectively with t6 according to synchronization message 3
Device time T5 and T6；Other video broadcasting condition synchronous method are consistent with the above.The synchronous method is thought in synchronization message twice
Between, propagation delay time of the message from terminal APP to HTTP server is constant.
The time synchronization of qos parameter D3 and realization qos parameter D3 and video playing log are obtained by parsing network data,
The standard test data set suitable for encrypted video QoE can be constructed.Standard test data set includes end message/video information
D4, user's operation behavior D5, MOS scoring D6, QoE parameter D7 and network QoS parameter D3；For the network QoS based on udp protocol
Parameter D3 is downlink network rate, further includes additional network RTT for the standard test data set qos parameter D3 based on TCP
And packet loss.
Claims (1)
1. a kind of construction method for the standard data set for evaluating and testing encrypted video QoE acquires nothing by way of mobile terminal drive test
Video, user and network data under gauze network in mobile video traffic watching process construct standard data set, it is characterised in that:
It is combined by distributed feedback drive test terminal with the acquisition of VPN centralization data, data acquisition platform is only in HTTP server
Video, user and network data needed for side can acquire building data set；HTTP server uses IP-based data acquisition side
Formula is with the encryption and non-encrypted mobile data services suitable for various IP/TCP and IP/UDP agreements；Concrete operation step are as follows:
(1) mobile terminal accesses data acquisition platform in such a way that VPN is acted on behalf of, and opens terminal APP selection video and is watched；
(2) it includes terminal brand and model, operating system, Termination ID, network operation that terminal APP, which is obtained used in viewing video,
Simultaneously video playing log is recorded in end message including quotient, network access mode, IP address of terminal and signal strength indication；Wherein
Network operator includes mobile, connection and telecommunications, and network access mode includes 3G, 4G and WIFI, and IP address of terminal is viewing view
The IP address of terminal when frequency, signal strength indication are quantified as 0~4 according to practical received signal strength from weak to strong；
(3) video information is obtained by terminal APP and video playing log is recorded, video information includes video ID, video name
Claim the playable duration with video；
(4) the user's operation behavior in video display process is recorded by terminal APP and broadcasting log is recorded, user's operation row
Being includes suspending, jumping forward, jump backward and stop to play；
(5) pass through the QoE parameter of terminal APP record mobile terminal network video to broadcasting log；The QoE parameter of video is mainly
The objective quality index of mobile video business, the first time slack including video prolongs, Caton information, video resolution and the video of video
Actual play duration；Wherein the first time slack of video prolongs finger user is played to since click and plays elapsed time, card
At the time of information includes video cardton and Caton duration；
(6) the MOS scoring that user is recorded by terminal APP, then uploads to HTTP server for journal file；Video playing is complete
At when terminal APP prompt user score, user is divided into from difference to favorable comment according to the experience in watching process, by video tastes
1~5；
(7) terminal APP provides automatic record screen function, and after the license of user, terminal APP can realize video display process
Record screen prompts user to upload to HTTP server after completion to be played；
(8) HTTP server crawl video-see during network data and with the terminal log of upload realization be associated with；When with
When selection one video of viewing of family, terminal APP initiates " selection video " message to server, and server is according to mobile terminal at this time
IP address grab network packet；When video playing is completed, terminal APP initiates " finishing playing " message to server, this
When server stop being the mobile terminal packet capturing, and the network data file of preservation is closed with the terminal log received
Connection；
(9) it data analytic construction standard data set: is grabbed first according to the HAS video that differing transmission protocols carry from HTTP server
Different qos parameters is extracted in the network data taken, comprising: it is real to extract downlink to the HAS video network data carried based on TCP
When rate, packet loss and round trip time parameter, downstream rate parameter is only extracted to the network data carried based on UDP；Then basis
Terminal APP timing is reported to the synchronization message of HTTP server, and time synchronization is realized in terminal log and qos parameter；Finally will
Include end message, video information, user's operation behavior, video QoE parameter, the standard of user MOS scoring and network QoS parameter
Data set saves as file or deposit database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810016344.XA CN108200471B (en) | 2018-01-08 | 2018-01-08 | A kind of construction method of standard data set that evaluating and testing encrypted video QoE |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810016344.XA CN108200471B (en) | 2018-01-08 | 2018-01-08 | A kind of construction method of standard data set that evaluating and testing encrypted video QoE |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108200471A CN108200471A (en) | 2018-06-22 |
| CN108200471B true CN108200471B (en) | 2019-08-16 |
Family
ID=62588596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810016344.XA Expired - Fee Related CN108200471B (en) | 2018-01-08 | 2018-01-08 | A kind of construction method of standard data set that evaluating and testing encrypted video QoE |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108200471B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109347726B (en) * | 2018-11-06 | 2021-02-09 | 西安海润通信技术有限公司 | Mobile phone internet network perception quality evaluation method based on WeChat public number |
| CN112887795B (en) * | 2021-01-26 | 2023-04-21 | 脸萌有限公司 | Video playing method, device, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104023225A (en) * | 2014-05-28 | 2014-09-03 | 北京邮电大学 | No-reference video quality evaluation method based on space-time domain natural scene statistics characteristics |
| CN105843876A (en) * | 2016-03-18 | 2016-08-10 | 合网络技术（北京）有限公司 | Multimedia resource quality assessment method and apparatus |
| CN106850298A (en) * | 2017-02-08 | 2017-06-13 | 中国联合网络通信集团有限公司 | IPTV service quality monitoring method, terminal and system |
| CN106998322A (en) * | 2017-02-20 | 2017-08-01 | 南京邮电大学 | A kind of stream sorting technique of the Mean Opinion Score characteristics of mean of use video traffic |
| CN107071399A (en) * | 2017-04-26 | 2017-08-18 | 华为技术有限公司 | The method for evaluating quality and device of a kind of encrypted video stream |
| CN107404646A (en) * | 2016-05-20 | 2017-11-28 | 华为技术有限公司 | The method, apparatus and headend of video quality assessment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102868666B (en) * | 2011-07-07 | 2015-09-23 | 北京东方文骏软件科技有限责任公司 | Based on the implementation method of the mutual stream media quality Surveillance of Consumer's Experience |
| CN102685790B (en) * | 2012-05-22 | 2014-09-10 | 北京东方文骏软件科技有限责任公司 | Method for evaluating QoE (Quality of Experience) of mobile streaming media service perception experience by simulating user behaviors |
| CN104349220A (en) * | 2014-11-25 | 2015-02-11 | 复旦大学 | Service quality monitoring system for intelligent television terminal |
| CN106470431B (en) * | 2015-08-14 | 2019-08-02 | 中国联合网络通信集团有限公司 | A kind of method and device for obtaining user and perceiving rate |
| CN106254153B (en) * | 2016-09-19 | 2019-12-10 | 腾讯科技（深圳）有限公司 | Network anomaly monitoring method and device |
-
2018
- 2018-01-08 CN CN201810016344.XA patent/CN108200471B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104023225A (en) * | 2014-05-28 | 2014-09-03 | 北京邮电大学 | No-reference video quality evaluation method based on space-time domain natural scene statistics characteristics |
| CN105843876A (en) * | 2016-03-18 | 2016-08-10 | 合网络技术（北京）有限公司 | Multimedia resource quality assessment method and apparatus |
| CN107404646A (en) * | 2016-05-20 | 2017-11-28 | 华为技术有限公司 | The method, apparatus and headend of video quality assessment |
| CN106850298A (en) * | 2017-02-08 | 2017-06-13 | 中国联合网络通信集团有限公司 | IPTV service quality monitoring method, terminal and system |
| CN106998322A (en) * | 2017-02-20 | 2017-08-01 | 南京邮电大学 | A kind of stream sorting technique of the Mean Opinion Score characteristics of mean of use video traffic |
| CN107071399A (en) * | 2017-04-26 | 2017-08-18 | 华为技术有限公司 | The method for evaluating quality and device of a kind of encrypted video stream |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108200471A (en) | 2018-06-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10841358B2 (en) | System and method for determining quality of a media stream | |
| Bronzino et al. | Inferring streaming video quality from encrypted traffic: Practical models and deployment experience | |
| CN104115448B (en) | The method and apparatus of monitoring network transmission characteristic | |
| CN104040953A (en) | Quality of user experience testing for video transmissions | |
| CN105024872B (en) | The method and device of applied in network performance test | |
| Casas et al. | Monitoring YouTube QoE: Is your mobile network delivering the right experience to your customers? | |
| CN108200471B (en) | A kind of construction method of standard data set that evaluating and testing encrypted video QoE | |
| KR102428093B1 (en) | Data processing methods, servers and data collection devices | |
| Ben Letaifa | Real time ml-based qoe adaptive approach in SDN context for HTTP video services | |
| Robitza et al. | Measuring YouTube QoE with ITU-T P. 1203 under constrained bandwidth conditions | |
| Baena et al. | Video Streaming and Cloud Gaming services over 4G and 5G: a complete network and service metrics dataset | |
| US11522939B2 (en) | Over-the-top media service testing and QoE issues isolation | |
| EP3985984A1 (en) | System and method for managing video streaming quality of experience | |
| Varvello et al. | Performance characterization of videoconferencing in the wild | |
| CN107659856A (en) | The acquisition method of mobile video business experience qualitative data collection based on user feedback | |
| Lyu et al. | Network Anatomy and Real-Time Measurement of Nvidia GeForce NOW Cloud Gaming | |
| Wu et al. | Resolution identification of encrypted video streaming based on http/2 features | |
| Wehner et al. | Beauty is in the eye of the smartphone holder a data driven analysis of youtube mobile qoe | |
| Göring et al. | A framework for QoE analysis of encrypted video streams | |
| Bartolec et al. | Impact of user playback interactions on in-network estimation of video streaming performance | |
| Bhargava et al. | Comparative evaluation of user perceived quality assessment of design strategies for HTTP-based adaptive streaming | |
| Suman et al. | Netflix, Amazon Prime, and YouTube: comparative study of streaming infrastructure and strategy | |
| Lopes et al. | A multi-layer probing approach for video over 5G in vehicular scenarios | |
| Stensen | Evaluating QoS and QoE Dimensions in Adaptive Video Streaming | |
| Metzger et al. | Context monitoring for improved system performance and QoE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190816 Termination date: 20220108 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |