WO2013147226A1 - 利用者体感品質推定装置、端末ボトルネック判定装置、類似操作抽出装置、及び方法、並びにプログラム - Google Patents
利用者体感品質推定装置、端末ボトルネック判定装置、類似操作抽出装置、及び方法、並びにプログラム Download PDFInfo
- Publication number
- WO2013147226A1 WO2013147226A1 PCT/JP2013/059680 JP2013059680W WO2013147226A1 WO 2013147226 A1 WO2013147226 A1 WO 2013147226A1 JP 2013059680 W JP2013059680 W JP 2013059680W WO 2013147226 A1 WO2013147226 A1 WO 2013147226A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- continuous area
- log
- continuous
- determination
- time
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3419—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/40—Data acquisition and logging
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/81—Threshold
Definitions
- the present invention according to the first aspect relates to a user experience quality estimation apparatus and method, and in particular, a user experience quality estimation apparatus that estimates the quality of an application experienced by a service user based on a user experience waiting time, and Regarding the method.
- the present invention according to the second aspect also relates to a terminal bottleneck determination apparatus and method, and more particularly, to a terminal bottleneck determination apparatus, method and program for determining a quality degradation factor when using an application.
- the present invention according to the third aspect relates to a similar operation extraction device and method in a quality degradation determination technique at the time of application execution, and in particular, an operation similar to an operation by a user given as an input at the time of application execution.
- the present invention relates to a similar operation extraction apparatus, method, and program for extraction.
- the following methods exist as a method for acquiring the user experience waiting time related to the application.
- the user's experience waiting time means the time from when the user clicks the screen until the execution result is displayed on the screen.
- (A) A method of acquiring and presenting an index close to the user experience waiting time such as the response time of an HTTP message exchanged between the terminal and the server (for example, see Non-Patent Document 1).
- a terminal bottleneck is determined as to whether or not a terminal has a quality deterioration factor when quality deterioration represented by a longer waiting time for the user is experienced.
- the “experience waiting time” here means a period of time until the execution result is displayed on the screen after the user clicks the screen.
- the application is characterized in that the software and data managed on the server side can be used by accessing the server via a Web browser, dedicated client software, etc. without installing the application on the user terminal. .
- Non-Patent Document 5 there is a problem that the user's bodily sensation quality is easily influenced by the delay of the server and the network, and it is difficult to guarantee a certain bodily bodily quality. Therefore, it is necessary to constantly monitor the user's bodily sensation quality and appropriately deal with the quality degradation. For example, there is a method in which the response time between the client and the server is constantly monitored, and when the response time becomes longer than the normal time, it is determined that the quality is deteriorated (for example, see Non-Patent Document 5).
- Non-Patent Document 6 describes the necessity of monitoring quality monitoring that takes into account user terminal factors, that is, the sensory waiting time.
- FIG. 1 shows a simplified operation when a user operation is performed.
- the time from the start of the user's operation to the time when the screen display is completed (t4-t0) is “Issuing a request from the terminal” ⁇ “Response from the server” ⁇ “Screen generation process at the terminal Is comprised.
- the terminal processing time (t4 ⁇ t3) is caused as an error.
- the conventional technique related to the second aspect has the following two problems.
- the conventional techniques (A) and (B) related to the second aspect have a problem that the correlation with the application is low.
- Resources such as CPU utilization are a measure of the load state of the terminal, but the correlation between this index and the application operation speed is weak except for a state where the terminal load is extremely high.
- a terminal bottleneck is suspected if the CPU usage rate at an appropriate time interval is always 100%, but it cannot be determined if it is 80% or 60%. Even if the CPU usage rate is 100%, there may be no problem in the application operation speed due to the process priority.
- the terminal specifications are the same, and it is difficult to correlate the extremely rough summarization of terminal specifications with the operation speeds of various application types and operation types.
- the related art relating to the third aspect has the following problems.
- the user experience waiting time varies depending on the operation performed by the user when executing the application. This is because the amount of processing generated by the operation and the complexity of the processing differ depending on the operation performed by the user when the application is executed, which leads to a difference in processing time and a difference in waiting time for the user experience. For this reason, when the waiting time is longer than usual, the method of determining quality degradation is a process that takes a long time if the operation is different from usual, especially when a complicated operation with a large amount of application processing is performed. In addition, since the user experience waiting time becomes longer than usual, it may be erroneously determined that quality degradation has occurred in spite of no quality degradation occurring in any of the server, network, and user terminal. . Therefore, in order to correctly determine the deterioration of the sensation quality, it is necessary to compare with the sensation waiting time when an operation similar to the past is performed.
- Non-Patent Document 7 (CA APM http://systemwalker.fujitsu.com/jp/caapm/) presents a method for monitoring server processing time in units of application methods, and the same method is executed in the past. It can be compared with the processing time.
- the processing time in the network and the user terminal also affects the user experience waiting time, it is not sufficient to simply monitor the server processing time.
- the contents of the method are monitored, the contents of the operation performed by the user are specified, and this method is not desirable from the viewpoint of protecting the personal information of the user.
- the present invention has been made in view of the above-described problems related to the first to third aspects.
- the first object of the present invention is based on the abstracted processing type (Network: data acquisition, Scripting: script execution, Rendering: screen drawing) acquired from an application such as a Web browser and the time data required. It is to provide a technology that enables a third party to estimate the experience waiting time of an application user as the experience quality.
- the second object of the present invention is that the terminal is based on the abstracted processing type (Network, Scripting, Rendering) acquired from an application such as a Web browser and the time data required for the processing, and the terminal has a quality of user experience quality. It is to provide a technology that enables a third party to determine whether or not it is a deterioration factor.
- Network Network, Scripting, Rendering
- a third object of the present invention is to perform an operation performed in the past similar to the operation performed by the user at the time of executing the application without monitoring the method contents in order to improve the quality degradation determination accuracy at the time of executing the application. It is to provide a technique that enables discrimination.
- a user experience quality estimation device for estimating a user experience waiting time of an application in a user terminal
- Data receiving means for acquiring the duration of each process of data acquisition, script execution, and screen drawing performed in the estimation target period in the user terminal as an estimation target log and storing it in a reception log storage means
- a log excluding unit that reads the estimation target log from the reception log storage unit, and outputs a log in which the data duration is shorter than a predetermined short time processing threshold or a log longer than a predetermined long time threshold
- Quantization means for calculating the number of each processing performed in a time slot of a fixed time as a multiplicity for the log output from the log exclusion means
- Continuous region extraction means for extracting a continuous region from the data quantized by the quantization means
- Have The continuous area extracting means includes A first continuous area extracting means for extracting a continuous area without distinguishing each process of the data acquisition, the script execution, and the screen drawing; or There is provided
- a quality degradation factor exists in a user terminal when using a service of an application provided from an application server.
- Quality degradation factor judgment device Data receiving means for acquiring the duration of each process of data acquisition, script execution, and screen drawing performed in the estimation target period in the user terminal as a determination target log and storing it in a reception log storage means; The judgment target log is compared with the reference log when the same operation as that for the application performed on the user terminal is performed, or the data of the experience cumulative distribution of the continuous area length of the script execution and screen drawing processing.
- Bottleneck determining means for determining an evaluation value and determining whether or not the user terminal has a quality degradation factor based on a comparison result between a total value of the evaluation values and a preset threshold value.
- a quality degradation factor determination device characterized by having the above.
- a similar operation extraction device that extracts an operation similar to an input operation that is an operation given as an input when executing an application.
- a feature amount calculation means for calculating a feature amount based on a log corresponding to a user experience waiting time cut out from only one of the application log and the network log, or a combination of both;
- Similarity calculation means for calculating the similarity between the feature quantity and the feature quantity saved in the past based on the calculated feature quantity, the feature quantity saved in the storage means in the past, and a predetermined similarity function
- a similar operation extracting device characterized by comprising:
- the present invention corresponding to the first object, at the start / end time of data acquisition (Network), script execution (Scripting), and screen rendering (Rendering) of an application such as a Web browser on the terminal
- Network data acquisition
- Scripting script execution
- Rendering screen rendering
- log data that is series information
- the estimation accuracy of the user experience quality (user experience waiting time) is improved by the present invention.
- terminals from other than user terminals can be used for a wide variety of applications. It is possible to always estimate the user experience waiting time in consideration of the influence of the side processing time, and it is possible to constantly monitor the user experience quality based on the user experience waiting time by a third party.
- the cause identifying operation is constructed in consideration of the quality deterioration of the terminal factor.
- the bottleneck determination reflecting the influence and the terminal state at the time of the failure can be realized, and the bottleneck determination can be made more accurate than the conventional one.
- a user for a similar past operation is determined by discriminating and extracting an operation performed in the past that is similar to the operation performed by the user when executing the application. It is possible to acquire the waiting time for the experience. By comparing the user experience waiting time at the time of execution of the application subject to quality degradation and the user experience waiting time for the similar past operation extracted above, it is not affected by the operation content, and is end-to-end. It is possible to detect only the longer waiting time of the sensation due to the quality degradation of the product, and the accuracy of the quality degradation determination is improved.
- Embodiment 2-1 of this invention It is an example of the data input into the bottleneck determination part in Embodiment 2-1 of this invention. It is a figure for demonstrating the reference determination in Embodiment 2-1 of this invention. It is a flowchart of the reference determination process in Embodiment 2-1 of this invention. It is sample data of various threshold evaluation in Embodiment 2-1 of this invention. This is an example of quantization in Embodiment 2-1 of the present invention. It is a flowchart of the continuous area
- Embodiment 2-2 of this invention It is an example of a structure of the terminal bottleneck determination apparatus in Embodiment 2-2 of this invention. It is a figure for demonstrating the non-reference determination in Embodiment 2-2 of this invention. It is an empirical cumulative distribution of continuous region length in the embodiment 2-2 of the present invention. It is an empirical cumulative distribution of continuous region length in the embodiment 2-2 of the present invention. It is a flowchart of the non-reference determination in Embodiment 2-2 of this invention. It is a specific example of the non-reference determination method in Embodiment 2-2 of this invention. It is a flowchart of the non-reference determination in Embodiment 2-2 of this invention. It is an example of evaluation in Embodiment 2-2 of the present invention. This is an evaluation method (No.
- time slot length of sensation waiting time ⁇ 10
- time slot length of sensation waiting time ⁇ 10
- the first embodiment is an embodiment corresponding to the first object.
- the processing type (Network, Scripting, Rendering) abstracted from the Web browser of the user terminal and the time data required are acquired in time series, and the browser application is used for the browser application after the operation by the user. (After a screen click or the like), the user's bodily sensation waiting time is estimated as the user bodily quality for the time from when the response of the browser application is received until the execution result is displayed on the screen.
- FIG. 4 shows an example of the system configuration of the present embodiment.
- the system includes a quality estimation apparatus 1100, a user terminal 1200, and a browser application server 1300.
- the user terminal 1200 includes a Web browser 1210, a browser log acquisition unit 1220, and a transmission unit 1230, and executes the content 1310 (application) of the application server 1300 on the Web browser 1210.
- the browser log acquisition unit 1220 acquires a browser processing information log, and the transmission unit 1230 transmits data to the quality estimation apparatus 1100.
- the quality estimation apparatus 1100 includes a reception unit 1110, a waiting time estimation unit 1120, and an estimation result storage / display unit 1130.
- the reception unit 1110 receives a log transmitted from the user terminal 1200, and the waiting time estimation unit 1120 The sensory waiting time is estimated, and the result is stored / displayed in the estimation result storage / display unit 1130.
- the browser application server 1300 has an HTTP server 1310, and content 1320 is stored in the storage means.
- content 1320 is stored in the storage means.
- an application related to a business including dynamic content is assumed as the browser application.
- the “browser application” may be referred to as a “Web application”.
- the first embodiment includes the embodiment 1-1 and the embodiment 1-2. First, Embodiment 1-1 will be described.
- FIG. 5 shows an example of the configuration of the quality estimation apparatus according to Embodiment 1-1 of the present invention.
- the quality estimation apparatus 1100 shown in the figure stores it in the reception log storage unit 1101.
- the data stored in the reception log storage unit 1101 includes all processes of data acquisition (Network), script execution (Scripting), and screen rendering (Rendering) performed by the Web browser 1210 of the user terminal 1200 during the estimation target period. Target the following times.
- Network time start / end time of data acquisition session
- Scripting time script execution time such as Javascript (registered trademark)
- Rendering time start-end time of screen drawing time Example of data stored in reception log storage unit 1101 in FIG. Show.
- the operation of the quality estimation apparatus 1100 is shown below.
- FIG. 7 is a flowchart of the operation of the quality estimation apparatus according to Embodiment 1-1 of the present invention.
- Step 1110 The waiting time estimation unit 1120 of the quality estimation apparatus 1100 reads log data from the reception log storage unit 1101. For processes with extremely short durations of Network, Scripting, and Rendering processes, the correlation with the user's sensory waiting time is low, so for processes that are shorter than the preset threshold L, from the read log data exclude.
- the threshold L to be set can be set for each process of Network, Scripting, and Rendering for each estimation target application.
- the threshold L is, for example, based on the relationship between the quality estimation result obtained by inputting sample data of various threshold evaluations as shown in FIG.
- a combination of duration values that most closely approximates the user's quality of experience is selected as a threshold value. For example, a value such as 5, 10, 20, 30,... Ms is input, and the result of quality estimation that best matches the user experience waiting time is selected. Specifically, about 1 million combinations including the parameters that appear later are set, 1 million ways of quality estimation are tried, and the combination of values closest to the user experience waiting time is used.
- the following threshold is set as the threshold L for the short-time processing for the application A.
- the threshold L is determined in advance and stored in a memory (not shown) in the waiting time estimation unit 1120.
- Network threshold 10ms Scripting threshold 5ms Rendering threshold 3ms At this time, when estimating the sensation waiting time of the application A, logs of processing times equal to or less than the above thresholds are excluded for Network, Scripting, and Rendering.
- Step 1120 Next, among the log data from which log data below the threshold value L is excluded in the processing of Step 1110 above, the processing in which the duration of Network, Scripting, and Rendering is extremely long is the user's sensory waiting time. Since the correlation is low, processing that takes a longer time than the preset threshold value H is excluded. Note that a threshold value H for each process of Network, Scripting, and Rendering can be set for each estimation target application.
- the threshold setting method is the same as in step 1110.
- the following threshold value H is set as a threshold value for the long-time processing for application A.
- the threshold value H is determined in advance and stored in a memory (not shown) of the waiting time estimation unit 1120.
- the quantization method is divided by a time slot of a fixed time, and the number of Network, Scripting, and Rendering processes performed in this time slot is calculated as multiplicity.
- the time slot length can be set according to the application.
- Fig. 9 shows an example of quantization.
- a time slot of 0.1 seconds is set for application B.
- For the application B log after the processing of steps 1110 and 1120, a time slot is set every 0.1 second, the number of processes in each slot is counted, and the multiplicity is calculated for each time slot.
- Step 1140 A continuous region is extracted from the data quantized in step 1130.
- evaluation is performed without distinguishing between Network, Scripting, and Rendering. Note that a method for distinguishing and evaluating Network, Scripting, and Rendering will be described later in Embodiment 1-2.
- FIG. 10 is a flowchart (part 1) of the continuous region extraction processing according to Embodiment 1-1 of the present invention.
- Step 1141) The input detection threshold value is set in a memory (not shown).
- Step 1142 The inputted sliding window length is set in a memory (not shown).
- Step 1143 The analysis is started from the first time slot.
- Step 1144) Extending the sliding window from the time slot, and if there is processing that exceeds the detection threshold in at least one time slot among the time slots in the sliding window (eg, detection in any one of Network, Scripting, and Rendering)
- the threshold value is exceeded, the time slot of the start point is determined to be within the continuous area. This determination process is performed while shifting the time slot one by one in the direction in which time elapses.
- Step 1146 A section in which sliding windows having time slots determined to be in a continuous area are continuous is determined as a continuous area.
- step 1146 An example of the determination method in step 1146 is shown in FIG. For example, when the sliding window length is 3 and the detection threshold is 3, if there is a time slot including a process exceeding the detection threshold in 3 slots in the elapsed time direction from the time slot, the time slot is within the continuous region. It is determined.
- FIG. 12 is a flowchart (part 2) of the continuous region extraction processing according to Embodiment 1-1 of the present invention.
- Step 1151) The input detection threshold value x is set in a memory (not shown).
- Step 1152 The input sliding window length y is set in a memory (not shown).
- Step 1153 The input continuous determination threshold value z is set in a memory (not shown).
- Step 1154 Advance time slot Ti by 1 in the direction in which time elapses.
- Step 1155 When the sliding window is extended from the time slot Ti as the starting point, if the existence ratio (ts / y) of the number of time slots (ts) exceeding the detection threshold x (ts / y) exceeds the continuous determination threshold z, Step 1156 If it does not exceed, return to Step 1154.
- the time slot exceeding the detection threshold (3) within 3 slots in the time lapse direction from the time slot (the number of time slots is If there are two or more (integer), the time slot is determined to be in the continuous area, and if there are one or less time slots exceeding the detection threshold, it is determined not to be in the continuous area.
- Step 1156 In the sliding window length section starting from the time slot Ti exceeding the continuous determination threshold, the area where the start point Ti continues is determined as a continuous area.
- Step 1150 The shaping process is performed on the continuous area extracted in Step 1140 according to the following rules.
- FIG. 14 shows a shaping example.
- Tmin is set for the purpose of excluding the waiting time (0.5 to 2 seconds) that the user cannot experience.
- Tmax aims to exclude situations where the system stops responding (freeze).
- the continuous area is stored in the storage means as the estimated waiting time or displayed on the display means. Note that continuous areas (estimated waiting time) are output for the continuous areas by the number of user operations. Specifically, when the application is operated 10 times, 10 continuous areas appear.
- Embodiment 1-2 In Embodiment 1-2, three methods for distinguishing and evaluating Network, Scripting, and Rendering in Step 1140 of FIG. 7 of Embodiment 1-1 will be described.
- step 1140 The configuration of the quality estimation apparatus and the processes other than step 1140 are the same as those in the embodiment 1-1, and thus the description thereof is omitted.
- Step 1140 of Embodiment 1-2 the method of distinguishing and evaluating Network, Scripting, and Rendering makes it possible to change the detection threshold and the like in each process of Network, Scripting, and Rendering, and extract the user experience waiting time The improvement of accuracy can be expected.
- the number of parameters that need to be set is larger than in “(2) Method of evaluating with time and multiplicity of sliding window continued (A-2)” in Embodiment 1-1.
- the first method (B-1) of the present embodiment will be described.
- FIG. 15 is a flowchart (part 1) of the continuous area extraction processing according to Embodiment 1-2 of the present invention.
- Step 1201 For each element of Network, Scripting, and Rendering, continuous area determination according to A-2 in Embodiment 1-1 is performed, and a continuous area of each element is extracted. To perform the same process as A-2, three parameters are required: sliding window length, detection threshold, and continuous determination threshold. Different values can be set for each element of Network, Scripting, and Rendering. It is.
- Step 1202 For each element of Network, Scripting, and Rendering, as shown in FIG. 16, the union of continuous areas is determined as a continuous area.
- FIG. 17 is a flowchart (part 2) of the continuous region extraction processing according to Embodiment 1-2 of the present invention.
- the area extracted by the processing of the flowchart is denoted as (B-2).
- Step 1211 For each element of Network, Scripting, and Rendering, A-2 continuous area determination processing in Embodiment 1-1 is performed to extract a continuous area of each element. This process requires three parameters, sliding window length, detection threshold, and continuous determination threshold, as in step 1201 of FIG. 15. However, different values can be set for each element of Network, Scripting, and Rendering. is there.
- Step 1212 A Scripting continuous area in the time slot range determined as a continuous area of Network processing or a time slot adjacent to the time slot range determined as a continuous area of Network processing (see FIG. 18). a) is extracted.
- Step 1213) The Rendering continuous area (b in FIG. 18) in the Scripting time slot range extracted in Step 1212 (b in FIG. 18) or the time slot adjacent to the Scripting time slot range is the start position. Extract.
- Step 1214 The logical sum of the Rendering continuous area extracted in Step 1213 and the Scripting and Network area that is the origin thereof is determined as an area (c in FIG. 18). When a plurality of areas overlap, the logical sum is set as a continuous area.
- FIG. 19 shows a flowchart (part 3) of the continuous region extraction processing in the embodiment of the present invention.
- the area extracted by the process of FIG. 17 is denoted as (B-2), and the area extracted by the process of the flowchart is denoted as (B-3).
- Step 1221 The process of B-2 shown in FIG. 17 is executed, and a continuous area configured in the order of Network ⁇ Scripting ⁇ Rendering is extracted.
- Step 1222 In the Network continuous area not extracted in Step 1221 (a in FIG. 20), the Rendering continuous area that starts within the time slot or adjacent to the end time slot of the Network continuous area (see FIG. 20). 20) b) is extracted.
- Step 1223 The logical sum of the Rendering continuous area extracted in Step 1222 and the Network continuous area that is the origin thereof is determined as a continuous area (c in FIG. 20).
- Step 1224 In the Scripting continuous area that has not been extracted in the processing of Step 1223 and B-2, the Rendering continuous area (d in FIG. 20) that starts adjacent to the end time slot of the Scripting continuous area is extracted.
- Step 1225 The logical sum of the Rendering continuous area extracted in Step 1224 and the Scripting continuous area that is the origin thereof is determined as a continuous area (e in FIG. 20).
- the user experience quality estimation apparatus shown in FIG. 5 can be realized by causing a computer to execute a program describing the processing contents described in the present embodiment. More specifically, the functions of each unit of the user experience quality estimation device are implemented by each unit using hardware resources such as a CPU, a memory, and a hard disk built in the computer constituting the user experience quality estimation device. This can be realized by executing a program corresponding to the processing to be performed.
- the program can be recorded on a computer-readable recording medium (portable memory or the like), stored, or distributed. It is also possible to provide the program through a network such as the Internet or electronic mail.
- the present embodiment is a user experience quality estimation program that causes a computer to function as a user experience quality estimation device for estimating a user experience wait time of an application in a user terminal.
- Data receiving means for acquiring the duration of each process of data acquisition, script execution, and screen drawing performed in the estimation target period in the user terminal as an estimation target log and storing it in a reception log storage means;
- a log excluding unit that reads the estimation target log from the reception log storage unit and outputs a log in which the data has a duration shorter than a predetermined short-time processing threshold or longer than a predetermined long-time threshold;
- Quantization means for calculating the number of processes performed in a time slot for a predetermined time as a multiplicity for the log output from the log exclusion means,
- a user experience quality estimation program that functions as a continuous region extraction unit that extracts a continuous region from data quantized by the quantization unit,
- the continuous area extraction means is a first continuous area extraction means for extracting a continuous area without distinguishing each process of the
- the second embodiment is an embodiment corresponding to the second object.
- FIG. 21 shows an example of the system configuration of the present embodiment.
- the system shown in the figure includes a terminal bottleneck determination device 2100, a user terminal 2200, and a browser application server 2300.
- the user terminal 2200 includes a Web browser 2210, a browser log acquisition unit 2220, and a transmission unit 2230, and executes the content 2310 (application) of the application server 2300 on the Web browser 2210.
- the browser log acquisition unit 2220 acquires a browser processing information log, and the transmission unit 2230 transmits data to the terminal bottleneck determination apparatus 2100.
- the terminal bottleneck determination apparatus 2100 includes a reception unit 2110, a bottleneck determination unit 2120, and a determination result storage / display unit 2130.
- the reception unit 2110 receives a log transmitted from the user terminal 2200, and the bottleneck determination unit The terminal bottleneck is determined at 2120, and the result is stored / displayed at the determination result storage / display unit 2130.
- the browser application server 2300 has an HTTP server 2310, and content 2320 is stored in the storage means.
- the content 2320 includes content including dynamic content and the like.
- the user terminal 2200 acquires the processing type (Network, Scripting, Rendering) abstracted from the Web browser and the time data required, and sends the data to the terminal bottleneck determination device 2100.
- processing type Network, Scripting, Rendering
- the terminal bottleneck determination device 2100 performs terminal bottleneck determination from the data.
- a reference determination method in which input data to be compared is given each time to perform bottleneck determination, and non-reference determination in which data to be compared is stored in advance and the stored results are compared There are two types of law.
- FIG. 22 shows a configuration example of the terminal bottleneck determination device in the embodiment of the present invention.
- a terminal bottleneck determination apparatus 2100 shown in the figure includes a reception log storage unit 2101, a reference log storage unit 2102, a reception unit 2110, a bottleneck determination unit 2120, and a determination result storage / display unit 2130.
- the reception log storage unit 2101 temporarily stores the following information acquired by the browser log acquisition unit 2220 of the user terminal 2200 and received by the reception unit 2110.
- FIG. 23 shows an example of stored data.
- the data shown in the figure is an example of a browser log from 0 second to 15 seconds after the start of the operation, and the browser log includes a start time (second), a duration (millisecond), and a processing type.
- the duration is a session duration in the network, and Rendering indicates a drawing time.
- a large amount of logs in which these processes are continued for about several ms to several tens of ms are output. For example, when the browser is operated for about 3 minutes, the log is output about 10,000 lines.
- the reference log storage unit 2102 temporarily stores a log (reference log) in which there is no disturbance factor in the user terminal 2200.
- a log reference log
- the reception log determination target log
- a log of the same operation as the reception log is required. If there is no log of the same operation, the operator or the like performs the same operation, accumulates the log, and compares it with the determination target log.
- the reference determination process of the bottleneck determination unit 2120 will be described below.
- FIG. 25 is a flowchart of reference determination in the embodiment 2-1 of the present invention.
- the receiving unit 2110 acquires the determination target log, stores it in the reception log storage unit 2101, collects the reference log by any of the methods described above, and stores it in the reference log storage unit 2102. To do.
- Step 2110) The bottleneck determination unit 2120 reads the logs from the reception log storage unit 2101 and the reference log storage unit 2102, respectively, and the processing with extremely short duration of Network, Scripting, and Rendering has a correlation with the user's waiting time. Since it is low, processing that takes a shorter time than a preset threshold is excluded from the log at this step.
- the threshold value is calculated empirically from sample data as shown in FIG. As the threshold value, for example, an arbitrary value is input, and the result of quality estimation is selected that most closely matches the user experience waiting time. Note that a threshold value for each process of Network, Scripting, and Rendering can be set for each estimation target application. As an example, an example of the short-time processing threshold for application A is shown below.
- Network threshold 10ms Scripting threshold: 5ms
- Rendering threshold 3ms
- Step 2120 Next, processing with an extremely long duration of Network, Scripting, and Rendering is excluded. Processing with extremely long durations of Network, Scripting, and Rendering takes a longer time than a preset threshold because the correlation with the user's waiting time is low. Note that a threshold value for each process of Network, Scripting, and Rendering can be set for each application to be determined. For example, Network threshold: 10000ms Scripting threshold: 30000ms Rendering threshold: 100000ms For Network, Scripting, and Rendering, logs with processing times equal to or greater than the above thresholds are excluded in this step.
- Step 2130 both the reference log and the determination target log from which short-time processing and long-time processing are excluded (steps 2110 and 2120) are evaluated by statistical values. Since the evaluation processing tends to increase the processing duration of Scripting and Rendering when the user terminal 2200 is a bottleneck, the following evaluation is performed.
- a) Evaluation by Scripting processing duration For each scripting process corresponding to each of the reference log and judgment target log, compare one of the following statistical values for the duration, and if the judgment target log is longer, it will be added as an evaluation of the judgment target log by the scripting process duration To do.
- the score value for each statistical value can be set individually. For example, the setting is as follows.
- Evaluation by Rendering processing duration For each corresponding Rendering process of the reference log and judgment target log, compare the same statistical value as the Scripting process for the duration, and if the judgment target log is longer, it is added as an evaluation of the judgment target log by the Rendering process duration To do.
- the score value for each statistical value can be set individually. Further, the present invention is not limited to this example, and an arbitrary statistical value can be set similarly to the above a).
- the total score obtained in this step is used in step 2170 for bottleneck determination based on the evaluation score.
- Step 2140 both the reference log and the determination target log are quantized.
- the data from which the short-time processing is excluded in step 2110 and the long-time processing is excluded in step 2120 is quantized at regular time intervals.
- the time is divided by time slots of a certain time, and the number of processings of Network, Scripting, and Rendering performed in this time slot is calculated as multiplicity.
- FIG. 27 shows an example of quantization. In the figure, a time slot of 0.1 seconds is set for application B. For the application B log, a time slot is set every 0.1 second, the number of processes in each slot is counted, and the multiplicity is calculated for each time slot.
- Step 2150 For each of the reference log and the judgment target log, continuous areas are extracted for both Scripting and Rendering. An example of continuous area extraction processing is shown below.
- FIG. 28 is a flowchart of continuous area extraction processing according to Embodiment 2-1 of the present invention.
- Step 2151 The input detection threshold value x is set in a memory (not shown).
- Step 2152 The inputted sliding window length y is set in a memory (not shown).
- Step 2153 The input continuous determination threshold value z is set in a memory (not shown).
- Step 2154 Advance time slot Ti by 1 in the direction in which time elapses.
- Step 2155 When the sliding window is extended with the time slot Ti as the starting point, if the existence ratio (ts / y) of the number of time slots (ts) exceeding the detection threshold x (ts / y) exceeds the continuous determination threshold z, Step 2156 If it does not exceed, return to Step 2154.
- Step 2156 It is determined that the time slot is within the range of the continuous area.
- FIG. 29 shows the determination method in step 2155 described above.
- the determination is made based on the existence ratio of time slots exceeding the detection threshold. For example, when the sliding window length is 3 and the continuous determination threshold is 0.5, if there are two or more time slots exceeding the detection threshold in the 3 slots in the elapsed time direction from the time slot, the time slot is determined to be in the continuous area. When the number of time slots is 1 or less, it is determined that the time slots are not continuous.
- Step 2160 evaluation is performed based on the statistical values of the continuous area length of Scripting and Rendering in the reference log and the determination target log. Since the continuous area length of Scripting and Rendering tends to be long when the user terminal 200 is a bottleneck, the following evaluation is performed.
- Evaluation by Rendering processing continuous area length The same statistical value as in the above Scripting process for each corresponding Rendering process continuous area of the reference log and the determination target log is compared. If the determination target log is longer, the evaluation is made as the evaluation of the determination target log based on the duration of the rendering process. In addition, the score value for each statistical value can be set individually.
- the total value of the above added points is used in the bottleneck determination in step 2170.
- Step 2170 Finally, based on the evaluation results up to steps 2130 and 2160, the bottleneck of the user terminal 2200 is determined.
- the evaluation based on the statistical value of the input information for the determination target log in step 2130 and the evaluation value based on the statistical value of the continuous region length in step 2160 are summed to determine whether the threshold is equal to or greater than a preset threshold. If there is a terminal bottleneck, it is determined.
- the evaluation based on the statistical value of the input information in step 2130 and the weighting of the statistical value of the continuous region length in step 2160 can be adjusted by the values added in step 2130 and step 2160.
- FIG. 30 shows an example of the determination.
- the evaluation result is 5 ⁇ 7. Determined as “terminal bottleneck”.
- the threshold value “10” for which the terminal bottleneck is strongly suspected and the threshold value “5” for which the terminal bottleneck is suspected are set, and the evaluation value is “12”, 10 ⁇ 12 Therefore, the evaluation result “highly suspected of terminal bottleneck” is obtained and output from the determination result storage / display unit 2130.
- the evaluation result is output to a storage medium such as a hard disk or a display means such as a display.
- Embodiment 2-1 the reference determination that requires that the determination target log and the log without the terminal disturbance factor (reference log) perform the same operation has been described. However, in the present embodiment, the determination target log And non-reference determination that does not require that the log without the factor of terminal disturbance perform the same operation.
- FIG. 31 shows a configuration example of the terminal bottleneck determination apparatus according to Embodiment 2-2 of the present invention.
- the terminal bottleneck determination apparatus shown in the figure is different from the terminal bottleneck determination apparatus shown in FIG. 22 in that a continuous area length experience cumulative distribution DB 2103 is used instead of the reference log DB 2102 shown in FIG.
- the continuous region length experience cumulative distribution DB 2103 is generated in advance by the continuous region length experience cumulative distribution DB generation unit 2140, and the continuous region length experience cumulative distribution DB 2103 includes Scripting when operating the application in a state where there is no terminal disturbance factor. Rendering duration is stored.
- the continuous area length experience cumulative distribution DB generation unit 2140 continuously executes Scripting and Rendering processing from a log of application operations performed in a state in which Scripting and Rendering logs can be acquired (operation log in a state where there is no terminal disturbance factor). The lengths are extracted, their cumulative experience distribution is obtained, and registered in the continuous region length experimental cumulative distribution DB 2103. In this application operation, the user does not repeat the same operation, but repeatedly performs various operations that would normally be used.
- the determination target data is a terminal bottleneck.
- the determination target log stored in the reception log storage unit 2101 stores log data generated by user operations in the user terminal 200, as in the case of the embodiment 2-1.
- the continuous region length experience cumulative distribution DB 2103 information is prepared in advance by the continuous region length experience cumulative distribution DB generation unit 2140 before performing the terminal bottleneck determination.
- the data stored in the DB 2103 is an empirical cumulative distribution of the Scripting continuous area length and the Rendering continuous area length when the application is operated, and the determination accuracy can be increased as the number of data increases.
- 33A and 33B show the empirical cumulative distribution of the continuous region length.
- FIG. 33A shows the experience cumulative distribution of the Scripting continuous region length
- FIG. 33B shows the experience cumulative distribution of the Rendering continuous region length. It is assumed that such a distribution is prepared before processing.
- the processing of the bottleneck determination unit 2120 in the present embodiment will be described. Note that the processing of the reception unit 2110 and the determination result storage / display unit 2130 is the same as that of the embodiment 2-1.
- the bottleneck determination unit 2120 performs the following processing.
- FIG. 34 is a flowchart of non-reference determination in the embodiment 2-2 of the present invention.
- Step 2210) In advance, the continuous region length experience cumulative distribution DB generation unit 2140 creates an experience cumulative distribution of the continuous region length as shown in FIGS. 33A and 33B and stores it in the continuous region length experience cumulative distribution DB 2103. .
- Step 2220 Processing similar to that of step 2110 in the embodiment 2-1 is performed.
- Step 2230 Processing similar to that of step 2120 in the embodiment 2-1 is performed.
- Step 2240 The same processing as step 2130 in the embodiment 2-1 is performed.
- Step 2250 Processing similar to that of step 2140 in the embodiment 2-1 is performed.
- Step 2260 The bottleneck determination unit 2120 determines whether the continuous region length obtained in the processing up to Step 2250 above and the experience cumulative distribution stored in the continuous region length experience cumulative distribution DB 2103 is up and down, and the determination result Is held in a memory (not shown).
- the determination method includes the following two methods.
- said (1), (2) is an example, What is necessary is just a method which can implement an up-down determination with experience cumulative distribution besides these.
- FIG. 35 shows an example of the determination method (1) above.
- the non-reference determination it is determined whether at most several tens of continuous region lengths included in the determination target log are located above or below the cumulative probability distribution.
- FIG. 36 is a flowchart of non-reference determination in Embodiment 2-2 of the present invention.
- Step 2261 The number of continuous area length data included in the determination target log is accumulated. That is, it is determined whether the individual data is above or below the cumulative probability distribution of the continuous region length empirical cumulative distribution DB 2103 to determine whether the entire data is above or below the distribution.
- Step 2262 If the individual data is above the cumulative ratio distribution, an evaluation value of +1 is assigned, and if it is below, an evaluation value of -1 is assigned. Specifically, as shown in FIG. 37, an evaluation value is obtained based on the distance from the experience cumulative distribution a at the same ratio value, or a ratio difference with the experience cumulative distribution a in the same number of seconds is used as the evaluation value. A method is conceivable. In addition to the distance, when evaluating the difference in seconds by aligning the cumulative ratios on the vertical axis, the evaluation is such that it is negative if it is on the right side of the experience cumulative distribution a, and positive if it is on the left side. Is also possible.
- Step 2263 The upper and lower judgment evaluation values for all data in the Scripting area length of the judgment target log are summed.
- Step 2264 If the total value is a positive number, it is determined to be above the experience cumulative distribution, and if it is a negative value, it is determined to be below.
- the first method is an example using hypothesis testing.
- FIG. 38 shows an evaluation method (No. 1) based on the Anderson Darling test according to the embodiment of the present invention.
- the second method is an example of using an empirical cumulative distribution of logs at normal times.
- the cumulative distribution function is also possible to estimate the cumulative distribution function as a smooth function based on the empirical cumulative distribution of the normal log that has undergone the processing of steps 2220 to 2250 (the estimated cumulative distribution). Can be calculated explicitly for each value of processing time).
- a smooth function is obtained as shown in FIG. 39B by using log data as an initial value and estimating the log data as a solution of a heat equation that imposes a boundary condition (non-negative).
- the third method is an example in which logs at the time of testing (logs other than logs acquired from user terminals) are used as the cumulative experience distribution.
- the fourth method is an evaluation based on the null hypothesis.
- the method is as follows: “The sample obtained by using the log at the time of the test of (2) of the first method is significantly larger than the distribution function obtained from the normal experience distribution of (1). Use the null hypothesis that it is not small. For the null hypothesis, a hypothesis test is performed based on the outputs of (1) and (2) of the first method (distribution function during normal time, empirical distribution during testing), and normality during testing Judge about. For example, the significance level is 5% according to Anderson-Darling test. In the first method (2), bootstrap is applied because the sample size is small. The bootstrap p-value is obtained by applying the following Anderson-Darling statistic A.
- the null hypothesis is rejected (with a significant difference), and it is determined that it is larger / smaller than usual.
- the null hypothesis cannot be rejected (it cannot be said that there is a significant difference), and the normal range is determined.
- Step 2270 Finally, based on the up / down determination result with the experience cumulative distribution stored in the memory (not shown), bottleneck determination as shown in FIG. 41 is performed.
- a threshold is set for the sum of the evaluation values in each of the Scripting process and the Rendering process. For example, when the bottleneck determination threshold is “ ⁇ 5” and the evaluation value is “2”, it is determined that the terminal bottleneck is not “high”, the threshold that strongly suspects the terminal bottleneck is “ ⁇ 10”, and the terminal If the threshold value is “ ⁇ 5” where the bottleneck is suspected and the obtained evaluation value is “ ⁇ 12”, it is determined that “the terminal bottleneck is strongly suspected”.
- step 2260 when the undersendering test is adopted in step 2260, it is determined as “terminal bottleneck” when it is determined to be below the cumulative experience distribution in each of the scripting process and the rendering process. In other cases, it is determined that the terminal is not a bottleneck. Alternatively, when it is determined that one of the Scripting process and the Rendering process is below the experience cumulative distribution, the terminal bottleneck is determined.
- the determination result obtained as described above is output to the storage means or the display means via the determination result storage / display unit 2130 as in the case of the embodiment 2-1.
- each component of the terminal bottleneck determination apparatus shown in FIG. 22 and FIG. 31 is constructed as a program and installed and executed on a computer used as the terminal bottleneck determination apparatus, or via a network. Can be distributed.
- each of the terminal bottleneck determination device shown in FIG. 22 and the terminal bottleneck determination device shown in FIG. 31 is realized by causing a computer to execute a program describing the processing contents described in this embodiment.
- the functions of each unit of the terminal bottleneck determination device are implemented by each unit using hardware resources such as a CPU, a memory, and a hard disk built in the computer constituting the terminal bottleneck determination device. It can be realized by executing a program corresponding to the process.
- the program can be recorded on a computer-readable recording medium (portable memory or the like), stored, or distributed. It is also possible to provide the program through a network such as the Internet or electronic mail.
- quality degradation that causes a computer to function as a quality degradation factor determination device for determining whether or not a quality degradation factor exists in a user terminal when using an application service provided from an application server.
- a factor determination program comprising: Data receiving means for acquiring the duration of each process of data acquisition, script execution, and screen drawing performed in the estimation target period in the user terminal as a determination target log and storing it in a reception log storage means; The judgment target log is compared with the reference log when the same operation as that for the application performed on the user terminal is performed, or the data of the experience cumulative distribution of the continuous area length of the script execution and screen drawing processing.
- Function as bottleneck determination means for determining an evaluation value and determining whether or not the user terminal has a quality deterioration factor based on a comparison result between a total value of the evaluation values and a preset threshold value A quality deterioration factor determination program is provided.
- the third embodiment is an embodiment corresponding to the third object.
- an operation performed by a user (hereinafter referred to as a “user input operation”) is performed in order to perform similarity determination of an operation performed by a user during application execution without monitoring the application in units of methods.
- a similar operation is extracted using the feature amount obtained from the log corresponding to the execution period at the time of the user input operation.
- user input operation refers to an operation performed on the application by the user through an external input / output interface such as login to a website or clicking on a link, and a similarity determination with an operation performed by the user in the past is performed.
- a target operation which is an operation performed by a user, and a process performed by an application in the background is excluded.
- the “process” refers to a process executed on the user terminal in accordance with the above user input operation, and a plurality of processes are executed in parallel with one normal user input operation.
- FIG. 42 shows a system configuration example according to the present embodiment.
- the system shown in the figure includes a similar operation extraction device 3100, a user terminal 3200, and an AP (application) server 3300.
- the similar operation extraction device 3100 includes an experiential waiting time estimation unit 3110, a log cutout unit 3120, a feature amount calculation unit 3130, a similarity calculation unit 3140, a similar operation extraction unit 3150, a storage unit 3160, and a storage unit 3170.
- the user terminal 3200 includes a transmission unit 3210, a log acquisition unit 3220, and a web browser 3230.
- the AP server 3300 includes an HTTP server 3310 and content 3320.
- the configuration in FIG. 42 is a configuration in which the similar operation extraction device 3100 and the user terminal 3200 are independent.
- the configuration is not limited to this example, and similar operation extraction is performed in the user terminal 3200 as shown in FIG.
- the device 3100 may be incorporated.
- FIG. 44 is a flowchart of processing from log acquisition to similar operation extraction in the present embodiment. Below, it demonstrates based on the structure of FIG.
- Step 3100 The log is acquired by the log acquisition unit 3220 of the user terminal 3200, and is transmitted to the similar operation extraction device 3100 via the transmission unit 3210.
- log refers to only one of the application log and the network log, or a combination of both. In the following description, it is assumed that both the application log and the network log are used.
- Step 3200 The sensation waiting time estimation unit 3110 of the similar operation extraction device 3100 estimates the sensation waiting time of the user from the application log acquired from the user terminal 3200 and stores it in the storage unit 3170.
- Step 3300 The log cutout unit 3120 cuts out the application log and the network log corresponding to the bodily sensation waiting time estimated by the bodily sensation waiting time estimation unit 3110.
- the feature amount calculation unit 3130 acquires at least one of the processing multiplicity graph calculated from the extracted application log and the received data amount calculated from the network log, and calculates the feature amount.
- Step 3500 The similarity calculation unit 3140 calculates the feature amount calculated from the feature amount calculated by the feature amount calculation unit 3130, the past feature amount stored in the storage unit 3170, and a predetermined similarity function. The similarity with the past feature amount is calculated.
- Step 3600 The similar operation extraction unit 3150 extracts an operation similar to the user input operation based on the similarity calculated by the similarity calculation unit 3140.
- the storage unit 3160 assigns an identification number to the feature quantity, and stores the combination of the identification number and the feature quantity in the storage unit 3170.
- the experience waiting time estimation unit 3110 of the similar operation extraction device 3100 estimates a synchronization processing execution section in which the multiplicity of processing performed by a Web browser, a client-dedicated application, or the like during application execution exceeds a threshold as “experience waiting time”.
- the user terminal 3200 may measure the actual waiting time for the experience separately and transmit it to the similar operation extraction device 3100 to use a substitute.
- a specific example of the method for estimating the sensation waiting time will be described below.
- logs before and after the user experience waiting time which are quality degradation determination targets, are acquired.
- the log acquisition unit 3220 uses a general analysis tool.
- application analysis tools include Dynatrace (registered trademark), Goole Developer (registered trademark), SpeedTracer (registered trademark), and network analysis tools include, for example, WireShark (registered trademark).
- the transmission unit 3210 of the user terminal 3200 transmits the acquired log to the similar operation extraction device 3100.
- the transmission unit 3210 of the user terminal 3200 can be replaced with a general device or program.
- the similar operation extraction device 3100 acquires a log from the transmission unit 3210 of the user terminal 3200.
- the experience waiting time estimation unit 3110 estimates the start time and end time of the user experience waiting time from the acquired log.
- the start time and the end time are expressed by the number of seconds that have elapsed since a certain point in time.
- a synchronization processing execution section in which the multiplicity of processing performed by an application such as a Web browser exceeds a threshold is estimated as a sensation waiting time.
- an estimation method of the sensation waiting time a method of obtaining and presenting an index close to the user sensation waiting time such as a response time of an HTTP message exchanged between the user terminal 3200 and the AP server 3300 (for example, CA Application Performance Management http://www.ca.com/jp/products/detail/CA-Application-Performance-Management/overview.aspx) or packets continuously between the user terminal 3200 and the AP server 3300
- an existing technique such as a method for estimating a user's sensation waiting time from a period during which data is transmitted and received (for example, Patent Document 1).
- the log cutout unit 3120 cuts out a log corresponding to the sensation waiting time from the start to the end of the sensation waiting time by the input operation from the acquired log. Specifically, the following procedure is used to cut out the log of the waiting part of the input operation.
- the log cutout unit 3120 acquires the start time and end time of the experience waiting time of the user input operation estimated by the experience waiting time estimation unit 3110.
- the feature amount calculation unit 3130 calculates a feature amount from the log cut out by the log cutout unit 3120.
- the feature amount corresponds to, for example, the amount of data received at the user terminal 3200 or the number, height, and position of the processing multiplicity peak calculated for each processing type, and may include at least one or more feature amounts. All of these may be included. Further, the amount of transmission data from the user terminal 3200, the number of processes for each processing type, the statistical value of the processing time, and the like may be used as the feature amount.
- the specific procedure for obtaining the number, height, width, and position of the processing multiplicity peak in the feature quantity calculation unit 3130 is as follows.
- the quantization is performed by dividing the application log into time slots of a certain time in time series, and Network (data acquisition), Scripting (script execution), For each process of rendering (screen rendering), a method of calculating the total number (multiplicity) of each process as a quantization value is used.
- the time slot may be determined by a relative time such as “the length of the waiting time of the experience ⁇ 50” or may be determined by an absolute time such as “0.1 second”.
- a plurality of time slots may be defined, and for each time slot, the multiplicity of each process of Network, Scripting, and Rendering may be quantized as a value.
- the time when the processing multiplicity becomes P min1 or more for the first time is determined as the peak start time, and the time when the processing multiplicity becomes P min2 or less after the start of the peak is determined as the peak end time.
- a peak height of P max or less is not counted as a peak.
- the peak height is the maximum multiplicity of processing from the peak start time to the peak end time.
- peak start and end For example, a peak start when the slope of the quantization graph is greater than or equal to a certain angle, and a peak end when the gradient is less than or equal to a certain angle, or a point where the graph has turned from decrease to increase are referred to as peak start and end.
- peak start and end There are ways to think.
- P min1 , P min2 , P max, and S inter are registered in advance in the storage unit 3170 and can be appropriately modified as necessary.
- the peak width is calculated in the form of (first peak width, second peak width, ).
- the peak width is determined by (peak end time) ⁇ (peak start time) at each peak.
- the peak position is calculated in the form of (first peak position, second peak position,).
- the peak position of each peak is represented by a relative position compared with the activation interval of the network processing, but from the set of peak start time and end time, or from the peak start time You may define so that it may represent with an absolute position, such as the time when processing multiplicity became the maximum until end time.
- the procedure for determining the relative position when the peak position is expressed as a relative position compared with the network processing activation section is as follows.
- the relative position representing the peak position of each peak is compared with the position of the activation interval of the network processing, “before the activation interval”, “the first half of the activation interval”, “ It is determined as one of “midboard”, “second half of the activation section”, “entire activation section”, “after the activation section”.
- the “activation interval” refers to a time zone in which a certain number of processes are performed, and in this embodiment, as an example, between the activation start time and the activation end time. Define as interval.
- each peak position is determined as a relative position compared with the network processing activation interval in the following procedure.
- an activation interval of Network processing is obtained.
- the time when the processing multiplicity becomes A min1 or more for the first time is the activation start time, the processing multiplicity becomes A min2 or less for the first time after the activation starts, and then the time for S continue or more,
- the time is determined as the activation end time. Note that the values of A min1 , A min2 , and S continue are registered in the storage unit 17 in advance, and can be appropriately modified as necessary.
- FIG. 47A “Before the activation interval”: When the peak end time is before the activation start time of Network processing;
- FIG. 47A (A) “After the activation interval”: When the peak start time is later than the activation end time of the network processing;
- FIG. 47B (C) “First half of activation interval”: (a) When the peak end time is earlier than 1/3 of the first half of the activation interval of Network processing except (a);
- FIG. 48A) (D) “Second half of the activation interval”: (a) Other than (a), when the peak start time is later than the last one third of the activation interval of the network processing; (FIG.
- the feature quantity calculation unit 3130 uses only the value calculated from the multiplicity graph of user terminal processing and the received data quantity as the feature quantity, it can be applied to various applications and touches the customer's personal information. There is nothing.
- the similarity calculation unit 3140 uses the feature amount calculated by the feature amount calculation unit 3130, the past feature amount stored in the storage unit 3170, and a feature stored in the past using a predetermined similarity function.
- the similarity with the calculated feature amount is calculated for all the amounts. Specifically, a value obtained by substituting the calculated feature amount S and the stored past feature amount R into the similarity function SM (S, R) is calculated as the similarity.
- the “similarity function” is composed of a function that represents the similarity between the feature quantities and a function that weights them. For example, when the peak of the communication processing amount is high, the similarity of the peak of the communication processing amount is heavily weighted, and when the peak is low, the similarity of the other feature amount is heavily weighted.
- the similarity calculation unit 3140 calculates several feature amounts from the graph, calculates the similarity between the feature amounts, and adds the weights together to calculate the similarity of the entire feature amount. As a result, there is no need to hold the information of the graph body, which can contribute to a reduction in the amount of stored data. In addition, since the graphs need not be compared at each time, but only the feature amounts need to be compared, the calculation amount at the time of calculating the similarity can be reduced.
- the similar operation extraction unit 3150 extracts an identification number stored as a pair with a feature amount whose similarity exceeds an extraction threshold among past feature amounts.
- the extraction threshold value may be determined as a relative value, such as “when included in the top 2% of all past feature values in which similarity is stored” or “when included in the top 100”. , “When the similarity exceeds 80” may be set as an absolute value.
- the storage unit 3160 gives an identification number to the calculated feature quantity not stored in the storage unit 3170, and stores the number and the feature quantity in the storage unit 3170.
- the identification number may be determined by the total number of features, or may be determined by the start time of the sensation waiting time at the time of performing the operation.
- identification number and the feature quantity are stored in the storage unit 3170 in association with the corresponding identification number. These may be extracted simultaneously with the identification number.
- the identification number is a unique value, and even if the feature value saved in the past and the calculated unsaved feature value are the same value, the identification number is a newly saved feature value. Is given a completely new unique identification number.
- the calculated value of one or more feature values is the value of each component constituting the feature value vector associated with the log that is the calculation source data, and the feature value stored in the storage unit 3170 is: This means the entire feature vector composed of all the constituent elements, and an identification number may be uniquely assigned to the entire feature vector composed of the feature value.
- step 3300 the application to be executed will be described using a Web browser as an example. The following operation will be described based on the flowchart of FIG. 44 described above. Hereinafter, step 3300 and subsequent steps will be described.
- Step 3300) Log logging:
- the log cutout unit 3120 receives the log acquired from the user terminal 3200 and the sensation waiting time estimated by the sensation waiting time estimation unit 3110 as inputs. For example, it is assumed that the application log shown in FIG. 49, the network log shown in FIG. 50, and the estimated experience waiting time shown in FIG. 51 are input.
- the log cutout unit 3120 cuts out the application log shown in FIG. 52 and the network log shown in FIG.
- the log extracted here is a log with a start time of 13.7 seconds or more and an end time of 15.8 seconds or less due to the waiting time.
- Step 3400 Feature amount calculation: It is assumed that the log (FIGS. 52 and 53) extracted in step 3300 is input to the feature amount calculation unit 3130.
- the time slot is “length of waiting time sensation ⁇ 50” and “length of feeling waiting time” ⁇ 10 ”is quantized with the value of the processing multiplicity for each processing type.
- the network processing peak number S nw_n the rendering processing peak number S ren_n
- the network processing peak height S nw_h the network processing peak height S nw_h , S ' nw_h , six peak widths S nw_w of Network processing, peak position S scr_p of Scripting processing, and only the peak height S nw_h_10 of Network processing is calculated for the latter.
- the information of the multiplicity graph obtained in the latter “experience waiting time ⁇ 10” time slot is used as additional information in the determination by the multiplicity graph obtained in the “experience waiting time ⁇ 50” time slot. It shall be used. Therefore, for the latter, not only all the feature values but only the peak height of the most dominant (influential) network processing is calculated and used as additional information.
- FIG. 54 shows the quantization result when the time slot is “length of experience waiting time ⁇ 50”
- FIG. 55 shows the quantization result when the time slot is “length of experience waiting time ⁇ 10”.
- Step 3500 Similarity Calculation
- Similarity function SM (S, R) A (S nw_h , R nw_h , S ' nw_h , R' nw_h ) * (b (S nw_h ) * B (S nw_n , R nw_n , S nw_h , R nw_h ) + c (S nw_h ) * C (S thr , R thr ) + d (S nw_h ) * D (S ren_n , R ren_n ) + e (S nw_h ) * E (S scr_p , R scr_p ) + f (S nw_h ) * F (S nw_h_10 , R nw_h_10 ) + g (S nw_h ) * G (S nw_w , R nw_w , S nw_h , R n
- Step 3600 Similar Operation Extraction
- the similar operation extraction unit 3150 arranges all the feature amounts stored in the storage unit 3170 in order of similarity with the feature amount obtained in step 3400. The result is shown in FIG. In FIG. 57, since there are 275 feature amounts stored in the storage unit 3170, the feature amounts included in the upper 2% of the similarity are Nos. 1, 148, 264, 164, and 56. . Therefore, these five are extracted as similar operations.
- the evaluation target of the validity of the identification result of the similar operation is a total of 276 operations and their feature values performed under the conditions shown in FIG.
- the arbitrary two operations indicate two operations randomly selected from 276 operations regardless of the type of operation and measurement conditions.
- Two arbitrary operations are extracted for the entire operation to be evaluated, and the probability that the result of determining whether or not the two operations are similar operations by the technique of the present embodiment is a threshold value.
- the results confirmed for each are shown in FIG. As shown in FIG. 59, the correct answer rate exceeds 80% for the similar operations and all the thresholds, and it was confirmed that the similar operations can be accurately identified by the technique of the present embodiment.
- each component of the similar operation extracting device shown in FIG. 42 or FIG. 43 is constructed as a program, installed in a computer used as the similar operation extracting device, executed, or distributed via a network. Is possible.
- the similar operation extraction apparatus shown in FIG. 42 or FIG. 43 can be realized by causing a computer to execute a program describing the processing contents described in the present embodiment.
- the function of each unit of the similar operation extraction device is a process performed by each unit using hardware resources such as a CPU, a memory, and a hard disk built in the computer constituting the similar operation extraction device. It can be realized by executing the corresponding program.
- the program can be recorded on a computer-readable recording medium (portable memory or the like), stored, or distributed. It is also possible to provide the program through a network such as the Internet or electronic mail.
- a similar operation extraction program for causing a computer to function as a similar operation extraction device that extracts an operation similar to an input operation that is given as an input when executing an application.
- the A feature amount calculating means for calculating a feature amount based on a log corresponding to a user experience waiting time cut out from only one of the application log and the network log, or a combination of both; Similarity calculation means for calculating the similarity between the feature quantity and the feature quantity saved in the past based on the calculated feature quantity, the feature quantity saved in the storage means in the past, and a predetermined similarity function ,
- a similar operation extraction program is provided.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Environmental & Geological Engineering (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
例えば、Windows(登録商標) OSに付属する「パフォーマンスモニタ」等のツールを用いて、CPU利用率や空きメモリ量が事前に規定した閾値を超えていないか確認を行う手法がある(例えば、特許文献2参照)。
CPU種別、搭載メモリ量等の端末スペック情報に基づいて、当該サービスに適合しているか否かを判定する手法がある(例えば、非特許文献2,3参照)。
当該サービスと近い動作を行うベンチマークソフトウェアを動作させ、そのスコアに基づいて当該サービスに適合しているか否かを判定する手法がある(例えば、非特許文献4参照)。
第1の側面に関する従来の技術には以下のような課題が存在する。
図1は、利用者の操作が行われた際の動作を単純化したものである。この例においては、利用者の操作開始から画面表示が完了する時刻までの時間(t4-t0)は、「端末からのリクエストの発出」⇒「サーバからの応答」⇒「端末での画面生成処理」で構成されている。従来技術においては、リクエスト発出からレスポンス受信までの時間(t3-t0)を検出するものであるため、端末処理時間(t4-t3)が誤差となって生じる。
従来の技術において、図2に示すように、利用者の操作時にサーバに対してリクエストが発出されたが動的コンテンツを構成要素に含むアプリケーションでは、処理がブラウザや専用のクライアントソフト内に閉じて、サーバへのリクエストが発生しないことも多く、従来技術では利用者の操作が行われたことを検知できない。
動的コンテンツを構成要素に含むアプリケーションにおいては、図3に示すように、利用者操作が起因の信号とブラウザや専用のクライアントソフト等の利用者操作以外が起因となった信号が混在する。利用者操作が起因となった信号を特定するためには、各アプリケーションの操作に応じた信号パターンを確認する必要がある。アプリケーションの数、並びに、各アプリケーション内操作の数は膨大であり、操作毎の信号パターンを確認することは極めて困難である。
第2の側面に関する従来の技術には以下の2つの課題がある。
第3の側面に関する従来の技術には以下の課題がある。
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を推定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段と、
前記受信ログ記憶手段から前記推定対象ログを読み出し、該データの継続時間が、所定の短時間処理閾値より短い、または、所定の長時間閾値より長いログを除外したログを出力するログ除外手段と、
前記ログ除外手段から出力されたログに対し、一定時間のタイムスロット内で行われた各処理の個数を多重度として算出する量子化手段と、
前記量子化手段で量子化されたデータから連続領域を抽出する連続領域抽出手段と、
を有し、
前記連続領域抽出手段は、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別せずに連続領域を抽出する第1の連続領域抽出手段、または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別して連続領域を抽出する第2の連続領域抽出手段と、を含むことを特徴とする利用者体感品質推定装置が提供される。
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を判定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段と、
前記利用者端末で行われるアプリケーションに対する操作と同様の操作を行った場合のリファレンスログ、または、スクリプト実行や画面描画の処理の連続領域長の経験累積分布のデータと、前記判定対象ログを比較し、評価値を求め、該評価値の合計値と予め設定されている閾値との比較結果に基づいて、該利用者端末に品質劣化要因があるか否かを判定するボトルネック判定手段と、を有することを特徴とする品質劣化要因判定装置が提供される。
アプリケーションログとネットワークログのいずれか一方のみ、もしくは両方の組み合わせから切り出されたユーザ体感待ち時間に対応するログに基づき、特徴量を算出する特徴量算出手段と、
算出した前記特徴量と過去に記憶手段に保存された特徴量、及び予め定められた類似度関数に基づき前記特徴量と前記過去に保存された特徴量との類似度を算出する類似度算出手段と、を有することを特徴とする類似操作抽出装置が提供される。
まず、第1の実施の形態を説明する。第1の実施の形態は、第1の目的に対応する実施の形態である。
第1の実施の形態は、実施の形態1-1、及び実施の形態1-2を含む。まず、実施の形態1-1を説明する。図5は、本発明の実施の形態1-1における品質推定装置の構成の一例を示す。
Scripting時間:Javascript(登録商標)等のスクリプト実行時間
Rendering時間:画面描画時間の開始-終了時間
受信ログ記憶部1101に格納されるデータの例を図6に示す。
Scripting閾値 5ms
Rendering閾値 3ms
このとき、アプリケーションAの体感待ち時間推定を行う際には、Network、Scripting、Renderingについて、上記各閾値以下の処理時間のログを除外する。
Scripting閾値 30000ms
Rendering閾値 100000ms
ステップ1130) 次に、上記のステップ1110、1120の処理を経たデータを一定時間間隔で量子化する。量子化の方法は、一定時間のタイムスロットで区切り、このタイムスロット内で行われたNetwork、Scripting、Renderingの各処理の個数を多重度として算出する。なお、タイムスロットの長さはアプリケーションに応じて設定可能である。
(2)スライディングウィンドウが継続した時間と多重度で評価する方法(A-2):
まず、上記の(1)の方法について説明する。
実施の形態1-2では、実施の形態1-1の図7のステップ1140において、Network、Scripting、Renderingを区別して評価する3つの方法について説明する。
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を推定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段、
前記受信ログ記憶手段から前記推定対象ログを読み出し、該データの継続時間が、所定の短時間処理閾値より短い、または、所定の長時間閾値より長いログを除外したログを出力するログ除外手段、
前記ログ除外手段から出力されたログに対し、一定時間のタイムスロット内で行われた各処理の個数を多重度として算出する量子化手段、
前記量子化手段で量子化されたデータから連続領域を抽出する連続領域抽出手段、として機能させる利用者体感品質推定プログラムであって、
前記連続領域抽出手段は、前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別せずに連続領域を抽出する第1の連続領域抽出手段、または、前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別して連続領域を抽出する第2の連続領域抽出手段と、を含む利用者体感品質推定プログラムが提供される。
次に、第2の実施の形態を説明する。第2の実施の形態は、第2の目的に対応する実施の形態である。
図22は、本発明の実施の形態における端末ボトルネック判定装置の構成例を示す。同図に示す端末ボトルネック判定装置2100は、受信ログ記憶部2101、リファレンスログ記憶部2102、受信部2110、ボトルネック判定部2120、判定結果格納・表示部2130から構成される。
・Scripting時間:Javascript(登録商標)等のスクリプト実行時間
・Rendering時間:画面描画時間の開始終了時間
図23に、格納されるデータ例を示す。同図に示すデータは、操作開始0秒後~15秒後までのブラウザログの例であり、当該ブラウザログは、開始時間(秒)、継続時間(ミリ秒)、処理種別から構成される。ここで、継続時間とは、ネットワークにおいてはセッション継続時間であり、Renderingは描画時間を示している。通常処理においてはこれらの処理が数ms~数十ms程度継続したログが大量に出力される。例えば、3分程度ブラウザを操作した際には、上記ログが一万行程度出力される。
Scripting閾値: 5ms
Rendering閾値: 3ms
アプリケーションAの体感待ち時間推定を行う際には、Network、Scripting、Renderingについて、上記各閾値以下の処理時間のログは当該ステップで除外する。
Network閾値: 10000ms
Scripting閾値: 30000ms
Rendering閾値: 100000ms
Network、Scripting、Renderingについて、上記各閾値以上の処理時間のログは当該ステップにおいて除外する。
リファレンスログ、判定対象ログそれぞれの対応する各Scripting処理について、継続時間に対する以下のいずれかの統計値を比較し、判定対象ログの方が長ければ、Scripting処理継続時間による判定対象ログの評価として加点する。なお、各統計値毎の加点値は個別に設定可能である。例えば、以下のように設定する。
・90%値 X%以下長ければ0点
・平均値 Z%以上短ければ-1点、W%(W>Z)以上短ければ-2点
なお、ここでは、代表的な統計値を提示したが、その他統計値を用いても良い。
リファレンスログ、判定対象ログそれぞれの対応する各Rendering処理について、継続時間に対する上記Scripting処理と同様の統計値を比較し、判定対象ログの方が長ければRendering処理継続時間による判定対象ログの評価として加点する。なお、各統計値毎の加点値は個別に設定可能である。また、この例に限定されることなく、上記a)と同様に任意の統計値を設定可能である。
リファレンスログ、判定対象ログそれぞれの対応する各Scripting連続領域長に対する以下の統計値を比較し、判定対象ログの方が長ければScripting連続領域長による判定対象ログの評価として加点する。なお、各統計値ごとの加点値は個別に設定可能である。
・90%値 X%以下長ければ0点
・平均値 Z%以上短ければ-1点、W%(W>Z)以上なら-2点
ここでは、代表的な統計値を示したが、この例に限定されることなく、その他統計値であってもよい。また、上記の例のXも様々な値を設定することが可能である。
リファレンスログ、判定対象ログそれぞれの対応する各Rendering処理連続領域に対する上記Scripting処理と同様の統計値を比較し、判定対象ログの方が長ければRendering処理継続時間による判定対象ログの評価として加点する。なお、各統計値毎の加点値は個別に設定可能である。
本実施の形態では、ボトルネック判定をノンリファレンス判定で実施する場合の例について説明する。
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を判定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段、
前記利用者端末で行われるアプリケーションに対する操作と同様の操作を行った場合のリファレンスログ、または、スクリプト実行や画面描画の処理の連続領域長の経験累積分布のデータと、前記判定対象ログを比較し、評価値を求め、該評価値の合計値と予め設定されている閾値との比較結果に基づいて、該利用者端末に品質劣化要因があるか否かを判定するボトルネック判定手段、として機能させるための品質劣化要因判定プログラムが提供される。
次に、第3の実施の形態を説明する。第3の実施の形態は、第3の目的に対応する実施の形態である。
(イ)「活性化区間より後」:ピーク開始時刻が、Network処理の活性化終了時刻よりも後であるとき;(図47B)
(ウ)「活性化区間の前半」:(ア)(イ)以外で、ピーク終了時刻が、Network処理の活性化区間の前半1/3よりも前であるとき;(図48A)
(エ)「活性化区間の後半」:(ア)(イ)以外で、ピーク開始時刻が、Network処理の活性化区間の後半1/3よりも後であるとき;(図48B)
(オ)「活性化区間の中盤」:(ア)~(エ)以外のとき;(図48C)
(3) 前記(2)で求めた、各ピークとNetwork処理の活性化区間との位置関係を、(1つ目のピークの位置関係,2つ目のピークの位置関係,…)の形で算出する。ただし、次の(a)~(c)に該当する場合は、以下の通りに出力することとする。
→ 上記に該当するピークの位置関係を、まとめて「Network処理の活性化区間全体」と出力する。
→ 上記に該当するピークの位置関係を、まとめて「活性化区間の前半」と出力する。
→ 上記に該当するピークの位置関係を、まとめて「活性化区間の後半」と出力する。例えば、上記(2)で1つ目のピークが「活性化区間より前」、2つ目のピークが「活性化区間の中盤」、3つ目のピークが「活性化区間の後半」に分類された場合は、(b)(c)に該当するので、出力は(「活性化区間より前」,「活性化区間の後半」)となる。
ログ切り出し部3120では、ユーザ端末3200から取得したログと体感待ち時間推定部3110において推定された体感待ち時間を入力とする。例えば、図49に示すアプリケーションログ、図50に示すネットワークログと、図51に示す推定体感待ち時間が入力されたとする。ログ切り出し部3120では、図52に示すアプリケーションログと、図53に示すネットワークログを切り出す。ここで切り出されるログは、体感待ち時間により、開始時刻が13.7秒以降かつ終了時刻が15.8秒以前のログである。
特徴量算出部3130にステップ3300で切り出されたログ(図52、図53)が入力されたとする。
ネットワークログから、ユーザ端末3200が受信したデータ量は、
507.97 + 304.45 = 812.42(Kbit) = 101.55(KB)
と与えられる。
本実施例では、タイムスロットとして「体感待ち時間の長さ÷50」の場合と「体感待ち時間の長さ÷10」の場合のそれぞれについて処理種別毎の処理多重度の値で量子化し、前者については、Network処理のピーク数Snw_nとRendering処理のピーク数Sren_n、Network処理のピーク高Snw_h, S'nw_h、Network処理のピーク幅Snw_w、Scripting処理のピーク位置Sscr_pの6つ、後者についてはNetwork処理のピーク高Snw_h_10のみを算出する。上記において、「体感待ち時間÷50」のタイムスロットで求めた多重度グラフだけでは判定が難しい操作があるため、「体感待ち時間÷10」のタイムスロットで求めた多重度グラフの情報を補足として用いている。なお、「体感待ち時間÷10」の場合は、全ての特徴量を算出せず、最も支配的であるネットワーク処理のピーク高のみを求めるものとする。
Network処理ではSinter=5, Pmin1=2, Pmin2=2, Pmax=2, Rendering処理ではSinter=5, Pmin1=0, Pmin2=0, Pmax =1とすると、図54の場合、Network処理のピーク数Snw_nは2、Rendering処理のピーク数Sren_nは0と与えられる。
図54の場合、Sinter=5, Pmin1=2, Pmin2=2, Pmax=2とするとNetwork処理のピーク高Snw_hは (5,14)、Sinter=5, Pmin1=0, Pmin2=0, Pmax=1とするとNetwork処理のピーク高S'nw_hは (14)である。当該例において、Pmin1=Pmin2=0としているため、多重度がはじめ0以上になったときにピーク開始、ピーク開始後に始めて0以下になったときにピーク終了とする。また、Pmax=1であるから、ピーク高が1未満のピークは無視する。よって、図54でピークとして数えられるのは、時刻13に始まって、時刻45で終了するピークのみで、このピークの高さは"14"となる。図55の場合、Sinter=5, Pmin1=2, Pmin2=2, Pmax=2とすると、ピーク高Snw_h_10は (15)と与えられる。
図54の場合、ピーク幅の組Snw_wは、(4,15)と与えられる。
(0) Scripting処理のピーク:
Sinter=5, Pmin1=1, Pmin2=0, Pmax=0とすると、図54におけるScripting処理のピークは1つで、開始時刻は21、終了時刻は47で与えられる。
Scontinue=5, Amin1=1, Amin2=0とすると、図54におけるNetwork処理の活性化開始時刻は14、活性化終了時刻は45と与えられる。
(0)で求めたScripting処理の各ピークと、(1)で求めたNetwork処理の活性化区間との位置関係は、(オ)「活性化区間の中盤」と与えられる。
(2)より、Sscr_pは(「活性化区間の中盤」)と与えられる。
類似度算出部3140に、ステップ3400で求めた特徴量の組S=(Sthr, Snw_n, Sren_n, Snw_h, S'nw_h, Snw_h_10, Snw_w, Sscr_p)と、図56に示すように、記憶部3170に保存されている特徴量の組R=(Rthr, Rnw_n, Rren_n, Rnw_h, R'nw_h, Rnw_h_10, Rnw_w, Rscr_p)、下記に示す類似度関数が与えられたとする。
{b(Snw_h) * B(Snw_n, Rnw_n, Snw_h, Rnw_h)+
c(Snw_h) * C(Sthr, Rthr) +
d(Snw_h) * D(Sren_n, Rren_n) +
e(Snw_h) * E(Sscr_p, Rscr_p) +
f(Snw_h) * F(Snw_h_10, Rnw_h_10) +
g(Snw_h) * G(Snw_w, Rnw_w, Snw_h, Rnw_h)}
ただし、各特徴量の類似度関数A~Gと、重みづけ関数 b~gは下記の通り定められているとする。
(i) 1 (0 < snw_h < 5かつ、rnw_h*0.8 ≦ snw_h ≦ rnw_h*1.2またはrnw_h-1 ≦ snw_h
≦ rnw_h+1の
(ii) 1 (snw_h = 0, s'nw_h > 0かつ、rnw_h=0, r'nw_h > 0のとき)
1 (snw_h = s'nw_h = 0かつ、r'nw_h = rnw_h=0のとき)
(iii) 0 (それ以外)
ここで、snw_h, s'nw_h, rnw_h, r'nw_hは、それぞれSnw_h, S'nw_h, Rnw_h, R'nw_hの最大値
B(Snw_n, Rnw_n, Snw_h, Rnw_h)=
(i) 1 (Snw_n = Rnw_n = 1のとき)
(ii) 1 (Snw_n, Rnw_n > 1で、かつ、snw_h2*0.9 ≦ rnw_h2 ≦ snw_h2*1.1またはsnw_h2-1 ≦
rnw_h2 ≦ snw_h2+1のとき)
(iii) 0 (それ以外)
ここで、snw_h2, rnw_h2は、それぞれSnw_h, Rnw_hの2番目に大きい値
C(Sthr, Rthr) =
(i) 1 (Sthr*0.8 ≦ Rthr ≦ Sthr*1.2のとき)
(ii) 0(それ以外)
D(Sren_n, Rren_n) =
(i) 1 (Sren_n, Rren_n ≧ 1またはSren_n = Rren_n = 0のとき)
(ii) 0(それ以外)
E(Sscr_p, Rscr_p) =
(i) 1 (Sscr_p = Rscr_pのとき)
(ii) 1 (Sscr_p, Rscr_pが共に「活性化区間より前」を含むとき、または、共に「活性化区間より後」を含むとき)
(iii) 0.3 ((i)以外で、Sscr_p, Rscr_pが共に「活性化区間より前」と「活性化区間より後」を含まないとき)
(iv) 0 (それ以外)
F(Snw_h_10, Rnw_h_10) =
(i) 1 (snw_h_10*0.9 ≦ rnw_h_10 ≦ snw_h_10*1.1またはsnw_h_10-1 ≦ rnw_h_10 ≦
snw_h_10+1のとき)
(ii) 0(それ以外)
ここで、snw_h_10, rnw_h_10は、それぞれSnw_h_10, Rnw_h_10の最大値
G(Snw_w, Rnw_w, Snw_h, Rnw_h) =
(i) 1 (snw_w*0.7 ≦ rnw_w ≦ snw_w*1.3またはsnw_w-1 ≦ rnw_w ≦ snw_w+1のとき)
(ii) 0 (それ以外)
ここで、snw_w, rnw_wは、それぞれSnw_w, Rnw_wのうち、ピーク高が最大になったときのピーク幅
b(Snw_h) =
(i) 30 (snw_h ≧ 5のとき)
(ii) 20 (0 < snw_h < 5のとき)
(iii) 0 (snw_h = 0のとき)
c(Snw_h) =
(i) 20 (snw_h ≧ 5のとき)
(ii) 20 (0 < snw_h < 5のとき)
(iii) 40 (snw_h = 0のとき)
d(Snw_h) =
(i) 10 (snw_h ≧ 5のとき)
(ii) 10 (0 < snw_h < 5のとき)
(iii) 30 (snw_h = 0のとき)
e(Snw_h) =
(i) 10 (snw_h ≧ 5のとき)
(ii) 20 (0 < snw_h < 5のとき)
(iii) 30 (snw_h = 0のとき)
f(Snw_h) =
(i) 20 (snw_h ≧ 5のとき)
(ii) 20 (0 < snw_h < 5のとき)
(iii) 0 (snw_h = 0のとき)
g(Snw_h) =
(i) 10 (snw_h ≧ 5のとき)
(ii) 10 (0 < snw_h < 5のとき)
(iii) 0 (snw_h = 0のとき)
このとき、例えば、ステップ3400で求めた特徴量と、図56のNo.1の特徴量の類似度は、1*{30 * 1 + 20 * 1 + 10 * 1 + 10 * 0 + 20 * 1 + 10 * 1} = 90と与えられる。
類似操作抽出部3150は、記憶部3170に保存されている特徴量すべてに対して、ステップ3400で求められた特徴量との類似度を高い順に並べる。この結果を図57に示す。図57において、記憶部3170に保存されている特徴量は275個であるので、類似度が上位2%に含まれる特徴量は、No. 1, 148, 264, 164, 56の5個である。したがって、この5個を類似操作として抽出する。
本実施の形態に係る技術を用いた類似操作の特定結果に対する妥当性の評価結果は次のとおりである。
アプリケーションログとネットワークログのいずれか一方のみ、もしくは両方の組み合わせから切り出されたユーザ体感待ち時間に対応するログに基づき、特徴量を算出する特徴量算出手段、
算出した前記特徴量と過去に記憶手段に保存された特徴量、及び予め定められた類似度関数に基づき前記特徴量と前記過去に保存された特徴量との類似度を算出する類似度算出手段、として機能させるための類似操作抽出プログラムが提供される。
1101 受信ログ記憶部
1110 受信部
1120 待ち時間推定部
1130 推定結果格納・表示部
1200 利用者端末
1210 Webブラウザ
1220 ブラウザログ取得部
1230 送信部
1300 ブラウザアプリケーションサーバ
1310 HTTPサーバ
1320 コンテンツ
2100 端末ボトルネック判定装置
2101 受信ログ記憶部
2102 リファレンスログ記憶部
2103 連続領域長経験累積分布DB
2110 受信部
2120 ボトルネック判定部
2130 判定結果格納・表示部
2140 連続領域長経験累積分布DB生成部
2200 利用者端末
2210 Webブラウザ
2220 ブラウザログ取得部
2230 送信部
2300 ブラウザアプリケーションサーバ
2310 HTTPサーバ
2320 コンテンツ
3100 類似操作抽出装置
3110 体感待ち時間推定部
3120 ログ切り出し部
3130 特徴量算出部
3140 類似度算出部
3150 類似操作抽出部
3160 保存部
3170 記憶部
3200 ユーザ端末
3210 送信部
3220 ログ取得部
3230 Webブラウザ
3300 APサーバ
3310 HTTPサーバ
3320 コンテンツ
Claims (26)
- 利用者端末におけるアプリケーションの利用者の体感待ち時間を推定するための利用者体感品質推定装置であって、
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を推定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段と、
前記受信ログ記憶手段から前記推定対象ログを読み出し、該データの継続時間が、所定の短時間処理閾値より短い、または、所定の長時間閾値より長いログを除外したログを出力するログ除外手段と、
前記ログ除外手段から出力されたログに対し、一定時間のタイムスロット内で行われた各処理の個数を多重度として算出する量子化手段と、
前記量子化手段で量子化されたデータから連続領域を抽出する連続領域抽出手段と、
を有し、
前記連続領域抽出手段は、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別せずに連続領域を抽出する第1の連続領域抽出手段、または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別して連続領域を抽出する第2の連続領域抽出手段と、
を含むことを特徴とする利用者体感品質推定装置。 - 前記第1の連続領域抽出手段は、
スライディングウィンドウ内のタイムスロットのうち少なくとも1つのタイムスロットにおいて、検知閾値以上の処理があれば、連続領域内と判定する処理を行い、連続領域内との判定がなされたスライディングウィンドウが連続した区間を連続領域とする第1の評価A手段、
または、
あるタイムスロットを始点とするスライディングウィンドウ長の区間内における、検知閾値を超えるタイムスロット数の存在比率が所定の連続判定閾値を超える場合に、当該タイムスロットを連続領域内であると判定する処理を各タイムスロットを始点として行い、連続領域内であると判定された始点のタイムスロットが連続する領域を連続領域とする第2の評価A手段
を含む請求項1記載の利用者体感品質推定装置。 - 前記第2の連続領域抽出手段は、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、各処理の連続領域の和集合を連続領域とする第1の評価B手段、
または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、前記データ取得処理の連続領域、当該データ取得処理の連続領域に隣接する前記スクリプト実行処理の連続領域、及び当該スクリプト実行処理の連続領域に隣接する前記画面描画処理の連続領域を抽出する第2の評価B手段、
または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、前記データ取得処理の連続領域、及び当該データ取得処理の連続領域に隣接する前記画面描画処理の連続領域を抽出し、更に、前記スクリプト実行処理の連続領域、及び当該スクリプト実行処理の連続領域に隣接する前記画面描画処理の連続領域を抽出する第3の評価B手段
を含む請求項1記載の利用者体感品質推定装置。 - 前記連続領域抽出手段で抽出された連続領域を、
前記連続領域抽出手段で抽出された前記連続領域の最終スロットと、次の連続領域の先頭スロットが連続する場合には、一つの連続領域とし、
前記一つの連続領域が、所定の閾値以下の場合は当該一つの連続領域を削除する、
または、
前記一つの連続領域が、所定の閾値以上場合には当該一つの連続領域を削除する、
ことにより連続領域を整形する連続領域整形手段を更に有する
請求項1記載の利用者体感品質推定装置。 - 利用者端末におけるアプリケーションの利用者の体感待ち時間を推定するための利用者体感品質推定方法であって、
データ受信手段が、前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を推定対象ログとして取得して受信ログ記憶手段に格納するデータ受信ステップと、
ログ除外手段が、前記受信ログ記憶手段から前記推定対象ログを読み出し、該データの継続時間が、所定の短時間処理閾値より短い、または、所定の長時間閾値より長いログを除外したログを出力するログ除外ステップと、
量子化手段が、前記ログ除外ステップで出力されたログに対し、一定時間のタイムスロット内で行われた各処理の個数を多重度として算出する量子化ステップと、
連続領域抽出手段が、前記量子化ステップで量子化されたデータから連続領域を抽出する連続領域抽出ステップと、
を行い、
前記連続領域抽出ステップにおいて、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別せずに連続領域を抽出する第1の連続領域抽出ステップと、または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理を区別して連続領域を抽出する第2の連続領域抽出ステップと、
を含むことを特徴とする利用者体感品質推定方法。 - 前記第1の連続領域抽出ステップにおいて、
スライディングウィンドウ内のタイムスロットのうち少なくとも1つのタイムスロットにおいて、検知閾値以上の処理があれば、連続領域内と判定する処理を行い、連続領域内との判定がなされたスライディングウィンドウが連続した区間を連続領域とする第1の評価Aステップ、
または、
あるタイムスロットを始点とするスライディングウィンドウ長の区間内における、検知閾値を超えるタイムスロット数の存在比率が所定の連続判定閾値を超える場合に、当該タイムスロットを連続領域内であると判定する処理を各タイムスロットを始点として行い、連続領域内であると判定された始点のタイムスロットが連続する領域を連続領域とする第2の評価Aステップを行う
請求項5記載の利用者体感品質推定方法。 - 前記第2の連続領域抽出ステップにおいて、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、各処理の連続領域の和集合を連続領域とする第1の評価Bステップ、
または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、前記データ取得処理の連続領域、当該データ取得処理の連続領域に隣接する前記スクリプト実行処理の連続領域、及び当該スクリプト実行処理の連続領域に隣接する前記画面描画処理の連続領域を抽出する第2の評価Bステップ、
または、
前記データ取得、前記スクリプト実行、前記画面描画の各処理に対して、連続領域を抽出し、前記データ取得処理の連続領域、及び当該データ取得処理の連続領域に隣接する前記画面描画処理の連続領域を抽出し、更に、前記スクリプト実行処理の連続領域、及び当該スクリプト実行処理の連続領域に隣接する前記画面描画処理の連続領域を抽出する第3の評価Bステップ
を行う請求項5記載の利用者体感品質推定方法。 - 前記連続領域抽出ステップで抽出された連続領域を、
前記連続領域抽出ステップで抽出された前記連続領域の最終スロットと、次の連続領域の先頭スロットが連続する場合には、一つの連続領域とし、
前記一つの連続領域が、所定の閾値以下の場合は当該一つの連続領域を削除する、
または、
前記一つの連続領域が、所定の閾値以上場合には当該一つの連続領域を削除する、
ことにより連続領域を整形して出力する連続領域整形ステップを更に行う、
請求項5記載の利用者体感品質推定方法。 - コンピュータを、
請求項1乃至4のいずれか1項に記載の利用者体感品質推定装置の各手段として機能させるための利用者体感品質推定プログラム。 - アプリケーションサーバから提供されるアプリケーションのサービス利用時に、品質劣化要因が利用者端末にあるか否かを判定するための品質劣化要因判定装置であって、
前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を判定対象ログとして取得して受信ログ記憶手段に格納するデータ受信手段と、
前記利用者端末で行われるアプリケーションに対する操作と同様の操作を行った場合のリファレンスログ、または、スクリプト実行や画面描画の処理の連続領域長の経験累積分布のデータと、前記判定対象ログを比較し、評価値を求め、該評価値の合計値と予め設定されている閾値との比較結果に基づいて、該利用者端末に品質劣化要因があるか否かを判定するボトルネック判定手段と、
を有することを特徴とする品質劣化要因判定装置。 - 前記ボトルネック判定手段は、
前記リファレンスログと前記判定対象ログのそれぞれ対応する処理の統計値を比較して、比較結果に応じて点数を付与し、該点数の合計値を求め、該合計値が、利用端末にボトルネックが存在することを意味する少なくとも1つの閾値を超える場合に利用端末にボトルネックがあると判定する手段を含む
請求項10記載の品質劣化要因判定装置。 - 通常利用されるであろう様々な操作を繰り返したスクリプト実行または画面描画の処理のログから該スクリプト実行または画面描画の処理の連続領域長の経験累積分布を、連続領域長記憶手段に登録する連続領域長DB生成手段を更に有し、
前記ボトルネック判定手段は、
前記判定対象ログに含まれる連続領域長データ数を累積比率化し、該累積比率化された判定対象ログのデータについて、前記連続領域長記憶手段の連続領域長の経験累積分布に対する上下関係を判定することによって利用者端末におけるボトルネックを判定する手段を含む
請求項10記載の品質劣化要因判定装置。 - 通常利用されるであろう様々な操作を繰り返したスクリプト実行または画面描画の処理のログから該スクリプト実行または画面描画の処理の連続領域長の経験累積分布を、連続領域長記憶手段に登録する連続領域長DB生成手段を更に有し、
前記ボトルネック判定手段は、
前記判定対象ログに含まれる連続領域長データ数を累積比率化し、該累積比率化された判定対象ログのデータについて、前記連続領域長記憶手段の連続領域長の経験累積分布を参照し、仮説検定を行うことにより、ボトルネックを判定する手段を含む
請求項10記載の品質劣化要因判定装置。 - アプリケーションサーバから提供されるアプリケーションのサービス利用時に、品質劣化要因が利用者端末にあるか否かを判定するための品質劣化要因判定方法であって、
データ受信手段が、前記利用者端末において推定対象期間に行われたデータ取得、スクリプト実行、画面描画の各処理の継続時間を判定対象ログとして取得して受信ログ記憶手段に格納するデータ受信ステップと、
ボトルネック判定手段が、前記利用者端末で行われるアプリケーションに対する操作と同様の操作を行った場合のリファレンスログ、または、スクリプト実行や画面描画の処理の連続領域長の経験累積分布のデータと、前記判定対象ログを比較し、評価値を求め、該評価値の合計値と予め設定されている閾値との比較結果に基づいて、該利用者端末に品質劣化要因があるか否かを判定するボトルネック判定ステップと、
を行うことを特徴とする品質劣化要因判定方法。 - 前記ボトルネック判定ステップにおいて、
前記リファレンスログと前記判定対象ログのそれぞれ対応する処理の統計値を比較して、比較結果に応じて点数を付与し、該点数の合計値を求め、該合計値が、利用端末にボトルネックが存在することを意味する少なくとも1つの閾値を超える場合に利用端末にボトルネックがあると判定する
請求項14記載の品質劣化要因判定方法。 - 通常利用されるであろう様々な操作を繰り返したスクリプト実行または画面描画の処理のログから該スクリプト実行または画面描画の処理の連続領域長の経験累積分布を連続領域長記憶手段に登録する連続領域長DB生成ステップを更に行い、
前記ボトルネック判定ステップにおいて、
前記判定対象ログに含まれる連続領域長データ数を累積比率化し、該累積比率化された判定対象ログのデータについて、前記連続領域長記憶手段の連続領域長の経験累積分布に対する上下関係を判定することによって利用者端末におけるボトルネックを判定する
請求項14記載の品質劣化要因判定方法。 - 通常利用されるであろう様々な操作を繰り返したスクリプト実行または画面描画の処理のログから該スクリプト実行または画面描画の処理の連続領域長の経験累積分布を連続領域長記憶手段に登録する連続領域長DB生成ステップを更に行い、
前記ボトルネック判定ステップにおいて、
前記判定対象ログに含まれる連続領域長データ数を累積比率化し、該累積比率化された判定対象ログのデータについて、前記連続領域長記憶手段の連続領域長の経験累積分布を参照し、仮説検定を行うことにより、ボトルネックを判定する
請求項14記載の品質劣化要因判定方法。 - コンピュータを、
請求項10乃至13のいずれか1項に記載の品質劣化要因判定装置の各手段として機能させるための品質劣化要因判定装置プログラム。 - アプリケーション実行にあたり、入力として与えられた操作である入力操作と類似する操作を抽出する類似操作抽出装置であって、
アプリケーションログとネットワークログのいずれか一方のみ、もしくは両方の組み合わせから切り出されたユーザ体感待ち時間に対応するログに基づき、特徴量を算出する特徴量算出手段と、
算出した前記特徴量と過去に記憶手段に保存された特徴量、及び予め定められた類似度関数に基づき前記特徴量と前記過去に保存された特徴量との類似度を算出する類似度算出手段と、
を有することを特徴とする類似操作抽出装置。 - 前記類似度算出手段によって算出された類似度から、入力操作と類似した操作を抽出する類似操作抽出手段を更に有する
請求項19記載の類似操作抽出装置。 - 前記特徴量に識別番号を与え、前記識別番号と前記特徴量を前記記憶手段に保存する手段を更に有する
請求項19または20記載の類似操作抽出装置。 - 前記特徴量算出手段は、
前記アプリケーションログから算出した値、もしくは、前記ネットワークログから算出した値、もしくはその双方を前記特徴量とする
請求項19記載の類似操作抽出装置。 - 前記類似度算出手段は、
前記特徴量算出手段で得られた前記特徴量と前記過去に保存された特徴量間の類似度を求め、重み付けして足し合わせた値を前記類似度とする手段を含む
請求項19記載の類似操作抽出装置。 - アプリケーション実行にあたり、入力として与えられた操作である入力操作と類似する操作を抽出する類似操作抽出方法であって、
特徴量算出手段、類似度算出手段、記憶手段を有する装置において、
前記特徴量算出手段が、アプリケーションログとネットワークログのいずれか一方のみ、もしくは両方の組み合わせから切り出されたユーザ体感待ち時間に対応するログに基づき、特徴量を算出する特徴量算出ステップと、
前記類似度算出手段が、前記特徴量算出ステップで算出された前記特徴量と過去に前記記憶手段に保存された特徴量、及び予め定められた類似度関数に基づき前記特徴量と前記過去に保存された特徴量との類似度を算出する類似度算出ステップと、
を行うことを特徴とする類似操作抽出方法。 - 類似操作抽出手段を更に有する装置において、
前記類似操作抽出手段が、前記類似度算出ステップによって算出された類似度から、入力操作と類似した操作を抽出するステップを、更に行う
請求項24記載の類似操作抽出方法。 - コンピュータを、
請求項19乃至23のいずれか1項に記載の類似操作抽出装置の各手段として機能させるための類似操作抽出プログラム。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201380017938.8A CN104246713B (zh) | 2012-03-30 | 2013-03-29 | 利用者体感品质推测装置、品质劣化主要原因判定装置、类似操作抽出装置及其方法 |
US14/389,034 US9794149B2 (en) | 2012-03-30 | 2013-03-29 | User experienced quality estimation apparatus, terminal bottleneck determination apparatus, similar operation extraction apparatus, method and program |
JP2014508214A JP5865486B2 (ja) | 2012-03-30 | 2013-03-29 | 利用者体感品質推定装置、端末ボトルネック判定装置、類似操作抽出装置、及び方法、並びにプログラム |
EP13769756.1A EP2838022B1 (en) | 2012-03-30 | 2013-03-29 | User sensory quality estimation device, terminal bottleneck determination device, similar operation extraction device, and methods and programs therefor |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012082803 | 2012-03-30 | ||
JP2012-082802 | 2012-03-30 | ||
JP2012-082803 | 2012-03-30 | ||
JP2012082802 | 2012-03-30 | ||
JP2012-234727 | 2012-10-24 | ||
JP2012234727 | 2012-10-24 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013147226A1 true WO2013147226A1 (ja) | 2013-10-03 |
Family
ID=49260466
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/059680 WO2013147226A1 (ja) | 2012-03-30 | 2013-03-29 | 利用者体感品質推定装置、端末ボトルネック判定装置、類似操作抽出装置、及び方法、並びにプログラム |
Country Status (5)
Country | Link |
---|---|
US (1) | US9794149B2 (ja) |
EP (1) | EP2838022B1 (ja) |
JP (1) | JP5865486B2 (ja) |
CN (1) | CN104246713B (ja) |
WO (1) | WO2013147226A1 (ja) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015195530A (ja) * | 2014-03-31 | 2015-11-05 | Kddi株式会社 | 通信行動分析装置 |
JP2016031720A (ja) * | 2014-07-30 | 2016-03-07 | 西日本電信電話株式会社 | 仮想化デスクトップ提供システム |
CN111858043A (zh) * | 2020-07-10 | 2020-10-30 | 海尔优家智能科技(北京)有限公司 | 服务请求的处理方法及装置、存储介质、电子装置 |
CN112905430A (zh) * | 2021-02-24 | 2021-06-04 | 浙江大华技术股份有限公司 | 日志输出方法、装置、存储介质及电子装置 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9740363B2 (en) * | 2013-10-02 | 2017-08-22 | Velocity Technology Solutions, Inc. | Methods and systems for managing community information |
US9940311B2 (en) | 2014-03-03 | 2018-04-10 | International Business Machines Corporation | Optimized read/write access to a document object model |
US10621849B2 (en) * | 2015-09-25 | 2020-04-14 | Intel Corporation | Alert system for internet of things (IoT) devices |
JP6710711B2 (ja) * | 2018-02-06 | 2020-06-17 | 日本電信電話株式会社 | 推定装置、推定方法及びプログラム |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003298655A (ja) | 2002-04-05 | 2003-10-17 | Nippon Telegr & Teleph Corp <Ntt> | サイト領域内ボトルネック特定方法 |
JP2009104267A (ja) * | 2007-10-22 | 2009-05-14 | Hitachi Ltd | ウェブアプリケーションの処理記録方法および処理記録装置 |
JP2010146146A (ja) * | 2008-12-17 | 2010-07-01 | Fujitsu Ltd | トランザクションモデル生成支援プログラム、トランザクションモデル生成支援装置、およびトランザクションモデル生成支援方法 |
JP2010146260A (ja) * | 2008-12-18 | 2010-07-01 | Hitachi Ltd | 再現処理方法、計算機システムおよびプログラム |
JP2011128969A (ja) * | 2009-12-18 | 2011-06-30 | Fujitsu Ltd | 運用管理プログラム、運用管理装置および運用管理方法 |
JP2011138422A (ja) * | 2009-12-29 | 2011-07-14 | Nippon Telegr & Teleph Corp <Ntt> | 行動パターン検出装置、行動パターン検出方法及び行動パターン検出プログラム |
JP2011142473A (ja) | 2010-01-06 | 2011-07-21 | Ntt Communications Kk | ユーザ待ち時間推定装置、ユーザ待ち時間推定方法、及びプログラム |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6792393B1 (en) | 2001-12-03 | 2004-09-14 | At&T Corp. | System and method for diagnosing computer system operational behavior |
JP2007221207A (ja) * | 2006-02-14 | 2007-08-30 | Hitachi Ltd | 管理装置及び通信システム |
US20100017486A1 (en) * | 2008-07-16 | 2010-01-21 | Fujitsu Limited | System analyzing program, system analyzing apparatus, and system analyzing method |
JP2010097285A (ja) * | 2008-10-14 | 2010-04-30 | Fujitsu Ltd | システム分析支援プログラム、システム分析支援装置、およびシステム分析支援方法 |
EP2350933A4 (en) | 2008-10-16 | 2012-05-23 | Hewlett Packard Development Co | APPLICATION PERFORMANCE ANALYSIS |
JP5468837B2 (ja) | 2009-07-30 | 2014-04-09 | 株式会社日立製作所 | 異常検出方法、装置、及びプログラム |
JP4927181B2 (ja) | 2010-01-06 | 2012-05-09 | エヌ・ティ・ティ・コミュニケーションズ株式会社 | ユーザ待ち時間推定装置、ユーザ待ち時間推定方法、及びプログラム |
-
2013
- 2013-03-29 CN CN201380017938.8A patent/CN104246713B/zh active Active
- 2013-03-29 EP EP13769756.1A patent/EP2838022B1/en active Active
- 2013-03-29 WO PCT/JP2013/059680 patent/WO2013147226A1/ja active Application Filing
- 2013-03-29 US US14/389,034 patent/US9794149B2/en active Active
- 2013-03-29 JP JP2014508214A patent/JP5865486B2/ja active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003298655A (ja) | 2002-04-05 | 2003-10-17 | Nippon Telegr & Teleph Corp <Ntt> | サイト領域内ボトルネック特定方法 |
JP2009104267A (ja) * | 2007-10-22 | 2009-05-14 | Hitachi Ltd | ウェブアプリケーションの処理記録方法および処理記録装置 |
JP2010146146A (ja) * | 2008-12-17 | 2010-07-01 | Fujitsu Ltd | トランザクションモデル生成支援プログラム、トランザクションモデル生成支援装置、およびトランザクションモデル生成支援方法 |
JP2010146260A (ja) * | 2008-12-18 | 2010-07-01 | Hitachi Ltd | 再現処理方法、計算機システムおよびプログラム |
JP2011128969A (ja) * | 2009-12-18 | 2011-06-30 | Fujitsu Ltd | 運用管理プログラム、運用管理装置および運用管理方法 |
JP2011138422A (ja) * | 2009-12-29 | 2011-07-14 | Nippon Telegr & Teleph Corp <Ntt> | 行動パターン検出装置、行動パターン検出方法及び行動パターン検出プログラム |
JP2011142473A (ja) | 2010-01-06 | 2011-07-21 | Ntt Communications Kk | ユーザ待ち時間推定装置、ユーザ待ち時間推定方法、及びプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP2838022A4 |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015195530A (ja) * | 2014-03-31 | 2015-11-05 | Kddi株式会社 | 通信行動分析装置 |
JP2016031720A (ja) * | 2014-07-30 | 2016-03-07 | 西日本電信電話株式会社 | 仮想化デスクトップ提供システム |
CN111858043A (zh) * | 2020-07-10 | 2020-10-30 | 海尔优家智能科技(北京)有限公司 | 服务请求的处理方法及装置、存储介质、电子装置 |
CN111858043B (zh) * | 2020-07-10 | 2024-03-22 | 海尔优家智能科技(北京)有限公司 | 服务请求的处理方法及装置、存储介质、电子装置 |
CN112905430A (zh) * | 2021-02-24 | 2021-06-04 | 浙江大华技术股份有限公司 | 日志输出方法、装置、存储介质及电子装置 |
CN112905430B (zh) * | 2021-02-24 | 2023-06-13 | 浙江大华技术股份有限公司 | 日志输出方法、装置、存储介质及电子装置 |
Also Published As
Publication number | Publication date |
---|---|
JP5865486B2 (ja) | 2016-02-17 |
EP2838022B1 (en) | 2019-12-25 |
EP2838022A4 (en) | 2016-08-10 |
US9794149B2 (en) | 2017-10-17 |
CN104246713A (zh) | 2014-12-24 |
CN104246713B (zh) | 2017-05-10 |
EP2838022A1 (en) | 2015-02-18 |
US20150074177A1 (en) | 2015-03-12 |
JPWO2013147226A1 (ja) | 2015-12-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5865486B2 (ja) | 利用者体感品質推定装置、端末ボトルネック判定装置、類似操作抽出装置、及び方法、並びにプログラム | |
CN107302547B (zh) | 一种web业务异常检测方法及装置 | |
CN109241343B (zh) | 一种刷量用户识别***、方法及装置 | |
US20170185758A1 (en) | Utilizing Behavioral Features To Identify Bot | |
US20180052755A1 (en) | System status visualization method and system status visualization device | |
KR100803889B1 (ko) | 클라이언트 단말로 제공되는 서비스 성능 분석 방법 및시스템 | |
US10866939B2 (en) | Alignment and deduplication of time-series datasets | |
US9244804B2 (en) | Techniques for gauging performance of services | |
CN107426136B (zh) | 一种网络攻击的识别方法和装置 | |
CN112437062B (zh) | 一种icmp隧道的检测方法、装置、存储介质和电子设备 | |
JP6196196B2 (ja) | ログ間因果推定装置、システム異常検知装置、ログ分析システム、及びログ分析方法 | |
JP2007243459A (ja) | トラヒック状態抽出装置及び方法ならびにコンピュータプログラム | |
CN113364784B (zh) | 检测参数生成方法、装置、电子设备及存储介质 | |
CN110808995A (zh) | 安全防护方法和装置 | |
CN113269378A (zh) | 一种网络流量处理方法、装置、电子设备和可读存储介质 | |
WO2021262344A1 (en) | Method and apparatus to detect scripted network traffic | |
CN111984848A (zh) | 一种基于分布式的网络自适应分类爬虫方法 | |
JP2011186712A (ja) | 性能分析装置及び性能分析方法及び性能分析プログラム | |
US20180039440A1 (en) | Non-transitory computer-readable recording medium, boundary value specifying method, and boundary value specifying apparatus | |
Ozonat | An information-theoretic approach to detecting performance anomalies and changes for large-scale distributed web services | |
Wang et al. | Detecting performance anomaly with correlation analysis for Internetware | |
CN116112209A (zh) | 漏洞攻击流量检测方法及装置 | |
CN113656314A (zh) | 压力测试处理方法及装置 | |
US9882927B1 (en) | Periodicity detection | |
KR20210132549A (ko) | 어노멀리 검출방법 및 그 장치 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13769756 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2014508214 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14389034 Country of ref document: US Ref document number: 2013769756 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |