CN111860081A

CN111860081A - Time series signal classification method and device and electronic equipment

Info

Publication number: CN111860081A
Application number: CN201910365000.4A
Authority: CN
Inventors: 赵元; 沈海峰
Original assignee: Beijing Didi Infinity Technology and Development Co Ltd
Current assignee: Beijing Didi Infinity Technology and Development Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2020-10-30

Abstract

The application provides a method and a device for classifying time series signals and electronic equipment, wherein the method comprises the following steps: acquiring a time sequence signal to be classified of a target object, and determining a probability value of a sampling data frame in the time sequence signal; moving a preset sliding window in the sampling data frames, and determining the probability value of the sliding window according to the probability value of each sampling data frame in the sliding window at the position of the sliding window after each movement; and determining the category information to which the time sequence signal belongs according to the probability value of the sliding window. The method and the device have the advantages that the preset sliding window moves in the sampling data frame in the time sequence signal, and for the position of the sliding window after each movement, the category information of the time sequence signal is determined by counting the probability value of the sampling data frame in the sliding window at the position. The time series signal classification method is simple and effective in calculation process and suitable for application scenes of sparse data.

Description

Time series signal classification method and device and electronic equipment

Technical Field

The present application relates to the technical field of signal classification, and in particular, to a method and an apparatus for classifying time series signals, and an electronic device.

Background

The time-series signal refers to a time-series signal sequence obtained at a certain frequency. Common time series signals are mainly referred to as audio signals and video signals. In a real scene, it is often necessary to perform category analysis on the entire time series signal, for example, to determine whether a driver has bad driving behaviors such as playing a mobile phone or not wearing a safety belt according to a driving monitoring video of the driver.

For the above-mentioned classification problem of time series signals, the prior art generally uses a special time series modeling method to classify the time series signals. The common time series modeling method mainly comprises the following steps: RNN (Recurrent neural Network) technologies such as LSTM (Long Short-term memory), GRU (Gated Recurrent Unit), and HMM (Hidden Markov Model), CRF (conditional random field), which are based on stochastic processes. Since the above method is complicated, it requires a lot of skill to make the model training converge. Meanwhile, because the parameters of the model in the method are more, the parameters in the model can be well fitted only by using massive training data to participate in training, and therefore, the method is not suitable for the condition that the training samples are extremely sparse.

In summary, the existing classification method for time series signals is complex in calculation and limited in application in scenes with sparse data.

Disclosure of Invention

In view of this, an object of the embodiments of the present application is to provide a method and an apparatus for classifying a time series signal, and an electronic device, where the method can obtain a sliding window by dividing the time series signal, and further count probability values of sampling data frames in the sliding window to determine a category of the time series signal, so as to determine an operation behavior of a target object according to the category of the time series signal, and the method is simple and effective in calculation and is suitable for an application scenario of sparse data.

According to one aspect of the present application, an electronic device is provided that may include one or more storage media and one or more processors in communication with the storage media. One or more storage media store machine-readable instructions executable by a processor. When the electronic device is operated, the processor communicates with the storage medium through the bus, and the processor executes the machine readable instructions to perform one or more of the following operations:

acquiring a time sequence signal to be classified of a target object, and determining a probability value of a sampling data frame in the time sequence signal; the sampled data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampled data frame belongs to the preset category information; moving a preset sliding window in the sampling data frames, and determining the probability value of the sliding window according to the probability value of each sampling data frame in the sliding window at the position of the sliding window after each movement; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information; determining the category information to which the time sequence signal belongs according to the determined probability value of the sliding window; the category information is used for determining the operation behavior of the target object.

In a preferred embodiment of the present application, determining probability values of sampled data frames in the time series signal comprises: sampling data frames in the time sequence signals according to a preset sampling frequency to obtain the sampled data frames; the number of the sampling data frames is multiple; and carrying out classification detection on each sampling data frame to obtain the probability value of each sampling data frame.

In a preferred embodiment of the present application, the classifying and detecting each of the sampled data frames, and obtaining the probability value of each of the sampled data frames includes: and classifying and detecting each sampling data frame by adopting a machine learning model to obtain the probability value of each sampling data frame.

In a preferred embodiment of the present application, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position includes: calculating the total probability value of all the sampling data frames in the sliding window according to the probability value of each sampling data frame in the sliding window at the position; calculating an average probability value of the sliding window in combination with the total probability value and the number of the sampled data frames in the sliding window; and taking the average probability value of the sliding window as the probability value of the sliding window.

In a preferred embodiment of the present application, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position includes: sorting the probability values of all the sampling data frames in the sliding window at the position to obtain a probability value sorting sequence; and taking the probability value positioned at the middle position in the probability value sequencing sequence as the probability value of the sliding window.

In a preferred embodiment of the present application, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position includes: carrying out weighted calculation on probability values of all the sampling data frames in the sliding window at the position to obtain weighted probability values; and taking the weighted probability value as the probability value of the sliding window.

In a preferred embodiment of the present application, determining the category information to which the time-series signal belongs according to the determined probability value of the sliding window includes: determining a probability value of the time sequence signal belonging to preset category information based on the determined probability value of the sliding window; and determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information.

In a preferred embodiment of the present application, determining the probability value of the time-series signal belonging to the preset category information based on the determined probability value of the sliding window includes: judging whether the determined probability value of the sliding window is greater than a first preset threshold value or not; if the determined probability value of the sliding window is larger than the first preset threshold value, taking the probability value of the sliding window as the probability value of the time sequence signal belonging to the preset category information; and if the determined value of the sliding window is smaller than or equal to the first preset threshold, determining the next sliding window of the sliding window, taking the next sliding window as the determined sliding window, and returning to the step of judging whether the probability value of the determined sliding window is larger than the first preset threshold until the sliding window with the probability value larger than the first preset threshold is determined.

In a preferred embodiment of the present application, determining the probability value of the time-series signal belonging to the preset category information based on the determined probability value of the sliding window further includes: if all the determined sliding windows do not contain the sliding window with the probability value larger than the first preset threshold value, determining a target sliding window with the highest probability value in all the determined sliding windows; and taking the probability value of the target sliding window as the probability value of the time sequence signal belonging to preset category information.

In a preferred embodiment of the present application, determining the category information of the time-series signal according to the probability value of the time-series signal belonging to each preset category information includes: and if the probability value of the time sequence signal is greater than a second preset threshold value, determining that the time sequence signal belongs to the preset category information, and determining the preset category information as the category information of the time sequence signal.

In a preferred embodiment of the present application, the number of the time-series signals is plural; determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information comprises: sequencing the time sequence signals according to a preset sequencing rule according to the probability value of each preset category information to which each time sequence signal belongs, and obtaining a sequencing sequence of the time sequence signals under each preset category information; according to the preset sorting rule, in the sorting sequence of the time sequence signals under each preset category information, determining the category information of the first N time sequence signals as the preset category information; n is a positive integer greater than 1.

In a preferred embodiment of the present application, the time-series signal includes: video or audio of the target object; the method further comprises the following steps: and determining the operation behavior of the target object based on the class information to which the time series signal belongs.

In a preferred embodiment of the present application, the time-series signal is a driving monitoring video or a driving monitoring audio of the target object during driving; the method further comprises the following steps: determining the driving behavior of the target object based on the class information to which the time-series signal belongs.

According to another aspect of the present application, there is also provided a time-series signal classification apparatus including: the device comprises an acquisition and determination unit, a classification unit and a classification unit, wherein the acquisition and determination unit is used for acquiring a time sequence signal to be classified of a target object and determining a probability value of a sampling data frame in the time sequence signal; the time sequence signal is used for determining the driving behavior of the target object, the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to preset category information; a moving and determining unit, configured to move in the sample data frame by using a preset sliding window, and determine, for the position of the sliding window after each movement, a probability value of the sliding window according to a probability value of each sample data frame in the sliding window at the position; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information; the first determining unit is used for determining the category information to which the time sequence signal belongs according to the determined probability value of the sliding window; the category information is used for determining the operation behavior of the target object.

In a preferred embodiment of the present application, the acquiring and determining unit includes: the sampling module is used for sampling the data frames in the time sequence signals according to a preset sampling frequency to obtain the sampled data frames; the number of the sampling data frames is multiple; and the classification detection module is used for performing classification detection on each sampling data frame to obtain the probability value of each sampling data frame.

In a preferred embodiment of the present application, the classification detection module is configured to: and classifying and detecting each sampling data frame by adopting a machine learning model to obtain the probability value of each sampling data frame.

In a preferred embodiment of the present application, the moving and determining unit includes: the first calculation module is used for calculating the total probability value of all the sampling data frames in the sliding window according to the probability value of each sampling data frame in the sliding window at the position; a second calculating module, configured to calculate an average probability value of the sliding window by combining the total probability value and the number of the sampled data frames in the sliding window; and the first setting module is used for taking the average probability value of the sliding window as the probability value of the sliding window.

In a preferred embodiment of the present application, the moving and determining unit includes: the sorting module is used for sorting the probability values of all the sampling data frames in the sliding window at the position to obtain a probability value sorting sequence; and the second setting module is used for taking the probability value positioned at the middle position in the probability value sequencing sequence as the probability value of the sliding window.

In a preferred embodiment of the present application, the moving and determining unit includes: the weighted calculation module is used for carrying out weighted calculation on the probability values of all the sampling data frames in the sliding window at the position to obtain weighted probability values; and the third setting module is used for taking the weighted probability value as the probability value of the sliding window.

In a preferred embodiment of the present application, the first determining unit includes: the first determining module is used for determining the probability value of the time sequence signal belonging to preset category information based on the determined probability value of the sliding window; and the second determining module is used for determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information.

In a preferred embodiment of the present application, the first determining module is further configured to: judging whether the determined probability value of the sliding window is greater than a first preset threshold value or not; if the determined probability value of the sliding window is larger than the first preset threshold value, taking the probability value of the sliding window as the probability value of the time sequence signal belonging to the preset category information; and if the determined value of the sliding window is smaller than or equal to the first preset threshold, determining the next sliding window of the sliding window, taking the next sliding window as the determined sliding window, and returning to the step of judging whether the probability value of the determined sliding window is larger than the first preset threshold until the sliding window with the probability value larger than the first preset threshold is determined.

In a preferred embodiment of the present application, the first determining module is further configured to: if all the determined sliding windows do not contain the sliding window with the probability value larger than the first preset threshold value, determining a target sliding window with the highest probability value in all the determined sliding windows; and taking the probability value of the target sliding window as the probability value of the time sequence signal belonging to preset category information.

In a preferred embodiment of the present application, the second determining module is further configured to: and if the probability value of the time sequence signal is greater than a second preset threshold value, determining that the time sequence signal belongs to the preset category information, and determining the preset category information as the category information of the time sequence signal.

In a preferred embodiment of the present application, the number of the time-series signals is plural; the second determination module is further to: sequencing the time sequence signals according to a preset sequencing rule according to the probability value of each preset category information to which each time sequence signal belongs, and obtaining a sequencing sequence of the time sequence signals under each preset category information; according to the preset sorting rule, in the sorting sequence of the time sequence signals under each preset category information, determining the category information of the first N time sequence signals as the preset category information; n is a positive integer greater than 1.

In a preferred embodiment of the present application, the time-series signal includes: video or audio of the target object; the device further comprises: a second determination unit, configured to determine an operation behavior of the target object based on the category information to which the time-series signal belongs.

In a preferred embodiment of the present application, the time-series signal is a driving monitoring video or a driving monitoring audio of the target object during driving; the device further comprises: a third determination unit configured to determine a driving behavior of the target object based on the category information to which the time-series signal belongs.

According to another aspect of the present application, there is also provided an electronic device including: the device comprises a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium are communicated through the bus, and the processor executes the machine-readable instructions to execute the steps of the method for classifying the time-series signals.

According to another aspect of the present application, there is also provided a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, performs the steps of the method for classifying a time-series signal as described above.

In the method, firstly, a time sequence signal to be classified of a target object is obtained, probability values of sampling data frames in the time sequence signal are determined, then, a preset sliding window moves in the sampling data frames, for the position of the sliding window after each movement, the probability value of the sliding window is determined according to the probability value of each sampling data frame in the sliding window at the position, and finally, category information to which the time sequence signal belongs is determined according to the probability value determined to the sliding window, so that the operation behavior of the target object is determined according to the category information. The method includes the steps that a preset sliding window moves in a sampling data frame in a time series signal, for the position of the sliding window after each movement, the probability value of the sliding window is determined by counting the probability value of the sampling data frame in the sliding window at the position, then the category information of the time series signal is determined according to the determined probability value of the sliding window, and finally the operation behavior of a target object is determined according to the category information of the time series signal. The time series signal classification method is simple and effective in calculation process and suitable for application scenes of sparse data.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application;

fig. 2 is a flowchart illustrating a method for classifying a time-series signal according to an embodiment of the present application;

fig. 3 is a schematic diagram illustrating a sampling data frame obtained by sampling a data frame in a time-series signal and a sliding window obtained after each movement is obtained by moving a preset sliding window in the sampling data frame according to an embodiment of the present application;

fig. 4 is a flowchart illustrating a method for determining probability values of sampled data frames in a time sequence signal according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a first method for determining a probability value of a sliding window according to an embodiment of the present disclosure;

FIG. 6 is a flowchart illustrating a second method for determining a probability value of a sliding window according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a third method for determining a probability value of a sliding window according to an embodiment of the present disclosure;

fig. 8 is a flowchart illustrating a method for determining category information to which a time-series signal belongs according to a determined probability value of a sliding window according to an embodiment of the present application;

fig. 9 is a diagram illustrating class information for determining a time series signal provided by an embodiment of the present application;

fig. 10 is a schematic diagram illustrating a device for classifying time-series signals according to an embodiment of the present application.

Detailed Description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.

In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.

Fig. 1 illustrates a schematic diagram of exemplary hardware and software components of an electronic device 100 that may implement the classification methods of time series signals provided herein, according to some embodiments of the present application.

The electronic device 100 may be a general purpose computer or a special purpose computer, both of which may be used to implement the time series signal classification method of the present application. Although only a single computer is shown, for convenience, the functions described herein may be implemented in a distributed fashion across multiple similar platforms to balance processing loads.

For example, the electronic device 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and a storage medium 140 of different form, such as a disk, ROM, or RAM, or any combination thereof. Illustratively, the computer platform may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented in accordance with these program instructions. The electronic device 100 also includes an Input/Output (I/O) interface 150 between the computer and other Input/Output devices (e.g., keyboard, display screen).

The storage medium 140 stores machine-readable instructions executable by the processor 120, when the electronic device is operated, the processor 120 communicates with the storage medium 140 through a bus, and the processor executes the machine-readable instructions to execute the steps of the method for classifying the time-series signals. In addition, the storage medium may also be referred to as a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, performs the steps of the method of classifying a time-series signal described below.

The obtaining and determining unit in the processor 120 is configured to obtain a time series signal to be classified of a target object, and determine a probability value of a sampled data frame in the time series signal; the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to the preset category information.

Then, the movement and determination unit in the processor 120 is configured to determine, based on movement in the sample data frame by using a preset sliding window, for the position of the sliding window after each movement, a probability value of the sliding window according to a probability value of each sample data frame in the sliding window at the position; and the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information.

Next, a first determining unit in the processor 120 determines category information to which the time series signal belongs according to the determined probability value of the sliding window; the category information is used for determining the operation behavior of the target object.

For ease of illustration, only one processor is depicted in electronic device 100. However, it should be noted that the electronic device 100 in the present application may also comprise a plurality of processors, and thus the steps performed by one processor described in the present application may also be performed by a plurality of processors in combination or individually. For example, if the processor of the electronic device 100 executes steps a and B, it should be understood that steps a and B may also be executed by two different processors together or separately in one processor. For example, a first processor performs step a and a second processor performs step B, or the first processor and the second processor perform steps a and B together.

See fig. 2 for a flow chart of a method of classifying a time series signal.

The method for classifying time series signals shown in fig. 2 can be applied to a terminal device and can also be applied to a server, and the method includes the following steps:

step S202, acquiring a time sequence signal to be classified of a target object, and determining a probability value of a sampling data frame in the time sequence signal; the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to the preset category information;

in this embodiment, the target object may be a driver or an operator (e.g., a bank operator, an operator, etc.), and the time-series signal to be classified of the target object may be a driving monitoring video of the driver or a service monitoring video of the operator when handling services. Of course, the present embodiment does not specifically limit the target object, and does not limit the time-series signal. The time-series signal may be any other time-series signal sequence obtained according to a certain frequency, and when the time-series signal is another time-series signal sequence obtained according to a certain frequency, the function thereof may be determined according to a specific application scenario when the time-series signal is classified.

For example, when the time-series signal is a driving monitoring video of the driver, the time-series signal may be used to determine the driving behavior of the driver; and when the time series signal is the service monitoring video of the service staff, the time series signal can be used for determining the service handling behavior of the service staff.

After the time sequence signal is obtained, further determining a probability value of a sampling data frame in the time sequence signal, wherein the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to the preset category information. For example, when the time-series signal is a driving monitoring video of the driver, the preset category information may be: the type of the mobile phone is called, the type of the safety belt is not tied according to the specification, and the like. The sampling data frame is a driving monitoring video frame obtained by sampling the driving monitoring video, and the probability value of the driving monitoring video frame is used for representing the probability that the driving monitoring video frame belongs to the category of the connecting and disconnecting mobile phone and the probability that the driving monitoring video frame belongs to the category of the safety belt which is not fastened according to the regulation.

It should be noted that the preset category information may be one or multiple, and this embodiment does not specifically limit the preset category information.

Hereinafter, a process of determining a probability value of a sampled data frame in a time series signal will be described in detail, and will not be described herein again.

Step S204, a preset sliding window moves in the sampling data frame, and for the position of the sliding window after each movement, the probability value of the sliding window is determined according to the probability value of each sampling data frame in the sliding window at the position; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information;

after the sampled data frame and the probability value of the sampled data frame are obtained, the sampled data frame is further moved in a preset sliding window.

When moving, a preset sliding window is obtained, the preset sliding window is determined by the sliding window step length and the sliding window length, and then the preset sliding window is moved in the sampling data frame.

As shown in fig. 3, fig. 3 shows a process of sampling a data frame in a time-series signal to obtain a sampled data frame, and a process of moving in the sampled data frame by a preset sliding window to obtain a sliding window after each movement.

As shown in fig. 3, the horizontal lines with arrows represent the time axis, and the upper row of the graph constituted by the time axis and rectangles on the time axis represents the time-series signal, wherein each rectangle represents one data frame in the time-series signal. After sampling the data frames in the time-series signal, the sampled data frames obtained by sampling are as rectangles in the second row in fig. 3, and the process of obtaining the rectangles in the second row by sampling the rectangles in the first row can be obtained, and the sampling frequency during sampling is 2 seconds per frame. After the sampling data frame is obtained, the sampling data frame is further moved in a preset sliding window, as can be seen from the second row in fig. 3, the length of the sliding window of the preset sliding window is 3, and the step length of the sliding window is 2, so that when moving in the sampling data frame, a sliding window at the position can be obtained every time the sliding window is moved. Of course, in this embodiment, the values of the sampling frequency, the sliding window length, and the sliding window step length are not specifically limited, and may be any other values.

After the sliding window is determined, the probability value of the sliding window can be determined according to the probability value of each sampling data frame in the sliding window at the position, and the process of determining the probability value of the sliding window is described in detail below, which is not described herein again.

Step S206, determining the category information of the time sequence signal according to the determined probability value of the sliding window; the category information is used to determine the operational behavior of the target object.

The process is described in detail below and will not be described herein.

The time series signal classification method can be applied to an application scene with sparse data. For example, in a driving monitoring video of a driver, the bad driving behaviors have sparseness, and the method can detect and classify the bad driving behaviors.

The above description briefly describes the classification method of the time-series signal according to the present invention, and the details thereof will be described in detail.

Due to the locality principle of the time series signal, in an actual application scenario, probability calculation and judgment are not required to be performed on each data frame in the time series signal. Therefore, subsequent calculation can be performed in the form of frame skipping sampling and stride sliding, so that a great deal of calculation cost is saved.

In an alternative embodiment of the present invention, referring to fig. 4, step S202, the determining a probability value of the sampled data frames in the time series signal comprises the following steps:

step S401, sampling a data frame in a time sequence signal according to a preset sampling frequency to obtain a sampled data frame; the number of the sampling data frames is multiple;

step S402, classifying and detecting each sampling data frame to obtain the probability value of each sampling data frame.

In implementation, a machine learning model may be used to perform classification detection on each sampled data frame, so as to obtain a probability value of each sampled data frame.

It should be noted that the machine learning model can be determined according to a specific application scenario. For example, if the classification detection is performed on the behavior of the driver for answering and calling the mobile phone, the machine learning model is a classification model for detecting the behavior of the driver for answering and calling the mobile phone; if the classification detection is carried out on the behavior that the driver does not fasten the safety belt according to the regulation, the machine learning model is a classification model for detecting the behavior that the driver does not fasten the safety belt according to the regulation, and the like.

When classification detection is performed by a machine learning model, due to the limitation of model performance, it is possible that erroneous judgment occurs in the classification detection of a certain sampling data frame. The inventor considers that the accuracy of the mode of determining the operation behavior of the target object only from the classification detection results of some sampled data frames is poor, and for this reason, the inventor designs the following three implementation schemes to smooth out the misjudgment results, so as to improve the classification accuracy, specifically, information fusion is performed by means of normalization of the probability values of the multi-frame sampled data frames in the sliding window, and the implementation scheme of the information fusion describes the following implementation schemes.

The first implementation scheme is as follows:

in an alternative embodiment of the present invention, referring to fig. 5, in step S204, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position includes the following steps:

step S501, calculating the total probability value of all the sampling data frames in the sliding window according to the probability value of each sampling data frame in the sliding window at the position;

step S502, calculating the average probability value of the sliding window by combining the total probability value and the number of the sampling data frames in the sliding window;

In step S503, the average probability value of the sliding window is used as the probability value of the sliding window.

As can be seen from the above description, the implementation scheme is to weaken the influence of the probability value of a certain error data frame by means of averaging, so that the whole time series signal is no longer classified by the influence of the probability value of the individual error data frame.

The second implementation scheme is as follows:

in an alternative embodiment of the present invention, referring to fig. 6, in step S204, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position further includes the following steps:

s601, sequencing the probability values of all the sampling data frames in the sliding window at the position to obtain a probability value sequencing sequence;

step S602, using the probability value located at the middle position in the probability value ranking sequence as the probability value of the sliding window.

It should be noted that, if the number of the probability values at the middle position in the probability value ranking sequence is two, the average value of the two probability values at the middle position may be taken, and the tie value is used as the probability value of the sliding window.

As can be seen from the above description, the second implementation scheme weakens the influence of the probability value of a certain erroneous data frame by taking the median, so that the whole time series signal is no longer influenced by the probability value of the individual erroneous data frame to cause the classification error.

The third implementation scheme is as follows:

in an alternative embodiment of the present invention, referring to fig. 7, in step S204, determining the probability value of the sliding window according to the probability value of each sampled data frame in the sliding window at the position further includes the following steps:

step S701, carrying out weighted calculation on probability values of all sampling data frames in the sliding window at the position to obtain weighted probability values;

in step S702, the weighted probability value is used as the probability value of the sliding window.

In implementation, a weight may be set in advance for the probability value of each sampled data frame in the sliding window, then a weighted probability value is obtained by performing weighted calculation on the probability value of the sampled data frame and the corresponding weight, and the obtained weighted probability value is used as the probability value of the sliding window.

As can be seen from the above description, the third implementation scheme is to weaken the influence of the probability value of a certain error data frame by means of weighted average, so that the whole time series signal is no longer classified by the influence of the probability values of the individual error data frames.

Of course, only the three implementation schemes are given in this embodiment, and other implementation schemes may be adopted in specific applications. For example, the maximum probability value and the minimum probability value of the sampled data frame in the sliding window are removed, the remaining probability values are averaged, and the obtained average value is used as the probability value of the sliding window. The present invention is not limited to the above three embodiments, and other embodiments derived from the above three embodiments are within the scope of the present invention.

The above description describes in detail a process of determining a probability value of a sliding window, and the following description describes in detail a process of determining category information to which a time-series signal belongs.

In an optional embodiment of the present invention, in step S206, determining the category information to which the time-series signal belongs according to the determined probability value of the sliding window includes the following steps:

firstly, determining the probability value of the time sequence signal belonging to the preset category information based on the determined probability value of the sliding window;

referring to fig. 8, the process specifically includes the following steps:

step S801, judging whether the probability value of the determined sliding window is greater than a first preset threshold value or not;

step S802, if the probability value of the sliding window is larger than a first preset threshold value, taking the probability value of the sliding window as the probability value of the preset category information to which the time sequence signal belongs;

it should be noted that, if the determined probability value of the sliding window is greater than the first preset threshold, the sliding window is not moved backward, that is, the probability value of the subsequent sliding window is not calculated, and the probability value of the sliding window greater than the first preset threshold is directly used as the probability value of the preset category information to which the time sequence signal belongs. By the scheme, subsequent redundant calculation can be reduced, and the calculation amount is greatly reduced. Therefore, the time sequence signal classification method is simple in calculation and good in accuracy.

Step S803, if the determined value of the sliding window is less than or equal to the first preset threshold, determining a next sliding window of the sliding window, taking the next sliding window as the determined sliding window, and returning to perform step S801 until determining the sliding window with the probability value greater than the first preset threshold.

Step S804, if all the determined sliding windows do not contain the sliding window with the probability value larger than the first preset threshold value, determining a target sliding window with the highest probability value in all the determined sliding windows;

step S805, the probability value of the target sliding window is used as the probability value of the preset category information to which the time series signal belongs.

That is, if the preset sliding window moves and traverses to all the sampled data frames, and all the determined sliding windows do not contain the sliding window with the probability value larger than the first preset threshold value, the target sliding window with the highest probability value is determined in all the determined sliding windows, and the probability value of the target sliding window is used as the probability value of the preset category information to which the time sequence signal belongs.

Expressed by a mathematical expression:

wherein l represents the sliding window length, P _iThe probability value of each sampling data frame in the sliding window is shown, E is the probability value of the time series signal belonging to the preset category information, and max (·) is the maximum value operation, which is only exemplified by the way of averaging.

Then, after the probability value of the time sequence signal belonging to the preset category information is determined according to the determined probability value of the sliding window, the category information of the time sequence signal can be determined according to the probability value of the time sequence signal belonging to each preset category information.

In specific implementation, any of the following schemes can be adopted:

and (I) if the probability value of the time sequence signal is greater than a second preset threshold value, determining that the time sequence signal belongs to preset category information, and determining the preset category information as the category information of the time sequence signal.

For ease of understanding, the following is exemplified:

for example, the preset category information is category information of the access handset, and if the probability value of the time-series signal is 0.8 and the second preset threshold value is 0.7, it is determined that the category information of the time-series signal belongs to the category of the access handset. The second preset threshold may be equal to the first preset threshold, or may not be equal to the first preset threshold, and the size of the second preset threshold is not specifically limited in the embodiment of the present invention. The second preset threshold is used for representing a threshold belonging to preset category information.

In the scheme (II), the number of the time series signals is multiple; sequencing the plurality of time sequence signals according to a preset sequencing rule according to the probability value of each preset category information to which each time sequence signal belongs, and obtaining a sequencing sequence of the time sequence signals under each preset category information; according to a preset sorting rule, in a sorting sequence of time sequence signals under each preset category information, determining category information of the first N time sequence signals as preset category information; n is a positive integer greater than 1, and as shown in fig. 9, which shows an ordered sequence of time-series signals, the category information of the top 5 time-series signals is determined as the preset category information.

For ease of understanding, this scheme is also illustrated below: assuming that the time-series signals are A, B and C, respectively, in the category information of the access handset, according to the probability values of the time-series signals of the category information of the access handset (for example, under the category information of the access handset, the probability value of the time-series signal a is 0.9, the probability value of the time-series signal B is 0.85, and the probability value of the time-series signal C is 0.8), the time-series signals are ranked in the order of the probability values from large to small, the ranking sequence of the time-series signals under the category information of the access handset is A, B, C, and the category information of the first 2 time-series signals (i.e., the time-series signal a and the time-series signal B) in the ranking sequence is determined as the category information of the access handset.

In this embodiment, the preset sorting rule refers to a sorting rule from large to small according to the probability value of each time series signal. In addition, the numerical value N is not particularly limited in the embodiment of the present invention, and may be other values.

In an alternative embodiment of the invention, the time series signal comprises: video or audio of the target object; after the category information to which the time-series signal belongs is obtained, the operation behavior of the target object is determined based on the category information to which the time-series signal belongs.

In a specific application scene, the time series signal is a driving monitoring video or a driving monitoring audio of a target object in the driving process; after the category information to which the time-series signal belongs is obtained, the driving behavior of the target object is determined based on the category information to which the time-series signal belongs.

For example, the category information to which the time-series signal belongs is category information of a call taker phone, and based on this, it can be determined that the driving behavior of the target object is a behavior of a call taker phone in the driving process of the driver.

Of course, the application scenario described above is not specifically limited in the embodiment of the present invention, and may be any other scenario.

The time series signal classification method has the advantages of small calculated amount and simple process on the premise of ensuring the classification accuracy, and can be applied to an application scene with sparse data.

Fig. 10 is a block diagram illustrating a time-series signal classification apparatus according to some embodiments of the present application, where the functions performed by the time-series signal classification apparatus correspond to the steps performed by the above-described method. The apparatus may be understood as the terminal device, the server, a processor of the terminal device, or a processor of the server, and of course, may also be understood as a component that is independent from the terminal device, the server, or the processor and implements the functions of the present application in the terminal device or under the control of the server, as shown in the figure, the apparatus for classifying the time series signals may include: an acquisition and determination unit 10, a movement and determination unit 20 and a first determination unit 30.

The acquiring and determining unit 10 is configured to acquire a time series signal to be classified of a target object, and determine a probability value of a sampling data frame in the time series signal; the time sequence signal is used for determining the driving behavior of a target object, the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to the preset category information;

A moving and determining unit 20, configured to move in the sample data frame by using a preset sliding window, and determine, for the position of the sliding window after each movement, a probability value of the sliding window according to a probability value of each sample data frame in the sliding window at the position; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information;

a first determining unit 30, configured to determine category information to which the time series signal belongs according to the determined probability value of the sliding window; the category information is used to determine the operational behavior of the target object.

Optionally, the obtaining and determining unit includes:

the sampling module is used for sampling the data frames in the time sequence signals according to a preset sampling frequency to obtain the sampled data frames; the number of the sampling data frames is multiple;

and the classification detection module is used for performing classification detection on each sampling data frame to obtain the probability value of each sampling data frame.

Optionally, the classification detection module is configured to:

and classifying and detecting each sampling data frame by adopting a machine learning model to obtain the probability value of each sampling data frame.

Optionally, the moving and determining unit includes:

the first calculation module is used for calculating the total probability value of all the sampling data frames in the sliding window according to the probability value of each sampling data frame in the sliding window at the position;

a second calculating module, configured to calculate an average probability value of the sliding window by combining the total probability value and the number of the sampled data frames in the sliding window;

and the first setting module is used for taking the average probability value of the sliding window as the probability value of the sliding window.

Optionally, the moving and determining unit includes:

The sorting module is used for sorting the probability values of all the sampling data frames in the sliding window at the position to obtain a probability value sorting sequence;

and the second setting module is used for taking the probability value positioned at the middle position in the probability value sequencing sequence as the probability value of the sliding window.

Optionally, the moving and determining unit includes:

the weighted calculation module is used for carrying out weighted calculation on the probability values of all the sampling data frames in the sliding window at the position to obtain weighted probability values;

and the third setting module is used for taking the weighted probability value as the probability value of the sliding window.

Optionally, the first determining unit includes:

the first determining module is used for determining the probability value of the time sequence signal belonging to preset category information based on the determined probability value of the sliding window;

and the second determining module is used for determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information.

Optionally, the first determining module is further configured to:

judging whether the determined probability value of the sliding window is greater than a first preset threshold value or not;

If the determined probability value of the sliding window is larger than the first preset threshold value, taking the probability value of the sliding window as the probability value of the time sequence signal belonging to the preset category information;

and if the determined value of the sliding window is smaller than or equal to the first preset threshold, determining the next sliding window of the sliding window, taking the next sliding window as the determined sliding window, and returning to the step of judging whether the probability value of the determined sliding window is larger than the first preset threshold until the sliding window with the probability value larger than the first preset threshold is determined.

Optionally, the first determining module is further configured to:

if all the determined sliding windows do not contain the sliding window with the probability value larger than the first preset threshold value, determining a target sliding window with the highest probability value in all the determined sliding windows;

and taking the probability value of the target sliding window as the probability value of the time sequence signal belonging to preset category information.

Optionally, the second determining module is further configured to:

and if the probability value of the time sequence signal is greater than a second preset threshold value, determining that the time sequence signal belongs to the preset category information, and determining the preset category information as the category information of the time sequence signal.

Optionally, the number of the time-series signals is plural; the second determination module is further to:

sequencing the time sequence signals according to a preset sequencing rule according to the probability value of each preset category information to which each time sequence signal belongs, and obtaining a sequencing sequence of the time sequence signals under each preset category information;

according to the preset sorting rule, in the sorting sequence of the time sequence signals under each preset category information, determining the category information of the first N time sequence signals as the preset category information; n is a positive integer greater than 1.

Optionally, the time series signal comprises: video or audio of the target object; the device further comprises:

a second determination unit, configured to determine an operation behavior of the target object based on the category information to which the time-series signal belongs.

Optionally, the time-series signal is a driving monitoring video or driving monitoring audio of the target object in the driving process; the device further comprises:

a third determination unit configured to determine a driving behavior of the target object based on the category information to which the time-series signal belongs.

The present application further provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods for classifying a time-series signal described above.

The modules may be connected or in communication with each other via a wired or wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may comprise a connection over a LAN, WAN, bluetooth, ZigBee, NFC, or the like, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for classifying a time series signal, comprising:

acquiring a time sequence signal to be classified of a target object, and determining a probability value of a sampling data frame in the time sequence signal; the sampled data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampled data frame belongs to the preset category information;

moving a preset sliding window in the sampling data frames, and determining the probability value of the sliding window according to the probability value of each sampling data frame in the sliding window at the position of the sliding window after each movement; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information;

Determining the category information to which the time sequence signal belongs according to the determined probability value of the sliding window; the category information is used for determining the operation behavior of the target object.

2. The method of claim 1, wherein determining probability values for sampled data frames in the time series of signals comprises:

sampling data frames in the time sequence signals according to a preset sampling frequency to obtain the sampled data frames; the number of the sampling data frames is multiple;

and carrying out classification detection on each sampling data frame to obtain the probability value of each sampling data frame.

3. The method of claim 2, wherein performing classification detection on each of the sampled data frames and obtaining a probability value for each of the sampled data frames comprises:

4. The method of claim 1, wherein determining the sliding window probability value according to the sliding window probability value for each sampled data frame in the sliding window at the position comprises:

calculating the total probability value of all the sampling data frames in the sliding window according to the probability value of each sampling data frame in the sliding window at the position;

Calculating an average probability value of the sliding window in combination with the total probability value and the number of the sampled data frames in the sliding window;

and taking the average probability value of the sliding window as the probability value of the sliding window.

5. The method of claim 1, wherein determining the sliding window probability value according to the sliding window probability value for each sampled data frame in the sliding window at the position comprises:

sorting the probability values of all the sampling data frames in the sliding window at the position to obtain a probability value sorting sequence;

and taking the probability value positioned at the middle position in the probability value sequencing sequence as the probability value of the sliding window.

6. The method of claim 1, wherein determining the sliding window probability value according to the sliding window probability value for each sampled data frame in the sliding window at the position comprises:

carrying out weighted calculation on probability values of all the sampling data frames in the sliding window at the position to obtain weighted probability values;

and taking the weighted probability value as the probability value of the sliding window.

7. The method of claim 1, wherein determining the category information to which the time-series signal belongs according to the determined probability value of the sliding window comprises:

Determining a probability value of the time sequence signal belonging to preset category information based on the determined probability value of the sliding window;

and determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information.

8. The method of claim 7, wherein determining the probability value of the time-series signal belonging to a preset category of information based on the determined probability value of the sliding window comprises:

9. The method of claim 8, wherein determining the probability value of the time-series signal belonging to a preset category of information based on the determined probability value of the sliding window further comprises:

10. The method of claim 7, wherein determining the category information of the time-series signal according to the probability value of the time-series signal belonging to each preset category information comprises:

11. The method according to claim 7, wherein the number of the time-series signals is plural;

determining the category information of the time sequence signal according to the probability value of the time sequence signal belonging to each preset category information comprises:

12. The method of claim 1, wherein the time series signal comprises: video or audio of the target object; the method further comprises the following steps:

and determining the operation behavior of the target object based on the class information to which the time series signal belongs.

13. The method of claim 1, wherein the time-series signal is a driving monitoring video or driving monitoring audio of the target object during driving; the method further comprises the following steps:

determining the driving behavior of the target object based on the class information to which the time-series signal belongs.

14. An apparatus for classifying a time-series signal, comprising:

The device comprises an acquisition and determination unit, a classification unit and a classification unit, wherein the acquisition and determination unit is used for acquiring a time sequence signal to be classified of a target object and determining a probability value of a sampling data frame in the time sequence signal; the time sequence signal is used for determining the driving behavior of the target object, the sampling data frame is obtained after sampling the data frame in the time sequence signal, and the probability value is used for representing the probability that the sampling data frame belongs to preset category information;

a moving and determining unit, configured to move in the sample data frame by using a preset sliding window, and determine, for the position of the sliding window after each movement, a probability value of the sliding window according to a probability value of each sample data frame in the sliding window at the position; the probability value of the sliding window is used for representing the probability that the sampling data frame in the sliding window belongs to the preset category information;

the first determining unit is used for determining the category information to which the time sequence signal belongs according to the determined probability value of the sliding window; the category information is used for determining the operation behavior of the target object.

15. The apparatus of claim 14, wherein the means for obtaining and determining comprises:

16. The apparatus of claim 15, wherein the classification detection module is configured to:

17. The apparatus of claim 14, wherein the movement and determination unit comprises:

18. The apparatus of claim 14, wherein the movement and determination unit comprises:

19. The apparatus of claim 14, wherein the movement and determination unit comprises:

20. The apparatus of claim 14, wherein the first determining unit comprises:

21. The apparatus of claim 20, wherein the first determining module is further configured to:

22. The apparatus of claim 21, wherein the first determining module is further configured to:

23. The apparatus of claim 20, wherein the second determining module is further configured to:

24. The apparatus of claim 20, wherein the number of the time-series signals is plural; the second determination module is further to:

25. The apparatus of claim 14, wherein the time series signal comprises: video or audio of the target object; the device further comprises:

26. The apparatus of claim 14, wherein the time-series signal is a driving monitoring video or driving monitoring audio of the target object during driving; the device further comprises:

27. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method of classifying a time-series signal according to any one of claims 1 to 13.

28. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the method for classifying a time-series signal according to any one of claims 1 to 13.