CN113206998B - Method and device for quality inspection of video data recorded by service - Google Patents

Method and device for quality inspection of video data recorded by service Download PDF

Info

Publication number
CN113206998B
CN113206998B CN202110486354.1A CN202110486354A CN113206998B CN 113206998 B CN113206998 B CN 113206998B CN 202110486354 A CN202110486354 A CN 202110486354A CN 113206998 B CN113206998 B CN 113206998B
Authority
CN
China
Prior art keywords
service
segment
quality inspection
video data
object detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110486354.1A
Other languages
Chinese (zh)
Other versions
CN113206998A (en
Inventor
沈超建
魏薇郦
刘金山
江文乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202110486354.1A priority Critical patent/CN113206998B/en
Publication of CN113206998A publication Critical patent/CN113206998A/en
Application granted granted Critical
Publication of CN113206998B publication Critical patent/CN113206998B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00Diagnosis, testing or measuring for television systems or their details
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272Voice signal separating
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/57Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for processing of video signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

The invention provides a method and a device for quality inspection of video data recorded by a service, which can be used in the technical field of cloud computing or other fields. On the other hand, the lightweight object detection algorithm has low requirement on computing power, and avoids influence on data recording operation caused by overlarge occupation of data recording terminal resources.

Description

Method and device for quality inspection of video data recorded by service
Technical Field
The invention relates to the technical field of cloud computing, in particular to a method and a device for quality inspection of video data recorded by a service.
Background
In order to protect the rights of consumers, according to the requirements of the supervision institution, when the financial institution sells products such as financing, fund and insurance, the financial institution needs to synchronously record the sound and video (short for data recording) in the selling process, and the situations of misleading the selling, privately selling 'flyer bill' and the like are prevented. The financial institution formulates data recording steps and standard conversational templates of each step when products are sold according to self business processes, and checks whether data recording videos meet requirements one by one after the data recording is finished.
In order to reduce the labor input of quality inspection, unify the quality inspection standard and improve the timeliness, partial mechanisms utilize the artificial intelligence technology to carry out real-time quality inspection on data recording, namely, one path of real-time audio and video stream is additionally obtained to carry out automatic quality inspection when the data is recorded, the inspection is completed before a customer leaves a business place as much as possible, the condition that the customer returns to supplementary recording again when the data is recorded and is not required is avoided, and therefore the customer experience and the business efficiency are improved.
The data recording steps and standard speech technology templates of different mechanisms are different, and a plurality of quality inspection points exist in voice and video respectively, such as: (1) voice quality inspection: confirming the identity of the customer, introducing by the customer manager the product issuing subject, the property of the warranty, the income level, the risk condition, the handling fee and the like; (2) video quality inspection: the customer manager shows the certificate, shows the product information, and has signature in the key link.
The data recording quality inspection needs to check whether relevant quality inspection points are in compliance one by one from the recording video, and the current universal data recording real-time quality inspection system has the following defects:
the video quality inspection is generally a mode of performing frame extraction and picture formation on a time period where a quality inspection point is located according to a segmentation result and then performing quality inspection on the picture. In order to ensure the accuracy of quality inspection, the frame extraction frequency cannot be too low, so that for a certain quality inspection point, frames may need to be extracted from videos of several minutes or even more than ten minutes, at this time, the number of extracted pictures may reach hundreds, the network pressure is large when the extracted pictures are transmitted to a quality inspection service end, the consumption of computing resources is large, and the consumption of quality inspection is long.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a device for quality inspection of video data recorded by a service, which are used for performing frame extraction on the video data.
In order to solve the technical problems, the invention provides the following technical scheme:
in a first aspect, the present invention provides a method for quality inspection of video data recorded in a service, including:
acquiring recorded video data of a service in progress;
determining the segment of the recorded video data according to the service segment information of the service;
performing frame extraction on the recorded video data aiming at the current segment to obtain a plurality of video images;
inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image;
and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
In a preferred embodiment, the method for quality inspection of video data recorded by a service further includes:
and determining the service segmentation information of each service according to the service rule and the service characteristics.
In a preferred embodiment, the determining the segment of the recorded video data according to the service segment information of the service includes:
and determining the section of the recorded video data according to the audio data corresponding to the recorded video data and the service section information.
In a preferred embodiment, the traffic segmentation information includes: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
In a preferred embodiment, the determining the segment of the recorded video data according to the audio data corresponding to the recorded video data and the service segment information includes:
comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information, and determining the identity information of the current audio producer;
and determining the current service segment according to the identity information, the current service running time and the duration of the tone.
In a preferred embodiment, the determining, according to the identity information, the current service running duration and the duration of the tone, the current service segment includes:
determining the service segmentation range according to the identity information;
according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range;
and matching the service segments in the range of the updated service segment according to the current service implementation duration and the service duration of all the service segments, and determining the service segment with the highest matching degree as the current service segment.
In a preferred embodiment, if the confidence degrees corresponding to all the video images of the current frame extraction are lower than a set threshold, or the current object detection result is a target-free object, the method further includes:
and executing iteration operation, performing frame extraction on the recorded video data again to obtain a plurality of video images, and inputting the plurality of video images and the current segment into a preset object detection model until the confidence coefficient output by the object detection model is higher than a set threshold value.
In a preferred embodiment, further comprising:
establishing the object detection model;
training the object detection model using a plurality of historical video images with labeled confidence and objects.
In a second aspect, the present invention provides a device for quality inspection of video data recorded in a service, including:
the acquisition module acquires recorded video data of a service in progress;
the segmentation determining module is used for determining the segments of the recorded video data according to the service segmentation information of the service;
the frame extracting module is used for extracting frames of the recorded video data aiming at the current segment to obtain a plurality of video images;
the input module is used for inputting the video images and the current segments into a preset object detection model, and the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image;
and the quality inspection module is used for performing quality inspection operation on the video image with the reliability higher than a set threshold value.
In a preferred embodiment, further comprising:
and the segmentation information determining module is used for determining the service segmentation information of each service according to the service rule and the service characteristics.
In a preferred embodiment, the segment determining module is specifically configured to determine the segment of the recorded video data according to the audio data corresponding to the recorded video data and the service segment information.
In a preferred embodiment, the traffic segmentation information includes: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
In a preferred embodiment, the service segment information includes identity information and tone information of each service segment audio producer, and the segment determining module includes:
the identity information determining unit is used for comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information and determining the identity information of the current audio producer;
and the service segment determining unit is used for determining the current service segment according to the identity information, the current service progress time and the duration of the tone.
In a preferred embodiment, the determining, according to the identity information, the current service running duration, and the duration of the tone, the current service segment includes:
determining the service segmentation range according to the identity information;
according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range;
and matching the service segments in the range of the updated service segment according to the current service carrying time and the service duration of all the service segments, and determining the service segment with the highest matching degree as the current service segment.
In a preferred embodiment, further comprising: and the iteration operation module executes iteration operation if the corresponding confidence degrees of all the currently framed video images are lower than a set threshold value or the current object detection result is no target object, performs frame extraction on the recorded video data again to obtain a plurality of video images, and inputs the plurality of video images and the current segments into a preset object detection model until the confidence degree output by the object detection model is higher than the set threshold value.
In a preferred embodiment, further comprising:
the model building module is used for building the object detection model;
and the model training module is used for training the object detection model by utilizing a plurality of historical video images with labeled confidence degrees and objects.
In a third aspect, the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the quality inspection method for video data recorded by a service when executing the program.
In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method for quality inspection of video data recorded by a service.
According to the technical scheme, the method and the device for quality inspection of the video data recorded by the service provided by the invention comprise the steps of firstly determining the segment of the recorded video data according to the service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the video images and the current segmentation into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than a set threshold value, wherein frame extraction is performed on video data in the current segment, then lightweight quality inspection is performed by using an object detection model, and compared with a general method for performing frame extraction on video in the whole technical segment, the method further accurately positions the time point of the occurrence of a quality inspection object by using a lightweight object detection algorithm, and only extracts a plurality of pictures for subsequent quality inspection, so that the number of pictures needing to be transmitted and detected is greatly reduced, the picture transmission pressure is reduced, the resource consumption and the quality inspection time consumption of quality inspection are reduced, and the quality inspection timeliness is further improved. On the other hand, the lightweight object detection algorithm has low calculation force requirement, and avoids influencing data recording operation due to overlarge occupied data recording terminal resources.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart illustrating a method for quality inspection of video data recorded by a service in an embodiment of the present invention.
Fig. 2 is a flowchart illustrating an embodiment of a video data quality inspection apparatus for service recording according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
It should be noted that the method and the device for quality inspection of video data recorded by a service disclosed by the present invention can be used in the technical field of cloud computing, and can also be used in any field except the technical field of cloud computing.
In one or more embodiments of the invention, the recorded data generally includes audio data and video data. The quality inspection contents of the audio and video data streams comprise that a customer manager confirms the identity of the customer and introduces the customer by self, product conditions (such as a release subject, a policy attribute, income level, risk conditions, commission charge and the like) are introduced, the customer manager and the customer are positioned on the same picture, key links have customer signatures, whether the customer manager says forbidden words or not and the like.
Video quality inspection generally adopts a mode of performing frame extraction on a time period where a quality inspection point is located to form a picture according to a segmentation result, and then performing quality inspection on the picture. In order to ensure the accuracy of quality inspection, the frame extraction frequency cannot be too low, so that for a certain quality inspection point, frames may need to be extracted from videos of several minutes or even more than ten minutes, at this time, the number of extracted pictures may reach hundreds, the network pressure is large when the extracted pictures are transmitted to the quality inspection service end, the consumption of computing resources is large, and the quality inspection consumes long time.
Based on the above content, the present invention first provides a service-recorded video data quality inspection apparatus for implementing the service-recorded video data quality inspection method provided in one or more embodiments of the present invention, where the service-recorded video data quality inspection apparatus may be in communication connection with a plurality of front-end recording devices, and the service-recorded video data quality inspection apparatus may specifically access the front-end recording devices through a dedicated network.
The quality inspection device for the video data recorded by the service acquires the recorded video data of the ongoing service; determining the segment of the recorded video data according to the service segment information of the service; aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
It is understood that the front-end recording device may include a device with recording and processing capabilities, such as a smart phone, a tablet electronic device, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), and the like, and may also include only a device with only a recording function, such as a recorder, and the present invention is not limited thereto.
In another practical application scenario, the part for performing quality inspection on the recorded data segments may be performed by the quality inspection apparatus as described in the above, or may be partially performed in the front-end recording device. Specifically, the selection may be performed according to the processing capability of the front-end recording device, the limitation of the user usage scenario, and the like. The invention is not limited in this regard.
The front-end recording device may have a communication module (i.e., a communication unit), and may be in communication connection with a quality inspection apparatus, so as to implement data transmission with the server. For example, the communication unit may send the recorded data segment quality inspection instruction to the quality inspection apparatus, so that the quality inspection apparatus performs the recorded data segment quality inspection according to the recorded data segment quality inspection instruction. The communication unit can also receive a quality inspection result returned by the quality inspection device.
The quality testing device and the front-end recording device can communicate with each other by using any suitable network protocol, including the network protocol which has not been developed at the filing date of the present invention. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.
According to the video data quality inspection device, the electronic equipment and the computer readable storage medium for the video data recorded by the service recorded video data quality inspection method, frames are extracted according to the video data in the current section, then the object detection model is used for light-weight quality inspection, and compared with a universal method for extracting the frames of the video in the whole technical section, the method and the device for the video data quality inspection further accurately position the time point when the quality inspection object appears through a light-weight object detection algorithm, and only a plurality of pictures are extracted for subsequent quality inspection, so that the number of the pictures needing to be transmitted and detected is greatly reduced, the transmission pressure of the pictures is reduced, the resource consumption and the quality inspection time are calculated through the quality inspection, and the timeliness of the quality inspection is further improved. On the other hand, the lightweight object detection algorithm has low calculation force requirement, and avoids influencing data recording operation due to overlarge occupied data recording terminal resources.
The following embodiments and application examples are specifically and respectively described.
In order to solve the problems that in the prior art, in order to ensure the quality inspection accuracy, the frame extraction frequency cannot be too low, and therefore, for a certain quality inspection point, it may be necessary to perform frame extraction on videos of several minutes or even more than ten minutes, at this time, the number of extracted pictures may reach hundreds, the network pressure is large when the extracted pictures are transmitted to a quality inspection service end, the consumption of computing resources is large, and the quality inspection time is long, the present invention provides an embodiment of a video data quality inspection method for service recording, which is shown in fig. 1, and specifically includes the following contents:
step S100: and acquiring the recorded video data of the ongoing service.
Step S200: and determining the segment of the recorded video data according to the service segment information of the service.
Step S300: and performing quality inspection on the recorded data according to the preset corresponding relation between the service segment and the quality inspection point and the segment where the service segment is located.
Step S400: and inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image.
Step S500: and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
It is understood that the service segmentation information can be determined and divided according to the service flow and service characteristics.
As can be seen from the above description, in the method for quality inspection of video data recorded in service according to the embodiment of the present invention, first, a segment where the video data is recorded is determined according to service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the video images and the current segmentation into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than the set threshold value, wherein the method comprises the steps of performing frame extraction on video data in the current segment, performing lightweight quality inspection by using an object detection model, and performing frame extraction on the video in the whole telephone segment. On the other hand, the lightweight object detection algorithm has low requirement on computing power, and avoids influence on data recording operation caused by overlarge occupation of data recording terminal resources.
In the present invention, the service segmentation information may be generated in advance, or may be generated online, and the present invention is not limited thereto, and in an embodiment, the steps of the present invention include a generation process of the service segmentation information, that is:
the quality inspection method for the video data recorded by the service further comprises the following steps:
and determining the service segmentation information of each service according to the service rule and the service characteristics.
It is to be understood that the service of the present invention may include a banking service in which the service segmentation information includes: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
In order to provide a specific method for determining service segments, in an embodiment of the method for quality inspection of video data recorded by a service provided by the present invention, a preferred manner for recording data segments is provided, where video data corresponds to audio data during recording, and the determining a segment of the recorded video data according to service segment information of the service includes:
and determining the section of the recorded video data according to the audio data corresponding to the recorded video data and the service section information.
Further, in an embodiment of the method for quality inspection of video data recorded in a service, the service segment information includes identity information and tone information of an audio producer of each service segment, and determining a segment of the recorded video data according to the audio data corresponding to the recorded video data and the service segment information includes:
comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information, and determining the identity information of the current audio producer;
and determining the current service segment according to the identity information, the current service running time and the duration of the tone.
In this embodiment, since the audio producer in each service segment is a client, a client manager, a system prompt, or the like, the present invention determines the identity according to the difference of the tone of the audio producer, and further determines the segment currently located based on the information such as the service progress duration and the tone duration.
To further illustrate a specific scheme of how to determine the current service segment in the above steps, in an embodiment of the video data quality inspection method for service recording provided by the present invention, a preferred mode of data quality inspection is provided, where determining the current service segment according to the identity information, the current service running time and the duration of the tone includes:
determining the service segmentation range according to the identity information;
according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range;
and matching the service segments in the range of the updated service segment according to the current service carrying time and the service duration of all the service segments, and determining the service segment with the highest matching degree as the current service segment.
In the embodiment, the preliminary range is determined through the identity information, then matching is carried out according to the tone duration and the duration of each service segment, if the matching degree is lower than a set threshold value, the time difference between the tone duration and the duration is overlarge, then the duration of the current service is matched with the duration in the service segment information, the service segment with the highest matching degree is selected, the obtained service segment is accurate, and manual service segmentation is not needed.
Further, in some embodiments, if the confidence degrees corresponding to all the video images of which the frame has been currently extracted are lower than the set threshold, or the current object detection result is a non-target object, the method further includes:
and executing iterative operation, performing frame extraction on the recorded video data again to obtain a plurality of video images, and inputting the plurality of video images and the current segment into a preset object detection model until the confidence coefficient output by the object detection model is higher than a set threshold value.
In the embodiment, the frame extraction accuracy can be greatly improved because the judgment can be carried out according to the confidence coefficient, and the frame extraction can be carried out for multiple times under the condition that the frame extraction is unqualified for the first time, so that the qualified frame extraction image is selected.
In some embodiments, the steps of the present invention further comprise:
establishing the object detection model;
training the object detection model using a plurality of historical video images with labeled confidence and objects.
In this embodiment, the object detection model is a machine learning model, and based on a lightweight object detection algorithm (e.g., YOLO-Tiny, nanoDet, etc.), a detection object (e.g., a certificate) related to a video quality inspection point is trained to obtain a detection model for checking whether a related object exists. For video data under a certain speech segmentation, frames are firstly extracted to be pictures (the frame extraction frequency can be controlled by parameters), then the pictures are sent to a trained lightweight object detection model, whether detection objects related to quality inspection points exist or not is detected, and the confidence coefficient exceeds a certain threshold value. When the number of the pictures meeting the requirement is larger than a certain number (the specific number can be set), the detection is not continued, and the detailed quality inspection is performed on the related pictures, for example: aiming at the quality inspection point whether the certificate is clear or not, the lightweight object detection model only detects whether the certificate exists or not, selects a plurality of pictures with confidence degrees higher than a threshold value and sends the pictures to the video quality inspection unit, and the definition detection is further implemented by the video quality inspection unit to judge whether the requirements of the quality inspection point are met or not. Compared with the general method of extracting frames from videos in the whole phone operation section, the method of the invention further accurately positions the time point of the appearance of the quality inspection object through the lightweight object detection algorithm, and extracts only a plurality of pictures for subsequent quality inspection, thereby greatly reducing the number of the pictures to be transmitted and detected, reducing the transmission pressure of the pictures, reducing the consumption of quality inspection computing resources and the time consumption of the quality inspection, and further improving the timeliness of the quality inspection. On the other hand, the lightweight object detection algorithm has low calculation force requirement, and the influence on the double-recording operation caused by overlarge double-recording terminal resource occupation is avoided.
Aiming at the picture obtained by frame extraction, performing quality inspection on video quality inspection points (such as whether certificates are displayed or not and whether certificate contents are clear) under the segment by adopting a target detection algorithm (such as YOLO, SSD and the like) and a behavior recognition algorithm (such as CNN + LSTM, two-Stream and the like) based on deep learning.
As can be seen from the above analysis, in the method for quality inspection of video data recorded by a service according to the embodiment of the present invention, first, a segment where the video data is recorded is determined according to service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the video images and the current segmentation into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than a set threshold value, wherein frame extraction is performed on video data in the current segment, then lightweight quality inspection is performed by using an object detection model, and compared with a general method for performing frame extraction on video in the whole technical segment, the method further accurately positions the time point of the occurrence of a quality inspection object by using a lightweight object detection algorithm, and only extracts a plurality of pictures for subsequent quality inspection, so that the number of pictures needing to be transmitted and detected is greatly reduced, the picture transmission pressure is reduced, the resource consumption and the quality inspection time consumption of quality inspection are reduced, and the quality inspection timeliness is further improved. On the other hand, the lightweight object detection algorithm has low calculation force requirement, and avoids influencing data recording operation due to overlarge occupied data recording terminal resources.
In terms of software, in order to solve the problem that in the prior art, in order to ensure the quality inspection accuracy, the frame extraction frequency cannot be too low, and therefore, for a certain quality inspection point, it may be necessary to perform frame extraction on videos of several minutes or even tens of minutes, at this time, several hundreds of frame-extracted pictures may be obtained, the network pressure is large when the pictures are transmitted to the quality inspection service end, the consumption of computing resources is large, and the quality inspection time is long, the present invention provides an embodiment of a video data quality inspection apparatus for performing service recording of all or part of the contents in the video data quality inspection method for performing service recording, and referring to fig. 2, the video data quality inspection apparatus for service recording specifically includes the following contents:
the acquisition module 10 acquires recorded video data of a service in progress;
the segmentation determining module 20 determines the segments of the recorded video data according to the service segmentation information of the service;
a frame extracting module 30, which extracts frames from the recorded video data for the current segment to obtain a plurality of video images;
an input module 40, configured to input the plurality of video images and the current segment into a preset object detection model, where the object detection model outputs an object detection result and a corresponding confidence of each video image;
and the quality inspection module 50 is used for performing quality inspection operation on the video images with the reliability higher than the set threshold value.
The embodiment of the video data quality inspection apparatus for service recording provided by the present invention can be specifically used for executing the processing flow of the embodiment of the video data quality inspection apparatus for service recording in the above embodiment, and the functions thereof are not described herein again, and reference may be made to the detailed description of the embodiment of the apparatus.
As can be seen from the above description, in the video data quality inspection apparatus for service recording provided in the embodiment of the present invention, first, the segment where the video data is recorded is determined according to the service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than a set threshold value, wherein frame extraction is performed on video data in the current segment, then lightweight quality inspection is performed by using an object detection model, and compared with a general method for performing frame extraction on video in the whole technical segment, the method further accurately positions the time point of the occurrence of a quality inspection object by using a lightweight object detection algorithm, and only extracts a plurality of pictures for subsequent quality inspection, so that the number of pictures needing to be transmitted and detected is greatly reduced, the picture transmission pressure is reduced, the resource consumption and the quality inspection time consumption of quality inspection are reduced, and the quality inspection timeliness is further improved. On the other hand, the lightweight object detection algorithm has low requirement on computing power, and avoids influence on data recording operation caused by overlarge occupation of data recording terminal resources.
In a preferred embodiment, further comprising:
and the segmentation information determining module is used for determining the service segmentation information of each service according to the service rule and the service characteristics.
In a preferred embodiment, the segment determining module is specifically configured to determine the segment of the recorded video data according to the audio data corresponding to the recorded video data and the service segment information.
In a preferred embodiment, the traffic segmentation information includes: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
In a preferred embodiment, the service segment information includes identity information and tone information of each service segment audio producer, and the segment determining module includes:
the identity information determining unit is used for comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information and determining the identity information of the current audio producer;
and the service segment determining unit is used for determining the current service segment according to the identity information, the current service running time and the duration of the tone.
In a preferred embodiment, the determining, according to the identity information, the current service running duration, and the duration of the tone, the current service segment includes:
determining the service segmentation range according to the identity information;
according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range;
and matching the service segments in the range of the updated service segment according to the current service carrying time and the service duration of all the service segments, and determining the service segment with the highest matching degree as the current service segment.
In a preferred embodiment, further comprising: and the iteration operation module executes iteration operation if the corresponding confidence degrees of all the currently framed video images are lower than a set threshold value or the current object detection result is no target object, performs frame extraction on the recorded video data again to obtain a plurality of video images, and inputs the plurality of video images and the current segments into a preset object detection model until the confidence degree output by the object detection model is higher than the set threshold value.
In a preferred embodiment, further comprising:
a model building module for building the object detection model;
and the model training module is used for training the object detection model by utilizing the labeled confidence coefficient and a plurality of historical video images of the object.
In terms of hardware, in order to solve the problem that in the prior art, in order to ensure the quality inspection accuracy, the frame extraction frequency cannot be too low, and therefore, for a certain quality inspection point, it may be necessary to perform frame extraction on videos of several minutes or even tens of minutes, at this time, several hundreds of frame-extracted pictures may be obtained, the network pressure is large when the pictures are transmitted to the quality inspection service end, the consumption of computing resources is large, and the quality inspection takes a long time, the present invention provides an embodiment of an electronic device for implementing all or part of the contents in the video data quality inspection method for service recording, where the electronic device specifically includes the following contents:
fig. 3 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present invention. As shown in fig. 3, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this FIG. 3 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the segment quality check function may be integrated into the central processor. Wherein the central processor may be configured to control:
step S100: and acquiring recorded video data of the ongoing service.
Step S200: and determining the segment of the recorded video data according to the service segment information of the service.
Step S300: and performing quality inspection on the recorded data according to the preset corresponding relation between the service segment and the quality inspection point and the segment where the service segment is located.
Step S400: and inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image.
Step S500: and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
As can be seen from the above description, in the electronic device provided in the embodiment of the present invention, first, the segment where the video data is recorded is determined according to the service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than the set threshold value, wherein the method comprises the steps of performing frame extraction on video data in the current segment, performing lightweight quality inspection by using an object detection model, and performing frame extraction on the video in the whole telephone segment. On the other hand, the lightweight object detection algorithm has low requirement on computing power, and avoids influence on data recording operation caused by overlarge occupation of data recording terminal resources.
In another embodiment, the video data quality inspection apparatus for service recording may be configured separately from the central processor 9100, for example, the video data quality inspection apparatus for service recording may be configured as a chip connected to the central processor 9100, and the quality inspection function of the recorded data segments is implemented by the control of the central processor.
As shown in fig. 3, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worthy to note that the electronic device 9600 also does not necessarily include all of the components shown in fig. 3; in addition, the electronic device 9600 may further include components not shown in fig. 3, which may be referred to in the prior art.
As shown in fig. 3, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 may be a solid-state memory, e.g., read Only Memory (ROM), random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
A plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, can be provided in the same electronic device based on different communication technologies. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all steps in the video data quality inspection method for service recording in the foregoing embodiment, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all steps of the video data quality inspection method for service recording whose main execution subject is a server or a client in the foregoing embodiment are implemented, for example, when the processor executes the computer program, the following steps are implemented:
step S100: and acquiring the recorded video data of the ongoing service.
Step S200: and determining the segment of the recorded video data according to the service segment information of the service.
Step S300: and performing quality inspection on the recorded data according to the preset corresponding relation between the service segment and the quality inspection point and the segment where the service segment is located.
Step S400: and inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image.
Step S500: and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
As can be seen from the above description, in the electronic device provided in the embodiment of the present invention, first, the segment where the video data is recorded is determined according to the service segment information of the service; then, aiming at the current segment, performing frame extraction on the recorded video data to obtain a plurality of video images; then inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image; and finally, performing quality inspection operation on the video image with the reliability higher than the set threshold value, wherein the method comprises the steps of performing frame extraction on video data in the current segment, performing lightweight quality inspection by using an object detection model, and performing frame extraction on the video in the whole telephone segment. On the other hand, the lightweight object detection algorithm has low requirement on computing power, and avoids influence on data recording operation caused by overlarge occupation of data recording terminal resources.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (12)

1. A method for quality inspection of video data recorded by a service, comprising:
acquiring recorded video data of a service in progress;
comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information, and determining the identity information of the current audio producer; determining the service segmentation range according to the identity information; according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range; matching the service segments in the range of the updated service segment according to the current service implementation duration and the service duration of all service segments, and determining the service segment with the highest matching degree as the current segment;
performing frame extraction on the recorded video data aiming at the current segment to obtain a plurality of video images;
inputting the plurality of video images and the current segment into a preset object detection model, wherein the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image;
and performing quality inspection operation on the video images with the reliability higher than a set threshold value.
2. The method of claim 1, wherein the method further comprises:
and determining the service segmentation information of each service according to the service rule and the service characteristics.
3. The method of claim 1, wherein the service segmentation information comprises: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
4. The method according to claim 1, wherein if the confidence levels corresponding to all video images of the current frame extraction are lower than a set threshold value, or the current object detection result is a non-target object, the method further comprises:
and executing iteration operation, performing frame extraction on the recorded video data again to obtain a plurality of video images, and inputting the plurality of video images and the current segment into a preset object detection model until the confidence coefficient output by the object detection model is higher than a set threshold value.
5. The method of claim 1, further comprising:
establishing the object detection model;
the object detection model is trained using a plurality of historical video images with annotated confidence levels and objects.
6. A device for quality control of video data recorded in a service, comprising:
the acquisition module acquires recorded video data of a service in progress;
the segmentation determining module is used for determining the segments of the recorded video data according to the service segmentation information of the service; wherein the segmentation determination module comprises: the identity information determining unit is used for comparing the tone information of the audio data corresponding to the currently recorded video data with the tone information in the service segmentation information and determining the identity information of the current audio producer; a service segment determining unit, which determines the current service segment according to the identity information, the current service running time and the duration of the tone; the service segmentation determining unit is specifically configured to determine a service segmentation range according to the identity information; according to the duration of the tone and the service duration of each service segment in the service segment range, eliminating the service segments with the matching degree lower than a set threshold value to obtain an updated service segment range; matching the service segments in the range of the updated service segment according to the current service carrying time and the service duration of all the service segments, and determining the service segment with the highest matching degree as the current segment;
the frame extracting module is used for extracting frames of the recorded video data aiming at the current segment to obtain a plurality of video images;
the input module is used for inputting the plurality of video images and the current segment into a preset object detection model, and the object detection model outputs an object detection result and a corresponding confidence coefficient of each video image;
and the quality inspection module is used for performing quality inspection operation on the video images with the reliability higher than a set threshold value.
7. The apparatus for quality inspection of service recorded video data according to claim 6, further comprising:
and the segmentation information determining module is used for determining the service segmentation information of each service according to the service rule and the service characteristics.
8. The apparatus of claim 6, wherein the service segmentation information comprises: the system comprises a client identity confirming section, a self-introduction section, a product type description section, a warranty and income description section, an investment range description section, a risk description section, a product deadline description section, a commission charge description section and a risk prompt section.
9. The apparatus for quality inspection of service recorded video data according to claim 6, further comprising: and the iteration operation module executes iteration operation if the confidence degrees corresponding to all the video images of which the frames are extracted currently are lower than a set threshold value or the current object detection result is no target object, extracts the frames of the recorded video data again to obtain a plurality of video images, and inputs the plurality of video images and the current segments into a preset object detection model until the confidence degree output by the object detection model is higher than the set threshold value.
10. The apparatus for quality inspection of video data recorded by a service as claimed in claim 6, further comprising:
a model building module for building the object detection model;
and the model training module is used for training the object detection model by utilizing a plurality of historical video images with labeled confidence degrees and objects.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the method for quality inspection of video data recorded by a service according to any one of claims 1 to 5.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for quality inspection of video data recorded by a service according to any one of claims 1 to 5.
CN202110486354.1A 2021-04-30 2021-04-30 Method and device for quality inspection of video data recorded by service Active CN113206998B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110486354.1A CN113206998B (en) 2021-04-30 2021-04-30 Method and device for quality inspection of video data recorded by service

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110486354.1A CN113206998B (en) 2021-04-30 2021-04-30 Method and device for quality inspection of video data recorded by service

Publications (2)

Publication Number Publication Date
CN113206998A CN113206998A (en) 2021-08-03
CN113206998B true CN113206998B (en) 2022-12-09

Family

ID=77028597

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110486354.1A Active CN113206998B (en) 2021-04-30 2021-04-30 Method and device for quality inspection of video data recorded by service

Country Status (1)

Country Link
CN (1) CN113206998B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114926464B (en) * 2022-07-20 2022-10-25 平安银行股份有限公司 Image quality inspection method, image quality inspection device and system in double-recording scene

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921206A (en) * 2018-06-15 2018-11-30 北京金山云网络技术有限公司 A kind of image classification method, device, electronic equipment and storage medium
CN109729383A (en) * 2019-01-04 2019-05-07 深圳壹账通智能科技有限公司 Double record video quality detection methods, device, computer equipment and storage medium
CN110245544A (en) * 2018-09-26 2019-09-17 浙江大华技术股份有限公司 A kind of method and device of determining dead ship condition
CN111383246A (en) * 2018-12-29 2020-07-07 杭州海康威视数字技术股份有限公司 Scroll detection method, device and equipment
CN111885375A (en) * 2020-07-15 2020-11-03 中国工商银行股份有限公司 Method, device, server and system for testing double-recorded video
CN112351337A (en) * 2021-01-04 2021-02-09 腾讯科技(深圳)有限公司 Video quality inspection method and device, computer equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201417093A (en) * 2012-10-19 2014-05-01 Hon Hai Prec Ind Co Ltd Electronic device with video/audio files processing function and video/audio files processing method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921206A (en) * 2018-06-15 2018-11-30 北京金山云网络技术有限公司 A kind of image classification method, device, electronic equipment and storage medium
CN110245544A (en) * 2018-09-26 2019-09-17 浙江大华技术股份有限公司 A kind of method and device of determining dead ship condition
CN111383246A (en) * 2018-12-29 2020-07-07 杭州海康威视数字技术股份有限公司 Scroll detection method, device and equipment
CN109729383A (en) * 2019-01-04 2019-05-07 深圳壹账通智能科技有限公司 Double record video quality detection methods, device, computer equipment and storage medium
CN111885375A (en) * 2020-07-15 2020-11-03 中国工商银行股份有限公司 Method, device, server and system for testing double-recorded video
CN112351337A (en) * 2021-01-04 2021-02-09 腾讯科技(深圳)有限公司 Video quality inspection method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN113206998A (en) 2021-08-03

Similar Documents

Publication Publication Date Title
KR101731404B1 (en) Voice and/or facial recognition based service provision
CN109993150B (en) Method and device for identifying age
CN111652093B (en) Text image processing method and device
CN111312283B (en) Cross-channel voiceprint processing method and device
CN111683285B (en) File content identification method and device, computer equipment and storage medium
CN110046571B (en) Method and device for identifying age
CN112233343A (en) Self-service terminal equipment service data processing method and device
CN111815169A (en) Business approval parameter configuration method and device
CN113095202A (en) Data segmentation method and device in double-record data quality inspection
JP2023526899A (en) Methods, devices, media and program products for generating image inpainting models
CN113206998B (en) Method and device for quality inspection of video data recorded by service
CN110008926B (en) Method and device for identifying age
CN111507698A (en) Processing method and device for transferring accounts, computing equipment and medium
CN113065879A (en) Data stream quality inspection method and system
CN113051924A (en) Method and system for segmented quality inspection of recorded data
CN114420135A (en) Attention mechanism-based voiceprint recognition method and device
CN110335237B (en) Method and device for generating model and method and device for recognizing image
CN115984853A (en) Character recognition method and device
CN112820298B (en) Voiceprint recognition method and device
CN112489662B (en) Method and apparatus for training speech processing model
CN111899411B (en) Bill data identification method and device
CN113206996B (en) Quality inspection method and device for service recorded data
CN115205009A (en) Account opening business processing method and device based on virtual technology
CN113206997B (en) Method and device for simultaneously detecting quality of multi-service recorded audio data
CN110399615B (en) Transaction risk monitoring method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant