CN115311606B - Classroom recorded video validity detection method - Google Patents
Classroom recorded video validity detection method Download PDFInfo
- Publication number
- CN115311606B CN115311606B CN202211219364.XA CN202211219364A CN115311606B CN 115311606 B CN115311606 B CN 115311606B CN 202211219364 A CN202211219364 A CN 202211219364A CN 115311606 B CN115311606 B CN 115311606B
- Authority
- CN
- China
- Prior art keywords
- video
- detection
- human body
- classroom
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Electrically Operated Instructional Devices (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a classroom recorded video validity detection method, which comprises the following steps: segmenting videos needing to be analyzed, carrying out audio detection on each segment of video, and executing subsequent steps after requirements are met; carrying out video detection, text detection, human body posture detection and human face detection on the segmented video meeting the audio detection requirement in sequence; obtaining the time length occupied by the human voice in the video, the frequency of the text appearing in the video, the maximum human body number detected and the frequency of the detected human face; and classifying the teaching video of the detection result of each video segment through the trained decision tree, classifying the whole video, and finally judging the effectiveness of the whole video. The invention combines the two key points of audio content and video content and a method for evaluating and scoring the effectiveness of classroom content by using a decision tree analysis method, thereby helping to screen out effective classrooms and reducing the waste of resources caused by the occupation of resources in ineffective classrooms.
Description
Technical Field
The invention relates to the technical field of video analysis, in particular to a detection method for effectiveness of classroom recorded video.
Background
Each classroom in the school can automatically record teaching courses according to the school timetables, and the school can keep effective teaching videos in the classroom; however, due to various reasons, part of classrooms may not have normal lessons, such as the teacher asking for leave, changing the time and place of lessons, the students taking their own study, and the people going to the internet at home due to epidemic situations, such recorded videos will become meaningless lessons after being recorded and stored, and most of the resources will be wasted to retain and process these invalid lessons; therefore, how to detect and judge the effectiveness of the classroom recorded video is a problem to be considered at present.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a method for detecting the effectiveness of classroom recorded videos, and solves the problem that the effectiveness of the traditional classroom recorded videos cannot be detected.
The purpose of the invention is realized by the following technical scheme: a detection method for effectiveness of classroom recorded video comprises the following steps:
s1, segmenting videos needing to be analyzed at fixed time intervals, firstly carrying out audio detection on each segment of video, and executing subsequent steps after requirements are met, otherwise, finishing the detection on the segment of video;
s2, sequentially carrying out video detection, text detection, human body posture detection and human face detection on the segmented video meeting the audio detection requirement;
s3, obtaining the duration occupied by the voice in the video, the number of times of the text appearing in the video, the maximum human body number and the number of times of the detected human face after the relevant detection is finished;
and S4, performing teaching video classification on the detection result of each video segment through the trained decision tree, classifying the whole video based on the segmented video classification result, and finally judging the effectiveness of the whole video.
The audio detection comprises: and detecting the audio of the segmented video through a voice detection algorithm, detecting the position and the duration of the voice in each segment of audio, if no voice appears in the current video segment, judging the video to be invalid classroom content directly without detecting the subsequent steps.
The video detection comprises: acquiring data of a certain frame of the video, comparing the data with the next frame of data, calculating the position and the size of an inconsistent area in a front picture and a rear picture, continuously detecting the picture change of the next frame of video if the current frame has no change, and indicating that activities exist in a classroom if the current frame has a change.
The text detection comprises: and intercepting a picture with the middle area of the current frame data being N, detecting a text through a pre-trained text detection model, and recording the time and the position of the text if the text is detected.
The human body posture detection comprises: and detecting the human body of the current frame by adopting a human body posture detection method, judging whether a person exists in the classroom picture of the current frame, and recording the positions of all effective human body postures if the human body posture is detected.
The face detection comprises the following steps: and judging whether a face facing a student seat appears in the current frame picture, if so, indicating that the face is a teacher, recording the position of the front face appearing in the current frame picture, and if not, indicating that the human body posture detected in the human body posture detection is a student, and indicating that the current frame picture is a student for self-study.
The detection method further comprises the following steps: calculating by using a large amount of effective and ineffective classroom video data to obtain four data of the time length occupied by the human voice in the video, the times of the text appearing in the video, the detected maximum human body number and the detected times of the human face appearing and whether the data is a data set of an effective teaching video, and training a classification decision tree by using a CART classification tree algorithm.
The invention has the following advantages: a method for detecting effectiveness of classroom recorded videos integrates detection of human voice, dynamic change in videos and detection results of texts, human bodies and human faces to judge whether the recorded classroom videos are effective classroom videos or not. The two key points of the audio content and the video content are combined, and a method for evaluating and scoring the effectiveness of the classroom content by using a decision tree analysis method is used, so that an effective classroom is screened out, and the waste of resources caused by the fact that resources are occupied by an ineffective classroom is reduced.
Drawings
FIG. 1 is a schematic flow chart of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings, but the scope of protection of the invention is not limited to the following.
As shown in fig. 1, the present invention specifically relates to a classroom recorded video validity detection method, which adopts multiple steps to detect and score audio and video in segmented video respectively by using multiple feature detection, so as to identify video segments which may be normal lectures of teachers. Then judging whether the whole video is an effective video for the teacher to give lessons according to the identification results of all the segments, wherein the effective video comprises the following contents:
firstly, segmenting the video to be analyzed at fixed time intervals, such as ten minutes. Then the audio and video in each segment are extracted respectively, and the segments are detected and scored independently. The segmented detection is performed to avoid the possible influence on the overall detection caused by a long quiet time period due to some reason (such as an in-class examination, etc.) in a classroom.
Step two: in the audio detection, the input audio is detected by using a human voice detection algorithm, and the position and the duration of the human voice in each audio segment are detected. And if no voice appears in the current video section, directly determining that the current section is invalid classroom content, and not performing detection of other steps, otherwise, entering video detection of the third step.
However, it is not enough to detect whether the classroom recorded video is effective or not by only depending on whether the voices appear or not, and various voices also appear in the classroom under some other conditions, for example, the voices of students can be generated in the study classroom, and voices such as conversation can be generated when the cleaning staff cleans the classroom. Therefore, the detection of the video content is also needed to comprehensively judge whether the video is the video for the teacher to give lessons. Therefore, subsequent testing is required for further determination.
Step three: in video detection, a video motion detection algorithm is used to detect a changed part in a video picture. Specifically, data of a first frame of the video is obtained, and then the next frame is taken as pixel comparison, and the position and the size of an inconsistent area in a front picture and a back picture are calculated. If the current frame has no change, continuously detecting the picture change of the next frame of video; if the current frame picture is changed, the fact that the activities exist in the classroom is shown, but whether the activities are teaching activities or other light and shadow changes or other activities need further analysis, and the next detection step is carried out.
Step four: in this step, text detection is performed in order to detect whether a teacher's blackboard writing or PPT text is present on the blackboard. If text information appears in the video, the possibility that the current video segment is an effective video for the teacher to give lessons increases. Because words which are meaningless to course analysis, such as recording time, may appear at the edge of the video, text detection is performed only by taking eighty percent of the middle of the current frame. And performing text detection on the current frame by adopting a pre-trained text detection model, and recording the time and the position of the text if the text is detected. And then, carrying out human body posture detection in the next step.
Step five: the human body posture method is adopted to carry out human body detection on the current frame, and the detection in the step is to judge whether people exist in a classroom or not, and the people are likely to be teachers and students in class. And if the human body posture is detected, recording the positions of all effective human body postures, and carrying out detection in the next step.
Step six: the detection of the step is to judge whether a face facing to a student seat appears, if the face facing to the student appears, the possibility that the face is a teacher is high, and the position of the face on the front side in the current frame is recorded; if there is no face facing the student, the person detected in the previous step is probably only the student who is self-learning in the classroom, and the video may not be the video of the normal teaching of the teacher.
Step seven: according to the detection of the steps, the following data can be obtained: (1) the time length of the human voice in the video; (2) the number of times the text appears in the video; (3) the maximum number of detected human bodies; and (4) detecting the number of times of the human face. Calculations were performed using a large amount of valid and invalid classroom video data to derive a data set of these four quantities and whether it was a valid lecture video. The classification decision tree is then trained using the CART classification tree algorithm. And classifying the teaching videos of the detection results of the video bands by using the trained decision tree.
Step eight: and classifying the whole video based on the classification result of the segmented video. The entire video can be determined to be an effective teaching video by simply using an empirical value for classification, for example, the proportion of the number of the segments determined to be effective teaching to the total number of the segments exceeds a certain empirical value (e.g., 60% -80%).
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (2)
1. A classroom recorded video validity detection method is characterized in that: the detection method comprises the following steps:
s1, segmenting videos needing to be analyzed at fixed time intervals, firstly carrying out audio detection on each segment of video, and executing subsequent steps after requirements are met, otherwise, finishing the detection on the segment of video;
s2, sequentially carrying out video detection, text detection, human body posture detection and human face detection on the segmented video meeting the audio detection requirement;
s3, obtaining the duration occupied by the voice in the video, the number of times of the text appearing in the video, the maximum human body number and the number of times of the detected human face after the relevant detection is finished;
s4, performing teaching video classification on the detection result of each video segment through the trained decision tree, classifying the whole video based on the classification result of the segmented video, and finally judging the effectiveness of the whole video;
the audio detection comprises: detecting the audio frequency of the segmented video through a voice detection algorithm, detecting the position and duration of the voice in each audio frequency, if no voice appears in the current video segment, judging the video segment as invalid classroom content directly without detecting the subsequent steps;
the video detection comprises: acquiring data of a certain frame of the video, comparing the data with the next frame of data to calculate the position and the size of an inconsistent area in a front picture and a rear picture, continuously detecting the picture change of the next frame of video if the current frame has no change, and indicating that activities exist in a classroom if the current frame has a change;
the text detection comprises: intercepting a picture with the middle area of the current frame data being N, carrying out text detection through a pre-trained text detection model, and recording the time and the position of the text if the text is detected;
the human body posture detection comprises: detecting the human body of the current frame by adopting a human body posture detection method, judging whether a human body exists in a classroom picture of the current frame, and recording the positions of all effective human body postures if the human body postures are detected;
the face detection comprises the following steps: and judging whether a face facing a student seat appears in the current frame, if so, indicating that the face is a teacher, recording the position of the face on the front side appearing in the current frame, and if not, indicating that the human body posture detected in the human body posture detection is a student, and indicating that the current frame is a study of the student.
2. The method for detecting effectiveness of video recorded in classroom according to claim 1, wherein: the detection method further comprises the following steps: calculating by using a large amount of effective and ineffective classroom video data to obtain four data of the time length occupied by the human voice in the video, the times of the text appearing in the video, the detected maximum human body number and the detected times of the human face appearing and whether the data is a data set of an effective teaching video, and training a classification decision tree by using a CART classification tree algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211219364.XA CN115311606B (en) | 2022-10-08 | 2022-10-08 | Classroom recorded video validity detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211219364.XA CN115311606B (en) | 2022-10-08 | 2022-10-08 | Classroom recorded video validity detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115311606A CN115311606A (en) | 2022-11-08 |
| CN115311606B true CN115311606B (en) | 2022-12-27 |
Family
ID=83866258
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211219364.XA Active CN115311606B (en) | 2022-10-08 | 2022-10-08 | Classroom recorded video validity detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115311606B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547248A (en) * | 2012-02-03 | 2012-07-04 | 深圳锐取信息技术股份有限公司 | Multi-channel real-time monitoring single-video-file recording method |
| CN102903265A (en) * | 2012-09-19 | 2013-01-30 | 河南智游网络技术有限公司 | Method for automatically recording classroom teaching |
| CN103428461A (en) * | 2013-08-16 | 2013-12-04 | 北京中广上洋科技股份有限公司 | System and method for recording teaching video |
| CN104539983A (en) * | 2014-12-26 | 2015-04-22 | 湖南亿谷信息科技发展有限公司 | Online class management system and method |
| CN105893500A (en) * | 2016-03-30 | 2016-08-24 | 苏州点通教育科技有限公司 | Intelligent real-time class recording system and method |
| CN109117731A (en) * | 2018-07-13 | 2019-01-01 | 华中师范大学 | A kind of classroom instruction cognitive load measuring system |
| CN109637211A (en) * | 2019-01-22 | 2019-04-16 | 合肥市云联鸿达信息技术有限公司 | A kind of full-automatic recording and broadcasting system |
| CN111429554A (en) * | 2020-03-26 | 2020-07-17 | 深圳壹账通智能科技有限公司 | Motion video data processing method and device, computer equipment and storage medium |
| CN111563452A (en) * | 2020-05-06 | 2020-08-21 | 南京师范大学镇江创新发展研究院 | Multi-human body posture detection and state discrimination method based on example segmentation |
| CN112055257A (en) * | 2019-06-05 | 2020-12-08 | 北京新唐思创教育科技有限公司 | Video classroom interaction method, device, equipment and storage medium |
| CN112703478A (en) * | 2018-09-11 | 2021-04-23 | 华为技术有限公司 | Data sharing method, graphical user interface, electronic device and system |
| CN113207033A (en) * | 2021-04-29 | 2021-08-03 | 读书郎教育科技有限公司 | System and method for processing invalid video clips recorded in intelligent classroom |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120002848A1 (en) * | 2009-04-16 | 2012-01-05 | Hill Daniel A | Method of assessing people's self-presentation and actions to evaluate personality type, behavioral tendencies, credibility, motivations and other insights through facial muscle activity and expressions |
| US8873813B2 (en) * | 2012-09-17 | 2014-10-28 | Z Advanced Computing, Inc. | Application of Z-webs and Z-factors to analytics, search engine, learning, recognition, natural language, and other utilities |
| CN110796005A (en) * | 2019-09-27 | 2020-02-14 | 北京大米科技有限公司 | Method, device, electronic equipment and medium for online teaching monitoring |
| CN111754368A (en) * | 2020-01-17 | 2020-10-09 | 天津师范大学 | College teaching evaluation method and college teaching evaluation system based on edge intelligence |
-
2022
- 2022-10-08 CN CN202211219364.XA patent/CN115311606B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547248A (en) * | 2012-02-03 | 2012-07-04 | 深圳锐取信息技术股份有限公司 | Multi-channel real-time monitoring single-video-file recording method |
| CN102903265A (en) * | 2012-09-19 | 2013-01-30 | 河南智游网络技术有限公司 | Method for automatically recording classroom teaching |
| CN103428461A (en) * | 2013-08-16 | 2013-12-04 | 北京中广上洋科技股份有限公司 | System and method for recording teaching video |
| CN104539983A (en) * | 2014-12-26 | 2015-04-22 | 湖南亿谷信息科技发展有限公司 | Online class management system and method |
| CN105893500A (en) * | 2016-03-30 | 2016-08-24 | 苏州点通教育科技有限公司 | Intelligent real-time class recording system and method |
| CN109117731A (en) * | 2018-07-13 | 2019-01-01 | 华中师范大学 | A kind of classroom instruction cognitive load measuring system |
| CN112703478A (en) * | 2018-09-11 | 2021-04-23 | 华为技术有限公司 | Data sharing method, graphical user interface, electronic device and system |
| CN109637211A (en) * | 2019-01-22 | 2019-04-16 | 合肥市云联鸿达信息技术有限公司 | A kind of full-automatic recording and broadcasting system |
| CN112055257A (en) * | 2019-06-05 | 2020-12-08 | 北京新唐思创教育科技有限公司 | Video classroom interaction method, device, equipment and storage medium |
| CN111429554A (en) * | 2020-03-26 | 2020-07-17 | 深圳壹账通智能科技有限公司 | Motion video data processing method and device, computer equipment and storage medium |
| CN111563452A (en) * | 2020-05-06 | 2020-08-21 | 南京师范大学镇江创新发展研究院 | Multi-human body posture detection and state discrimination method based on example segmentation |
| CN113207033A (en) * | 2021-04-29 | 2021-08-03 | 读书郎教育科技有限公司 | System and method for processing invalid video clips recorded in intelligent classroom |
Non-Patent Citations (1)
| Title |
|---|
| "Auto-Tracking Camera System for Remote Learning Using Face Detection and Hand Gesture Recognition Based on Convolutional Neural Network";Daniel Imanuel Sutanto等;《2021 1st International Conference on Computer Science and Artificial Intelligence (ICCSAI)》;20211124;第451-457页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115311606A (en) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108648757B (en) | Analysis method based on multi-dimensional classroom information | |
| CN108182649A (en) | For the intelligent robot of Teaching Quality Assessment | |
| Akram et al. | Effectiveness of online teaching during COVID-19 | |
| Asgari et al. | Copus, portaal, or dart? Classroom observation tool comparison from the instructor user’s perspective | |
| Hagans | A response-to-intervention approach to decreasing early literacy differences in first graders from different socioeconomic backgrounds: Evidence for the intervention validity of the DIBELS | |
| CN115311606B (en) | Classroom recorded video validity detection method | |
| Rahayuningsih | THE EFFECT OF READING HABIT AND VOCABULARY MASTERY TOWARDS STUDENTS’READING COMPREHENSION (Survey at State Senior High School in Central Jakarta) | |
| Tan et al. | Exploring the relationship between foreign language anxiety, gender, years of learning English and learners’ oral English achievement amongst Chinese college students | |
| CN111667128A (en) | Teaching quality assessment method, device and system | |
| Schmeichel et al. | Why has there never been a woman president in the United States? An inquiry lesson | |
| Yulianti et al. | Comparison of the effectiveness of certainty factor vs dempster-shafer in the determination of the adolescent learning styles | |
| CN114944089A (en) | Remote education training system and method based on user behavior analysis | |
| Shields et al. | How Kindergarten Entry Assessments Are Used in Public Schools and How They Correlate with Spring Assessments. REL 2017-182. | |
| CN111950472A (en) | Teacher grinding evaluation method and system | |
| Takahashi et al. | Improvement of detection for warning students in e-learning using web cameras | |
| Shenoy et al. | A Study on various Applications of Computer Vision for Teaching Learning in Classroom | |
| Ardini et al. | The Correlation between the Use of Audio-Visual Learning Media and Children's Listening Skill in Suwawa Selatan | |
| CN115081923A (en) | Method and device for evaluating intelligent teaching quality | |
| Jones | Learning Modalities--Should They Be Considered?. | |
| Putri | The Correlation between Students’ Language Awareness and Learning Styles toward Their TOEFL Listening Skill | |
| Wang et al. | Facial expressions and politeness effect in foreign language training system | |
| Syamsinar | Improving the Listening Achievement of the Students of Smu Negeri 1 Sungguminasa Through the Use of Kangguru English Learning Packages | |
| Perez et al. | Image Creation Intervention: Effectiveness on the Reading Comprehension of Learners | |
| Juntong | Application of Artificial Neural Network in Student's Behavior Analysis in English Class | |
| الأحمري et al. | Integrating behavioural and occupational therapy using a computer visual system to reduce sensory stimulation in children with autism spectrum disorder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |