WO2024108512A1

WO2024108512A1 - Class note generation method and apparatus, device, and storage medium

Info

Publication number: WO2024108512A1
Application number: PCT/CN2022/134186
Authority: WO
Inventors: 尹志超
Original assignee: 广州视源电子科技股份有限公司; 广州视睿电子科技有限公司
Priority date: 2022-11-24
Filing date: 2022-11-24
Publication date: 2024-05-30

Abstract

The present application discloses a class note generation method and apparatus, a device, and a storage medium. The method comprises: analyzing a teacher's lesson teaching audio/video to determine a first occurrence time period of a teacher's lesson teaching behavior, and determining a first teaching content in the first occurrence time period according to the first occurrence time period and a teaching content video; analyzing students' lesson attending videos to determine a second occurrence time period of the students' lesson attending behaviors, and determining a second teaching content in the second occurrence time period according to the second occurrence time period and the teaching content video; and according to the first teaching content and the second teaching content of the same student, generating electronic class notes corresponding to the students, determining identity information of the students corresponding to the electronic class notes, and recording the identity information in the corresponding electronic class notes. By means of the technical means, students can quickly improve knowledge defects and deeply learn key knowledge on the basis of personal class notes, thereby effectively improving their after-class review efficiency.

Description

课堂笔记生成方法、装置、设备及存储介质Classroom note generation method, device, equipment and storage medium

技术领域Technical Field

本申请涉及智能课堂技术领域，尤其涉及一种课堂笔记生成方法、装置、设备及存储介质。The present application relates to the field of smart classroom technology, and in particular to a classroom note generation method, device, equipment and storage medium.

背景技术Background technique

课堂笔记是帮助学生对老师授课内容进行课后复习以及深度理解的重要学习手段，优秀的课堂笔记可以更有效地帮助学生巩固课堂知识，提高学习成绩。电子课堂笔记是基于课堂内容自动生成的课堂笔记，电子课堂笔记无需学生手动记录课堂笔记，使得学生将注意力放在老师授课内容上，提高学生的课堂听课效率和注意力。Class notes are an important learning tool to help students review and deeply understand the teacher's teaching content after class. Excellent class notes can more effectively help students consolidate classroom knowledge and improve their academic performance. Electronic class notes are class notes automatically generated based on class content. Electronic class notes do not require students to manually record class notes, allowing students to focus on the teacher's teaching content, improving students' classroom listening efficiency and attention.

在现有技术中，基于课堂的音频信息和板书图像信息生成该节课的课堂笔记，以便学生在课后通过课堂笔记巩固该节课的课堂知识。但现有课堂笔记是基于老师的授课内容生成的广泛笔记，导致学生的课后复习效率较低。In the prior art, class notes are generated based on the audio information and blackboard image information of the class, so that students can consolidate the classroom knowledge of the class through the class notes after class. However, the existing class notes are extensive notes generated based on the teacher's teaching content, resulting in low efficiency of students' after-class review.

发明内容Summary of the invention

本申请提供一种课堂笔记生成方法、装置、设备及存储介质，通过分析学生和老师的行为举止，确定每个学生的课堂知识短板和课堂重点知识，并根据课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记，便于学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习，有效提高学生的课后复习效率，解决了现有技术中学生无法基于电子课堂笔记针对性学习的问题。The present application provides a method, device, equipment and storage medium for generating classroom notes. By analyzing the behaviors of students and teachers, the classroom knowledge shortcomings and key classroom knowledge of each student are determined, and the students' personal classroom notes are generated in a targeted manner based on the classroom knowledge shortcomings and key classroom knowledge. This facilitates students to quickly make up for their knowledge shortcomings and deepen their study of key knowledge based on their personal classroom notes, effectively improves the students' after-class review efficiency, and solves the problem in the prior art that students cannot learn in a targeted manner based on electronic classroom notes.

第一方面，本申请提供了一种课堂笔记生成方法，包括：In a first aspect, the present application provides a method for generating classroom notes, comprising:

对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容；Analyze the audio and video of the teacher's teaching to determine a first time period of the teacher's teaching behavior, and determine the first teaching content within the first time period according to the first time period and the teaching content video;

对所述学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据所述第二发生时间段和所述教学内容视频确定所述第二发生时间段内的第二教学内容；Analyze the student lecture video to determine a second occurrence time period of each student's lecture behavior, and determine a second teaching content within the second occurrence time period according to the second occurrence time period and the teaching content video;

根据所述第一教学内容和同一学生的所述第二教学内容生成对应学生的电子课堂笔记，并确定各个所述电子课堂笔记对应学生的身份信息，将所述身份信息记录对应的电子课堂笔记中。Generate electronic classroom notes for the corresponding student based on the first teaching content and the second teaching content of the same student, determine identity information of the student corresponding to each electronic classroom note, and record the identity information in the corresponding electronic classroom notes.

第二方面，本申请提供了一种课堂笔记生成***，包括采集设备和课堂笔记生成设备，其中：In a second aspect, the present application provides a class note generation system, including a collection device and a class note generation device, wherein:

所述采集设备用于，采集老师授课音视频、学生听课视频和教学内容视频，并将所述老师授课音视频、所述学生听课视频和所述教学内容视频发送至所述处理设备；The acquisition device is used to collect audio and video of the teacher's lecture, video of the students listening to the lecture and video of the teaching content, and send the audio and video of the teacher's lecture, video of the students listening to the lecture and video of the teaching content to the processing device;

所述课堂笔记生成设备用于，接收所述采集设备发送的所述老师授课音视频、所述学生听课视频和所述教学内容视频；对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容；对所述学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据所述第二发生时间段和所述教学内容视频确定所述第二发生时间段内的第二教学内容；根据所述第一教学内容和同一学生的所述第二教学内容生成对应学生的电子课堂笔记，并确定各个所述电子课堂笔记对应学生的身份信息，将所述身份信息记录对应的电子课堂笔记中。The classroom note generating device is used to receive the teacher's teaching audio and video, the student's lecture video and the teaching content video sent by the acquisition device; analyze the teacher's teaching audio and video to determine the first time period of the teacher's teaching behavior, and determine the first teaching content within the first time period according to the first time period and the teaching content video; analyze the student's lecture video to determine the second time period of each student's lecture behavior, and determine the second teaching content within the second time period according to the second time period and the teaching content video; generate electronic classroom notes for the corresponding students according to the first teaching content and the second teaching content of the same student, and determine the identity information of the students corresponding to each of the electronic classroom notes, and record the identity information in the corresponding electronic classroom notes.

第三方面，本申请提供了一种课堂笔记生成设备，包括：In a third aspect, the present application provides a class note generation device, comprising:

一个或多个处理器；存储装置，存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如第一方面所述的课堂笔记生成方法。One or more processors; a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the classroom note generating method as described in the first aspect.

第四方面，本申请提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如第一方面所述的课堂笔记生成方法。In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the classroom note generation method as described in the first aspect.

本申请通过对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据第一发生时间段和教学内容视频确定第一发生时间段内的第一教学内容；对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容；根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应学生的身份信息，将身份信息记录对应的电子课堂笔记中。通过上述技术手段，分析学生和老师在课堂上的行为举止，确定每个学生的课堂知识短板和课堂重点知识，并根据课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记，以便学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习，有效提高学生的课后复习效率，解决了现有技术中学生无法基于电子课堂笔记针对性学习的问题。This application analyzes the audio and video of the teacher's lecture to determine the first time period of the teacher's lecture behavior, and determines the first teaching content within the first time period according to the first time period and the teaching content video; analyzes the student's lecture video to determine the second time period of each student's lecture behavior, and determines the second teaching content within the second time period according to the second time period and the teaching content video; generates the electronic classroom notes of the corresponding students according to the first teaching content and the second teaching content of the same student, and determines the identity information of the students corresponding to each electronic classroom note, and records the identity information in the corresponding electronic classroom notes. Through the above technical means, the behavior of students and teachers in the classroom is analyzed to determine the classroom knowledge shortcomings and classroom key knowledge of each student, and the personal classroom notes of students are generated in a targeted manner according to the classroom knowledge shortcomings and classroom key knowledge, so that students can quickly make up for their own knowledge shortcomings and deepen their study of key knowledge based on their personal classroom notes, effectively improve the efficiency of students' after-school review, and solve the problem that students in the prior art cannot learn in a targeted manner based on electronic classroom notes.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1是本申请实施例提供的课堂笔记生成***的结构示意图；FIG1 is a schematic diagram of the structure of a classroom note generation system provided in an embodiment of the present application;

图2是本申请实施例提供的一种课堂笔记生成方法的流程图；FIG2 is a flow chart of a method for generating classroom notes provided in an embodiment of the present application;

图3是本申请实施例提供的采集设备的安装示意图；FIG3 is a schematic diagram of the installation of a collection device provided in an embodiment of the present application;

图4是本申请实施例提供的对老师授课音视频进行分析的流程图；FIG4 is a flow chart of analyzing the audio and video of a teacher's lecture provided by an embodiment of the present application;

图5是本申请实施例提供的确定第一教学内容的流程图；FIG5 is a flow chart of determining the first teaching content provided by an embodiment of the present application;

图6是本申请实施例提供的第一视频帧的示意图；FIG6 is a schematic diagram of a first video frame provided in an embodiment of the present application;

图7是本申请实施例提供的对学生听课视频进行分析的流程图；7 is a flow chart of analyzing a student lecture video provided by an embodiment of the present application;

图8是本申请实施例提供的生成电子课堂笔记的流程图；FIG8 is a flow chart of generating electronic classroom notes provided in an embodiment of the present application;

图9是本申请实施例提供的一种课堂笔记生成装置的结构示意图；FIG9 is a schematic diagram of the structure of a classroom note generation device provided in an embodiment of the present application;

图10是本申请实施例提供的一种课堂笔记生成设备的结构示意图。FIG10 is a schematic diagram of the structure of a classroom note generating device provided in an embodiment of the present application.

具体实施方式Detailed ways

为了使本申请的目的、技术方案和优点更加清楚，下面结合附图对本申请具体实施例作进一步的详细描述。可以理解的是，此处所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部内容。在更加详细地讨论示例性实施例之前应当提到的是，一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各项操作(或步骤)描述成顺序的处理，但是其中的许多操作可以被并行地、并发地或者同时实施。此外，各项操作的顺序可以被重新安排。当其操作完成时处理可以被终止，但是还可以具有未包括在附图中的附加步骤。处理可以对应于方法、函数、规程、子例程、子程序等等。In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processing or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and/or" in the specification and claims represents at least one of the connected objects, and the character "/" generally indicates that the objects associated with each other are in an "or" relationship.

在现有技术中，获取录像装置采集的课堂音频信号和课堂图像数据，对课堂音频信号进行语音识别得到文本数据，对课堂图像数据进行图像处理得到图像笔记，将图像数据***文本数据中，得到文本笔记。由于课堂音频信号是老师授课音频，课堂图像数据是老师授课使用的课件，因此现有技术生成的文本笔记记录的是整个课堂的教学内容，学生拿到该文本笔记时无法确定课堂的重点教学内容，而只能按照文本笔记对所有教学内容进行复习，课后复习效率较低。而且该文本笔记是基于老师的授课内容生成的广泛笔记，缺少基于学生的课堂学习情况生成的个性笔记，学生无法基于现有课堂笔记针对性地补齐自己的知识短板，导致课后复习效率较低。In the prior art, the classroom audio signal and classroom image data collected by the video recording device are obtained, the classroom audio signal is subjected to voice recognition to obtain text data, the classroom image data is subjected to image processing to obtain image notes, and the image data is inserted into the text data to obtain text notes. Since the classroom audio signal is the audio of the teacher's lecture, and the classroom image data is the courseware used by the teacher for lectures, the text notes generated by the prior art record the teaching content of the entire class. When students get the text notes, they cannot determine the key teaching content of the class, but can only review all the teaching content according to the text notes, and the after-class review efficiency is low. Moreover, the text notes are broad notes generated based on the teacher's teaching content, and lack personalized notes generated based on the students' classroom learning situation. Students cannot make up for their knowledge shortcomings based on the existing classroom notes, resulting in low after-class review efficiency.

针对上述现有技术存在的技术问题，本实施例提供一种课堂笔记生成方法，旨在通过分析学生和老师双方在课堂上实时的行为举止，确定每个学生的课堂知识短板和课堂重点知识，并根据课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记及对应的学习资料，便于学生有效提高课后复习效率。In response to the technical problems existing in the above-mentioned prior art, the present embodiment provides a method for generating classroom notes, which aims to determine each student's classroom knowledge shortcomings and classroom key knowledge by analyzing the real-time behavior of both students and teachers in the classroom, and generate students' personal classroom notes and corresponding learning materials in a targeted manner based on the classroom knowledge shortcomings and classroom key knowledge, so as to help students effectively improve their after-class review efficiency.

本实施例中提供的课堂笔记生成方法可以由课堂笔记生成设备执行，该课堂笔记生成设备可以通过软件和/或硬件的方式实现，该课堂笔记生成设备可以是两个或多个物理实体构成，也可以是一个物理实体构成。例如课堂笔记生成设备可以是课堂笔记生成***中的处理设备，课堂笔记生成***旨在用于生成每个学生个性化的电子课堂笔记，处理设备可以是服务器等计算能力较强的电子设备。The classroom note generation method provided in this embodiment can be executed by a classroom note generation device, which can be implemented by software and/or hardware, and can be composed of two or more physical entities, or can be composed of one physical entity. For example, the classroom note generation device can be a processing device in a classroom note generation system, which is intended to generate personalized electronic classroom notes for each student, and the processing device can be an electronic device with strong computing power such as a server.

课堂笔记生成设备安装有至少一类操作***，其中，操作***包括但不限定于安卓***、Linux***及Windows***。课堂笔记生成设备可以基于操作***安装至少一个应用程序，应用程序可以为操作***自带的应用程序，也可以为从第三方设备或者服务器中下载的应用程序。在该实施例中，课堂笔记生成设备至少有可以执行课堂笔记生成方法的应用程序。The class note generation device is installed with at least one type of operating system, wherein the operating system includes but is not limited to Android system, Linux system and Windows system. The class note generation device can install at least one application based on the operating system, and the application can be an application that comes with the operating system or an application downloaded from a third-party device or server. In this embodiment, the class note generation device has at least an application that can execute the class note generation method.

为便于理解，本实施例以课堂笔记生成***中的处理设备为执行课堂笔记生成方法的主体为例进行描述。For ease of understanding, this embodiment is described by taking the processing device in the class note generation system as an example of the main body executing the class note generation method.

在一实施例中，图1是本申请实施例提供的课堂笔记生成***的结构示意图。如图1所示，课堂笔记生成***包括采集设备、处理设备、教师设备、教务管理***和教学资源中心。采集设备包括教师摄像机、教师麦克风、学生摄像机和黑板摄像机，采集设备通过教师摄像机、教师麦克风、学生摄像机和黑板摄像机采集学生听课视频、老师授课音视频和教学内容视频，并将采集到的视频数据传输至处理设备。处理设备分别对视频数据中学生和老师的行为举止进行分析，确定每个学生的课堂知识短板和课堂重点知识。处理设备从教务管理***中获取学生的身份信息，并根据各个学生的身份信息、课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记，便于学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习。处理设备还可以根据学生的课堂知识短板从教学资源中心获取教案、课件、知识点讲解内容和习题等教育资源信息以记录在学生的个人课堂笔记中，以便学生加强对课堂知识短板的学习。处理设备还可将实时监控到各种学生的行为发送给教师设备，以便老师可以通过教师设备了解各个学生对课堂知识的学生情况。相比于现有技术，本申请中的电子课堂笔记针对性记录了不同学生的课堂知识短板和课堂重点知识，便于学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习，有效提高学生的课后复习效率，解决了现有技术中学生无法基于电子课堂笔记针对性学习的问题。In one embodiment, FIG. 1 is a structural diagram of a classroom note generation system provided by an embodiment of the present application. As shown in FIG. 1 , the classroom note generation system includes a collection device, a processing device, a teacher device, an educational management system, and a teaching resource center. The collection device includes a teacher camera, a teacher microphone, a student camera, and a blackboard camera. The collection device collects student lecture videos, teacher lecture audio and video, and teaching content videos through the teacher camera, the teacher microphone, the student camera, and the blackboard camera, and transmits the collected video data to the processing device. The processing device analyzes the behaviors of students and teachers in the video data respectively to determine the classroom knowledge shortcomings and classroom key knowledge of each student. The processing device obtains the identity information of the students from the educational management system, and generates the students' personal classroom notes in a targeted manner according to the identity information, classroom knowledge shortcomings, and classroom key knowledge of each student, so as to facilitate students to quickly make up for their own knowledge shortcomings and deepen their study of key knowledge based on their personal classroom notes. The processing device can also obtain educational resource information such as lesson plans, courseware, knowledge point explanation content, and exercises from the teaching resource center according to the students' classroom knowledge shortcomings to record them in the students' personal classroom notes, so that students can strengthen their study of classroom knowledge shortcomings. The processing device can also send the real-time monitoring of various student behaviors to the teacher's device, so that the teacher can understand the student status of each student through the teacher's device. Compared with the prior art, the electronic classroom notes in this application specifically record the classroom knowledge shortcomings and key knowledge of different students, which is convenient for students to quickly make up for their own knowledge shortcomings and deepen their study of key knowledge based on their personal classroom notes, effectively improving the efficiency of students' after-class review, and solving the problem that students in the prior art cannot learn in a targeted manner based on electronic classroom notes.

图2给出了本申请实施例提供的一种课堂笔记生成方法的流程图。参考图2，该课堂笔记生成方法具体包括：FIG2 is a flow chart of a method for generating classroom notes provided in an embodiment of the present application. Referring to FIG2 , the method for generating classroom notes specifically includes:

S110、对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容。S110. Analyze the audio and video of the teacher's teaching to determine a first time period of the teacher's teaching behavior, and determine the first teaching content within the first time period based on the first time period and the teaching content video.

在本实施例中，老师授课音视频包括老师授课视频和老师授课音频。参考图1，采集设备包括教师摄像机、教师麦克风、学生摄像机和黑板摄像机。图3是本申请实施例提供的采集设备的安装示意图。如图3所示，教师摄像机16安装在教室11的后方并朝向讲台13，教师麦克风安装在讲台13上，学生摄像机15安装在教室11的右前方并朝向学生区域14，黑板摄像机安装在教室11的左方并朝向智能黑板12。在每节课的上课时间点采集设备控制教师摄像机16、教师麦克风、学生摄像机15和黑板摄像机17开始工作，并在下课时间点控制教师摄像机16、教师麦克风、学生摄像机15和黑板摄像机17停止工作，以使教师摄像机16、教师麦克风、学生摄像机15和黑板摄像机17分别采集到该节课的老师授课视频、老师授课音频、学生听课视频和教学内容视频。采集设备将实时采集到的老师授课视频、老师授课音频、学生听课视频和教学内容视频发送至处理设备。其中，智能黑板是集黑板、展台和智能交互大板于一体的设备，黑板摄像机采集的教学内容视频包含有板书画面、课件画面和展台投影画面。In this embodiment, the teacher's teaching audio and video includes the teacher's teaching video and the teacher's teaching audio. Referring to Figure 1, the acquisition device includes a teacher's camera, a teacher's microphone, a student's camera and a blackboard camera. Figure 3 is a schematic diagram of the installation of the acquisition device provided in the embodiment of the present application. As shown in Figure 3, the teacher's camera 16 is installed at the rear of the classroom 11 and faces the podium 13, the teacher's microphone is installed on the podium 13, the student's camera 15 is installed in the right front of the classroom 11 and faces the student area 14, and the blackboard camera is installed on the left side of the classroom 11 and faces the smart blackboard 12. At the class time of each class, the acquisition device controls the teacher's camera 16, the teacher's microphone, the student's camera 15 and the blackboard camera 17 to start working, and controls the teacher's camera 16, the teacher's microphone, the student's camera 15 and the blackboard camera 17 to stop working at the end of get out of class time, so that the teacher's camera 16, the teacher's microphone, the student's camera 15 and the blackboard camera 17 respectively collect the teacher's teaching video, the teacher's teaching audio, the student's class video and the teaching content video of the class. The acquisition device sends the real-time collected teacher's teaching video, the teacher's teaching audio, the student's class video and the teaching content video to the processing device. Among them, the smart blackboard is a device that integrates a blackboard, a booth and a smart interactive board. The teaching content video captured by the blackboard camera includes blackboard images, courseware images and booth projection images.

由于老师授课音视频记录了教师在该节课堂上的书写板书、讲解课件知识点和讲解习题等授课行为，教学内容视频记录了在该节课堂上的大板播放的课件、黑板书写的板书或展台展示的教材或作业等教学内容。因此处理设备可基于老师授课音视频中教师的各种授课行为和教学内容视频中对应的教学内容，确定出该节课的重要知识点，以便后续将该节课堂的重要知识点记录在电子课堂笔记中，便于学生在课后复习该电子课堂笔记中的重要知识点，快速掌握重点知识点。Since the teacher's teaching audio and video records the teacher's teaching behaviors such as writing on the blackboard, explaining the courseware knowledge points and explaining the exercises in the class, the teaching content video records the teaching content such as the courseware played on the big board in the class, the blackboard writing on the blackboard, or the teaching materials or homework displayed on the booth. Therefore, the processing device can determine the important knowledge points of the class based on the teacher's various teaching behaviors in the teacher's teaching audio and video and the corresponding teaching content in the teaching content video, so as to subsequently record the important knowledge points of the class in the electronic class notes, so that students can review the important knowledge points in the electronic class notes after class and quickly master the key knowledge points.

在一实施例中，老师授课行为包括强调行为、板书行为和反复讲解行为中的至少一个。强调行为是指教师语音说明当前授课的知识点为重点知识时的行为，板书行为是指教师在黑板上书写知识点时的行为，反复讲解行为是指教师重复多次讲解同一知识点的行为。可理解，当教师非常重视某一知识点，会通过强调、板书和反复讲解等行为去加深学生对该知识点的印象，以便学生掌握该知识点。因此当老师授课视频中的教师出现上述三种授课行为时，即可确定对应时间段内的授课内容为重要知识点，可将该重要知识点记录到电子课堂笔记中。在该实施例中，图4是本申请实施例提供的对老师授课音视频进行分析的流程图。如图4所示，该对老师授课音视频进行分析的步骤具体包括S1101-S1103：In one embodiment, the teacher's teaching behavior includes at least one of emphasizing behavior, writing on the blackboard, and repeatedly explaining behavior. Emphasizing behavior refers to the behavior of the teacher's voice explaining that the knowledge point currently being taught is the key knowledge, writing on the blackboard behavior refers to the behavior of the teacher writing the knowledge point on the blackboard, and repeatedly explaining behavior refers to the behavior of the teacher repeatedly explaining the same knowledge point. It can be understood that when a teacher attaches great importance to a certain knowledge point, he will deepen the students' impression of the knowledge point through behaviors such as emphasizing, writing on the blackboard, and repeatedly explaining, so that students can master the knowledge point. Therefore, when the teacher in the teacher's teaching video shows the above three teaching behaviors, it can be determined that the teaching content in the corresponding time period is an important knowledge point, and the important knowledge point can be recorded in the electronic classroom notes. In this embodiment, Figure 4 is a flowchart of analyzing the audio and video of the teacher's teaching provided in the embodiment of the present application. As shown in Figure 4, the steps of analyzing the audio and video of the teacher's teaching specifically include S1101-S1103:

S1101、对老师授课视频中老师的板书动作进行识别，确定出包含板书动作的第一目标视频，将第一目标视频的时间段作为板书行为的第一发生时间段。S1101. Identify the teacher's blackboard writing action in the teacher's teaching video, determine a first target video containing the blackboard writing action, and use the time period of the first target video as the first occurrence time period of the blackboard writing behavior.

示例性的，将老师授课视频中的各个视频帧输入预先训练的第一神经网络模型中，得到第一神经网络模型输出的各个视频帧是否包含有板书动作的预测结果。将包含有板书动作的连续视频帧作为一个第一目标视频，连续视频帧的时间戳即为对应第一目标视频的时间段。在第一目标视频的时间段内，老师有做出书写板书的动作，因此可将第一目标视频的时间段作为对应板书行为的第一发生时间段，以便后续根据该第一发生时间段从教学内容视频中对应获取到老师书写的板书内容。Exemplarily, each video frame in the teacher's teaching video is input into a pre-trained first neural network model to obtain a prediction result of whether each video frame output by the first neural network model contains a blackboard writing action. The continuous video frames containing the blackboard writing action are used as a first target video, and the timestamps of the continuous video frames are the time period corresponding to the first target video. During the time period of the first target video, the teacher has made the action of writing on the blackboard, so the time period of the first target video can be used as the first occurrence time period of the corresponding blackboard writing behavior, so that the corresponding blackboard writing content written by the teacher can be obtained from the teaching content video according to the first occurrence time period.

S1102、将老师授课音频转换为文本数据，对文本数据进行自然语义分析，确定出包含强调内容的第一目标文本，将第一目标文本对应的音频时间段作为强调行为的第一发生时间段。S1102, converting the teacher's lecture audio into text data, performing natural semantic analysis on the text data, determining a first target text containing emphasized content, and using the audio time period corresponding to the first target text as the first occurrence time period of the emphasized behavior.

示例性的，当文本数据中包含有“重点”、“重要知识点”“经常考”等词语时，自然语义分析算法可以分析出该文本数据中有强调知识点的内容，因此可从文本数据中获取到包含强调内容的第一目标文本。根据第一目标文本在文本数据中的位置，确定出第一目标文本对应老师授课音频中的音频时间段。在第一目标文本对应的音频时间段内，老师有做出强调当前教学内容的动作，因此可将音频时间段作为对应强调行为的第一发生时间段，以便后续根据第一发生时间段从教学内容视频中对应获取到老师强调的教学内容。For example, when the text data contains words such as "key points", "important knowledge points", and "frequently tested", the natural semantic analysis algorithm can analyze that the text data contains content that emphasizes knowledge points, and thus the first target text containing emphasized content can be obtained from the text data. According to the position of the first target text in the text data, the audio time period in the teacher's lecture audio corresponding to the first target text is determined. In the audio time period corresponding to the first target text, the teacher has made an action to emphasize the current teaching content, so the audio time period can be used as the first occurrence time period of the corresponding emphasis behavior, so that the teaching content emphasized by the teacher can be obtained from the teaching content video according to the first occurrence time period.

S1103、对文本数据进行自然语义分析，确定出包含反复讲解内容的第二目标文本，将第二目标文本对应的音频时间段，作为反复讲解行为的第一发生时间段。S1103, performing natural semantic analysis on the text data to determine a second target text containing repeated explanation content, and using the audio time period corresponding to the second target text as the first occurrence time period of the repeated explanation behavior.

示例性的，当文本数据中包含有多段重复的语句时，自然语义分析算法可以分析出该文本数据中有反复讲解知识点的内容，因此可从文本数据中获取到包含反复讲解内容的第二目标文本。根据第二目标文本在文本数据中的位置，确定出第二目标文本对应老师授课音频中的音频时间段。在第二目标文本对应的音频时间段内，老师有做出反复讲解当前教学内容的动作，因此可将音频时间段作为对应反复讲解行为的第一发生时间段，以便后续根据第一发生时间段从教学内容视频中对应获取到老师反复讲解的教学内容。Exemplarily, when the text data contains multiple repeated sentences, the natural semantic analysis algorithm can analyze that the text data contains content that repeatedly explains knowledge points, so the second target text containing the repeatedly explained content can be obtained from the text data. According to the position of the second target text in the text data, the audio time period in the teacher's teaching audio corresponding to the second target text is determined. In the audio time period corresponding to the second target text, the teacher has made an action of repeatedly explaining the current teaching content, so the audio time period can be used as the first occurrence time period corresponding to the repeated explanation behavior, so that the teaching content repeatedly explained by the teacher can be obtained from the teaching content video according to the first occurrence time period.

在该实施例中，当文本数据不包含反复讲解内容和强调内容时，可通过自然语义分析算法确定出文本数据包含的知识点内容，若该知识点为教案上的重要知识点，则可以从文本数据中确定出包含知识点内容的第三目标文本。确定第三目标文本对应的音频时间段，根据该音频时间段从教学内容视频中获取到老师讲解重要知识点的教学内容，将该教学内容记录到电子课堂笔记中，可有利于学生重点学习重要知识点的教学内容。In this embodiment, when the text data does not contain repeated explanations and emphasized contents, the knowledge point contents contained in the text data can be determined by a natural semantic analysis algorithm. If the knowledge point is an important knowledge point in the teaching plan, a third target text containing the knowledge point contents can be determined from the text data. The audio time period corresponding to the third target text is determined, and the teaching content of the teacher explaining the important knowledge points is obtained from the teaching content video according to the audio time period, and the teaching content is recorded in the electronic classroom notes, which can help students focus on learning the teaching content of the important knowledge points.

进一步的，图5是本申请实施例提供的确定第一教学内容的流程图。如图5所示，该确定第一教学内容的步骤具体包括S1104-S1105：Further, FIG5 is a flowchart of determining the first teaching content provided by an embodiment of the present application. As shown in FIG5, the step of determining the first teaching content specifically includes S1104-S1105:

S1104、根据第一发生时间段从教学内容视频中截取对应的第一视频帧，并对第一视频帧的内容进行分析，确定第一视频帧包含的第一知识点内容。S1104. Capture a corresponding first video frame from the teaching content video according to the first occurrence time period, and analyze the content of the first video frame to determine the first knowledge point content included in the first video frame.

S1105、将第一视频帧与第一知识点内容，作为对应第一发生时间段内的第一教学内容。S1105. Use the first video frame and the first knowledge point content as the first teaching content corresponding to the first occurrence time period.

示例性的，教师在授课时会采用课件、板书和展台这三种方式去配合讲解知识点，由于老师授课音视频和教学内容视频是同步采集的，老师授课音视频中教师正在讲解重点知识时，相应的，同一时间段内教学内容视频的第一视频帧中包含有对应重点知识内容的课件、黑板或展台的画面。因此可基于强调行为、板书行为和反复讲解行为的第一发生时间段，从教学内容视频中截取对应时间段内的第一视频帧。对第一视频帧的文字内容进行识别，并根据文字内容确定出该节课的重要知识点，并将重要知识点和对应的第一视频帧的画面作为对应行为的第一教学内容。在该实施例中，图6是本申请实施例提供的第一视频帧的示意图。第一视频帧示出了课件画面，假设基于强调行为的第一发生时间段在教学内容视频中获取到图6所示的课件画面，对该课件画面的文字内容进行识别，确定出该节课的重要知识点为两、就、哪、宽、顶、睛、肚、皮、孩和跳这些初识生字。将该课件画面和对应知识点作为对应强调行为的第一教学内容，以便后续将该课件画面和对应知识点记录在电子课堂笔记中。Exemplarily, when teaching, teachers will use courseware, blackboard writing and booths to explain knowledge points. Since the audio and video of the teacher's teaching and the teaching content video are collected synchronously, when the teacher is explaining the key knowledge in the audio and video of the teacher's teaching, the first video frame of the teaching content video in the same time period contains the courseware, blackboard or booth pictures of the corresponding key knowledge content. Therefore, based on the first occurrence time period of the emphasizing behavior, the blackboard writing behavior and the repeated explanation behavior, the first video frame in the corresponding time period can be intercepted from the teaching content video. The text content of the first video frame is identified, and the important knowledge points of the class are determined according to the text content, and the important knowledge points and the corresponding first video frame picture are used as the first teaching content of the corresponding behavior. In this embodiment, Figure 6 is a schematic diagram of the first video frame provided by the embodiment of the present application. The first video frame shows the courseware picture. It is assumed that the courseware picture shown in Figure 6 is obtained in the teaching content video based on the first occurrence time period of the emphasizing behavior, and the text content of the courseware picture is identified, and the important knowledge points of the class are determined to be the first recognition words such as two, just, which, wide, top, eyes, belly, skin, child and jump. The courseware screen and the corresponding knowledge points are used as the first teaching content of the corresponding emphasized behavior, so that the courseware screen and the corresponding knowledge points can be recorded in the electronic classroom notes later.

S120、对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容。S120. Analyze the video of students listening to lectures to determine the second time period of each student's listening behavior, and determine the second teaching content within the second time period based on the second time period and the teaching content video.

同样的，学生听课视频记录了每个学生在该节课堂上的记笔记、起立回答和表情疑惑等听课行为，教学内容视频记录了在该节课堂上的大板播放的课件、黑板书写的板书或展台展示的教材或作业等教学内容。因此处理设备中的学生听课行为分析子***可基于学生听课视频中每个学生的各种听课行为和教学内容视频中对应的教学内容，确定出该节课中每个学生的短板知识点，以便后续将每个学生的短板知识点对应记录在个人的电子课堂笔记中，便于学生在课后重点学习自身的短板知识点，快速补齐短板知识点。Similarly, the video of students listening to lectures records each student's listening behaviors such as taking notes, standing up to answer questions, and expressing doubts in class, and the video of teaching content records the teaching content such as the courseware played on the big board, the blackboard writing, or the teaching materials or homework displayed on the booth in the class. Therefore, the student listening behavior analysis subsystem in the processing device can determine the weak points of each student in the class based on the various listening behaviors of each student in the student listening video and the corresponding teaching content in the teaching content video, so that each student's weak points can be recorded in the personal electronic class notes later, so that students can focus on learning their weak points after class and quickly make up for their weak points.

在一实施例中，学生行为包括握笔记录行为、表情疑惑行为和答题行为中的至少一个。握笔记录行为是指学生握笔书写笔记时的行为，表情疑惑行为是指学生脸上出现疑惑表情时的行为，答题行为是指学生起立回答课堂问题时的行为。可理解，当学生不理解某一知识点时，会通过记笔记、疑惑表情和答题错误等行为去表达自身对该知识点的掌握程度不高。因此当第二视频流中的学生出现上述三种听课行为时，即可确定对应时间段内的教学内容为该学生的短板知识点。在该实施例中，图7是本申请实施例提供的对学生听课视频进行分析的流程图。如图7所示，该对学生听课视频进行分析的步骤具体包括S1201-S1203：In one embodiment, student behavior includes at least one of holding a pen to record behavior, expressing doubtful expressions, and answering questions. Holding a pen to record behavior refers to the behavior of students holding a pen to write notes, expressing doubtful expressions refers to the behavior when a doubtful expression appears on the student's face, and answering questions refers to the behavior of students standing up to answer classroom questions. It is understandable that when students do not understand a certain knowledge point, they will express their low level of mastery of the knowledge point through behaviors such as taking notes, expressing doubtful expressions, and answering questions incorrectly. Therefore, when students in the second video stream exhibit the above three listening behaviors, it can be determined that the teaching content in the corresponding time period is the student's weak knowledge point. In this embodiment, Figure 7 is a flowchart of analyzing student listening videos provided in an embodiment of the present application. As shown in Figure 7, the steps of analyzing student listening videos specifically include S1201-S1203:

S1201、对学生听课视频中学生的握笔记录动作进行识别，确定出包含握笔记录动作的第二目标视频，将第二目标视频的时间段作为握笔记录行为的第二发生时间段。S1201. Identify the pen-holding and recording actions of students in the video of students listening to a lecture, determine a second target video containing the pen-holding and recording actions, and use the time period of the second target video as the second occurrence time period of the pen-holding and recording behavior.

示例性的，将学生听课视频中的各个视频帧输入预先训练的第二神经网络模型中，得到第二神经网络模型输出的各个视频帧中哪个学生有握笔记录动作的预测结果。将包含有同一学生的握笔记录动作的连续视频帧作为该学生的一个第二目标视频，连续视频帧的时间戳即为对应第二目标视频的时间段。在第二目标视频的时间段内，该学生有做出握笔记录的动作，因此可将第二目标视频的时间段作为该学生对应握笔记录行为的第二发生时间段，以便后续根据该第二发生时间段从教学内容视频中对应获取到学生握笔记录的教学内容。Exemplarily, each video frame in the student lecture video is input into a pre-trained second neural network model to obtain a prediction result of which student has a pen-holding and recording action in each video frame output by the second neural network model. The continuous video frames containing the same student's pen-holding and recording action are used as a second target video of the student, and the timestamps of the continuous video frames are the time period corresponding to the second target video. During the time period of the second target video, the student has a pen-holding and recording action, so the time period of the second target video can be used as the second occurrence time period of the student's corresponding pen-holding and recording behavior, so that the corresponding teaching content of the student's pen-holding and recording behavior can be obtained from the teaching content video according to the second occurrence time period.

S1202、对学生听课视频中学生的站立动作进行识别，确定出包含站立动作的第三目标视频，将第三目标视频的时间段作为答题行为的第二发生时间段。S1202. Identify the standing action of the students in the video of the students listening to the lecture, determine a third target video containing the standing action, and use the time period of the third target video as the second occurrence time period of the answering behavior.

示例性的，将学生听课视频中的各个视频帧输入预先训练的第三神经网络模型中，得到第三神经网络模型输出的各个视频帧中哪个学生有站立动作的预测结果。将包含有同一学生的站立动作的连续视频帧作为该学生的一个第三目标视频，连续视频帧的时间戳即为对应第三目标视频的时间段。在第三目标视频的时间段内，该学生有做出站立回答问题的动作，因此可将第三目标视频的时间段作为该学生对应答题行为的第二发生时间段，以便后续根据该第二发生时间段从教学内容视频中对应获取到学生答题的教学内容。Exemplarily, each video frame in the student lecture video is input into a pre-trained third neural network model to obtain a prediction result of which student has a standing action in each video frame output by the third neural network model. The continuous video frames containing the standing action of the same student are used as a third target video of the student, and the timestamps of the continuous video frames are the time period corresponding to the third target video. During the time period of the third target video, the student has the action of standing up to answer questions, so the time period of the third target video can be used as the second occurrence time period of the student's answering behavior, so that the corresponding teaching content of the student's answering can be obtained from the teaching content video according to the second occurrence time period.

S1203、对学生听课视频中学生的面部疑惑表情进行识别，确定出包含面部疑惑表情的第四目标视频，将第四目标视频的时间段，分别作为表情疑惑行为的第二发生时间段。S1203, identifying the puzzled facial expressions of students in the video of students listening to a lecture, determining a fourth target video containing the puzzled facial expressions, and using the time period of the fourth target video as the second occurrence time period of the puzzled facial expressions.

示例性的，将学生听课视频中的各个视频帧输入预先训练的第四神经网络模型中，得到第四神经网络模型输出的各个视频帧中哪个学生的面部出现疑惑表情的预测结果。将包含有同一学生的面部疑惑表情的连续视频帧作为该学生的一个第四目标视频，连续视频帧的时间戳即为对应第四目标视频的时间段。在第四目标视频的时间段内，该学生对当前教学内容产生疑惑的表情，因此可将第四目标视频的时间段作为该学生对应表情疑惑行为的第二发生时间段，以便后续根据该第二发生时间段从教学内容视频中对应获取到学生疑惑的教学内容。Exemplarily, each video frame in the student's lecture video is input into a pre-trained fourth neural network model to obtain a prediction result of which student has a puzzled expression on his face in each video frame output by the fourth neural network model. Continuous video frames containing the same student's puzzled facial expression are used as a fourth target video of the student, and the timestamps of the continuous video frames are the time period corresponding to the fourth target video. During the time period of the fourth target video, the student has a puzzled expression on the current teaching content, so the time period of the fourth target video can be used as the second occurrence time period of the student's corresponding expression of puzzlement, so that the corresponding teaching content that the student is puzzled about can be obtained from the teaching content video according to the second occurrence time period.

需要说明的，由于学生听课视频同时拍摄了多个学生，在确定出某个学生有握笔记录行为、表情疑惑行为或答题行为后，可从学生听课视频的某一帧中截取出该学生的人脸图像，或者从某一帧中确定出该学生在教室中的座位信息。将该学生的所有听课行为与该学生的座位信息或者人脸图像关联保存，以便后续根据座位信息或人脸图像确定出各个学生听课行为对应的学生的身份信息。It should be noted that since the video of students attending classes simultaneously captures multiple students, after determining that a certain student has taken notes with a pen, has a puzzled expression, or has answered questions, the student's face image can be captured from a certain frame of the video of the student attending class, or the student's seat information in the classroom can be determined from a certain frame. All the student's listening behaviors are associated with the student's seat information or face image and saved, so that the identity information of the student corresponding to each student's listening behavior can be determined later based on the seat information or face image.

同样的，学生听课视频和教学内容视频是同步采集的，学生听课视频中学生表达对教学内容不理解时，相应的，同一时间段内教学内容视频的第二视频帧中包含有对应教学内容的课件、黑板或展台。因此可基于握笔记录行为、表情疑惑行为和答题行为的第二发生时间段，从教学内容视频中截取对应时间段内的第二视频帧。对第二视频帧的文字内容进行识别，并根据文字内容确定出该节课中对应学生的短板知识点，并将短板知识点和对应的第二视频帧的画面作为该学生的对应行为的第二知识点内容。Similarly, the video of students listening to lectures and the video of teaching content are collected synchronously. When students express that they do not understand the teaching content in the video of students listening to lectures, the second video frame of the teaching content video in the same time period will contain the courseware, blackboard or exhibition stand of the corresponding teaching content. Therefore, based on the second time period of the pen-holding recording behavior, the expression of doubt and the answering behavior, the second video frame in the corresponding time period can be intercepted from the teaching content video. The text content of the second video frame is recognized, and the weak knowledge points of the corresponding students in the class are determined according to the text content, and the weak knowledge points and the corresponding second video frame are used as the second knowledge point content of the corresponding behavior of the student.

S130、根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应的学生信息，将学生信息记录对应的电子课堂笔记中。S130, generating electronic classroom notes corresponding to the students according to the first teaching content and the second teaching content of the same student, determining the student information corresponding to each electronic classroom note, and recording the student information in the corresponding electronic classroom notes.

示例性的，将同一学生的握笔记录行为、表情疑惑行为和答题行为的第二发生时间段和对应的第二教学内容关联记录在该学生的电子课堂笔记中。将教师的强调行为、板书行为和反复知识点行为的第一发生时间段和对应的第一教学内容关联记录在每个学生的电子课堂笔记中。将学生的个性笔记内容和老师的大众笔记内容分别作为短板知识内容和重点知识内容，以便学生可以针对电子课堂笔记中有效补齐自身的短板知识和深入学习重点知识，提高学生的学习效率，快速掌握各种知识点。Exemplarily, the second time period of the same student's pen-holding recording behavior, expression of doubt and answering behavior and the corresponding second teaching content are recorded in the student's electronic classroom notes. The first time period of the teacher's emphasis behavior, blackboard writing behavior and repeated knowledge point behavior and the corresponding first teaching content are recorded in each student's electronic classroom notes. The student's personal note content and the teacher's public note content are respectively used as weak knowledge content and key knowledge content, so that students can effectively make up for their own weak knowledge and in-depth study of key knowledge in the electronic classroom notes, improve students' learning efficiency, and quickly master various knowledge points.

在该实施例中，图8是本申请实施例提供的生成电子课堂笔记的流程图。如图8所示，该生成电子课堂笔记的步骤具体包括S1301-S1302：In this embodiment, FIG8 is a flowchart of generating electronic classroom notes provided by an embodiment of the present application. As shown in FIG8, the steps of generating electronic classroom notes specifically include S1301-S1302:

S1301、根据第一发生时间段和第二发生时间段，确定老师授课行为与同一学生的学生听课行为的先后发生顺序。S1301. Determine the order of occurrence of the teacher's teaching behavior and the student's listening behavior of the same student according to the first occurrence time period and the second occurrence time period.

S1302、按照先后发生顺序，依次将老师授课行为和对应的第一教学内容以及学生听课行为与对应的第二教学内容，记录在对应学生的电子课堂笔记中。S1302. Record the teacher's teaching behavior and the corresponding first teaching content, and the student's listening behavior and the corresponding second teaching content in the electronic classroom notes of the corresponding students in the order of occurrence.

示例性的，将某个学生在该节课上的握笔记录行为、表情疑惑行为和答题行为的第二发生时间段以及该节课上老师的强调行为、板书行为和反复讲解行为的第一发生时间段，将各个行为进行排序。按照各个行为的先后发生顺序，将该学生的握笔记录行为、表情疑惑行为和答题行为与对应的第二发生时间段以及第二知识点内容，关联记录在该学生的电子课堂笔记。按照各个行为的先后发生顺序，将老师的强调行为、板书行为和反复讲解行为与对应的第一发生时间段以及第一知识点内容，关联记录在该学生的电子课堂笔记中。可理解，教师在课堂上会按照各个知识点之间的逻辑关系进行讲解，如果学生在课后复习时按照教师讲解的顺序依次去学生各个知识点，能够很好掌握各个知识点之间的逻辑关系，有利学生更好地学习各种知识点。Exemplarily, the second time period of a student's pen-holding and recording behavior, expression of doubt, and answering behavior in the class, as well as the first time period of the teacher's emphasis, blackboard writing, and repeated explanation in the class, are sorted. According to the order of occurrence of each behavior, the student's pen-holding and recording behavior, expression of doubt, and answering behavior are associated with the corresponding second time period and the second knowledge point content and recorded in the student's electronic class notes. According to the order of occurrence of each behavior, the teacher's emphasis, blackboard writing, and repeated explanation behavior are associated with the corresponding first time period and the first knowledge point content and recorded in the student's electronic class notes. It is understandable that the teacher will explain in class according to the logical relationship between each knowledge point. If students go to each knowledge point in turn according to the order of the teacher's explanation during after-class review, they can master the logical relationship between each knowledge point well, which is conducive to students learning various knowledge points better.

在本实施例中，身份信息是指学生的姓名、班级、人脸图像、教室座位、邮件地址或学生账号等信息。由于分析学生听课视频中各个学生的听课行为时，只能确定出教室哪个座位或者哪个人脸的学生有做出学生听课行为，却无法确定学生的姓名和班级等身份信息。因此在生成某个学生的电子课堂笔记后，可确定出该电子课堂笔记对应学生的身份信息，以便后续学生可以根据身份信息查询到自己的电子课堂笔记。In this embodiment, the identity information refers to the student's name, class, face image, classroom seat, email address or student account information. When analyzing the listening behavior of each student in the student listening video, it is only possible to determine which seat or which face of the student in the classroom has performed the student listening behavior, but it is impossible to determine the student's name and class and other identity information. Therefore, after generating a student's electronic classroom notes, the identity information of the student corresponding to the electronic classroom notes can be determined, so that subsequent students can query their own electronic classroom notes based on the identity information.

示例性的，参考图1，教务管理***可以是存储有各种学生的身份信息的信息数据库，处理设备可从教务管理***获取对应班级的所有学生的身份信息。处理设备从教务管理***获取到的人脸图像与各个学生听课行为关联保存的人脸图像进行对比，确定出各个学生听课行为对应的学生的身份信息，或者将该座位信息与各个学生听课行为关联保存的座位信息进行对比，确定出各个学生听课行为对应学生的身份信息。将学生听课行为对应的身份信息作为该学生听课行为对应生成的电子课堂笔记对应学生的身份信息，并将该身份信息记录在对应的电子课堂笔迹中。例如，学生听课行为A和学生听课行为B与人脸图像A关联保存，学生听课行为A和学生听课行为B对应的第二教学内容记录在电子课堂笔迹A中。处理设备从教务管理***获取到学生A的人脸图像B后，确定学生A与人脸图像B中的人脸相同，则可确定学生听课行为A和学生听课行为B为学生A在该节课做出的行为，进而将学生A的姓名和班级记录在电子课堂笔迹A。Exemplarily, referring to FIG1 , the teaching management system may be an information database storing identity information of various students, and the processing device may obtain the identity information of all students in the corresponding class from the teaching management system. The processing device compares the facial image obtained from the teaching management system with the facial image stored in association with each student's listening behavior to determine the identity information of the student corresponding to each student's listening behavior, or compares the seat information with the seat information stored in association with each student's listening behavior to determine the identity information of the student corresponding to each student's listening behavior. The identity information corresponding to the student's listening behavior is used as the identity information of the student corresponding to the electronic classroom notes generated corresponding to the student's listening behavior, and the identity information is recorded in the corresponding electronic classroom handwriting. For example, student listening behavior A and student listening behavior B are stored in association with facial image A, and the second teaching content corresponding to student listening behavior A and student listening behavior B is recorded in electronic classroom handwriting A. After the processing device obtains the facial image B of student A from the academic affairs management system, it determines that the face of student A is the same as the face in facial image B. It can then determine that student listening behavior A and student listening behavior B are the behaviors performed by student A in that class, and then record student A's name and class in the electronic classroom handwriting A.

在该实施例中，处理设备在确定出各个电子课堂笔迹对应学生的身份信息后，可根据身份信息中的邮箱地址或者学生账号，将电子课堂笔记发送至对应的学生设备。学生可以从学生设备获取到该节课的电子课堂笔记，提高了学生获取到电子课堂笔记的效率，提高了学生学习的便捷性。In this embodiment, after determining the identity information of the students corresponding to each electronic classroom handwriting, the processing device can send the electronic classroom notes to the corresponding student device according to the email address or student account in the identity information. Students can obtain the electronic classroom notes of the class from the student device, which improves the efficiency of students obtaining electronic classroom notes and improves the convenience of students' learning.

在一实施例中，参考图1，处理设备可以将课堂上学生的各种行为发送至教师设备，以便老师通过教师设备查看学生对当前教学内容的知识点的掌握情况。在该实施例中，处理设备在识别到学生听课视频中各个学生的握笔记录行为、表情疑惑行为和/或答题行为后，统计当前出现握笔记录行为、表情疑惑行为和/或答题行为的学生数量，并将学生数量发送至教师设备，以使教师设备展示出现握笔记录行为、表情疑惑行为和/或答题行为的学生数量。可理解，握笔记录行为、表情疑惑行为和答题行为对应的学生数量可表征学生对教师当前教学内容的知识点的掌握情况。老师可根据学生数量对授课节奏进行调整，便于学生适应教师的授课节奏，提高学生的听讲效率。例如，如果当前的表情疑惑行为的学生数量较多时，教师可仔细讲解当前教学内容的知识点，以便学生能够在课堂上掌握该知识点。In one embodiment, referring to FIG1 , the processing device can send various behaviors of students in the classroom to the teacher's device so that the teacher can view the students' mastery of the knowledge points of the current teaching content through the teacher's device. In this embodiment, after identifying the pen-holding recording behavior, expression of doubt behavior and/or answering behavior of each student in the student's lecture video, the processing device counts the number of students who currently have pen-holding recording behavior, expression of doubt behavior and/or answering behavior, and sends the number of students to the teacher's device so that the teacher's device displays the number of students who have pen-holding recording behavior, expression of doubt behavior and/or answering behavior. It can be understood that the number of students corresponding to the pen-holding recording behavior, expression of doubt behavior and answering behavior can represent the students' mastery of the knowledge points of the teacher's current teaching content. The teacher can adjust the teaching rhythm according to the number of students, so that the students can adapt to the teacher's teaching rhythm and improve the students' listening efficiency. For example, if the number of students with doubtful expressions is large, the teacher can carefully explain the knowledge points of the current teaching content so that the students can master the knowledge points in class.

在一实施例中，处理设备中的电子课堂笔记生成子***可根据表情疑惑行为对应的第二教学内容中的第二知识点内容，获取第二知识点内容对应的教学资源信息，并将教学资源信息记录在对应学生的电子课堂笔记中，教学资源信息包括课件、教案、知识点讲解内容和习题中的至少一个。其中，第二知识点内容是指对第二教学内容进行分析得到的第二教学内容包含的知识点内容。可理解，学生出现表情疑惑行为可能是因为教师讲解当前教学内容的知识点时不够详细，此时可辅助其他的教学资源信息以补全对应知识的详细解释，便于学生更好理解和掌握教学内容的各种知识点。参考图1，教学资源中心可以是存储有教案、课件、知识点讲解内容和习题等教育资源信息的资源数据库，处理设备可根据第二知识点内容的知识点从教学资源中心中对应查询该第二知识点内容对应的课件、教案、知识点讲解内容和习题，并将课件、教案、知识点讲解内容和习题和对应的第二知识点内容记录在一起，以便学生在课后复习到该知识点时可通过课件、教案和知识点讲解内容深入理解该知识点，并通过习题巩固该知识点。例如，假设第二知识点内容是图6示出的初识生字，可根据这些初识生字获取到与对应的拼音习题以及这些初识生字的朗读视频的地址，并将拼音习题和视频地址记录在电子课堂笔记对应的初识生字下方，以便学生通过练习电子课堂笔记中的拼音习题，或通过视频地址打开对应的朗读视频回顾这些初识生字的读音，加强对初识生字的学习。In one embodiment, the electronic classroom note generation subsystem in the processing device can obtain the teaching resource information corresponding to the second knowledge point content in the second teaching content corresponding to the facial expression and puzzlement behavior, and record the teaching resource information in the electronic classroom notes of the corresponding student, and the teaching resource information includes at least one of courseware, lesson plan, knowledge point explanation content and exercises. Among them, the second knowledge point content refers to the knowledge point content contained in the second teaching content obtained by analyzing the second teaching content. It is understandable that the student's facial expression and puzzlement behavior may be because the teacher is not detailed enough when explaining the knowledge points of the current teaching content. At this time, other teaching resource information can be used to supplement the detailed explanation of the corresponding knowledge, so that students can better understand and master the various knowledge points of the teaching content. With reference to FIG1 , the teaching resource center can be a resource database storing educational resource information such as lesson plans, courseware, knowledge point explanation content and exercises, and the processing device can query the courseware, lesson plan, knowledge point explanation content and exercises corresponding to the second knowledge point content from the teaching resource center according to the knowledge point of the second knowledge point content, and record the courseware, lesson plan, knowledge point explanation content and exercises together with the corresponding second knowledge point content, so that when students review the knowledge point after class, they can deeply understand the knowledge point through the courseware, lesson plan and knowledge point explanation content, and consolidate the knowledge point through exercises. For example, assuming that the second knowledge point content is the first-known new words shown in FIG6 , the addresses of the corresponding pinyin exercises and the reading videos of these first-known new words can be obtained according to these first-known new words, and the pinyin exercises and video addresses are recorded below the first-known new words corresponding to the electronic classroom notes, so that students can review the pronunciation of these first-known new words by practicing the pinyin exercises in the electronic classroom notes, or by opening the corresponding reading videos through the video addresses, and strengthen the learning of the first-known new words.

综上，本申请实施例提供的课堂笔记生成方法，通过对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据第一发生时间段和教学内容视频确定第一发生时间段内的第一教学内容；对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容；根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应学生的身份信息，将身份信息记录对应的电子课堂笔记中。通过上述技术手段，分析学生和老师在课堂上的行为举止，确定每个学生的课堂知识短板和课堂重点知识，并根据课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记，以便学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习，有效提高学生的课后复习效率，解决了现有技术中学生无法基于电子课堂笔记针对性学习的问题。In summary, the classroom notes generation method provided by the embodiment of the present application determines the first time period of the teacher's teaching behavior by analyzing the audio and video of the teacher's teaching, and determines the first teaching content within the first time period according to the first time period and the teaching content video; analyzes the student's listening video to determine the second time period of each student's student listening behavior, and determines the second teaching content within the second time period according to the second time period and the teaching content video; generates the electronic classroom notes of the corresponding students according to the first teaching content and the second teaching content of the same student, and determines the identity information of the students corresponding to each electronic classroom note, and records the identity information in the corresponding electronic classroom notes. Through the above technical means, the behavior of students and teachers in the classroom is analyzed, the classroom knowledge shortcomings and classroom key knowledge of each student are determined, and the personal classroom notes of students are generated in a targeted manner according to the classroom knowledge shortcomings and classroom key knowledge, so that students can quickly make up for their own knowledge shortcomings and deepen their study of key knowledge based on personal classroom notes, effectively improve the efficiency of students' after-school review, and solve the problem that students in the prior art cannot learn in a targeted manner based on electronic classroom notes.

在上述实施例的基础上，图9为本申请实施例提供的一种课堂笔记生成装置的结构示意图。参考图9，本实施例提供的课堂笔记生成装置具体包括：第一行为分析模块21、第二行为分析模块22和笔记生成模块23。Based on the above embodiment, Fig. 9 is a schematic diagram of the structure of a class note generation device provided in the embodiment of the present application. Referring to Fig. 9, the class note generation device provided in the present embodiment specifically includes: a first behavior analysis module 21, a second behavior analysis module 22 and a note generation module 23.

其中，第一行为分析模块，被配置为对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据第一发生时间段和教学内容视频确定第一发生时间段内的第一教学内容；The first behavior analysis module is configured to analyze the teacher's teaching audio and video, determine the first occurrence time period of the teacher's teaching behavior, and determine the first teaching content within the first occurrence time period according to the first occurrence time period and the teaching content video;

第二行为分析模块，被配置为对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容；The second behavior analysis module is configured to analyze the student listening video, determine the second occurrence time period of each student's student listening behavior, and determine the second teaching content within the second occurrence time period according to the second occurrence time period and the teaching content video;

笔记生成模块，被配置为根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应学生的身份信息，将身份信息记录对应的电子课堂笔记中。The note generation module is configured to generate electronic classroom notes for the corresponding students based on the first teaching content and the second teaching content of the same student, determine the identity information of the students corresponding to each electronic classroom note, and record the identity information in the corresponding electronic classroom notes.

在上述实施例的基础上，老师授课行为包括强调行为、板书行为和反复讲解行为中的至少一个；相应的，第一行为分析模块，包括：板书行为识别单元，被配置为对老师授课视频中老师的板书动作进行识别，确定出包含板书动作的第一目标视频，将第一目标视频的时间段作为板书行为的第一发生时间段；强调行为识别单元，被配置为将老师授课音频转换为文本数据，对文本数据进行自然语义分析，确定出包含强调内容的第一目标文本，将第一目标文本对应的音频时间段作为强调行为的第一发生时间段；反复讲解行为识别单元，被配置为对文本数据进行自然语义分析，确定出包含反复讲解内容的第二目标文本，将第二目标文本对应的音频时间段，作为反复讲解行为的第一发生时间段。On the basis of the above embodiments, the teacher's teaching behavior includes at least one of emphasis behavior, blackboard writing behavior and repeated explanation behavior; accordingly, the first behavior analysis module includes: a blackboard writing behavior recognition unit, which is configured to recognize the teacher's blackboard writing action in the teacher's teaching video, determine the first target video containing the blackboard writing action, and use the time period of the first target video as the first occurrence time period of the blackboard writing behavior; an emphasis behavior recognition unit, which is configured to convert the teacher's teaching audio into text data, perform natural semantic analysis on the text data, determine the first target text containing emphasis content, and use the audio time period corresponding to the first target text as the first occurrence time period of the emphasis behavior; the repeated explanation behavior recognition unit, which is configured to perform natural semantic analysis on the text data, determine the second target text containing repeated explanation content, and use the audio time period corresponding to the second target text as the first occurrence time period of the repeated explanation behavior.

在上述实施例的基础上，学生听课行为包括握笔记录行为、表情疑惑行为和答题行为中的至少一个；相应的，第二行为分析模块，包括：握笔行为识别单元，被配置为对学生听课视频中学生的握笔记录动作进行识别，确定出包含握笔记录动作的第二目标视频，将第二目标视频的时间段作为握笔记录行为的第二发生时间段；答题行为识别单元，被配置为对学生听课视频中学生的站立动作进行识别，确定出包含站立动作的第三目标视频，将第三目标视频的时间段作为答题行为的第二发生时间段；表情疑惑行为识别单元，被配置为对学生听课视频中学生的面部疑惑表情进行识别，确定出包含面部疑惑表情的第四目标视频，将第四目标视频的时间段，分别作为表情疑惑行为的第二发生时间段。On the basis of the above embodiments, the student's listening behavior includes at least one of the following: pen-holding and recording behavior, facial expression and puzzled behavior, and answering question behavior; accordingly, the second behavior analysis module includes: a pen-holding behavior recognition unit, configured to recognize the student's pen-holding and recording action in the student's listening video, determine a second target video containing the pen-holding and recording action, and use the time period of the second target video as the second time period of occurrence of the pen-holding and recording behavior; a question-answering behavior recognition unit, configured to recognize the student's standing action in the student's listening video, determine a third target video containing the standing action, and use the time period of the third target video as the second time period of occurrence of the answering behavior; a facial expression and puzzled behavior recognition unit, configured to recognize the student's facial expression of puzzled expression in the student's listening video, determine a fourth target video containing the facial expression of puzzled expression, and use the time period of the fourth target video as the second time period of occurrence of the facial expression and puzzled behavior.

在上述实施例的基础上，第一行为分析模块，包括：视频帧截取单元，被配置为根据第一发生时间段从教学内容视频中截取对应的第一视频帧，并对第一视频帧的内容进行分析，确定第一视频帧包含的第一知识点内容；教学内容确定单元，被配置为将第一视频帧与第一知识点内容，作为对应第一发生时间段内的第一教学内容。Based on the above embodiment, the first behavior analysis module includes: a video frame capture unit, which is configured to capture the corresponding first video frame from the teaching content video according to the first occurrence time period, and analyze the content of the first video frame to determine the first knowledge point content contained in the first video frame; a teaching content determination unit, which is configured to use the first video frame and the first knowledge point content as the first teaching content corresponding to the first occurrence time period.

在上述实施例的基础上，笔记生成模块，包括：顺序确定单元，被配置为根据第一发生时间段和第二发生时间段，确定老师授课行为与同一学生的学生听课行为的先后发生顺序；教学内容记录单元，被配置为按照先后发生顺序，依次将老师授课行为和对应的第一教学内容以及学生听课行为与对应的第二教学内容，记录在对应学生的电子课堂笔记中。Based on the above embodiment, the note generation module includes: a sequence determination unit, which is configured to determine the sequence of occurrence of the teacher's teaching behavior and the student's listening behavior of the same student according to the first occurrence time period and the second occurrence time period; a teaching content recording unit, which is configured to record the teacher's teaching behavior and the corresponding first teaching content and the student's listening behavior and the corresponding second teaching content in the electronic classroom notes of the corresponding students in sequence according to the sequence of occurrence.

在上述实施例的基础上，笔记生成模块，包括：笔记补充单元，被配置为根据表情疑惑行为对应的第二教学内容中的第二知识点内容，获取第二知识点内容对应的教学资源信息，并将教学资源信息记录在对应学生的电子课堂笔记中，教学资源信息包括课件、教案、知识点讲解内容和习题中的至少一个。Based on the above embodiment, the note generation module includes: a note supplement unit, which is configured to obtain teaching resource information corresponding to the second knowledge point content in the second teaching content corresponding to the facial expression and doubt behavior, and record the teaching resource information in the corresponding student's electronic classroom notes, the teaching resource information including at least one of courseware, lesson plans, knowledge point explanation content and exercises.

在上述实施例的基础上，课堂笔记生成装置还包括：课堂反馈模块，被配置为在对学生听课视频进行分析之后，统计当前出现握笔记录行为、表情疑惑行为和/或答题行为的学生数量，并将学生数量发送至教师设备，以使教师设备展示出现握笔记录行为、表情疑惑行为和/或答题行为的学生数量。Based on the above embodiment, the classroom note generating device also includes: a classroom feedback module, which is configured to count the number of students who are currently holding a pen to record, expressing doubts and/or answering questions after analyzing the student listening video, and send the number of students to the teacher's device, so that the teacher's device can display the number of students who are currently holding a pen to record, expressing doubts and/or answering questions.

在上述实施例的基础上，课堂笔记生成装置还包括：笔记发送模块，被配置为在将身份信息记录对应的电子课堂笔记之后，根据身份信息中的邮箱地址或学生账号，将电子课堂笔记发送至对应的学生设备。Based on the above embodiment, the classroom note generating device also includes: a note sending module, which is configured to send the electronic classroom notes to the corresponding student device according to the email address or student account in the identity information after recording the electronic classroom notes corresponding to the identity information.

上述，本申请实施例提供的课堂笔记生成装置，通过对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据第一发生时间段和教学内容视频确定第一发生时间段内的第一教学内容；对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容；根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应学生的身份信息，将身份信息记录对应的电子课堂笔记中。通过上述技术手段，分析学生和老师在课堂上的行为举止，确定每个学生的课堂知识短板和课堂重点知识，并根据课堂知识短板和课堂重点知识针对性生成学生的个人课堂笔记，以便学生基于个人课堂笔记快速补齐自己的知识短板和深入对重点知识的学习，有效提高学生的课后复习效率，解决了现有技术中学生无法基于电子课堂笔记针对性学习的问题。As mentioned above, the classroom note generation device provided by the embodiment of the present application determines the first time period of the teacher's teaching behavior by analyzing the audio and video of the teacher's teaching, and determines the first teaching content within the first time period according to the first time period and the teaching content video; analyzes the student's listening video, determines the second time period of each student's student's listening behavior, and determines the second teaching content within the second time period according to the second time period and the teaching content video; generates the electronic classroom notes of the corresponding students according to the first teaching content and the second teaching content of the same student, and determines the identity information of the students corresponding to each electronic classroom note, and records the identity information in the corresponding electronic classroom notes. Through the above technical means, the behavior of students and teachers in the classroom is analyzed, the classroom knowledge shortcomings and classroom key knowledge of each student are determined, and the personal classroom notes of students are generated in a targeted manner according to the classroom knowledge shortcomings and classroom key knowledge, so that students can quickly make up for their own knowledge shortcomings and deepen their study of key knowledge based on personal classroom notes, effectively improve the efficiency of students' after-school review, and solve the problem that students in the prior art cannot learn in a targeted manner based on electronic classroom notes.

本申请实施例提供的课堂笔记生成装置可以用于执行上述实施例提供的课堂笔记生成方法，具备相应的功能和有益效果。The classroom notes generating device provided in the embodiment of the present application can be used to execute the classroom notes generating method provided in the above embodiment, and has corresponding functions and beneficial effects.

图10是本申请实施例提供的一种课堂笔记生成设备的结构示意图，参考图10，该课堂笔记生成设备包括：处理器31、存储器32、通信装置33、输入装置34及输出装置35。该课堂笔记生成设备中处理器31的数量可以是一个或者多个，该课堂笔记生成设备中的存储器32的数量可以是一个或者多个。该课堂笔记生成设备的处理器31、存储器32、通信装置33、输入装置34及输出装置35可以通过总线或者其他方式连接。FIG10 is a schematic diagram of the structure of a class note generating device provided by an embodiment of the present application. Referring to FIG10 , the class note generating device includes: a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 in the class note generating device can be one or more, and the number of memories 32 in the class note generating device can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the class note generating device can be connected via a bus or other means.

存储器32作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本申请任意实施例的课堂笔记生成方法对应的程序指令/模块(例如，课堂笔记生成装置中的第一行为分析模块21、第二行为分析模块22和笔记生成模块23)。存储器32可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序；存储数据区可存储根据设备的使用所创建的数据等。此外，存储器32可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器可进一步包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions/modules corresponding to the classroom note generation method of any embodiment of the present application (for example, the first behavior analysis module 21, the second behavior analysis module 22 and the note generation module 23 in the classroom note generation device). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

通信装置33用于进行数据传输。The communication device 33 is used for data transmission.

处理器31通过运行存储在存储器32中的软件程序、指令以及模块，从而执行设备的各种功能应用以及数据处理，即实现上述的课堂笔记生成方法。The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, realizes the above-mentioned classroom note generation method.

输入装置34可用于接收输入的数字或字符信息，以及产生与设备的用户设置以及功能控制有关的键信号输入。输出装置35可包括显示屏等显示设备。The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

上述提供的课堂笔记生成设备可用于执行上述实施例提供的课堂笔记生成方法，具备相应的功能和有益效果。The classroom note generation device provided above can be used to execute the classroom note generation method provided in the above embodiment, and has corresponding functions and beneficial effects.

本申请实施例还提供一种包含计算机可执行指令的存储介质，计算机可执行指令在由计算机处理器执行时用于执行一种课堂笔记生成方法，该课堂笔记生成方法包括：对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据第一发生时间段和教学内容视频确定第一发生时间段内的第一教学内容；对学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据第二发生时间段和教学内容视频确定第二发生时间段内的第二教学内容；根据第一教学内容和同一学生的第二教学内容生成对应学生的电子课堂笔记，并确定各个电子课堂笔记对应学生的身份信息，将身份信息记录对应的电子课堂笔记中。An embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute a class note generation method when executed by a computer processor. The class note generation method includes: analyzing the audio and video of the teacher's lecture to determine a first time period of the teacher's teaching behavior, and determining the first teaching content within the first time period based on the first time period and the teaching content video; analyzing the student listening video to determine the second time period of each student's student listening behavior, and determining the second teaching content within the second time period based on the second time period and the teaching content video; generating electronic class notes for the corresponding students based on the first teaching content and the second teaching content of the same student, and determining the identity information of the students corresponding to each electronic class note, and recording the identity information in the corresponding electronic class notes.

存储介质——任何的各种类型的存储器设备或存储设备。术语“存储介质”旨在包括：安装介质，例如CD-ROM、软盘或磁带装置；计算机***存储器或随机存取存储器，诸如DRAM、DDR RAM、SRAM、EDO RAM，兰巴斯(Rambus)RAM等；非易失性存储器，诸如闪存、磁介质(例如硬盘或光存储)；寄存器或其它相似类型的存储器元件等。存储介质可以还包括其它类型的存储器或其组合。另外，存储介质可以位于程序在其中被执行的第一计算机***中，或者可以位于不同的第二计算机***中，第二计算机***通过网络(诸如因特网)连接到第一计算机***。第二计算机***可以提供程序指令给第一计算机用于执行。术语“存储介质”可以包括驻留在不同位置中(例如在通过网络连接的不同计算机***中)的两个或更多存储介质。存储介质可以存储可由一个或多个处理器执行的程序指令(例如具体实现为计算机程序)。Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

当然，本申请实施例所提供的一种包含计算机可执行指令的存储介质，其计算机可执行指令不限于如上的课堂笔记生成方法，还可以执行本申请任意实施例所提供的课堂笔记生成方法中的相关操作。Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application is not limited to the above-mentioned classroom note generation method, and the computer executable instructions can also execute the relevant operations in the classroom note generation method provided in any embodiment of the present application.

上述实施例中提供的课堂笔记生成装置、存储介质及课堂笔记生成设备可执行本申请任意实施例所提供的课堂笔记生成方法，未在上述实施例中详尽描述的技术细节，可参见本申请任意实施例所提供的课堂笔记生成方法。The classroom note generation device, storage medium and classroom note generation equipment provided in the above embodiments can execute the classroom note generation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the classroom note generation method provided in any embodiment of the present application.

上述仅为本申请的较佳实施例及所运用的技术原理。本申请不限于这里的特定实施例，对本领域技术人员来说能够进行的各种明显变化、重新调整及替代均不会脱离本申请的保护范围。因此，虽然通过以上实施例对本申请进行了较为详细的说明，但是本申请不仅仅限于以上实施例，在不脱离本申请构思的情况下，还可以包括更多其他等效实施例，而本申请的范围由权利要求的范围决定。The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

一种课堂笔记生成方法，其特征在于，包括：A method for generating classroom notes, characterized by comprising:

对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容；Analyze the audio and video of the teacher's teaching to determine a first time period of the teacher's teaching behavior, and determine the first teaching content within the first time period according to the first time period and the teaching content video;

对所述学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据所述第二发生时间段和所述教学内容视频确定所述第二发生时间段内的第二教学内容；Analyze the student lecture video to determine a second occurrence time period of each student's lecture behavior, and determine a second teaching content within the second occurrence time period according to the second occurrence time period and the teaching content video;

根据所述第一教学内容和同一学生的所述第二教学内容生成对应学生的电子课堂笔记，并确定各个所述电子课堂笔记对应学生的身份信息，将所述身份信息记录对应的电子课堂笔记中。Generate electronic classroom notes corresponding to the student according to the first teaching content and the second teaching content of the same student, determine the identity information of the student corresponding to each electronic classroom note, and record the identity information in the corresponding electronic classroom notes.
根据权利要求1所述的课堂笔记生成方法，其特征在于，所述老师授课行为包括强调行为、板书行为和反复讲解行为中的至少一个；The class note generation method according to claim 1 is characterized in that the teacher's teaching behavior includes at least one of emphasizing behavior, writing on the blackboard behavior, and repeatedly explaining behavior;

所述对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，包括以下至少之一：The step of analyzing the teacher's teaching audio and video to determine the first time period of the teacher's teaching behavior includes at least one of the following:

对老师授课视频中老师的板书动作进行识别，确定出包含板书动作的第一目标视频，将所述第一目标视频的时间段作为所述板书行为的第一发生时间段；Recognize the teacher's blackboard writing action in the teacher's teaching video, determine a first target video containing the blackboard writing action, and use the time period of the first target video as the first occurrence time period of the blackboard writing behavior;

将老师授课音频转换为文本数据，对所述文本数据进行自然语义分析，确定出包含强调内容的第一目标文本，将所述第一目标文本对应的音频时间段作为所述强调行为的第一发生时间段；Convert the teacher's teaching audio into text data, perform natural semantic analysis on the text data, determine a first target text containing emphasized content, and use the audio time period corresponding to the first target text as the first occurrence time period of the emphasized behavior;

对所述文本数据进行自然语义分析，确定出包含反复讲解内容的第二目标文本，将所述第二目标文本对应的音频时间段，作为所述反复讲解行为的第一发生时间段。A natural semantic analysis is performed on the text data to determine a second target text containing repeated explanation content, and an audio time period corresponding to the second target text is used as a first occurrence time period of the repeated explanation behavior.
根据权利要求1所述的课堂笔记生成方法，其特征在于，所述学生听课行为包括握笔记录行为、表情疑惑行为和答题行为中的至少一个；The class note generation method according to claim 1 is characterized in that the student's listening behavior includes at least one of a pen-holding behavior, a puzzled expression behavior, and a question-answering behavior;

所述对所述学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，包括以下至少之一：The analyzing the video of the students listening to the class to determine the second time period of occurrence of each student's listening behavior includes at least one of the following:

对所述学生听课视频中学生的握笔记录动作进行识别，确定出包含握笔记录动作的第二目标视频，将所述第二目标视频的时间段作为所述握笔记录行为的第二发生时间段；Identify the student's pen-holding and recording action in the video of the student listening to the class, determine a second target video containing the pen-holding and recording action, and use the time period of the second target video as the second occurrence time period of the pen-holding and recording behavior;

对所述学生听课视频中学生的站立动作进行识别，确定出包含站立动作的第三目标视频，将所述第三目标视频的时间段作为所述答题行为的第二发生时间段；Recognize the standing action of the student in the video of the student listening to the class, determine a third target video containing the standing action, and use the time period of the third target video as the second occurrence time period of the answering behavior;

对所述学生听课视频中学生的面部疑惑表情进行识别，确定出包含面部疑惑表情的第四目标视频，将所述第四目标视频的时间段，分别作为所述表情疑惑行为的第二发生时间段。The puzzled facial expressions of the students in the video of the students listening to the class are identified, a fourth target video containing the puzzled facial expressions is determined, and the time period of the fourth target video is used as the second time period of occurrence of the puzzled facial expressions.
根据权利要求1所述的课堂笔记生成方法，其特征在于，所述根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容，包括：The class note generation method according to claim 1 is characterized in that the step of determining the first teaching content within the first occurrence time period according to the first occurrence time period and the teaching content video comprises:

根据所述第一发生时间段从所述教学内容视频中截取对应的第一视频帧，并对所述第一视频帧的内容进行分析，确定所述第一视频帧包含的第一知识点内容；According to the first occurrence time period, a corresponding first video frame is intercepted from the teaching content video, and the content of the first video frame is analyzed to determine the first knowledge point content contained in the first video frame;

将所述第一视频帧与所述第一知识点内容，作为对应所述第一发生时间段内的第一教学内容。The first video frame and the first knowledge point content are used as the first teaching content corresponding to the first occurrence time period.
根据权利要求1所述的课堂笔记生成方法，其特征在于，所述将所述第一教学内容和所述第二教学内容组合，生成各个学生对应的电子课堂笔记，包括：The class notes generation method according to claim 1 is characterized in that the step of combining the first teaching content and the second teaching content to generate electronic class notes corresponding to each student comprises:

根据所述第一发生时间段和所述第二发生时间段，确定所述老师授课行为与同一学生的学生听课行为的先后发生顺序；Determining the order of occurrence of the teacher's teaching behavior and the student's listening behavior of the same student according to the first occurrence time period and the second occurrence time period;

按照所述先后发生顺序，依次将所述老师授课行为和对应的第一教学内容以及所述学生听课行为与对应的第二教学内容，记录在对应学生的电子课堂笔记中。According to the order of occurrence, the teacher's teaching behavior and the corresponding first teaching content, as well as the student's listening behavior and the corresponding second teaching content, are recorded in the electronic classroom notes of the corresponding students.
根据权利要求3所述的课堂笔记生成方法，其特征在于，所述基于每个学生的学生行为信息和老师行为信息，生成对应学生的电子课堂笔记，包括：The class notes generation method according to claim 3 is characterized in that the step of generating electronic class notes for a corresponding student based on the student behavior information and teacher behavior information of each student comprises:

根据所述表情疑惑行为对应的第二教学内容中的第二知识点内容，获取所述第二知识点内容对应的教学资源信息，并将所述教学资源信息记录在对应学生的电子课堂笔记中，所述教学资源信息包括课件、教案、知识点讲解内容和习题中的至少一个。According to the second knowledge point content in the second teaching content corresponding to the facial expression and puzzled behavior, obtain the teaching resource information corresponding to the second knowledge point content, and record the teaching resource information in the electronic classroom notes of the corresponding student, wherein the teaching resource information includes at least one of courseware, lesson plans, knowledge point explanation content and exercises.
根据权利要求3所述的课堂笔记生成方法，其特征在于，在所述对所述学生听课视频进行分析之后，还包括：The method for generating class notes according to claim 3 is characterized in that, after analyzing the video of the student listening to the class, it also includes:

统计当前出现所述握笔记录行为、所述表情疑惑行为和/或所述答题行为的学生数量，并将所述学生数量发送至教师设备，以使所述教师设备展示出现所述握笔记录行为、所述表情疑惑行为和/或所述答题行为的学生数量。The number of students who are currently performing the pen-holding recording behavior, the puzzled expression behavior and/or the question-answering behavior is counted, and the number of students is sent to the teacher's device so that the teacher's device displays the number of students who are currently performing the pen-holding recording behavior, the puzzled expression behavior and/or the question-answering behavior.
根据权利要求1所述的课堂笔记生成方法，其特征在于，在所述将所述身份信息记录对应的电子课堂笔记之后，还包括：The method for generating classroom notes according to claim 1, characterized in that after recording the identity information in the corresponding electronic classroom notes, it further comprises:

根据所述身份信息中的邮箱地址或学生账号，将所述电子课堂笔记发送至对应的学生设备。The electronic classroom notes are sent to the corresponding student device according to the email address or student account in the identity information.
一种课堂笔记生成***，其特征在于，包括采集设备和处理设备，其中：A class note generation system, characterized in that it includes a collection device and a processing device, wherein:

所述采集设备用于，采集老师授课音视频、学生听课视频和教学内容视频，并将所述老师授课音视频、所述学生听课视频和所述教学内容视频发送至所述处理设备；The acquisition device is used to collect audio and video of the teacher's lecture, video of the students listening to the lecture and video of the teaching content, and send the audio and video of the teacher's lecture, video of the students listening to the lecture and video of the teaching content to the processing device;

所述处理设备用于，接收所述采集设备发送的所述老师授课音视频、所述学生听课视频和所述教学内容视频；对老师授课音视频进行分析，确定老师授课行为的第一发生时间段，并根据所述第一发生时间段和教学内容视频确定所述第一发生时间段内的第一教学内容；对所述学生听课视频进行分析，确定每个学生的学生听课行为的第二发生时间段，并根据所述第二发生时间段和所述教学内容视频确定所述第二发生时间段内的第二教学内容；根据所述第一教学内容和同一学生的所述第二教学内容生成对应学生的电子课堂笔记，并确定各个所述电子课堂笔记对应学生的身份信息，将所述身份信息记录对应的电子课堂笔记中。The processing device is used to receive the teacher's teaching audio and video, the student's listening video and the teaching content video sent by the acquisition device; analyze the teacher's teaching audio and video to determine the first time period of the teacher's teaching behavior, and determine the first teaching content within the first time period based on the first time period and the teaching content video; analyze the student's listening video to determine the second time period of each student's listening behavior, and determine the second teaching content within the second time period based on the second time period and the teaching content video; generate electronic classroom notes for the corresponding students based on the first teaching content and the second teaching content of the same student, and determine the identity information of the students corresponding to each of the electronic classroom notes, and record the identity information in the corresponding electronic classroom notes.
根据权利要求9所述的课堂笔记生成***，其特征在于，所述老师授课行为包括强调行为、板书行为和反复讲解行为中的至少一个；The class note generation system according to claim 9 is characterized in that the teacher's teaching behavior includes at least one of an emphasis behavior, a blackboard writing behavior, and a repeated explanation behavior;

相应的，所述处理设备还用于，对老师授课视频中老师的板书动作进行识别，确定出包含板书动作的第一目标视频，将所述第一目标视频的时间段作为所述板书行为的第一发生时间段；将老师授课音频转换为文本数据，对所述文本数据进行自然语义分析，确定出包含强调内容的第一目标文本，将所述第一目标文本对应的音频时间段作为所述强调行为的第一发生时间段；对所述文本数据进行自然语义分析，确定出包含反复讲解内容的第二目标文本，将所述第二目标文本对应的音频时间段，作为所述反复讲解行为的第一发生时间段。Correspondingly, the processing device is also used to identify the teacher's blackboard writing action in the teacher's teaching video, determine the first target video containing the blackboard writing action, and use the time period of the first target video as the first time period for the blackboard writing behavior to occur; convert the teacher's teaching audio into text data, perform natural semantic analysis on the text data, determine the first target text containing emphasized content, and use the audio time period corresponding to the first target text as the first time period for the emphasized behavior to occur; perform natural semantic analysis on the text data, determine the second target text containing repeated explanation content, and use the audio time period corresponding to the second target text as the first time period for the repeated explanation behavior to occur.
根据权利要求9所述的课堂笔记生成***，其特征在于，所述学生听课行为包括握笔记录行为、表情疑惑行为和答题行为中的至少一个；The class note generation system according to claim 9 is characterized in that the student's listening behavior includes at least one of a pen-holding behavior, a puzzled expression behavior, and a question-answering behavior;

相应的，所述处理设备还用于，对所述学生听课视频中学生的握笔记录动作进行识别，确定出包含握笔记录动作的第二目标视频，将所述第二目标视频的时间段作为所述握笔记录行为的第二发生时间段；对所述学生听课视频中学生的站立动作进行识别，确定出包含站立动作的第三目标视频，将所述第三目标视频的时间段作为所述答题行为的第二发生时间段；对所述学生听课视频中学生的面部疑惑表情进行识别，确定出包含面部疑惑表情的第四目标视频，将所述第四目标视频的时间段，分别作为所述表情疑惑行为的第二发生时间段。Correspondingly, the processing device is also used to identify the student's pen-holding and recording action in the student listening to the lecture video, determine a second target video containing the pen-holding and recording action, and use the time period of the second target video as the second time period of occurrence of the pen-holding and recording behavior; identify the student's standing action in the student listening to the lecture video, determine a third target video containing the standing action, and use the time period of the third target video as the second time period of occurrence of the answering behavior; identify the student's facial expression of doubt in the student listening to the lecture video, determine a fourth target video containing the facial expression of doubt, and use the time period of the fourth target video as the second time period of occurrence of the expression of doubt behavior.
根据权利要求11所述的课堂笔记生成***，其特征在于，所述课堂笔记生成***还包括教师设备，其中：The class note generation system according to claim 11, characterized in that the class note generation system also includes a teacher device, wherein:

所述处理设备还用于，统计当前出现所述握笔记录行为、所述表情疑惑行为和/或所述答题行为的学生数量，并将所述学生数量发送至所述教师设备；The processing device is also used to count the number of students who currently have the pen-holding recording behavior, the expression of doubt behavior and/or the answering behavior, and send the number of students to the teacher device;

所述教师设备用于，展示出现所述握笔记录行为、所述表情疑惑行为和/或所述答题行为的学生数量。The teacher device is used to display the number of students who have performed the pen-holding recording behavior, the puzzled expression behavior and/or the question-answering behavior.
一种课堂笔记生成设备，其特征在于，包括：一个或多个处理器；存储装置，存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-8任一所述的课堂笔记生成方法。A classroom note generating device, characterized in that it comprises: one or more processors; a storage device storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the classroom note generating method as described in any one of claims 1-8.
一种包含计算机可执行指令的存储介质，其特征在于，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-8任一所述的课堂笔记生成方法。A storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the classroom note generation method as described in any one of claims 1-8 when executed by a computer processor.