CN114241549A - Remote reading accompanying system - Google Patents
Remote reading accompanying system Download PDFInfo
- Publication number
- CN114241549A CN114241549A CN202111382974.7A CN202111382974A CN114241549A CN 114241549 A CN114241549 A CN 114241549A CN 202111382974 A CN202111382974 A CN 202111382974A CN 114241549 A CN114241549 A CN 114241549A
- Authority
- CN
- China
- Prior art keywords
- face
- student
- learning
- module
- teacher
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a remote reading accompanying system, which comprises: the first camera module is arranged at the student end and used for acquiring a first face image of the student end; the second camera module is arranged at the student end and used for acquiring a learning scene image of the student end; the transmission module is arranged at the student end and used for realizing browsing of learning videos and traversal and uploading of student data and problems of the students based on the Flex technology of J2 EE; a teacher end for: acquiring a learning interface, a first face image and a learning scene image of a student end in real time; determining reply information according to the data and the questions of the students, and returning the reply information to the student end; and the family terminal is used for acquiring the interactive information of the teacher and the students and the learning information of the students in real time. The system can be used for carrying out targeted teaching according to the states of students, is convenient for the students to learn on line, is beneficial for parents to know the interactive information between the students and teachers and the learning information of the students, and improves the teaching effect.
Description
Technical Field
The invention relates to the technical field of remote education, in particular to a remote accompanying reading system.
Background
Distance education, also called modern distance education as network education, is performed in some documents that the education department has already finished. The teaching mode is a teaching mode using transmission media such as televisions, the Internet and the like, breaks through the boundary of time and space, and is different from the traditional teaching mode in school accommodation. Students using this teaching mode can attend class anytime and anywhere because they do not need to attend class at a specific place. Students can also learn with different channels through television broadcasting, internet, guidance lines, classmates, professors (letters), and the like, which is a new concept generated after the modern information technology is applied to education, namely, education developed by using network technology and environment. The convenience brought to people by various advantages of distance education also prompts people to explore the field more deeply. Network education based on different platforms and different development tools is endless.
The reading accompanying system is developed on the basis of remote education, and is an education form that students and teachers, students and education organizations mainly adopt various media ways to carry out system teaching and communication, and parents can participate in the teaching accompanying system in the media way. The accompanying reading system refers to education in which courses are delivered outside the campus via audio, video (live or video) and computer technologies including real-time and non-real-time.
In the prior art, a teacher can not carry out targeted teaching according to the state of a student, the teaching effect is poor, planning learning can not be carried out according to the interest and hobbies of the student, the online learning of the student is not facilitated, and the parents can not know the interactive information of the student and the teacher and the learning information of the student.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the art described above. Therefore, the invention aims to provide a remote reading accompanying system, which can carry out targeted teaching according to the states of students, is convenient for the students to learn on line, is also beneficial for parents to know the interactive information between the students and teachers and the learning information of the students, and improves the teaching effect.
In order to achieve the above object, an embodiment of the present invention provides a remote reading accompanying system, including:
the first camera module is arranged at the student end and used for acquiring a first face image of the student end;
the second camera module is arranged at the student end and used for acquiring a learning scene image of the student end;
the transmission module is arranged at the student end and used for realizing the browsing of learning videos and the traversal and uploading of student data and problems of students based on the F lex technology of J2 EE;
a teacher end for:
acquiring a learning interface, a first face image and a learning scene image of a student end in real time;
determining reply information according to the data and the questions of the students, and returning the reply information to the student end;
and the family terminal is used for acquiring the interactive information of the teacher and the students and the learning information of the students in real time.
According to some embodiments of the present invention, the teacher end performs reading-accompanying AI intelligent analysis on the first face image, performs face detection and analysis, facial feature positioning, face search, and face comparison on the first face image based on a face recognition system, determines a face feature in the first face image, determines a face frame on the first face image, and sends an alarm prompt when it is determined that the face feature is absent in the face frame.
According to some embodiments of the invention, the interaction between the student side and the teacher side is implemented based on a Java + Spring architecture.
According to some embodiments of the present invention, the Spring architecture includes a Spring Core module, a Spring AOP module, a Spring ORM module, a Spring DAO module, a Spring Web module, a Spring Context module, and a Spring Web MVC module.
According to some embodiments of the present invention, the business architecture of the face recognition system comprises:
sample marking, including a face recognition sample tool, face region detection sample marking and face characteristic point calibration sample marking;
model training, including human face region detection model testing, human face region detection model training, human face characteristic point calibration model evaluation, human face characteristic point calibration model training, human face comparison model evaluation and human face comparison model training;
and the model application comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
According to some embodiments of the invention, the technical architecture of the face recognition system comprises:
the infrastructure layer comprises a CPU, a GPU, a cloud computing and big data;
a learning framework layer comprising machine learning, deep learning and computer vision library;
the algorithm model layer comprises a human face region detection algorithm model, a human face feature point detection algorithm model, a human face alignment algorithm model and a human face living body detection algorithm model;
the computer vision technology layer comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
According to some embodiments of the invention, the application architecture of the face recognition system comprises:
the system comprises a client, a face acquisition module and a face registration module, wherein the face acquisition module is used for realizing a face acquisition function and a face registration function, and the face acquisition function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face identification interface calling; the face registration function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face registration interface calling;
the server is used for realizing the human face characteristic point detection function, the human face characteristic point alignment function, the human face comparison function, the blink recognition function and the mouth opening recognition function and providing a human face recognition service interface and a human face registration service interface;
and the data terminal is used for realizing management and maintenance of data resources and model resources, and comprises a registered first face image library, a registered face label library, a face region detection model, a face characteristic point labeling model and a face comparison model.
In one embodiment, the method further comprises:
the first building module is used for obtaining first login information of the student end and building a first block node of a first block chain according to the first login information;
the first determining module is used for determining student behaviors according to the learning interface, the first face image and the learning scene image of the student end and recording the student behaviors on a first block node;
the second building module is used for obtaining second login information of the teacher end and building a second block node of a second block chain according to the second login information;
the second determining module is used for determining teacher behaviors according to the teaching interface of the teacher end, the second face image and the teaching scene image and recording the teacher behaviors on a second block node;
the third determining module is used for extracting the characteristics of the student behaviors stored in the first block node and determining student characteristic vectors;
the fourth determining module is used for extracting the characteristics of the teacher behaviors stored in the second block nodes and determining teacher characteristic vectors;
and the cross-chain interaction module is used for enabling the student characteristic vectors to participate in the second block chain, matching the student characteristic vectors with the teacher characteristic vectors on the second block nodes of the second block chain to obtain a plurality of matching degrees, and taking the teacher corresponding to the teacher characteristic vector with the highest matching degree as a target teacher of the student.
In one embodiment, the method further comprises:
the fifth determining module is used for acquiring a plurality of student characteristic vectors, performing cluster analysis on the plurality of student characteristic vectors, dividing the plurality of student characteristic vectors into a plurality of cluster sets, and taking the cluster set with the largest number of student characteristic vectors as a target cluster set;
the sixth determining module is used for determining the clustering center of the target clustering set and determining the feature vector of the target student according to the clustering center;
the seventh determining module is used for acquiring teacher behaviors of the current teacher and determining feature vectors of the current teacher;
the eighth determining module is used for calculating the matching degree of the target student characteristic vector and the current teacher characteristic vector, and sending prompt information to the teacher end when the matching degree is determined to be smaller than the preset matching degree;
and the teacher end corrects the teaching behavior according to the feature vectors of the target students.
In one embodiment, the method further comprises:
the system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for acquiring learning course information in a preset time period, establishing a learning management model according to the learning course information, and dividing the learning management model according to the types of learning courses for marking;
the simulation module is used for inputting the learning characteristic vector of the student end and the course plan of the student into a pre-trained learning simulation model and outputting the completion degree and the concentration degree of the student on the course plan;
the extraction module is used for replanning the curriculum plan according to the completion degree and the concentration degree of the student on the curriculum plan, determining a target curriculum plan and extracting preset learning progress information;
and the marking module is used for determining actual learning progress information according to a learning interface of a student end, comparing the actual learning progress information with preset learning progress information, determining a learning plan with delayed progress according to a comparison result, and marking different early warning colors in the learning management model.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a block diagram of a remote companion reading system according to one embodiment of the invention;
FIG. 2 is a schematic diagram of Flex technology according to one embodiment of the invention;
FIG. 3 is a schematic view of a Spring frame according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of the business architecture of a face recognition system according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of the technical architecture of a face recognition system according to one embodiment of the invention;
FIG. 6 is a schematic diagram of a remote companion reading system architecture according to one embodiment of the invention;
fig. 7 is a schematic diagram of an application architecture of a face recognition system according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
As shown in fig. 1, an embodiment of the present invention provides a remote reading accompanying system, including:
the first camera module is arranged at the student end and used for acquiring a first face image of the student end;
the second camera module is arranged at the student end and used for acquiring a learning scene image of the student end;
the transmission module is arranged at the student end and used for realizing browsing of learning videos and traversal and uploading of student data and problems of the students based on the Flex technology of J2 EE;
a teacher end for:
acquiring a learning interface, a first face image and a learning scene image of a student end in real time;
determining reply information according to the data and the questions of the students, and returning the reply information to the student end;
and the family terminal is used for acquiring the interactive information of the teacher and the students and the learning information of the students in real time.
The working principle and the beneficial effects of the technical scheme are as follows: the Internet technology is used, so that online learning of students, online tutoring of teachers, real-time online monitoring of parents are facilitated, indexes of students use the Flex technology of J2EE, the students can browse videos to be learned conveniently, and information and problems of the students can be uploaded in a traversing manner; the teacher end indexes are that a learning interface, a first face image and a learning scene image of a student are seen in real time, and the learning scene image comprises a picture written by a student desk; the index of the captain end is to see the interactive picture of the teacher and the students and the picture of the learning condition of the students in real time, and the captain end can also consult with the teacher in a one-to-one way when having problems. Student end, with the help of professional hardware and live technical advantage, through scientific and technological enabling education, break geographical limitation, let more students enjoy higher-quality education, through the authority management and control, the speed of counting deducting, the detection student is on the net, and the network video of seeing in guaranteeing to attend a lesson is smooth and easy, and the network video class divide into the high definition, and the standard definition is ordinary, can supply the student to independently select automatically, makes panorama immersive study experience, takes the mysterious of student's guidance knowledge. Teacher's end, the long-range online companion of subject teacher reads, and professional point is dialled, and the teacher knows that the student learns the problem and accomplishes thousand people thousand faces to different students, helps the student to master the thinking method, and the real academic meeting is understood, lets the student only learn not, only practises this practise, finishes on class, and the teacher continues to plan every student's course after studying. The household terminal can know the learning condition of students in real time and is convenient to contact with accompanying teachers. The system can be used for carrying out targeted teaching according to the states of students, is convenient for the students to learn on line, is beneficial for parents to know the interactive information between the students and teachers and the learning information of the students, and improves the teaching effect.
As shown in fig. 2, the remote reading accompanying system provided by the present invention employs technologies such as JAVA + J2EE + FLEX + Mysql, and J2EE is a set of technical architecture completely different from the traditional application development, and includes many components, which can simplify and standardize the development and deployment of the application system, thereby improving portability, safety and reuse value. The core of the J2EE is a set of technical specifications and guidelines, in which various components, service architectures and technical layers contained in the guidelines all have common standards and specifications, so that various platforms conforming to the J2EE architecture have good compatibility, and the dilemma that information products used at the back end of an enterprise cannot be compatible with each other and the inside or outside of the enterprise is difficult to communicate with each other in the past is solved. The J2EE component differs from the "standard" Java classes in that: it is assembled in a J2EE application, in a fixed format and complying with the J2EE specification, managed by the J2EE server. The J2EE specification defines the J2EE component as follows: client applications and applets are components that run on the client; java servlets and Java Server Pages (JSPs) are Web components that run on the Server side; an Enterprise Java Bean (EJB) component is a business component that runs on the server side. Flex is a presentation server (presentation service) published by Macromedia, which is an application of java web container or net server, and generates a corresponding swf file according to an mxml file (a pure xml description file and an actions script), and transmits the swf file to a client, and the swf file is interpreted and executed by a flash player or a shockwave player of the client, so that the client experiences rich in customer experience and is very suitable for cloud computing.
FLEX has the following advantages:
1) flex is a presentation level solution for enterprise-level rich Internet applications;
2) flex is an application framework;
3) the Flex-series product includes a compiler tool and IDE, and can be viewed using a Flash Player plug-in of a browser by writing MXML (an XML-like markup language) and ActionScript (AS, script language of Flex, ported from Flash) codes, generating a SWF file with the compiler. Almost every user browser has a Flash Player plug-in installed;
4) an XML language (MXML) describing an application program interface;
5) an ECMA standard script language (ActionScript) for processing events of users and systems and constructing a complex data model;
6) a class library;
7) an instant service at runtime;
8) a compiler for generating SWF files from MXML files.
MySQL is a small relational database management system, and supports various operating systems such as FreeBSD, Linux, MAC, Windows and the like and other large databases.
The advantages are as follows:
1. the core thread used by the system is a complete multithread and supports a multiprocessor;
2. there are various column types: 1. 2, 3, 4, and 8 byte length signed/unsigned integers, FLOAT, DOUBLE, CHAR, VARCHAR, TEXT, BLOB, DATE, TIME, DATETIME, TIMETAMMP, YEAR, and ENUM types;
3. it implements the SQL function library through a highly optimized class library and as fast as they can, usually without any memory allocation after query initialization. No memory leak exists;
4. GROUP BY and ORDER BY clauses that fully support SQL, support aggregation functions (COUNT (), COUNT (DISTINCT), AVG (), STD (), SUM (), MAX (), and MIN ()). You can mix tables from different databases in the same query;
5. LEFT0UTER JOIN and ODBC supporting ANSI SQL;
6. all columns have default values. You can INSERT a subset of the list columns with INSERT, those columns that do not have explicit given values set to their province values;
MySQL can work on different platforms. C, C + +, Java, Perl, PHP, Python, and TCL APIs are supported.
According to some embodiments of the present invention, the teacher end performs reading-accompanying AI intelligent analysis on the first face image, performs face detection and analysis, facial feature positioning, face search, and face comparison on the first face image based on a face recognition system, determines a face feature in the first face image, determines a face frame on the first face image, and sends an alarm prompt when it is determined that the face feature is absent in the face frame.
The working principle and the beneficial effects of the technical scheme are as follows: AI intelligent analysis and study convey the teacher end with first face image in real time at the student end, and the teacher end can analyze student's face identification, discovers that the student has the condition of leaving, and the interface of student's control can turn red, has audible alarm and reminds the teacher more than 5 minutes. Concentration is the most main index affecting student achievement and is one of the key factors of human intelligent behaviors. In particular, in recent years, attention of parents to students has been paid widely. Many evaluation methods are proposed for concentration assessment, including questionnaires, physiological observation methods, computer vision and other methods. In the primary and secondary school teaching class, the concentration of students is an important factor influencing the achievement and teaching effect of students, so that the students are concerned by a plurality of parents. At present, most of online learning is the condition of concentration degree in class of students judged subjectively by naked eyes of teachers, the mode not only occupies the class time of teachers and disperses the attention of teachers, but also has low accuracy rate of judgment of naked eyes for students, and the assessment effect of the class effect is poor. In particular, the attention of the students in class cannot be counted and analyzed in real time, thereby generating a certain hysteresis. Under the condition of rapid development of artificial intelligence technology, machine vision is a branch of artificial intelligence and is always a research hotspot in academic circles, the principle of machine learning is that a machine replaces human eyes to acquire information and judge the information, the machine vision converts the acquired information into digital image signals, and information required by researchers is extracted by applying the related technology of digital image processing. The AI face recognition is read in an accompanying manner, a first face image is taken through machine vision, then facial features are analyzed to acquire relevant information and timely correct problems of students in the course of lessons, the contents spoken by a teacher are understood by the students, the teaching effect of the teacher is achieved, the concentration degree of the students is solved, and the whole situation of the students in class can be analyzed through AI intelligent analysis.
In an embodiment, the teaching of the student side and the teacher side is presented in a live video mode, the remote reading accompanying system is connected with Tencent Real-Time audio and video (TRTC), the multi-protocol streaming capability of second-level second, average 400ms first frame Time, RTMP/SRT and the like can be ensured, RTMP is matched with QUIC to effectively reduce 10% of Kadunn, and through the tests of first-line cities and second-line cities of Beijing Shanghai, Guangzhou, Shenzhen, Nanjing, Chengdu, Hangzhou and the like, the Time delay of the teacher and the students can ensure the interactive live broadcast within 1 second, so that the Communication between the teacher and the students is smoother, and the Real-Time video Communication can be basically ensured.
According to some embodiments of the invention, the interaction between the student side and the teacher side is implemented based on a Java + Spring architecture.
The working principle and the beneficial effects of the technical scheme are as follows: the interaction adopts Java + Spring architecture, and Spring is an open source framework and is created for solving the development complexity of enterprise application programs. One of the main advantages of the framework is its layered architecture, which allows customers to choose which component to use, while providing an integrated framework for J2EE application development. The functions of the Spring framework can be used in any J2EE server, most of which are also applicable in an unmanaged environment. The core key points of Spring are as follows: reusable business and data access objects that are not tied to a particular J2EE service are supported, and such objects can be reused across different J2EE environments (Web or EJB), independent applications, test environments.
According to some embodiments of the present invention, the Spring architecture includes a Spring Core module, a Spring AOP module, a Spring ORM module, a Spring DAO module, a Spring Web module, a Spring Context module, and a Spring Web MVC module.
The working principle and the beneficial effects of the technical scheme are as follows: as shown in fig. 3, each of the modules (or components) that make up the Spring framework may exist alone or in combination with one or more other modules. The function of each module is as follows: the Spring Core module is a Core container and provides basic functions of a Spring framework. The main component of the core container is the beans factory, which is a factory-mode implementation. BeanFactory uses a control reversal (IOC) mode to separate the configuration and dependency specifications of an application from the actual application code. The Spring Context module is a configuration file as a Spring Context, and provides Context information to the Spring framework. Spring context includes enterprise services such as JNDI, EJB, email, internationalization, check and scheduling functions. And the Spring AOP module directly integrates the aspect-oriented programming function into a Spring framework through the configuration management characteristics. Therefore, any object managed by the Spring framework can be easily made to support AOP. The Spring AOP module provides transaction management services for objects in Spring-based applications. By using Spring AOP, declarative transaction management can be integrated into an application without relying on EJB components. Spring DAO module: the JDBC DAO abstraction layer provides a meaningful exception hierarchy that can be used to manage exception handling and error messages thrown by different database vendors. The exception hierarchy simplifies error handling and greatly reduces the amount of exception code that needs to be written (e.g., open and close connections). JDBC-oriented exceptions to Spring DAO follow a generic DAO exception hierarchy. Spring ORM module: the Spring frame inserts a plurality of ORM frames, thereby providing an object relation tool of the ORM, wherein the object relation tool comprises JDO, Hibernate and iBatis SQL Map. All of which follow the Spring's generic transaction and DAO exception hierarchy. The spring-web module is a one-stop framework, provides a complete set of solutions from a performance layer (springmvc) to a business layer (spring) and then to a data layer, is a container for managing beans, and can be a general term including many open source items. The Spring-webMVC is a Web framework added on the basis of Spring functions, a ring-Web must be relied on by the ring-webMVC firstly, and the ring-webMVC only provides support for a Spring presentation layer and only provides one open source project.
In one embodiment, after the face is detected based on the reading accompanying AI intelligent analysis, the face can be analyzed to obtain 72 key points such as eye, mouth, nose contours and the like, and the 72 key points can be positioned to accurately identify various face attributes such as sex, age, expression and the like. The technology can adapt to various actual environments such as large-angle side faces, sheltering, blurring and expression change, and provides data basis for concentration degree analysis in the later period.
As shown in fig. 4, according to some embodiments of the present invention, a service architecture of the face recognition system includes:
sample marking, including a face recognition sample tool, face region detection sample marking and face characteristic point calibration sample marking;
model training, including human face region detection model testing, human face region detection model training, human face characteristic point calibration model evaluation, human face characteristic point calibration model training, human face comparison model evaluation and human face comparison model training;
and the model application comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
The beneficial effects of the above technical scheme are that: the face recognition system is accurately constructed and trained, and the recognition accuracy is improved conveniently.
As shown in fig. 5, according to some embodiments of the present invention, the technical architecture of the face recognition system includes:
the infrastructure layer comprises a CPU, a GPU, a cloud computing and big data;
a learning framework layer comprising machine learning, deep learning and computer vision library;
the algorithm model layer comprises a human face region detection algorithm model, a human face feature point detection algorithm model, a human face alignment algorithm model and a human face living body detection algorithm model;
the computer vision technology layer comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
The working principle and the beneficial effects of the technical scheme are as follows: the technical architecture of the face recognition system can be divided into four levels: the system comprises an infrastructure layer, a learning framework layer, an algorithm model layer and a computer vision technology layer. The infrastructure layer mainly comprises a CPU/GPU/cloud computing and big data, wherein the GPU is the highest in correlation degree with the face recognition project, and the corresponding development framework is cuda. The learning framework layer mainly comprises Opencv, Dlib, TensorFlow and Keras related to computer vision. The key technology related to the algorithm model layer mainly comprises a face region detection algorithm model (Hog/CNN), a face feature point detection algorithm model (ResNet/CNN), a face alignment algorithm model, a face verification algorithm model and a living body detection algorithm model. The computer vision technology layer mainly comprises real-time video acquisition, real-time image capture, real-time face detection, real-time face characteristic point calibration, real-time face characteristic point alignment, real-time face comparison, real-time blink recognition, real-time mouth opening recognition and the like. The technical framework of the face recognition system is built through the infrastructure layer, the learning framework layer, the algorithm model layer and the computer vision technical layer, so that the stable operation of the technical framework is ensured, and the reliability is improved.
As shown in fig. 6, the remote reading accompanying system architecture is designed to be divided into a media support layer for supporting streaming media and supporting a plurality of streaming media formats in order to promote the reading accompanying work of teachers, students and parents. A support layer: and system development management, operation deployment and asynchronous communication are supported. Version library: and the CVS is used as a code version management library to realize code version management. The form based on live video is developed, carries out face identification based on AI intelligent analysis, acquires relevant information, realizes teacher, the head of a family's work of accompanying reading, improves online education effect, and then improves student's learning effect.
As shown in fig. 7, according to some embodiments of the present invention, an application architecture of the face recognition system includes:
the system comprises a client, a face acquisition module and a face registration module, wherein the face acquisition module is used for realizing a face acquisition function and a face registration function, and the face acquisition function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face identification interface calling; the face registration function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face registration interface calling;
the server is used for realizing the human face characteristic point detection function, the human face characteristic point alignment function, the human face comparison function, the blink recognition function and the mouth opening recognition function and providing a human face recognition service interface and a human face registration service interface;
and the data terminal is used for realizing management and maintenance of data resources and model resources, and comprises a registered first face image library, a registered face label library, a face region detection model, a face characteristic point labeling model and a face comparison model.
The working principle and the beneficial effects of the technical scheme are as follows: the face recognition system adopts a C/S/D framework and is divided into a client, a server and a data end, so that the application accuracy of the face recognition system is improved.
The invention aims to solve the problems of communication among teachers, students and parents, mutual teaching, accompanying, answering and confusion, and enable teachers to accompany students to become a habit for learning. The system can find out the interest and knowledge points of different students and the teaching contents of thousands of people and thousands of faces, and shows different courses for different students. The learning machine can intelligently analyze learning data to find out knowledge points which are not met by students, the students only practice the learning, a planner teacher is arranged behind the learning machine, the students learn intelligently at home, the students love learning, and the learning is more interesting.
The reading accompanying system judges whether the user needs to be adjusted or not by collecting various learning information of the learning environment of the current user, which is acquired by the intelligent equipment, so as to improve the current learning environment; the learning content and learning process monitoring module identifies and judges the learning content and the learning state of the current user and records the detailed learning process of the user; and denoising and cleaning the collected data at the learning content and learning process monitoring module, and performing index classified storage on the data. Through the intelligent analysis module, according to the data of the data center, intelligent analysis is carried out, knowledge points are associated, knowledge induction is carried out, and the mastering condition of the knowledge points of the user is accurately judged.
The mission of the remote accompanying reading system is scientific and technological enabling, so that learning becomes a good experience. The reading accompanying system becomes the habit of students. The parents are not anxious, and the parents can grow together in the growth process of accompanying the students. Striving to become the leading education intelligent platform in China, being dedicated to providing the independent study integration solution for the student, let the student love to study, let the head of a family relieved eliminate anxiety, promote student's learning efficiency, for school, training institution provides wisdom classroom integration solution, promotes accurate teaching.
In one embodiment, the face recognition is performed by using an open-source or third-party face recognition analysis system, geometric features such as distances, areas, angles and the like are used, and geometric relationships among face features such as eyes, a nose, a mouth and the like are used.
In an embodiment, the attention degree analysis in the AI intelligent analysis and learning is performed by using a neural network algorithm, or by using an algorithm of bayesian, K-nearest neighbor, and support vector machine.
In one embodiment, the method further comprises:
the first building module is used for obtaining first login information of the student end and building a first block node of a first block chain according to the first login information;
the first determining module is used for determining student behaviors according to the learning interface, the first face image and the learning scene image of the student end and recording the student behaviors on a first block node;
the second building module is used for obtaining second login information of the teacher end and building a second block node of a second block chain according to the second login information;
the second determining module is used for determining teacher behaviors according to the teaching interface of the teacher end, the second face image and the teaching scene image and recording the teacher behaviors on a second block node;
the third determining module is used for extracting the characteristics of the student behaviors stored in the first block node and determining student characteristic vectors;
the fourth determining module is used for extracting the characteristics of the teacher behaviors stored in the second block nodes and determining teacher characteristic vectors;
and the cross-chain interaction module is used for enabling the student characteristic vectors to participate in the second block chain, matching the student characteristic vectors with the teacher characteristic vectors on the second block nodes of the second block chain to obtain a plurality of matching degrees, and taking the teacher corresponding to the teacher characteristic vector with the highest matching degree as a target teacher of the student.
The working principle of the technical scheme is as follows: the first building module is used for obtaining first login information of the student end and building a first block node of a first block chain according to the first login information; each first blocknode on the first blockchain corresponds to one first login information, namely, identity information management of each student is realized. The first determining module is used for determining student behaviors according to the learning interface, the first face image and the learning scene image of the student end and recording the student behaviors on a first block node; the second building module is used for obtaining second login information of the teacher end and building a second block node of a second block chain according to the second login information; and each second block node on the second block chain corresponds to one second login information, namely, the identity information management of each teacher is realized. The second determining module is used for determining teacher behaviors according to the teaching interface of the teacher end, the second face image and the teaching scene image and recording the teacher behaviors on a second block node; the third determining module is used for extracting the characteristics of the student behaviors stored in the first block node and determining student characteristic vectors; the fourth determining module is used for extracting the characteristics of the teacher behaviors stored in the second block nodes and determining teacher characteristic vectors; and the cross-chain interaction module is used for enabling the student characteristic vectors to participate in the second block chain, matching the student characteristic vectors with the teacher characteristic vectors on the second block nodes of the second block chain to obtain a plurality of matching degrees, and taking the teacher corresponding to the teacher characteristic vector with the highest matching degree as a target teacher of the student.
The beneficial effects of the above technical scheme are that: based on the block chain technology, the first login information and the second login information are accurately recorded, the student identity information and the teacher identity information are effectively managed, the corresponding relation between the student behaviors and the first block nodes and the corresponding relation between the teacher behaviors and the second block nodes are determined, based on the cross-chain interaction technology, the most appropriate teacher is selected for the students as a target teacher according to the characteristics of the students, the learning interaction times between the target teacher and the students are increased, the reading accompanying effect of the students and the online teaching effect of the students are improved, and the one-to-one teaching effect of the teachers and the students is improved.
In one embodiment, the method further comprises:
the fifth determining module is used for acquiring a plurality of student characteristic vectors, performing cluster analysis on the plurality of student characteristic vectors, dividing the plurality of student characteristic vectors into a plurality of cluster sets, and taking the cluster set with the largest number of student characteristic vectors as a target cluster set;
the sixth determining module is used for determining the clustering center of the target clustering set and determining the feature vector of the target student according to the clustering center;
the seventh determining module is used for acquiring teacher behaviors of the current teacher and determining feature vectors of the current teacher;
the eighth determining module is used for calculating the matching degree of the target student characteristic vector and the current teacher characteristic vector, and sending prompt information to the teacher end when the matching degree is determined to be smaller than the preset matching degree;
and the teacher end corrects the teaching behavior according to the feature vectors of the target students.
The working principle of the technical scheme is as follows: the fifth determining module is used for acquiring a plurality of student characteristic vectors, performing cluster analysis on the plurality of student characteristic vectors, dividing the plurality of student characteristic vectors into a plurality of cluster sets, and taking the cluster set with the largest number of student characteristic vectors as a target cluster set; the sixth determining module is used for determining the clustering center of the target clustering set and determining the feature vector of the target student according to the clustering center; the seventh determining module is used for acquiring teacher behaviors of the current teacher and determining feature vectors of the current teacher; the eighth determining module is used for calculating the matching degree of the target student characteristic vector and the current teacher characteristic vector, and sending prompt information to the teacher end when the matching degree is determined to be smaller than the preset matching degree; and the teacher end corrects the teaching behavior according to the feature vectors of the target students.
The beneficial effects of the above technical scheme are that: when carrying out online teaching to a plurality of students simultaneously based on a teacher, revise the teaching action based on target student's eigenvector, when realizing teaching to a plurality of students, adapt to most of students' study custom, improve online teaching effect.
In one embodiment, the method further comprises:
the system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for acquiring learning course information in a preset time period, establishing a learning management model according to the learning course information, and dividing the learning management model according to the types of learning courses for marking;
the simulation module is used for inputting the learning characteristic vector of the student end and the course plan of the student into a pre-trained learning simulation model and outputting the completion degree and the concentration degree of the student on the course plan;
the extraction module is used for replanning the curriculum plan according to the completion degree and the concentration degree of the student on the curriculum plan, determining a target curriculum plan and extracting preset learning progress information;
and the marking module is used for determining actual learning progress information according to a learning interface of a student end, comparing the actual learning progress information with preset learning progress information, determining a learning plan with delayed progress according to a comparison result, and marking different early warning colors in the learning management model.
The working principle of the technical scheme is as follows: the system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for acquiring learning course information in a preset time period, establishing a learning management model according to the learning course information, and dividing the learning management model according to the types of learning courses for marking; the simulation module is used for inputting the learning characteristic vector of the student end and the course plan of the student into a pre-trained learning simulation model and outputting the completion degree and the concentration degree of the student on the course plan; the extraction module is used for replanning the curriculum plan according to the completion degree and the concentration degree of the student on the curriculum plan, determining a target curriculum plan and extracting preset learning progress information; and the marking module is used for determining actual learning progress information according to a learning interface of a student end, comparing the actual learning progress information with preset learning progress information, determining a learning plan with delayed progress according to a comparison result, and marking different early warning colors in the learning management model.
The beneficial effects of the above technical scheme are that: the method and the system realize accurate management of the actual learning progress information of the students based on the learning management model, are convenient for displaying the comparison result between the actual learning progress information and the preset learning progress information and carrying out early warning prompt, are convenient for the students to adjust the learning plans and the progress of the students in time, and ensure that all learning courses are finished within the preset time period. The learning characteristic vector of the student end and the course plan of the student are identified based on the learning simulation model, the completion degree and the concentration degree of the student on the course plan are accurately determined, then re-planning is carried out, an accurate target course plan is obtained, personalized characteristics are customized according to the individuation of the student, the learning interest of the student is convenient to improve, and the learning effect is improved.
In one embodiment, the method further comprises:
an auto-answer module to:
when the condition of a teacher at a teacher end is determined to be abnormal, problem information of students is obtained, and the problem information is analyzed to obtain a character string;
matching the character strings with preset character strings corresponding to the questions in the database, calculating the matching degree, determining the questions corresponding to the preset character strings with the matching degree larger than the preset matching degree as target questions, acquiring target answers of the target questions and returning the target answers to the student.
Calculating the matching degree of the character string and a preset character string corresponding to the problems in the database, wherein the matching degree comprises the following steps:
wherein, P is the matching degree of the character string and the preset character string; q1The number of semantics included for the string; q2The number of semantics included for the preset string; siIs the ith sub-string in the character string; giThe ith sub-string in the preset character string is selected; d is the semantic distance between the character string and the preset character string; n is the number of the substrings included by the character string and is also the number of the substrings included by the preset character string, and the number of the substrings included by the character string is equal to the number of the substrings included by the preset character string.
The working principle and the beneficial effects of the technical scheme are as follows: an auto-answer module to: when the condition of a teacher at a teacher end is determined to be abnormal, problem information of students is obtained, and the problem information is analyzed to obtain a character string; matching the character strings with preset character strings corresponding to the questions in the database, calculating the matching degree, determining the questions corresponding to the preset character strings with the matching degree larger than the preset matching degree as target questions, acquiring target answers of the target questions and returning the target answers to the student. When the teacher status at the teacher end is determined to be abnormal, specifically, the teacher may suddenly leave something. The problem of the student is answered automatically based on the automatic answering module, so that the problem that the study time of the student is delayed due to abnormity of a teacher end is avoided, the student is answered and puzzled, the study progress of the student is guaranteed, and the study effect is improved. Based on the formula, the matching degree of the preset character strings corresponding to the character strings and the questions in the database is accurately calculated, so that the accuracy of determining the target questions is improved, and the students can be answered accurately.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A remote reading accompanying system, comprising:
the first camera module is arranged at the student end and used for acquiring a first face image of the student end;
the second camera module is arranged at the student end and used for acquiring a learning scene image of the student end;
the transmission module is arranged at the student end and used for realizing browsing of learning videos and traversal and uploading of student data and problems of the students based on the Flex technology of J2 EE;
a teacher end for:
acquiring a learning interface, a first face image and a learning scene image of a student end in real time;
determining reply information according to the data and the questions of the students, and returning the reply information to the student end;
and the family terminal is used for acquiring the interactive information of the teacher and the students and the learning information of the students in real time.
2. The remote reading accompanying system of claim 1, wherein the teacher end performs reading accompanying AI intelligent analysis on the first face image, performs face detection and analysis, facial positioning, face search and face comparison on the first face image based on a face recognition system, determines face features in the first face image, determines a face frame on the first face image, and issues an alarm prompt when it is determined that the face features are absent in the face frame.
3. The remote reading accompanying system of claim 1, wherein the interaction between the student side and the teacher side is implemented based on Java + Spring architecture.
4. The remote reading accompanying system of claim 3, wherein the Spring architecture comprises a Spring Core module, a Spring AOP module, a Spring ORM module, a Spring DAO module, a Spring Web module, a Spring Context module, a Spring Web MVC module.
5. The remote reading accompanying system of claim 2, wherein the business architecture of the face recognition system comprises:
sample marking, including a face recognition sample tool, face region detection sample marking and face characteristic point calibration sample marking;
model training, including human face region detection model testing, human face region detection model training, human face characteristic point calibration model evaluation, human face characteristic point calibration model training, human face comparison model evaluation and human face comparison model training;
and the model application comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
6. The remote reading accompanying system of claim 2, wherein the technical architecture of the face recognition system comprises:
the infrastructure layer comprises a CPU, a GPU, a cloud computing and big data;
a learning framework layer comprising machine learning, deep learning and computer vision library;
the algorithm model layer comprises a human face region detection algorithm model, a human face feature point detection algorithm model, a human face alignment algorithm model and a human face living body detection algorithm model;
the computer vision technology layer comprises real-time feature alignment, real-time video acquisition, real-time face comparison, real-time image snapshot, real-time blink recognition, real-time face detection, real-time mouth opening recognition and real-time feature calibration.
7. The remote reading accompanying system of claim 2, wherein the application architecture of the face recognition system comprises:
the system comprises a client, a face acquisition module and a face registration module, wherein the face acquisition module is used for realizing a face acquisition function and a face registration function, and the face acquisition function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face identification interface calling; the face registration function comprises real-time video acquisition, real-time face region detection, real-time face region snapshot and face registration interface calling;
the server is used for realizing the human face characteristic point detection function, the human face characteristic point alignment function, the human face comparison function, the blink recognition function and the mouth opening recognition function and providing a human face recognition service interface and a human face registration service interface;
and the data terminal is used for realizing management and maintenance of data resources and model resources, and comprises a registered first face image library, a registered face label library, a face region detection model, a face characteristic point labeling model and a face comparison model.
8. The remote reading accompanying system of claim 1, further comprising:
the first building module is used for obtaining first login information of the student end and building a first block node of a first block chain according to the first login information;
the first determining module is used for determining student behaviors according to the learning interface, the first face image and the learning scene image of the student end and recording the student behaviors on a first block node;
the second building module is used for obtaining second login information of the teacher end and building a second block node of a second block chain according to the second login information;
the second determining module is used for determining teacher behaviors according to the teaching interface of the teacher end, the second face image and the teaching scene image and recording the teacher behaviors on a second block node;
the third determining module is used for extracting the characteristics of the student behaviors stored in the first block node and determining student characteristic vectors;
the fourth determining module is used for extracting the characteristics of the teacher behaviors stored in the second block nodes and determining teacher characteristic vectors;
and the cross-chain interaction module is used for enabling the student characteristic vectors to participate in the second block chain, matching the student characteristic vectors with the teacher characteristic vectors on the second block nodes of the second block chain to obtain a plurality of matching degrees, and taking the teacher corresponding to the teacher characteristic vector with the highest matching degree as a target teacher of the student.
9. The remote reading accompanying system of claim 8, further comprising:
the fifth determining module is used for acquiring a plurality of student characteristic vectors, performing cluster analysis on the plurality of student characteristic vectors, dividing the plurality of student characteristic vectors into a plurality of cluster sets, and taking the cluster set with the largest number of student characteristic vectors as a target cluster set;
the sixth determining module is used for determining the clustering center of the target clustering set and determining the feature vector of the target student according to the clustering center;
the seventh determining module is used for acquiring teacher behaviors of the current teacher and determining feature vectors of the current teacher;
the eighth determining module is used for calculating the matching degree of the target student characteristic vector and the current teacher characteristic vector, and sending prompt information to the teacher end when the matching degree is determined to be smaller than the preset matching degree;
and the teacher end corrects the teaching behavior according to the feature vectors of the target students.
10. The remote reading accompanying system of claim 8, further comprising:
the system comprises an establishing module, a processing module and a processing module, wherein the establishing module is used for acquiring learning course information in a preset time period, establishing a learning management model according to the learning course information, and dividing the learning management model according to the types of learning courses for marking;
the simulation module is used for inputting the learning characteristic vector of the student end and the course plan of the student into a pre-trained learning simulation model and outputting the completion degree and the concentration degree of the student on the course plan;
the extraction module is used for replanning the curriculum plan according to the completion degree and the concentration degree of the student on the curriculum plan, determining a target curriculum plan and extracting preset learning progress information;
and the marking module is used for determining actual learning progress information according to a learning interface of a student end, comparing the actual learning progress information with preset learning progress information, determining a learning plan with delayed progress according to a comparison result, and marking different early warning colors in the learning management model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111382974.7A CN114241549A (en) | 2021-11-22 | 2021-11-22 | Remote reading accompanying system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111382974.7A CN114241549A (en) | 2021-11-22 | 2021-11-22 | Remote reading accompanying system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114241549A true CN114241549A (en) | 2022-03-25 |
Family
ID=80750247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111382974.7A Withdrawn CN114241549A (en) | 2021-11-22 | 2021-11-22 | Remote reading accompanying system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241549A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557428A (en) * | 2024-01-11 | 2024-02-13 | 深圳市华视圣电子科技有限公司 | Teaching assistance method and system based on AI vision |
-
2021
- 2021-11-22 CN CN202111382974.7A patent/CN114241549A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557428A (en) * | 2024-01-11 | 2024-02-13 | 深圳市华视圣电子科技有限公司 | Teaching assistance method and system based on AI vision |
| CN117557428B (en) * | 2024-01-11 | 2024-05-07 | 深圳市华视圣电子科技有限公司 | Teaching assistance method and system based on AI vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110991381B (en) | Real-time classroom student status analysis and indication reminding system and method based on behavior and voice intelligent recognition | |
| CN109359215A (en) | Video intelligent method for pushing and system | |
| Bidwell et al. | Classroom analytics: Measuring student engagement with automated gaze tracking | |
| LU100314B1 (en) | Method and system for predicting academic achievements of students based on naive bayesian model | |
| CN106648082A (en) | Intelligent service device capable of simulating human interactions and method | |
| CN112184500A (en) | Extraclass learning tutoring system based on deep learning and knowledge graph and implementation method | |
| CN114693488B (en) | Smart campus management method and system based on 5G network | |
| CN113506624B (en) | Autism children cognitive ability evaluation intervention system based on hierarchical generalization push logic | |
| WO2019169338A1 (en) | Systems and methods for automated content evaluation and delivery | |
| CN109637207A (en) | A kind of preschool education interactive teaching device and teaching method | |
| US20180204480A1 (en) | Cognitive training system | |
| CN107845300B (en) | Real-time monitoring safety examination system | |
| WO2022170985A1 (en) | Exercise selection method and apparatus, and computer device and storage medium | |
| CN113656749A (en) | Intelligent education management method and system based on big data and readable storage medium | |
| CN110245253A (en) | A kind of Semantic interaction method and system based on environmental information | |
| Rama et al. | A study on causal rule discovery with PC algorithm | |
| CN114241549A (en) | Remote reading accompanying system | |
| Villegas-Ch et al. | Identification of emotions from facial gestures in a teaching environment with the use of machine learning techniques | |
| CN110111011A (en) | A kind of quality of instruction monitoring and managing method, device and electronic equipment | |
| Ermatita et al. | Improving student’s cognitive performance during the pandemic through a machine learning-based virtual museum | |
| CN111583739A (en) | VR-based multimedia teaching system | |
| Chen | Application of Mobile Learning in Higher English Education Systems Using Cognitive Web Services. | |
| CN113268512B (en) | Enterprise post professional skill training system based on internet platform | |
| CN111460245A (en) | Multi-dimensional crowd characteristic measuring method | |
| CN115640403A (en) | Knowledge management and control method and device based on knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220325 |