CN113099254B - Online teaching method, system, equipment and storage medium for regional variable resolution - Google Patents

Online teaching method, system, equipment and storage medium for regional variable resolution Download PDF

Info

Publication number
CN113099254B
CN113099254B CN202110348149.9A CN202110348149A CN113099254B CN 113099254 B CN113099254 B CN 113099254B CN 202110348149 A CN202110348149 A CN 202110348149A CN 113099254 B CN113099254 B CN 113099254B
Authority
CN
China
Prior art keywords
image
area
image information
information
resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110348149.9A
Other languages
Chinese (zh)
Other versions
CN113099254A (en
Inventor
孙高朝
杨福强
涂序文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Penguin Network Technology Co ltd
Original Assignee
Shenzhen Penguin Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Penguin Network Technology Co ltd filed Critical Shenzhen Penguin Network Technology Co ltd
Priority to CN202110348149.9A priority Critical patent/CN113099254B/en
Publication of CN113099254A publication Critical patent/CN113099254A/en
Application granted granted Critical
Publication of CN113099254B publication Critical patent/CN113099254B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/22Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/08Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234345Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a regional variable resolution online teaching method, a system, equipment and a storage medium, wherein the method comprises the following steps: the image acquisition equipment acquires teaching videos, and establishes a video group with n continuous frames in video image information of the teaching videos, wherein n is more than or equal to 2; respectively carrying out portrait image recognition on the pictures in each video group to obtain a first region, carrying out image-text recognition to obtain a second region, overlapping the first region and the second region to obtain partition information for distinguishing a third region from a fourth region except the third region; according to the partition information, obtaining first image information from the image of a third area in each frame of the video group, compressing the image of a fourth area to obtain second image information, and sending the second image information to a receiving end; each image displayed by the receiving end is obtained by combining the first image information and the second image information. The invention can reduce the flow cost of online teaching and is beneficial to online teaching in a lower cost or poorer network environment.

Description

Online teaching method, system, equipment and storage medium for regional variable resolution
Technical Field
The invention relates to the field of education advertisement delivery, in particular to an on-line teaching method, system, equipment and storage medium with variable resolution in different areas.
Background
With rapid development of information technology, especially from the Internet to the mobile Internet, living, working and learning modes crossing space and time are created, and the knowledge acquisition mode is radically changed. The teaching and learning can be free from the limitation of time, space and place conditions, and the knowledge acquisition channel is flexible and diversified. On-line education, i.e., e-Learning, or distance education, on-line Learning, is generally referred to as a network-based Learning behavior in the current concept, and is similar to the network training concept.
The online education platform, i.e. the online training system, is tool software for implementing online training and online education, and is a remote online education college which can be customized and expanded by applying network technology and software technology. The system helps industries or enterprises to quickly build a proprietary knowledge base system through simple and easy courseware, test question importing and manufacturing functions, and provides functions of training requirement investigation, training target setting, course system design, training plan management, training process monitoring, assessment and the like to help clients to efficiently implement staff training and assessment tasks.
The current online education is mainly carried out through a high-resolution video, high bandwidth and flow are needed, but images of a large part of areas in the video do not contribute to teaching, in many cases, the content of instructors and blackboard writing in a picture only occupies about 50% of the picture area, the flow cost is very high under the conditions of completely using the high-resolution and high code rate, and the cost of online education websites is greatly increased. If the low resolution or the low code rate is used, the writing content and the expression, action and the like of the instructor are ambiguous, the learning experience is affected, and a dilemma is caused.
Therefore, the invention provides a regional variable resolution online teaching method, a system, equipment and a storage medium.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide the regional variable resolution online teaching method, system, equipment and storage medium, which overcome the difficulties in the prior art, reduce the flow cost of online teaching and are beneficial to online teaching in a lower cost or poorer network environment.
The embodiment of the invention provides an online teaching method with variable resolution in different areas, which is applied to an online education platform and comprises the following steps:
s101, an image acquisition device acquires teaching videos, a video group is built in video image information of the teaching videos in continuous n frames, and n is more than or equal to 2;
s102, respectively carrying out portrait image recognition on the pictures in each video group to obtain a first area, and carrying out image-text recognition to obtain a second area;
s103, overlapping the first area and the second area to obtain a third area, and obtaining partition information for distinguishing the positions of the third area and the fourth area in a fourth area outside the third area;
s104, according to the partition information, obtaining first image information from the image of a third area in the picture of each frame in the video group, compressing the image of a fourth area to obtain second image information, and sending the second image information to a receiving end;
s105, each image displayed by the receiving end is obtained by combining the first image information and the second image information.
Preferably, in step S102, the first frame of picture in each video group is respectively identified by a portrait image and a picture-text, and a first area of the local image corresponding to the instructor and a second area of the local image corresponding to the blackboard writing are established in the first frame of picture.
Preferably, in the step S103, a third area in the first frame is obtained after the first area and the second area are overlapped, a fourth area except for the third area in the first frame is obtained, and partition information for distinguishing the positions of the third area and the fourth area is created.
Preferably, in the step S103, the original resolution of each frame of the video group is maintained in the first image information.
Preferably, in the step S104, the image of the second image information is compressed, the pixels covered by the second area and the contour are maintained unchanged, the resolution of the picture in the second area is reduced, and the second image information with the same timestamp as the first image information is obtained, and the image resolution of the second image information is lower than the image resolution of the first image information.
Preferably, the image resolution is arranged from high to low in resolution at the following level: 8K, 6K, 4K, 1080P, 720P, 480P, 360P, the level of image resolution of the first image information being higher than the level of image resolution of the second image information.
Preferably, in the step S104, the image of the second image information is compressed, so as to maintain the covered pixels and the contour of the second area unchanged, and reduce the code rate of the picture in the second area.
Preferably, first image information is obtained for the image of the third area in the first frame picture in each video group, and second image information is obtained by compressing the image of the fourth area, and is sent and displayed at the receiving end;
and obtaining first image information of only the images of the third area in the subsequent frame pictures arranged behind the first frame picture in each video group, and sending the first image information to a receiving end, wherein each image displayed by the receiving end is obtained by combining the first image information of the subsequent frame pictures and the second image information of the first frame picture.
Preferably, in the step S102, the image capturing device includes a camera and an infrared sensor for detecting, and the actual distance parameter between each image pixel and the image capturing device is obtained through the coaxial shooting of the infrared sensor and the camera;
and adjusting the range of the first area and the second area by the actual distance parameters of all the image pixels covered by the first area and the actual distance parameters of all the image pixels covered by the second area, which are obtained by portrait image identification, wherein the actual distance parameters of all the image pixels covered by the second area are larger than the actual distance parameters of all the image pixels covered by the first area.
The embodiment of the invention also provides a regional variable resolution online teaching system for realizing the regional variable resolution online teaching method, which comprises the following steps:
the image acquisition module is used for acquiring teaching videos by the image acquisition equipment, and establishing a video group by using continuous n frames in video image information of the teaching videos, wherein n is more than or equal to 2;
the image recognition module is used for respectively carrying out portrait image recognition on the pictures in each video group to obtain a first area and carrying out image-text recognition to obtain a second area;
the image partitioning module is used for overlapping the first area and the second area to obtain a third area, and a fourth area outside the third area to obtain partitioning information for distinguishing the positions of the third area and the fourth area;
the image compression module is used for obtaining first image information from the image of a third area in the picture of each frame in the video group according to the partition information, compressing the image of a fourth area to obtain second image information and transmitting the second image information to a receiving end;
and the image display module is used for combining each image displayed by the receiving end with the first image information and the second image information.
The embodiment of the invention also provides a regional variable resolution online teaching device, which comprises:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the above-described split-area variable resolution online teaching method via execution of the executable instructions.
The embodiment of the invention also provides a computer readable storage medium for storing a program which when executed realizes the steps of the above-mentioned regional variable resolution online teaching method.
According to the method, the system, the equipment and the storage medium for the regional variable resolution online teaching, real-time regional identification can be carried out on the picture, the first image information with original image quality is built based on the region of the instructor and the blackboard writing in the image, the compressed second image information with low image quality is built on other regions in the image, and the video formed by the first image information and the second image information is displayed at the user side, so that the flow cost of the online teaching is greatly reduced on the premise of not reducing the main educational content in the picture, and the online teaching is facilitated in a lower cost or poorer network environment.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings.
FIG. 1 is a flow chart of the split zone variable resolution online teaching method of the present invention.
Fig. 2 to 16 are schematic diagrams of implementation of the split-area variable resolution online teaching method of the present invention.
FIG. 17 is a schematic diagram of the architecture of the zoned variable resolution online teaching system of the present invention.
Fig. 18 is a schematic structural view of the split area variable resolution on-line teaching apparatus of the present invention. And
fig. 19 is a schematic structural view of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus a repetitive description thereof will be omitted.
FIG. 1 is a flow chart of a first split area variable resolution online teaching method of the present invention. As shown in fig. 1, the first split-area variable-resolution online teaching method of the present invention includes the following steps:
s101, the image acquisition equipment acquires teaching videos, a video group is built in video image information of the teaching videos in continuous n frames, and n is more than or equal to 2.
S102, respectively carrying out portrait image recognition on the pictures in each video group to obtain a first area, and carrying out image-text recognition to obtain a second area.
S103, overlapping the first area and the second area to obtain a third area, and obtaining a fourth area outside the third area to obtain partition information for distinguishing the positions of the third area and the fourth area.
S104, obtaining first image information from the image of the third area in the picture of each frame in the video group according to the partition information, compressing the image of the fourth area to obtain second image information, and sending the second image information to the receiving end.
S105, each image displayed by the receiving end is obtained by combining the first image information and the second image information.
The invention aims to establish the first image information with original image quality based on the areas of the instructor and the blackboard writing in the image by carrying out real-time partition identification on the image, and establish the compressed second image information with low image quality for other areas in the image, and display the video formed by the first image information and the second image information together at the user side, thereby greatly reducing the flow cost of online teaching on the premise of not reducing the main education content in the image and being beneficial to carrying out online teaching in lower cost or poorer network environment.
In a preferred embodiment, in step S102, the first frame of picture in each video group is respectively identified by portrait image and graphic context, and a first area corresponding to a local image where the instructor is located and a second area corresponding to a local image where the blackboard writing is located are established in the first frame of picture. The portrait image recognition may use an existing portrait image recognition model to convolve the graphics, etc., to obtain an area about the portrait in the first frame. Similarly, the area related to the text in the first frame of picture is identified by adopting the existing pattern of image-text identification, which is not limited to this.
In a preferred embodiment, in step S103, a third region in the first frame is obtained after the first region and the second region are overlapped, and a fourth region in the first frame excluding the third region is obtained, and partition information for distinguishing the positions of the third region and the fourth region is created. The original resolution of each frame of the video group may be maintained in the first image information. By overlapping the first area and the second area, the complete coverage range of the first area and the second area, that is, the third area where the content (instructor and blackboard writing) with high image quality is needed is obtained, so that the influence of the overlapping condition is avoided, and then the fourth area outside the third area is not very concerned by the instructor, so that the content with low image quality can be displayed, but not limited to.
In a preferred embodiment, in step S104, the image of the second image information is compressed, the pixels covered by the second area and the contour are maintained unchanged, the resolution of the picture in the second area is reduced, and the second image information with the same time stamp as the first image information is obtained, and the image resolution of the second image information is lower than the image resolution of the first image information. The image resolution is arranged from high to low according to the resolution: 8K, 6K, 4K, 1080P, 720P, 480P, 360P, the level of the image resolution of the first image information is higher than the level of the image resolution of the second image information, and may be higher by one level, two levels, three levels, or the like, without being limited thereto. The difference between the first data capacity of the original image data of the second area and the second data capacity of the compressed second image information is the saved data stream, and the larger the difference is, the more obvious the throttling effect is.
In a preferred embodiment, in step S104, the image of the second image information is compressed, and the covered pixels and the contour of the second area are maintained unchanged, so as to reduce the code rate of the picture in the second area. In this embodiment, first image information is obtained for an image of a third area in a first frame of picture in each video group, and second image information is obtained by compressing an image of a fourth area, and is sent and displayed at a receiving end. And obtaining first image information of images of the third area in the subsequent frame pictures arranged behind the first frame picture in each video group, and sending the first image information to a receiving end, wherein each image displayed by the receiving end is obtained by combining the first image information of the subsequent frame pictures and the second image information of the first frame picture. Because only the first image information and the second image information of the first frame picture are transmitted to the receiving end in the video group, and the subsequent frame pictures arranged behind the first frame picture only transmit the first image information to the receiving end, the code rate in the second area is greatly reduced, the data flow is greatly reduced, the speed of video compression processing is accelerated, and the fluency of video is maintained.
In a preferred embodiment, in step S102, the image capturing device includes a camera and an infrared sensor for detecting, and the actual distance parameter between each image pixel and the image capturing device is obtained through the imaging of the infrared sensor and the camera coaxially. And adjusting the range of the first area and the second area by the actual distance parameters of all the image pixels covered by the first area and the actual distance parameters of all the image pixels covered by the second area, which are obtained by the portrait image identification, wherein the actual distance parameters of all the image pixels covered by the second area are larger than the actual distance parameters of all the image pixels covered by the first area. In order to optimize the boundaries between the different recognition areas and also to prevent the instructor from misrecognition of the character-bearing clothing, the accuracy of the extent of the first and second areas is optimized with the aid of infrared sensors.
Fig. 2 to 16 are schematic diagrams of implementation of the split-area variable resolution online teaching method of the present invention. The implementation of the invention is described in detail below with reference to fig. 2 to 16.
As shown in fig. 2, the instructor 12 and the blackboard writing 13 used by the instructor 12 are photographed in a 4K format by the image pickup device 11, real-time online teaching video is generated, and transmitted to the mobile phones 3 of the respective students through the server 4 for viewing.
As shown in fig. 3, a video group is built up from video image information of a teaching video in 12 consecutive frames, and the current online teaching video includes two video groups T100 and T200. Wherein T100 comprises T1 to T12, and total 12 frames of pictures. T200 includes T13 to T24 for a total of 12 frames.
As shown in fig. 4 and 5, taking a first frame of picture T1 in a video group T100 as an example, performing portrait image recognition and picture-text recognition on pictures in each video group, and establishing a first area 21 corresponding to a local image where a instructor 12 is located and a second area 22 corresponding to a local image where an blackboard writing 13 is located in the first frame of picture.
As shown in fig. 6, 7, and 8, the third region 23 is obtained by overlapping the first region 21 and the second region 22, and the fourth region 24 excluding the third region 23 is obtained in the first frame T1, the partition information 10 for distinguishing the third region 23 and the fourth region position 24 is obtained, and a time stamp is generated. Since only 12 consecutive frames of the video group T100 have a short time interval therebetween, the partition information 10 is obtained by sequentially partitioning only the first frame of the video group T100, and the subsequent frame after the first frame is not partitioned but refers to the partition information 10 obtained by the first frame to obtain the third region 23 and the fourth region 24 in each frame, and the first image information 11 and the second image information 12 are generated, respectively. The image of the third area 23 in the picture of each frame in the video group is obtained as the first image information 11 according to the partition information, the image of the fourth area 24 is compressed to obtain the second image information 12, and the second image information 12 is compressed to 1080P format, and the partition information 10, the first image information 11, the second image information 12 are transmitted to the receiving end 3. Each frame of picture displayed by the receiving end 3 is combined by the first image information 11 and the second image information 12 having the same time stamp so as to ensure the accuracy of the image combination.
Referring to the flow of the video group T100 in the video group T200, as shown in fig. 9 to 13, first, a first area 21 corresponding to a partial image where the instructor 12 is located and a second area 22 corresponding to a partial image where the blackboard writing 13 is located are created for a first frame T13 in the video group T200 to overlap the first area 21 and the second area 22 to obtain a third area 23, and a fourth area 24 excluding the third area 23 in the first frame T13 to obtain partition information 10 that distinguishes the third area 23 and the fourth area position 24, and a time stamp is generated. The image of the third area 23 in the picture of each frame in the video group is obtained as the first image information 11 according to the partition information, the image of the fourth area 24 is compressed to obtain the second image information 12, and the second image information 12 is compressed to 1080P format, and the partition information 10, the first image information 11, the second image information 12 are transmitted to the receiving end 3. Each frame of picture displayed by the receiving end 3 is obtained by combining the first image information 11 and the second image information 12 with the same time stamp, so as to ensure the accuracy of image combination, which is not described herein again.
Referring to fig. 14, in a preferred embodiment, the image capturing device 11 includes a camera and an infrared sensor detection 14, by which an actual distance parameter of each image pixel from the image capturing device 11 is obtained by taking a picture coaxially with the camera. The range of the first area and the second area is adjusted by the actual distance parameters of all image pixels covered by the first area obtained by portrait image identification and the actual distance parameters of all image pixels covered by the second area obtained by graphic identification, since the instructor 12 must stand before the board 13, the actual distance parameters of all image pixels covered by the second area are larger than the actual distance parameters of all image pixels covered by the first area. In order to optimize the boundary between different recognition areas and also to prevent the instructor 12 from misrecognition of the clothing with characters, with the aid of the infrared sensor, the part of the second area behind the instructor 12 that is in the same plane and has the largest area is determined as the optimized second area, so that the accuracy of the ranges of the first area and the second area is optimized.
Referring to fig. 15 and 16, in a preferred embodiment, the data stream may be further reduced by changing the code rate of the local area of the picture, and the partition information 10, the first image information 11, and the second image information 12 are obtained for each video group by arranging the first frame picture T1, and the process is as described above and will not be repeated here. Then, the first image information 11 is obtained for only the image of the third region 23 (without any processing of the fourth region 24) in each of the subsequent frames T2 to T12 arranged after the first frame T1 in each video group, and is transmitted to the receiving terminal 3, and each image displayed by the receiving terminal 3 is obtained by combining the first image information 11 of each of the subsequent frames T2 to T12 and the second image information 12 of the first frame. Because only the first image information and the second image information of the first frame T1 picture are transmitted to the receiving end 3 in the video group, and the subsequent frames T2 to T12 pictures arranged after the first frame T1 picture only transmit the first image information to the receiving end 3, the code rate in the second area is greatly reduced, the data flow is greatly reduced, the speed of video compression processing is accelerated, and the overall fluency of the video is maintained.
Fig. 17 is a schematic diagram of the architecture of the zoned variable resolution online teaching system 5 of the present invention. As shown in fig. 17, the embodiment of the present invention further provides a regional variable resolution online teaching system 5, which is configured to implement the above-mentioned regional variable resolution online teaching method, where the regional variable resolution online teaching system 5 includes:
the image acquisition module 51 is used for acquiring teaching videos by image acquisition equipment, and establishing a video group by using continuous n frames in video image information of the teaching videos, wherein n is more than or equal to 2;
the image recognition module 52 is used for respectively carrying out portrait image recognition on the pictures in each video group to obtain a first area and carrying out image-text recognition to obtain a second area;
the image partitioning module 53, overlapping the first area and the second area to obtain a third area, and obtaining partition information for distinguishing the positions of the third area and a fourth area except the third area;
the image compression module 54 obtains first image information from the image of the third area in the picture of each frame in the video group according to the partition information, compresses the image of the fourth area to obtain second image information, and sends the second image information to the receiving end;
the image display module 55 combines each image displayed on the receiving side with the first image information and the second image information.
In a preferred embodiment, the first frame of picture in each video group is subjected to portrait image recognition and graphic recognition respectively, and a first area corresponding to the local image of the instructor and a second area corresponding to the local image of the blackboard writing are established in the first frame of picture.
In a preferred embodiment, the third region in the first frame is obtained after the first region and the second region are overlapped, and the fourth region in the first frame excluding the third region is obtained, and partition information for distinguishing the positions of the third region and the fourth region is created.
In a preferred embodiment, the original resolution of each frame of the video group is maintained in the first image information.
In a preferred embodiment, the image of the second image information is compressed, the pixels covered by the second area and the contour are maintained unchanged, the resolution of the picture in the second area is reduced, the second image information with the same time stamp as the first image information is obtained, and the image resolution of the second image information is lower than the image resolution of the first image information.
In a preferred embodiment, the image resolution is ranked at a level of resolution from high to low: 8K, 6K, 4K, 1080P, 720P, 480P, 360P, the level of image resolution of the first image information is higher than the level of image resolution of the second image information.
In a preferred embodiment, the image of the second image information is compressed, the covered pixels and the contour of the second area are maintained unchanged, and the code rate of the picture in the second area is reduced.
In a preferred embodiment, first image information is obtained for the image of the third area in the first frame picture in each video group, and second image information is obtained by compressing the image of the fourth area, and is sent and displayed on the receiving end;
and obtaining first image information of images of the third area in the subsequent frame pictures arranged behind the first frame picture in each video group, and sending the first image information to a receiving end, wherein each image displayed by the receiving end is obtained by combining the first image information of the subsequent frame pictures and the second image information of the first frame picture.
In a preferred embodiment, the image acquisition device comprises a camera and an infrared sensor for detection, and the actual distance parameter between each image pixel and the image acquisition device is obtained through the coaxial shooting of the infrared sensor and the camera;
and adjusting the range of the first area and the second area by the actual distance parameters of all the image pixels covered by the first area and the actual distance parameters of all the image pixels covered by the second area, which are obtained by the portrait image identification, wherein the actual distance parameters of all the image pixels covered by the second area are larger than the actual distance parameters of all the image pixels covered by the first area.
The regional variable-resolution online teaching system 5 can establish the first image information of original image quality based on the region of the instructor and the blackboard writing in the image by carrying out real-time regional identification on the picture, establish the compressed second image information of low image quality for other regions in the image, and display the video formed by the first image information and the second image information together at the user side, thereby greatly reducing the flow cost of online teaching on the premise of not reducing the main educational content in the picture and being beneficial to carrying out online teaching in lower cost or poorer network environment.
The embodiment of the invention also provides the regional variable-resolution online teaching equipment which comprises a processor. A memory having stored therein executable instructions of a processor. Wherein the processor is configured to execute the steps of the split area variable resolution online teaching method via execution of the executable instructions.
As described above, this embodiment can create the first image information of original image quality based on the region of the instructor and the blackboard writing in the image by performing real-time partition recognition on the screen, create the compressed second image information of low image quality for the other region in the image, and display the video composed of the first image information and the second image information together at the user side, thereby greatly reducing the flow cost of online teaching without reducing the main educational content in the screen, and being beneficial to online teaching in a lower cost or worse network environment.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" platform.
Fig. 18 is a schematic structural view of the split area variable resolution on-line teaching apparatus of the present invention. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 18. The electronic device 600 shown in fig. 18 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 18, the electronic device 600 is in the form of a general purpose computing device. Components of electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including memory unit 620 and processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code executable by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above-described electronic prescription flow processing method section of the present specification. For example, the processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 6201 and/or cache memory unit 6202, and may further include Read Only Memory (ROM) 6203.
The storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 600, and/or any device (e.g., router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 650. Also, electronic device 600 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage platforms, and the like.
The embodiment of the invention also provides a computer readable storage medium for storing a program, and the steps of the regional variable resolution online teaching method are realized when the program is executed. In some possible embodiments, the aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the electronic prescription stream processing method section of this specification, when the program product is run on the terminal device.
As described above, this embodiment can create the first image information of original image quality based on the region of the instructor and the blackboard writing in the image by performing real-time partition recognition on the screen, create the compressed second image information of low image quality for the other region in the image, and display the video composed of the first image information and the second image information together at the user side, thereby greatly reducing the flow cost of online teaching without reducing the main educational content in the screen, and being beneficial to online teaching in a lower cost or worse network environment.
Fig. 19 is a schematic structural view of a computer-readable storage medium of the present invention. Referring to fig. 19, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In summary, the method, the system, the equipment and the storage medium for the regional variable resolution online teaching can be used for establishing the first image information of original image quality based on the region of a teacher and a blackboard writing in an image through real-time regional identification of the picture, establishing the compressed second image information of low image quality for other regions in the image, and displaying the video formed by the first image information and the second image information together at the user side, so that the flow cost of online teaching is greatly reduced on the premise of not reducing the main educational content in the picture, and the online teaching is beneficial to being performed in lower cost or poorer network environment.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.

Claims (11)

1. The regional variable-resolution online teaching method is applied to an online education platform and is characterized by comprising the following steps of:
s101, acquiring teaching videos by image acquisition equipment, and establishing a video group with n frames in video image information of the teaching videos, wherein n is more than or equal to 2, wherein the teaching videos are real-time online teaching videos generated by shooting a teacher through the image acquisition equipment and writing on a blackboard used by the teacher, the image acquisition equipment comprises a camera and an infrared sensor for detection, and the real distance parameters of each image pixel and the image acquisition equipment are obtained through the detection through the coaxial shooting of the infrared sensor and the camera;
s102, respectively carrying out portrait image recognition on pictures in each video group to obtain a first area, carrying out image-text recognition to obtain a second area, wherein the first area is a teacher area in the pictures, the second area is a blackboard writing area in the pictures, and the actual distance parameters of all image pixels covered by the first area and the actual distance parameters of all image pixels covered by the second area obtained through portrait image recognition are adjusted to be larger than the actual distance parameters of all image pixels covered by the first area;
s103, overlapping the first area and the second area to obtain a third area, and obtaining partition information for distinguishing the positions of the third area and the fourth area in a fourth area outside the third area;
s104, according to the partition information, obtaining first image information from the image of a third area in the picture of each frame in the video group, compressing the image of a fourth area to obtain second image information, and sending the second image information to a receiving end; and
s105, each image displayed by the receiving end is obtained by combining the first image information and the second image information.
2. The method according to claim 1, wherein in step S102, the first frame of each video group is respectively identified by a portrait image and a graphic, and a first area of a local image corresponding to the instructor and a second area of a local image corresponding to the blackboard writing are established in the first frame.
3. The method for on-line teaching of regional variable resolution according to claim 1, wherein in step S103, a third region in the first frame is obtained after the first region and the second region are overlapped, a fourth region in the first frame excluding the third region is obtained, and partition information for distinguishing the positions of the third region and the fourth region is created.
4. The method according to claim 1, wherein in step S103, the original resolution of each frame of the video group is maintained in the first image information.
5. The method according to claim 1, wherein in step S104, the image of the second image information is compressed, the pixels and the contours covered by the second area are maintained unchanged, the resolution of the picture in the second area is reduced, and the second image information with the same timestamp as the first image information is obtained, wherein the image resolution of the second image information is lower than the image resolution of the first image information.
6. The method for on-line teaching of regional variable resolution according to claim 5, wherein the image resolution is arranged in a level from high to low according to the resolution: 8K, 6K, 4K, 1080P, 720P, 480P, 360P, the level of image resolution of the first image information being higher than the level of image resolution of the second image information.
7. The method according to claim 2, wherein in step S104, the image of the second image information is compressed, the covered pixels and the contour of the second area are maintained unchanged, and the code rate of the picture in the second area is reduced.
8. The method for regional variable resolution online teaching of claim 7,
obtaining first image information from the images of the third area in the first frame of picture in each video group, compressing the images of the fourth area to obtain second image information, and sending and displaying the second image information on the receiving end;
and obtaining first image information of only the images of the third area in the subsequent frame pictures arranged behind the first frame picture in each video group, and sending the first image information to a receiving end, wherein each image displayed by the receiving end is obtained by combining the first image information of the subsequent frame pictures and the second image information of the first frame picture.
9. An on-line teaching system for regional variable resolution, which is used for realizing the on-line teaching method for regional variable resolution according to claim 1, and is characterized by comprising the following steps:
the image acquisition device acquires teaching videos, a video group is established in video image information of the teaching videos in continuous n frames, and n is more than or equal to 2, wherein the teaching videos are real-time online teaching videos generated by shooting a teacher through the image acquisition device and writing on a blackboard used by the teacher, the image acquisition device comprises a camera and an infrared sensor for detection, and the real distance parameters of each image pixel and the image acquisition device are obtained through the detection through the coaxial shooting of the infrared sensor and the camera;
the image recognition module is used for respectively carrying out portrait image recognition on the pictures in each video group to obtain a first area and carrying out image-text recognition to obtain a second area, wherein the first area is a instructor area in the pictures, the second area is a board writing area in the pictures, the actual distance parameters of all image pixels covered by the first area and the actual distance parameters of all image pixels covered by the second area are obtained through portrait image recognition, the ranges of the first area and the second area are adjusted, and the actual distance parameters of all image pixels covered by the second area are larger than the actual distance parameters of all image pixels covered by the first area;
the image partitioning module is used for overlapping the first area and the second area to obtain a third area, and a fourth area outside the third area to obtain partitioning information for distinguishing the positions of the third area and the fourth area;
the image compression module is used for obtaining first image information from the image of a third area in the picture of each frame in the video group according to the partition information, compressing the image of a fourth area to obtain second image information and transmitting the second image information to a receiving end;
and the image display module is used for combining each image displayed by the receiving end with the first image information and the second image information.
10. An on-line teaching device with variable resolution in different areas, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the split area variable resolution online teaching method of any of claims 1 to 8 via execution of the executable instructions.
11. A computer-readable storage medium storing a program, wherein the program when executed implements the steps of the split-area variable resolution online teaching method according to any one of claims 1 to 8.
CN202110348149.9A 2021-03-31 2021-03-31 Online teaching method, system, equipment and storage medium for regional variable resolution Active CN113099254B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110348149.9A CN113099254B (en) 2021-03-31 2021-03-31 Online teaching method, system, equipment and storage medium for regional variable resolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110348149.9A CN113099254B (en) 2021-03-31 2021-03-31 Online teaching method, system, equipment and storage medium for regional variable resolution

Publications (2)

Publication Number Publication Date
CN113099254A CN113099254A (en) 2021-07-09
CN113099254B true CN113099254B (en) 2023-10-17

Family

ID=76671772

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110348149.9A Active CN113099254B (en) 2021-03-31 2021-03-31 Online teaching method, system, equipment and storage medium for regional variable resolution

Country Status (1)

Country Link
CN (1) CN113099254B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114241521A (en) * 2021-12-13 2022-03-25 北京华夏电通科技股份有限公司 Method, device and equipment for identifying court trial video picture normal area
CN114915771A (en) * 2022-04-26 2022-08-16 深圳市企鹅网络科技有限公司 Online teaching method, system, equipment and storage medium based on image superposition

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107944351B (en) * 2017-11-07 2020-08-04 深圳市易成自动驾驶技术有限公司 Image recognition method, image recognition device and computer-readable storage medium
CN110572579B (en) * 2019-09-30 2021-09-14 联想(北京)有限公司 Image processing method and device and electronic equipment
CN111242962A (en) * 2020-01-15 2020-06-05 中国平安人寿保险股份有限公司 Method, device and equipment for generating remote training video and storage medium
CN111787398A (en) * 2020-06-24 2020-10-16 浙江大华技术股份有限公司 Video compression method, device, equipment and storage device
CN111741274B (en) * 2020-08-25 2020-12-29 北京中联合超高清协同技术中心有限公司 Ultrahigh-definition video monitoring method supporting local amplification and roaming of picture

Also Published As

Publication number Publication date
CN113099254A (en) 2021-07-09

Similar Documents

Publication Publication Date Title
CN113099254B (en) Online teaching method, system, equipment and storage medium for regional variable resolution
CN111242962A (en) Method, device and equipment for generating remote training video and storage medium
US11812152B2 (en) Method and apparatus for controlling video frame image in live classroom
US11599974B2 (en) Joint rolling shutter correction and image deblurring
CN103794089A (en) Online teaching method and device with photo taking function
WO2023125181A1 (en) Image processing method and apparatus, electronic device, and storage medium
CN114267213B (en) Real-time demonstration method, device, equipment and storage medium for practical training
CN114007098B (en) Method and device for generating 3D holographic video in intelligent classroom
CN110933350A (en) Electronic cloud mirror recording and broadcasting system, method and device
CN105721837A (en) Student self-adaptive learning system and method
CN111161592B (en) Classroom supervision method and supervising terminal
CN115460353B (en) Teaching tracking camera equipment configuration method and device, electronic equipment and medium
CN114095747B (en) Live broadcast interaction system and method
CN112291445B (en) Image processing method, device, equipment and storage medium
CN109257546A (en) A kind of image display method, device, electronic equipment and medium
CN111787226B (en) Remote teaching method, device, electronic equipment and medium
CN112116852B (en) Remote experiment method, device and terminal equipment
CN103533384A (en) Image processing method, image restoration method, devices and system
CN210072615U (en) Immersive training system and wearable equipment
CN115410232B (en) Blackboard writing snapshot method and device, electronic equipment and readable storage medium
CN113141464B (en) Camera control method, device, medium and electronic equipment
CN210694201U (en) Double-screen live broadcast acquisition system and double-screen live broadcast playing system
CN210119873U (en) Supervision device based on VR equipment
CN111415635B (en) Large-screen display method, device, medium and electronic equipment
CN112732381B (en) Desktop data acquisition method and system for online class

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20220317

Address after: 518057 1801, block B, building 1, Shenzhen International Innovation Valley, Dashi 1st Road, Xili community, Xili street, Nanshan District, Shenzhen, Guangdong

Applicant after: SHENZHEN PENGUIN NETWORK TECHNOLOGY Co.,Ltd.

Address before: 200030 unit 01, room 801, 166 Kaibin Road, Xuhui District, Shanghai

Applicant before: Shanghai Ping An Education Technology Co.,Ltd.

GR01 Patent grant
GR01 Patent grant