CN111246125B - Multi-channel video stream synthesis method and device - Google Patents
Multi-channel video stream synthesis method and device Download PDFInfo
- Publication number
- CN111246125B CN111246125B CN202010055011.5A CN202010055011A CN111246125B CN 111246125 B CN111246125 B CN 111246125B CN 202010055011 A CN202010055011 A CN 202010055011A CN 111246125 B CN111246125 B CN 111246125B
- Authority
- CN
- China
- Prior art keywords
- motion
- action
- image sequence
- motion image
- matched
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001308 synthesis method Methods 0.000 title description 7
- 230000033001 locomotion Effects 0.000 claims abstract description 269
- 230000009471 action Effects 0.000 claims abstract description 192
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 99
- 238000004590 computer program Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000012360 testing method Methods 0.000 description 8
- 239000000126 substance Substances 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- ZCYVEMRRCGMTRW-UHFFFAOYSA-N 7553-56-2 Chemical compound [I] ZCYVEMRRCGMTRW-UHFFFAOYSA-N 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 239000011630 iodine Substances 0.000 description 3
- 229910052740 iodine Inorganic materials 0.000 description 3
- 239000007788 liquid Substances 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- VZGDMQKNWNREIO-UHFFFAOYSA-N tetrachloromethane Chemical compound ClC(Cl)(Cl)Cl VZGDMQKNWNREIO-UHFFFAOYSA-N 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 239000007864 aqueous solution Substances 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Circuits (AREA)
Abstract
The embodiment of the invention provides a method and a device for synthesizing multi-channel video streams, wherein the method comprises the following steps: acquiring a plurality of paths of video streams; identifying a sequence of action images from the multi-channel video stream; determining a motion image sequence matched with a preset motion combination from the motion image sequences of the multi-channel video stream; and displaying the action image sequence matched with the preset action combination. All actions of a certain examinee and the connection actions among different videos are not required to be searched according to a time axis, all the actions in different videos of the certain examinee are spliced, and the scoring mode that one examinee corresponds to one video is achieved.
Description
Technical Field
The present invention relates to the multimedia technology field, and in particular, to a method and an apparatus for synthesizing multiple video streams.
Background
In the practical training skill examination, an examination mode of recording videos and grading according to the recorded videos is adopted, in the mode, the examination actions of the examinees need to be recorded, and then the actions of the examinees are graded according to the played videos.
In many skill examinations, video recording is needed to be carried out on multiple steps of an examinee, and generally, multiple paths of cameras are adopted for continuous recording, so that a scoring teacher needs to open and repeatedly play back multiple videos recorded by the multiple paths of cameras for multiple times, and only after the joint between the action of the examinee and different steps is determined, one examinee can be scored to obtain a final scoring result; and because video transmission and storage have the time difference's problem, also have inconsistent process between a plurality of videos that multichannel camera recorded, the mr of grading often is difficult to look for all actions of finding a certain examinee in unison according to the time axis, and is difficult to find the linking action between two adjacent camera videos, and to the mr of grading, the process of grading is wasted time and energy again, increases the mr's of grading work load.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a multi-channel video stream composing method and a corresponding multi-channel video stream composing apparatus that overcome or at least partially solve the above problems.
In order to solve the above problem, an embodiment of the present invention discloses a method for synthesizing multiple video streams, where the method includes:
acquiring a plurality of paths of video streams;
identifying a sequence of action images from the multi-channel video stream;
determining a motion image sequence matched with a preset motion combination from motion image sequences of the multi-path video stream;
and displaying the motion image sequence matched with the preset motion combination.
Optionally, the preset action combinations include multiple groups of action combinations; the determining a motion image sequence matched with a preset motion combination from motion image sequences of the multi-path video stream comprises:
determining motion image sequences matched with all groups of motion combinations from motion image sequences of the multi-path video stream, and determining the arrangement sequence of the groups of motion combinations;
and generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations.
Optionally, the determining, from the motion image sequences of the multiple video streams, a motion image sequence matching each set of motion combinations includes:
respectively extracting action image sequences matched with all groups of action combinations from the action image sequences of each path of video stream;
when the motion image sequences matched with the same group of motion combinations comprise motion image sequences from one video stream, determining motion image sequences matched with the same group of motion combinations from the motion image sequences from the one video stream;
and/or when the motion image sequences matched with the same group of motion combinations comprise motion image sequences from at least two video streams, determining the motion image sequences matched with the same group of motion combinations from the motion image sequences from at least two video streams.
Optionally, the determining a motion image sequence matching the same set of motion combinations from the determining the motion image sequences from the at least two video streams includes:
alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
and determining the motion image sequence matched with the same group of motion combinations by adopting the motion image sequence obtained by alternate extraction.
The embodiment of the invention also discloses a device for synthesizing the multi-path video streams, which comprises:
the multi-channel video stream acquisition module is used for acquiring multi-channel video streams;
the motion image sequence identification module is used for identifying a motion image sequence from the multi-channel video stream;
the motion image sequence determining module is used for determining a motion image sequence matched with a preset motion combination from the motion image sequences of the multi-channel video stream;
and the action image sequence display module is used for displaying the action image sequence matched with the preset action combination.
Optionally, the motion image sequence determination module comprises:
the action image sequence determining sub-module is used for determining an action image sequence matched with each group of action combinations from the action image sequences of the multi-path video stream and determining the arrangement sequence of the plurality of groups of action combinations;
and the action image sequence generation sub-module is used for generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations.
Optionally, the motion image sequence determination sub-module includes:
the motion image sequence extraction unit is used for extracting motion image sequences matched with all groups of motion combinations from the motion image sequences of each video stream;
a first motion image sequence determining unit, configured to determine, when a motion image sequence matching a same set of motion combinations includes a motion image sequence from one video stream, a motion image sequence matching the same set of motion combinations from the motion image sequence from the one video stream;
and the second motion image sequence determining unit is used for determining the motion image sequence matched with the same group of motion combinations from the motion image sequences of at least two video streams when the motion image sequence matched with the same group of motion combinations comprises the motion image sequences of at least two video streams.
Optionally, the second motion image sequence determination unit includes:
the motion image sequence extraction subunit is used for alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
and the motion image sequence generation subunit is used for determining the motion image sequence matched with the same group of motion combinations by adopting the motion image sequence obtained by the alternate extraction.
The embodiment of the invention also discloses an electronic device, which comprises: a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program when executed by the processor implementing the steps of the multi-path video stream composition method.
The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program realizes the steps of the multi-path video stream synthesis method when being executed by a processor.
The embodiment of the invention has the following advantages:
the invention provides a method and a device for synthesizing multi-path video streams, which match action image sequences in each path of video stream with each group of action combinations respectively, and synthesize the action image sequences matched with each group of action combinations into one path of video stream, wherein the synthesized video stream is an action image sequence matched with all the action combinations. All actions of a certain examinee and the connection actions among different videos are not required to be searched according to a time axis, all the actions in different videos of the certain examinee are spliced, and the scoring mode that one examinee corresponds to one video is achieved.
Drawings
FIG. 1 is a flow chart illustrating the steps of a first embodiment of a method for synthesizing multiple video streams according to the present invention;
FIG. 2 is a flowchart illustrating the steps of a second embodiment of a method for synthesizing multiple video streams;
fig. 3 is a block diagram of a multi-channel video stream synthesizing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
The core idea of the embodiment of the invention is to use a plurality of cameras to shoot all actions of examinee in the training skill examination process, adopt the modes of picture tracking switching and picture synthesis for pictures in a plurality of paths of video streams shot by the plurality of cameras, track and switch the examinee actions of different training steps, namely, action image sequences matched with the training steps, and synthesize all the actions of examinee training in the plurality of paths of video streams according to different training steps, thereby facilitating the scoring of teachers.
Referring to fig. 1, a flowchart illustrating steps of a first embodiment of a method for synthesizing multiple video streams according to the present invention is shown, which may specifically include the following steps:
the method comprises the steps of adopting a picture tracking switching and picture synthesizing mode to pictures in a plurality of paths of video streams shot by a plurality of cameras, carrying out tracking switching on examinee actions in different practical training steps, namely action image sequences matched with the practical training steps, and synthesizing all actions of the examinee practical training in the paths of video streams according to different practical training steps.
In an embodiment of the present invention, since the frames in the multiple paths of video streams shot by the multiple cameras need to be switched by frame tracking and frame synthesis, the terminal may first obtain the multiple paths of video streams shot by the multiple cameras, where the terminal may receive, in real time, the multiple paths of video streams currently and respectively shot by the multiple cameras at the same time, and may also receive the multiple paths of video streams shot by the multiple cameras at the same time.
in an embodiment of the present invention, after receiving multiple video streams shot by multiple cameras, the terminal may identify multiple sets of motion image sequences in the multiple video streams, so as to match the multiple sets of motion image sequences according to a step order, and determine a motion image sequence corresponding to the step order.
103, determining a motion image sequence matched with a preset motion combination from the motion image sequences of the multiple paths of video streams;
in an embodiment of the present invention, after the terminal identifies the plurality of sets of motion image sequences from the multi-channel video stream, a motion image sequence matching a preset motion combination may be determined from the plurality of sets of motion image sequences. The preset action combination can be an action combination of all practical training steps, so as to determine and display an action image sequence corresponding to all practical training steps. It should be noted that the preset action combination may be an action combination of a single practical training step, an action combination of multiple practical training steps, or an action combination of all practical training steps, which is not limited in this embodiment of the present invention.
And 104, displaying the motion image sequence matched with the preset motion combination.
In an embodiment of the present invention, a terminal may first obtain multiple paths of video streams captured by multiple cameras, and after identifying multiple sets of motion image sequences in the multiple paths of video streams, may determine a motion image sequence matching all motion combinations from the multiple sets of motion image sequences; after determining the motion image sequences matching all the motion combinations, the motion image sequences corresponding to all the training steps may be presented.
The embodiment of the invention respectively matches the action image sequence in each path of video stream with each group of action combination, and synthesizes the action image sequence matched with each group of action combination into one path of video stream, wherein the synthesized video stream is the action image sequence matched with all the action combinations. All actions of a certain examinee and the connection actions among different videos are not required to be searched uniformly according to a time axis, all the actions in different videos of the certain examinee are spliced, and the scoring mode that one examinee corresponds to one video is achieved.
Referring to fig. 2, a flowchart illustrating steps of a second embodiment of the method for synthesizing multiple video streams of the present invention is shown, which may specifically include the following steps:
the multi-camera real-training test system has the advantages that the multiple cameras can be used for shooting the actions of the examinees in the real-training test and generating the multi-channel video streams, the terminal can simultaneously receive the multi-channel video streams shot by the multiple cameras, so that the examinee actions corresponding to the real-training steps can be tracked and synthesized by the multi-channel video streams received at the same time, and one channel of video streams corresponding to the real-training test of a certain examinee is generated. It should be noted that the multiple video streams received by the terminal at the same time may be video streams recorded by multiple cameras in real time, and may also try on video streams recorded by multiple cameras for test takers.
In an embodiment of the invention, a terminal receives multiple paths of video streams shot by multiple cameras at the same time, firstly, the practical training step in the practical training skill examination can be determined, then multiple cameras are installed according to the practical training step, the multiple cameras can record multiple steps of students continuously and respectively at the same time, namely, different videos are recorded by different cameras and are aimed at different actions, and finally, the recorded multiple paths of video streams can be transmitted to the terminal. It should be noted that the number of the installed cameras may be 4 or 8, which is not limited in the present invention; and a plurality of cameras are installed according to the training steps, and can be installed according to different angles or different areas, or can not be installed according to angles or areas, which is not limited in the embodiment of the invention.
after receiving the multiple paths of video streams transmitted by the multiple cameras, the terminal can identify the action image sequences of the multiple paths of video streams, namely, the action video streams in the multiple paths of video streams can be intercepted, so that the action video streams correspond to the practical training steps, and the action video streams are laid out according to the specific practical training steps.
In an embodiment of the present invention, since a video stream is composed of images that are played continuously, one of the images corresponds to one image frame in the video stream, that is, a plurality of image frames may be included in a video stream; then, the method of identifying the motion image sequence of the multi-channel video stream may be identified by an inter-frame difference method.
Firstly, decoding a multi-path video stream, restoring the multi-path video stream into a plurality of groups of initial image frames forming the multi-path video stream, obtaining pixel information of a plurality of images in the plurality of groups of image frames after obtaining the plurality of groups of image frames corresponding to the multi-path video stream, wherein the pixel information can be pixel brightness difference, the number of continuous pixels and the like, and when the pixel information reaches a set threshold value, determining the image frame as an image frame with action; since the video stream is composed of images played continuously, a plurality of continuous image frames with motion determined can also be called as a sequence of images with motion, i.e. a sequence of motion images.
in an embodiment of the invention, after identifying the motion image sequences of the multiple paths of video streams, the terminal can determine the motion image sequences matched with the training step from the motion image sequences of the multiple paths of video streams; the determined motion image sequence matched with the practical training step may be a motion image sequence matched with all practical training steps, that is, the determined motion image sequence is a motion image sequence which is finally laid out according to the practical training steps.
In an embodiment of the present invention, the preset action combination includes a plurality of action combinations; step 203 may include the following sub-steps:
a substep S11, determining the action image sequence matched with each group of action combination from the action image sequences of the multi-channel video stream, and determining the arrangement sequence of the plurality of groups of action combinations;
in an embodiment of the invention, the terminal needs to determine an action image sequence which is matched with all practical training steps and is finally laid out according to the practical training steps from the action image sequences of the multi-path video stream; firstly, the terminal can determine an action image sequence matched with each practical training step from the action image sequences of the multi-channel video streams, and determine the arrangement sequence of each practical training step, so that the determined action image sequences matched with each practical training step are synthesized according to the arrangement sequence of the practical training steps, and finally, the action image sequences which are well laid out according to the practical training steps are obtained.
In one embodiment of the present invention, the sub-step S11 may include the following sub-steps:
a substep S111 of extracting a motion image sequence matched with each group of motion combinations from the motion image sequences of each video stream;
after the terminal identifies each group of action image sequences from each path of video stream, the action image sequences corresponding to each practical training step can be extracted for the group of action image sequences corresponding to a certain video stream; namely, according to each training step, each group of motion image sequences identified in each path of video stream is classified, and the motion image sequences correspond to the training steps.
In one embodiment of the invention, a plurality of cameras are used for recording practical training operation of practical training skill examination of an examinee, and it is assumed that a terminal receives 4 paths of video streams, a motion image sequence identified for a1 st path of video stream is a group A, a motion image sequence identified for a2 nd path of video stream is a group B, a motion image sequence identified for a3 rd path of video stream is a group C, and a motion image sequence identified for a4 th path of video stream is a group D; assuming that the training step is 5 steps in total, and action image sequences corresponding to the 5 steps of training step exist in 4 paths of video streams, and classifying each group of action image sequences identified in each path of video stream, extracting an action image sequence A1 matched with the 1 st step, an action image sequence A2 matched with the 2 nd step, an action image sequence A3 matched with the 3 rd step, an action image sequence A4 matched with the 4 th step and an action image sequence A5 matched with the 5 th step from the A group of action image sequences; similarly, the motion image sequences of other paths of video streams are classified according to the training steps, B1-B5 are extracted from the group B motion image sequences, C1-C5 are extracted from the group C motion image sequences, and D1-D5 are extracted from the group D motion image sequences.
A substep S112, when the motion image sequence matched with the same group of motion combinations comprises a motion image sequence from one video stream, determining a motion image sequence matched with the same group of motion combinations from the motion image sequence from the one video stream;
when the terminal classifies each group of motion image sequences identified in each path of video stream according to the training step, in one case, for a certain training step, when the motion image sequence corresponding to the training step is only from one path of video stream, the motion image sequence from the video stream can be directly determined to be the motion image sequence matched with the training step.
In an embodiment of the present invention, if, for the 5 th training step, only the group B motion image sequences identified by the 2 nd video stream have a motion image sequence corresponding to the 5 th training step, it is directly determined that the motion image sequence matched with the 5 th training step is B5; at this time, the other video streams do not exist in the motion image series corresponding to the training step of step 5, i.e., A5, C5 and D5 do not exist.
And/or, when the motion image sequences matching the same set of motion combinations include motion image sequences from at least two video streams, determining motion image sequences matching the same set of motion combinations from the motion image sequences from the at least two video streams.
In an embodiment of the present invention, when the terminal classifies each group of motion image sequences identified in each video stream according to a practical training step, in another case, for a practical training step, when the motion image sequence corresponding to the practical training step is from at least two video streams, it cannot be directly determined that the motion image sequence in the video stream is a motion image sequence matching the practical training step, but it is required to process the motion image sequence corresponding to the practical training step in the at least two video streams, and determine the processed motion image sequence as the motion image sequence matching the practical training step.
In one embodiment of the present invention, the sub-step S113 may include the following sub-steps:
a substep S1131, alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
the terminal needs to process the action image sequences corresponding to the training step in at least two video streams, and for a certain training step, the processing method can be to alternately extract the action image sequences matched with the training step from at least two video streams so as to splice and synthesize the alternately extracted action image sequences.
In an embodiment of the present invention, the terminal alternately extracts the motion image sequences, and if for the 2 nd training step, the a group motion image sequence identified by the 1 st channel video stream, the B group motion image sequence identified by the 2 nd channel video stream, and the D group motion image sequence identified by the 4 th channel video stream all have motion image sequences corresponding to the 2 nd training step, at this time, it is necessary to alternately extract the motion image sequences A2, B2, and D2 corresponding to the 2 nd training step in each group, so as to splice and synthesize the alternately extracted motion image sequences, and obtain a motion image sequence matched with the 2 nd training step.
The terminal can play the motion image sequences A2, B2 and D2 at the same time, place A2 in the main picture, and when the playing time of A2, B2 and D2 reaches 5s, if B2 and D2 are not played completely, that is, other motions continue, switch the main picture to play B2 after 5 s; at this time, if the D2 has not been played, the main picture is switched to the D2 played for 10 seconds; judging once every second, switching once every 5 seconds, and switching main pictures in turn for A2, B2 and D2 if other actions continue; finally, the action image sequences alternately extracted on the main picture can be spliced and synthesized to generate an action image sequence matched with the 2 nd training step. If the motion stops in the middle, that is, if the motion image sequence A2, B2 or D2 is played completely in the middle, the motion time needs to be recalculated; and the motion image sequences which are alternately extracted are spliced and synthesized, and the time error of splicing and synthesizing between two different motion image sequences is only 1s, so that the problem of time difference in motion connection and video splicing and synthesizing is solved.
And a sub-step S1132 of determining a motion image sequence matching the same set of motion combinations by using the motion image sequence obtained by the alternate extraction.
In an embodiment of the invention, the terminal alternately extracts the motion image sequences matched with a certain training step from at least two video streams, and splices and synthesizes the alternately extracted motion image sequences, so as to determine the motion image sequences matched with each training step under the condition that the motion image sequences corresponding to a certain training step are at least from two video streams.
And a substep S12 of generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations.
In an embodiment of the invention, after the terminal determines the action image sequence matched with each training step and determines the arrangement sequence of each training step, the terminal can splice and synthesize the action image sequence matched with each training step according to the arrangement sequence of the training steps to generate the action image sequence which is finally laid out according to the training steps.
And step 204, displaying the action image sequence which is finally laid out according to the training step.
In an embodiment of the invention, the generated action image sequence which is finally laid out according to the training steps is displayed, and a plurality of paths of video streams recorded in the training test of each examinee are combined into one path of video stream, so that a teacher can conveniently look up and score.
According to the embodiment of the invention, the action image sequence in each path of video stream is respectively matched with each practical training step, and the action image sequence matched with each practical training step is combined into one path of video stream, wherein the combined video stream is the action image sequence which is finally laid out according to the practical training steps. All actions of a certain examinee and the connection actions among different videos are not required to be searched uniformly according to a time axis, all the actions in different videos of the certain examinee are spliced, and the scoring mode that one examinee corresponds to one video is achieved.
To facilitate understanding of the solution of the present invention, the following description is made in conjunction with an application scenario.
In an embodiment of the present invention, an application scenario of the multi-channel video stream synthesis method is further provided, where the application scenario is applied to an examinee for performing a practical training skill examination, and a device required by the practical training skill examination room includes a camera for recording various practical training steps, a terminal device for receiving camera information and simultaneously performing decoding, calculation, encoding and output, and a display for displaying a final synthesis picture.
In an application scenario, assuming that a practical training skill test is a chemical practical training test, the problem is to extract iodine from iodine water, and the practical training steps can be divided into the following 4 steps: 1) Measuring 10mL of iodine saturated aqueous solution by using a measuring cylinder, pouring the solution into a separating funnel, then injecting 4mL of carbon tetrachloride, and covering a glass plug; 2) Pressing the mouth of the separating funnel with right hand, holding the piston part with left hand, inverting the separating funnel, vibrating to make the two liquids contact fully, opening the piston after vibration to discharge the gas in the funnel; 3) Placing the separating funnel on an iron support, and standing; 4) After the liquids are layered, the glass plug on the neck of the separating funnel is opened, or the groove (or the small hole) on the plug is aligned with the small hole on the funnel, and then the piston below the separating funnel is unscrewed, so that the lower layer of liquid slowly flows down along the wall of the beaker.
When an examinee performs a chemical practical training test, recording practical training operation of the examinee, and synthesizing the recorded multi-channel video stream, wherein the process can be as follows:
1) Constructing 4 paths of cameras according to the training steps of the training skill examination of the examinee, and respectively and simultaneously recording the training operation of the examinee during the chemical training examination; 2) Setting a terminal to simultaneously receive video streams recorded by 4 paths of cameras, wherein the 4 paths of video streams are recorded according to different actions of examinees; 3) Decoding 4 paths of video streams recorded by the 4 paths of cameras; 4) Determining the video stream with action in the 4 paths of video streams through an algorithm, such as measuring, pouring, oscillating and other actions, and corresponding the video stream with the action to the chemical practical training step; 5) Setting the layout of the video stream with actions: arranging pictures of the video stream with the determined action according to the chemical practical training step; 6) Synthesizing 4 paths of video streams subjected to picture arrangement into 1 path of video stream; 7) And outputting the video streams synthesized into 1 path to a display so that a teacher can consult and grade the training operation of the examinees.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 3, a block diagram of a multi-channel video stream synthesizing apparatus according to an embodiment of the present invention is shown, which may specifically include the following modules:
a multi-channel video stream obtaining module 301, configured to obtain a multi-channel video stream;
an action image sequence identification module 302, configured to identify an action image sequence from the multi-channel video stream;
the motion image sequence recognition module 303 is configured to determine a motion image sequence matched with a preset motion combination from motion image sequences of the multiple video streams;
and the motion image sequence identification module 304 is configured to display the motion image sequence matched with the preset motion combination.
In an embodiment of the present invention, the preset action combination includes a plurality of action combinations, and the action image sequence determining module 303 may include the following sub-modules:
the action image sequence determining sub-module is used for determining an action image sequence matched with each group of action combinations from the action image sequences of the multi-path video stream and determining the arrangement sequence of the plurality of groups of action combinations;
and the action image sequence generation sub-module is used for generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations.
In one embodiment of the invention, the motion image sequence determination submodule may include the following units:
the motion image sequence extraction unit is used for extracting motion image sequences matched with all groups of motion combinations from the motion image sequences of each video stream;
a first motion image sequence determining unit, configured to determine, when a motion image sequence matching a same set of motion combinations includes a motion image sequence from one video stream, a motion image sequence matching the same set of motion combinations from the motion image sequence from the one video stream;
and the second motion image sequence determining unit is used for determining the motion image sequence matched with the same group of motion combinations from the motion image sequences of at least two video streams when the motion image sequence matched with the same group of motion combinations comprises the motion image sequences of at least two video streams.
In one embodiment of the invention, the second motion image sequence determination unit may comprise the following sub-units:
the motion image sequence extraction subunit is used for alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
and the motion image sequence generation subunit is used for determining the motion image sequence matched with the same group of motion combinations by adopting the motion image sequence obtained by the alternate extraction.
For the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.
An embodiment of the present invention further provides an electronic device, including:
the multi-path video stream synthesis method comprises a processor, a memory and a computer program which is stored in the memory and can run on the processor, wherein when the computer program is executed by the processor, each process of the multi-path video stream synthesis method embodiment is realized, the same technical effect can be achieved, and in order to avoid repetition, the details are not repeated.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the above-mentioned embodiment of the multi-channel video stream synthesis method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one of skill in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or terminal apparatus that comprises the element.
The above detailed description is provided for a multi-channel video stream synthesizing method and a multi-channel video stream synthesizing device, and the principle and the implementation of the present invention are explained by applying specific examples, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (8)
1. A method for synthesizing multiple video streams, the method comprising:
acquiring a plurality of paths of video streams; the multi-path video stream comprises multi-path video streams of different angles or different areas;
identifying a sequence of action images from the multi-channel video stream;
determining a motion image sequence matched with a preset motion combination from the motion image sequences of the multi-channel video stream; displaying the action image sequence matched with the preset action combination;
the preset action combination comprises an action combination used for representing one or more training steps; the determining a motion image sequence matched with a preset motion combination from motion image sequences of the multi-path video stream comprises:
determining motion image sequences matched with all groups of motion combinations from the motion image sequences of the multi-path video stream, and determining the arrangement sequence of the groups of motion combinations;
generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations; the action image sequence matched with the preset action combination comprises an action image sequence alternately extracted from the action image sequence matched with the at least two video streams on the main picture, wherein the action image sequence alternately extracted is realized by switching the action image sequence which is not played on the main picture but is not played completely to the main picture after the at least two action image sequences which are simultaneously played reach a certain playing time in turn.
2. The method of claim 1, wherein determining from the sequence of motion images of the multi-channel video stream a sequence of motion images that matches each set of motion combinations comprises:
respectively extracting action image sequences matched with all groups of action combinations from the action image sequences of each path of video stream;
when the motion image sequences matched with the same group of motion combinations comprise motion image sequences from one video stream, determining motion image sequences matched with the same group of motion combinations from the motion image sequences from the one video stream;
and/or when the motion image sequences matched with the same group of motion combinations comprise motion image sequences from at least two video streams, determining the motion image sequences matched with the same group of motion combinations from the motion image sequences from at least two video streams.
3. The method of claim 2, wherein said determining from said sequence of motion images from said at least two video streams a sequence of motion images matching said same set of motion combinations comprises:
alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
and determining the motion image sequence matched with the same group of motion combinations by adopting the motion image sequence obtained by alternate extraction.
4. A multi-channel video stream synthesizing apparatus, comprising:
the multi-channel video stream acquisition module is used for acquiring multi-channel video streams;
the motion image sequence identification module is used for identifying a motion image sequence from the multi-channel video stream;
the motion image sequence determining module is used for determining a motion image sequence matched with a preset motion combination from the motion image sequences of the multi-channel video stream; the action image sequence display module is used for displaying the action image sequence matched with the preset action combination;
the preset action combination comprises an action combination used for representing one or more practical training steps, and the action image sequence determining module comprises:
the action image sequence determining submodule is used for determining an action image sequence matched with each group of action combinations from the action image sequences of the multi-path video stream and determining the arrangement sequence of the plurality of groups of action combinations;
the action image sequence generation sub-module is used for generating an action image sequence matched with a preset action combination according to the arrangement sequence of the plurality of groups of action combinations by adopting the action image sequence matched with each group of action combinations; the action image sequence matched with the preset action combination comprises an action image sequence alternately extracted from the action image sequence matched with the at least two video streams on the main picture, wherein the action image sequence alternately extracted is realized by switching the action image sequence which is not played on the main picture but is not played completely to the main picture after the at least two action image sequences which are simultaneously played reach a certain playing time in turn.
5. The apparatus of claim 4, wherein the motion image sequence determination sub-module comprises:
the motion image sequence extraction unit is used for extracting motion image sequences matched with all groups of motion combinations from the motion image sequences of each video stream;
a first motion image sequence determining unit, configured to determine, when a motion image sequence matching a same set of motion combinations includes a motion image sequence from one video stream, a motion image sequence matching the same set of motion combinations from the motion image sequence from the one video stream;
and the second motion image sequence determining unit is used for determining the motion image sequence matched with the same group of motion combinations from the motion image sequences of at least two video streams when the motion image sequence matched with the same group of motion combinations comprises the motion image sequences of at least two video streams.
6. The apparatus according to claim 5, wherein the second motion image sequence determination unit comprises:
the motion image sequence extraction subunit is used for alternately extracting motion image sequences from the motion image sequences matched by the at least two video streams;
and the motion image sequence generation subunit is used for determining the motion image sequence matched with the same group of motion combinations by adopting the motion image sequence obtained by the alternate extraction.
7. An electronic device, comprising: processor, memory and a computer program stored on the memory and capable of running on the processor, the computer program when executed by the processor implementing the steps of the multi-path video stream composing method according to any of claims 1 to 3.
8. A computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, carries out the steps of the multi-path video stream composing method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055011.5A CN111246125B (en) | 2020-01-17 | 2020-01-17 | Multi-channel video stream synthesis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055011.5A CN111246125B (en) | 2020-01-17 | 2020-01-17 | Multi-channel video stream synthesis method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111246125A CN111246125A (en) | 2020-06-05 |
| CN111246125B true CN111246125B (en) | 2022-11-01 |
Family
ID=70877974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010055011.5A Active CN111246125B (en) | 2020-01-17 | 2020-01-17 | Multi-channel video stream synthesis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111246125B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3046319A1 (en) * | 2015-01-19 | 2016-07-20 | Thomson Licensing | Method for generating an HDR image of a scene based on a tradeoff between brightness distribution and motion |
| EP3346445B1 (en) * | 2017-01-05 | 2021-11-10 | Université de Toulon | Methods and devices for extracting an object from a video sequence |
| CN107197172A (en) * | 2017-06-21 | 2017-09-22 | 北京小米移动软件有限公司 | Net cast methods, devices and systems |
| CN108881927B (en) * | 2017-11-30 | 2020-06-26 | 视联动力信息技术股份有限公司 | Video data synthesis method and device |
| CN110166650B (en) * | 2019-04-29 | 2022-08-23 | 北京百度网讯科技有限公司 | Video set generation method and device, computer equipment and readable medium |
-
2020
- 2020-01-17 CN CN202010055011.5A patent/CN111246125B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111246125A (en) | 2020-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107105310B (en) | Human image replacing method and device in video live broadcast and recording and broadcasting system | |
| US20220254158A1 (en) | Learning situation analysis method, electronic device, and storage medium | |
| CN112562720A (en) | Lip-synchronization video generation method, device, equipment and storage medium | |
| CN111901572B (en) | Multi-channel video stream synthesis method, device, equipment and storage medium | |
| WO2021098616A1 (en) | Motion posture recognition method, motion posture recognition apparatus, terminal device and medium | |
| US11847726B2 (en) | Method for outputting blend shape value, storage medium, and electronic device | |
| US20210104169A1 (en) | System and method for ai based skill learning | |
| CN113691836A (en) | Video template generation method, video generation method and device and electronic equipment | |
| CN109598188A (en) | Information-pushing method, device, computer equipment and storage medium | |
| CN108391077A (en) | A kind of simple Classic Course video automatic shooting record device | |
| CN111144356A (en) | Teacher sight following method and device for remote teaching | |
| KR20150084586A (en) | Kiosk and system for authoring video lecture using virtual 3-dimensional avatar | |
| CN112511818B (en) | Video playing quality detection method and device | |
| CN112887790A (en) | Method for fast interacting and playing video | |
| CN112055257B (en) | Video classroom interaction method, device, equipment and storage medium | |
| CN111353439A (en) | Method, device, system and equipment for analyzing teaching behaviors | |
| CN106658037A (en) | Live video method and apparatus of multiple video streams | |
| CN113220940B (en) | Video classification method, device, electronic equipment and storage medium | |
| CN111246125B (en) | Multi-channel video stream synthesis method and device | |
| CN111722964A (en) | Screen-casting delay testing method and terminal | |
| CN113784059A (en) | Video generation and splicing method, equipment and storage medium for clothing production | |
| CN108647710A (en) | A kind of method for processing video frequency, device, computer and storage medium | |
| CN116704405B (en) | Behavior recognition method, electronic device and storage medium | |
| CN115278297B (en) | Data processing method, device, equipment and storage medium based on drive video | |
| CN107491934B (en) | 3D interview system based on virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |