WO2018095003A1 - 多媒体数据的实时传输方法及装置 - Google Patents
多媒体数据的实时传输方法及装置 Download PDFInfo
- Publication number
- WO2018095003A1 WO2018095003A1 PCT/CN2017/086495 CN2017086495W WO2018095003A1 WO 2018095003 A1 WO2018095003 A1 WO 2018095003A1 CN 2017086495 W CN2017086495 W CN 2017086495W WO 2018095003 A1 WO2018095003 A1 WO 2018095003A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- somatosensory
- control data
- data packet
- live broadcast
- somatosensory control
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000005540 biological transmission Effects 0.000 title claims description 7
- 230000003238 somatosensory effect Effects 0.000 claims abstract description 364
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 9
- 238000004088 simulation Methods 0.000 abstract description 15
- 238000004891 communication Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 241000282414 Homo sapiens Species 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000011022 operating instruction Methods 0.000 description 2
- 230000008786 sensory perception of smell Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 208000002193 Pain Diseases 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000392 somatic effect Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2408—Monitoring of the upstream path of the transmission network, e.g. client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
- H04N21/25891—Management of end-user data being end-user preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
Definitions
- the invention belongs to the technical field of networks, and in particular relates to a real-time transmission method and device for multimedia data.
- Multimedia refers to a human-computer interactive information communication and communication medium that combines two or more media, including text, pictures, sounds, animations, and movies, as well as interactive functions provided by the program.
- multimedia technology can not only bring visual and auditory experiences to users through computer, but also simulate other senses of human beings, including touch, power, warmth, wetness and wind.
- the sense of smell such as the sense of smell, is simulated by all human beings to give the user an interactive experience of somatosensory.
- the realization principle of the somatosensory sensing is to send a somatosensory control signal to the somatosensory sensing device attached to the human body surface to control the somatosensory sensing device to perform various types of somatosensory feedback to the human body.
- the force feedback control signal controls the magnitude and duration of the force generated by the force sensor located at the hand, thereby bringing a handshake behavior with the game.
- the technician found that the prior art has at least the following technical defects:
- a large part of the existing video files are streamed to the user for online play, and the existing video content providers and various types of somatosensory device suppliers independently provide services, video content and somatosensory sensing control signals.
- the format is also different, so the prior art cannot match the somatosensory data in real time for the video played online, thereby reducing the interactivity of the online scene.
- the embodiments of the present invention provide a real-time transmission method and device for multimedia data, so as to solve the problem that the prior art cannot match the somatosensory data in real time for the video played online, thereby reducing the interaction of the online scene.
- an embodiment of the present invention provides a real-time sending method for multimedia data, where the method includes:
- the live broadcast terminal is instructed to listen;
- the live control terminal When the live control terminal generates the somatosensory control command, the somatosensory control data packet is generated according to the somatosensory control command, and the image frame currently collected by the live broadcast terminal is marked;
- the generating the somatosensory control data packet according to the somatosensory control instruction includes:
- the M and the N are each an integer greater than or equal to 1.
- the generating the somatosensory control data packet according to the somatosensory control command further includes:
- each data bit corresponding to the control information is written into the preset character in the somatosensory control data packet;
- the i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
- the preset character is 0.
- a second aspect is a real-time transmitting device for multimedia data, the device comprising:
- the monitoring unit is configured to perform instruction monitoring on the live terminal during the live broadcast of the video
- a generating unit configured to generate a somatosensory control data packet according to the somatosensory control command when the live broadcast terminal generates the somatosensory control command, and mark an image frame currently collected by the live broadcast terminal;
- An association unit configured to associate the somatosensory control data packet with the image frame
- a sending unit configured to send the image frame to the user terminal in real time through a video live broadcast platform, and simultaneously send the somatosensory control data packet associated with the image frame to the user terminal by using the video live broadcast platform, And causing the user terminal to synchronously output the somatosensory control data packet to the somatosensory sensing device while outputting the image frame to the video playing device.
- the generating unit includes:
- a first writing subunit configured to write the acquired somatosensory control data of the M body points into the somatosensory control data packet
- the M and the N are each an integer greater than or equal to 1.
- the generating unit further includes:
- a second writing subunit for controlling information of the jth type of somatosensory sensing device that does not acquire the i-th body point, each data bit corresponding to the control information in the somatosensory control data packet Write preset characters;
- the i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
- the preset character is 0.
- the live control terminal can generate and transmit the somatosensory control data to the user terminal in real time, so that the user terminal can experience the somatosensory simulation matching the live video content while watching the live video, greatly Improve the interactivity of online scenes.
- an embodiment of the present invention provides a real-time output method for multimedia data, where the method includes:
- the image broadcast frame is received by the live broadcast platform in real time;
- the somatosensory control data packet that is synchronously sent by the live broadcast terminal is received by the video live broadcast platform, where the somatosensory control data packet is broadcasted by the live broadcast terminal. Generated according to the somatosensory control command;
- the image frame is output to the video playback device, and the somatosensory control data packet is synchronously output to the somatosensory sensing device.
- the synchronous outputting the somatosensory control data packet to the somatosensory sensing device includes:
- the M, the N, and the i are all integers greater than or equal to 1, and the i is less than or equal to M.
- the synchronizing output of the somatosensory control data packet to the somatosensory sensing device further includes:
- control information corresponding to the jth type of the somatosensory sensing device of the i-th body point in the somatosensory control data packet is a preset character in each data bit, the somatosensory control data packet is associated with While the image frame is being played, the output of the control information is stopped;
- the j is an integer greater than or equal to 1, and the j is less than or equal to N.
- the preset character is 0.
- an embodiment of the present invention provides a real-time output device for multimedia data, where the device includes:
- a first receiving unit configured to receive, by the video live broadcast platform, an image frame sent by the live broadcast terminal in real time during the live broadcast of the video
- a second receiving unit configured to: when the image frame is associated with a somatosensory control data packet, receive, by the video live broadcast platform, the somatosensory control data packet that is synchronously sent by the live broadcast terminal, where the somatosensory control data packet is sent by the live broadcast
- the terminal is generated according to the somatosensory control instruction during the live broadcast of the video;
- an output unit configured to output the image frame to the video playing device, and synchronously output the somatosensory control data packet to the somatosensory sensing device.
- the output unit includes:
- a first parsing subunit configured to parse the body sense control data of the M body points from the somatosensory control data packet
- a second parsing subunit configured to parse control information of the N types of the somatosensory sensing devices from the parsed somatosensory control data of the i-th body point;
- a first output control subunit configured to output the parsed control information to the N types of somatosensory sensing devices on the i-th body point;
- the M, the N, and the i are all integers greater than or equal to 1, and the i is less than or equal to M.
- the output unit further includes:
- a second output control subunit configured to: in the somatosensory control data packet, the control information corresponding to the jth type of the somatosensory sensing device of the i-th body point is a preset character in each data bit, And stopping outputting the control information while playing the image frame related to the somatosensory control data packet;
- the j is an integer greater than or equal to 1, and the j is less than or equal to N.
- the preset character is 0.
- the embodiment of the invention can experience the somatosensory simulation matching the live content of the video while watching the live video, and greatly improves the interactivity of the online scene.
- FIG. 1 is a flowchart of implementing a real-time sending method of multimedia data according to an embodiment of the present invention
- FIG. 2 is a specific implementation flowchart of a real-time sending method S102 of multimedia data according to an embodiment of the present invention
- FIG. 3 is a structural block diagram of a real-time transmitting apparatus for multimedia data according to an embodiment of the present invention.
- FIG. 4 is a schematic diagram of a computing node according to an embodiment of the present invention.
- FIG. 5 is a flowchart of implementing a real-time output method of multimedia data according to an embodiment of the present invention
- FIG. 6 is a flowchart of a specific implementation of a real-time output method S503 for multimedia data according to an embodiment of the present invention
- FIG. 7 is a structural block diagram of a real-time output device for multimedia data according to an embodiment of the present invention.
- FIG. 8 is a schematic diagram of another computing node according to an embodiment of the present invention.
- the real-time transmission scheme of the multimedia data mentioned in the embodiment of the present invention can insert the somatosensory control data into the live video stream at any time during the live broadcast of the video, and transmit the somatosensory control data along with the live video stream to the live broadcast platform.
- the user terminal is configured to enable the user terminal to simultaneously experience the somatosensory feedback during the live broadcast of the video.
- the somatosensory control data is used to control the somatosensory sensing device, and the somatosensory control data is sent from the somatosensory control device in the form of a somatosensory control signal and transmitted to the somatosensory sensor electrically connected to the somatosensory control device. In the device.
- the somatosensory control device and one or more somatosensory sensing devices can be combined with the wearable product, for example, attaching the somatosensory control device and the plurality of somatosensory sensing devices to the wearable body that wraps the entire body of the user.
- the somatosensory control device outputs the somatosensory control data to the plurality of somatosensory sensing devices, so that the plurality of somatosensory sensing devices perform a somatosensory simulation on the user based on the received somatosensory control data.
- the live broadcast terminal performs instruction monitoring.
- the live video broadcast mainly refers to the live broadcast using the Internet and streaming media technology.
- the live broadcast terminal can refer to the acquisition terminal for collecting image signals and sound signals, and can be a professional audio and video capture camcorder, and can also be a smart phone.
- a smart terminal device with a camera function such as a tablet.
- the video live broadcast process mainly refers to the image and sound digital signals collected by the audio and video collection terminal being generated by the streaming media encoder, and transmitted to the user terminal through the video live broadcast platform.
- the live broadcast terminal when the live broadcast terminal is in the live broadcast process, the live broadcast terminal performs instruction monitoring to detect whether the user has issued a somatosensory control command through the live broadcast terminal.
- the live broadcast terminal detects that the live control terminal generates the somatosensory control command
- the somatosensory control data packet is generated according to the somatosensory control command, and the image frame currently collected by the live broadcast terminal is marked.
- the live terminal provides an operation menu for issuing the somatosensory control command on the display interface, and the user can issue a somatosensory control command through the operation menu at any time.
- the somatosensory control data is carried in the order for controlling the body sensing device.
- the somatosensory control data packet is generated according to the monitored somatosensory control command, and an image frame is marked in the live video stream collected by the live broadcast terminal, where the image frame is an image frame acquired when the somatosensory control command is monitored.
- each of the somatosensory control data packets may carry the somatosensory control data of one or more body points, and the somatosensory control data of each body point includes one or more Control information for a type of somatosensory sensing device.
- the process of generating a sense control packet is as follows:
- S201 Acquire somatic control data of M body points, and the somatosensory control data of each body point includes control information of N types of somatosensory sensing devices.
- M and the N are integers greater than or equal to 1.
- each of the somatosensory control data packets is used to control a somatosensory sensing device on a body point to generate a somatosensory simulation when the image frame is played;
- M is greater than 1 and N is equal to 1
- each of the somatosensory control data packets is used to control the same somatosensory sensing device on the plurality of body points to simultaneously generate the somatosensory simulation when the image frame is played;
- both M and N are greater than At 1 o'clock, each of the somatosensory control data packets is used to control a plurality of somatosensory sensing devices on a plurality of body points to simultaneously generate a somatosensory simulation.
- At least one or more types of somatosensory sensing devices as listed in Table 1 may be integrated according to product requirements:
- a frame format of a somatosensory control data packet is also proposed, and the frame format comprises a start control frame as a frame header and a somatosensory data frame.
- the start control frame includes 4 bytes of control data, which are control frame byte 1, control frame byte 2, control frame byte 3, and control frame byte. 4.
- Each control frame byte can be written into the corresponding data content according to the needs of the somatosensory control.
- the control frame byte can be used to indicate the number of body points covered by the somatosensory control data packet, or to indicate the somatosensory control data packet. Whether it needs to be shielded by the control device.
- the somatosensory data frame carries the somatosensory control data of several kinds of somatosensory sensors sequentially written, and also reserves the data bits for writing the somatosensory control data of several other somatosensory sensors according to development needs in the future:
- the somatosensory control data corresponding to each of the somatosensory sensors may be used to indicate the somatosensory sensor related somatosensory control mode, or to describe the somatosensory sensor related somatosensory function.
- each of the somatosensory sensors corresponds to 4 bytes of somatosensory control data, which are respectively a somatosensory control mode byte 1, a somatosensory function byte 1, a somatosensory function byte 2, and a somatosensory function byte 3 .
- Table 4 shows an example of the frame structure of a complete somatosensory control data packet.
- the frame header of the somatosensory control data packet is a 4-byte initial control frame, and the somatosensory data frame is sequentially written.
- the somatosensory control data of 13 kinds of somatosensory sensing devices can further write the somatosensory control data of the seven kinds of somatosensory sensing devices, and the somatosensory control data of each somatosensory sensing device is 4 byte.
- Table 5 shows the somatosensory control data frame structures corresponding to the 13 somatosensory sensing preambles mentioned in Table 1:
- the somatosensory control data packet is generated by using the frame format described above, on the one hand, the somatosensory control data of a plurality of different types of somatosensory sensing devices can be integrated into a unified data format. Effectively eliminating the fragmentation phenomenon of the sensing device due to different suppliers; on the other hand, writing the somatosensory control data of a plurality of different types of somatosensory sensing devices into one data packet for transmission, To some extent, the occurrence of packet loss during data transmission is reduced, and the reliability of data communication is improved.
- each data bit corresponding to the control information is written into the preset character in the somatosensory control data packet;
- the i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
- a character is preset, and if a body point is not matched with a certain type of somatosensory simulation, in the somatosensory control data packet, each of the types of somatosensory sensing devices corresponding to the body point is occupied.
- the data bits are written to the preset character. For example, if the somatosensory control command issued by the user only indicates that force feedback is needed, and no other somatosensory simulation is required, all the body points are all except the data bits occupied by the force feedback device in the somatosensory control data packet. The data bits are written to the preset character.
- the frame writing scheme of the above-mentioned somatosensory control data packet can ensure that the data lengths of all the somatosensory control data packets are consistent, so that the data length is the same for each generated somatosensory control data packet, which is helpful in the data verification process. Quickly discover the packet loss situation.
- the preset character is 0.
- the somatosensory control data packet is associated with the image frame.
- the somatosensory control data packet generated in S102 is associated with the image frame marked in S102, and the association relationship exists once determined, even after the video live broadcast process ends, from the video live broadcast platform through the user terminal.
- the association relationship always exists to ensure that the somatosensory simulation always corresponds to the same live video stream progress during the live broadcast and playback.
- the image frame is sent to the user terminal in real time through the video live broadcast platform, and the somatosensory control data packet associated with the image frame is synchronously sent to the user terminal through the video live broadcast platform, so that The user terminal synchronously outputs the somatosensory control data packet to the somatosensory sensing device while outputting the image frame to the video playback device.
- the data is transmitted in real time.
- the video broadcast platform (server) is broadcasted by the video live broadcast platform to all user terminals watching the live video broadcast. Therefore, when the image frame is sent to the user terminal in real time through the video live broadcast platform, the image frame is also associated with the image frame.
- the somatosensory control data packet is synchronously broadcasted to all user terminals watching the live video broadcast through the same video live broadcast platform, so that when the user terminal watches the live video broadcast, the user terminal can simultaneously play the live video stream while simultaneously performing the somatosensory simulation. In order to truly restore the somatosensory control intention of the live terminal user.
- FIG. 3 is a structural block diagram of a real-time transmitting apparatus for multimedia data provided by an embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.
- the apparatus includes:
- the monitoring unit 31 performs instruction monitoring on the live broadcast terminal during the live broadcast of the video
- the generating unit 32 when the live broadcast terminal detects that the live control terminal generates the somatosensory control command, generates a somatosensory control data packet according to the somatosensory control command, and marks the image frame currently collected by the live broadcast terminal;
- the associating unit 33 associates the somatosensory control data packet with the image frame
- the sending unit 34 sends the image frame to the user terminal in real time through the video live broadcast platform, and simultaneously sends the somatosensory control data packet associated with the image frame to the user terminal through the video live broadcast platform, so that The user terminal synchronously outputs the somatosensory control data packet to the somatosensory sensing device while outputting the image frame to the video playback device.
- the generating unit 31 includes:
- the M and the N are each an integer greater than or equal to 1.
- the generating unit 31 further includes:
- the second writing subunit if the jth type of somatosensory sensing device of the i-th body point is not obtained Setting control information, in the somatosensory control data packet, each data bit corresponding to the control information is written into a preset character;
- the i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
- the preset character is 0.
- FIG. 4 is a schematic diagram of a computing node 400 according to an embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.
- the computing node 400 may be a host server that includes computing power, or a personal computer PC, or a portable computer or terminal that can be carried.
- the specific embodiment of the present invention does not limit the specific implementation of the computing node.
- Compute node 400 includes:
- a processor 410 a communication interface 420, a memory 430, and a bus 440.
- the processor 410, the communication interface 420, and the memory 430 complete communication with each other via the bus 440.
- the communication interface 420 is configured to communicate with a network element, such as a virtual machine management center, shared storage, and the like.
- the processor 410 is configured to execute a program.
- the program can include program code, the program code including computer operating instructions.
- the processor 410 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
- CPU Central Processing Unit
- ASIC Application Specific Integrated Circuit
- the memory 430 is configured to store a program.
- the memory 430 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.
- the program may be specifically configured to perform a real-time sending method of multimedia data, where the method includes:
- the live broadcast terminal is instructed to listen;
- the live control terminal When the live control terminal generates the somatosensory control command, the somatosensory control data packet is generated according to the somatosensory control command, and the image frame currently collected by the live broadcast terminal is marked;
- the generating the somatosensory control data packet according to the somatosensory control instruction comprises:
- the M and the N are each an integer greater than or equal to 1.
- the generating the somatosensory control data packet according to the somatosensory control instruction further includes:
- each data bit corresponding to the control information is written into the preset character in the somatosensory control data packet;
- the i and the j are integers greater than or equal to 1, and the i is less than or equal to the M, and the j is less than or equal to the N.
- the preset character is 0.
- the live control terminal can generate and transmit the somatosensory control data to the user terminal in real time, so that the user terminal can experience the somatosensory simulation matching the live video content while watching the live video, greatly Improve the interactivity of online scenes.
- the live broadcast platform receives the image frame sent by the live broadcast terminal.
- the somatosensory control data packet that is synchronously sent by the live broadcast terminal is received by the video live broadcast platform, where the somatosensory control data packet is The live broadcast terminal is generated according to the somatosensory control instruction during the live broadcast of the video.
- the image frame is output to the video playback device, and the somatosensory control data packet is synchronously outputted to the somatosensory sensing device.
- the user terminal receives the live video broadcast frame from the video live broadcast platform in the manner of watching the live video broadcast in the prior art, and as described in the foregoing embodiment, if the live broadcast terminal user is at a certain moment
- the somatosensory control command transmits the somatosensory control data packet generated according to the sensing control command and the image frame collected at the moment to the user terminal through the video live broadcast platform, and the user terminal outputs the image frame to the video playing device.
- the somatosensory control data packet is also synchronously outputted to the somatosensory sensing device.
- each of the somatosensory control data packets includes control information of N types of body sensing devices on M body points, as shown in FIG. 6, the synchronous output station
- the somatosensory control data packet to the somatosensory sensing device includes:
- the somatosensory control data of the M body points are parsed from the somatosensory control data packet.
- the parsed control information is respectively output to N types of somatosensory sensing devices on the i-th body point.
- the M, the N, and the i are all integers greater than or equal to 1, and the i is less than or equal to M.
- the somatosensory control data can be accurately outputted to a specific somatosensory sensing device located at a specific body point, and then for the whole body M A total of M*N individual sensing devices on the body point can simultaneously realize the output of the somatosensory simulation in the process of playing related image frames, bringing a full-scale somatosensory experience.
- a character is preset on the basis of the above-described somatosensory control packet frame format, and if a body point is not matched to a certain type of somatosensory simulation, the somatosensory control packet is Writing, each data bit occupied by the body sensing device corresponding to the body point to the preset character, for example, writing a preset character 0 to ensure the data length of all the somatosensory control data packets Consistent. Then, under the solution, the synchronously outputting the somatosensory control data packet to the somatosensory sensing device further includes:
- control information corresponding to the jth type of the somatosensory sensing device of the i-th body point in the somatosensory control data packet is a preset character in each data bit, the somatosensory control data packet is associated with While the image frame is being played, the output of the control information is stopped;
- the j is an integer greater than or equal to 1, and the j is less than or equal to N.
- the somatosensory control device detects that the control information corresponding to a specific type of somatosensory sensing device on a specific body point is a preset character in each data bit occupied by the somatosensory data packet, then at the moment Stopping outputting control information to the somatosensory sensing device on the body point, so that although the data length of the somatosensory control data packet associated with each image frame is the same, the somatosensory control device can still be based on each data bit.
- the upper data content determines whether to output control information or stop outputting control information, thereby flexibly implementing output control of multiple body points and multiple body sensing devices.
- the live control terminal can generate and transmit the somatosensory control data to the user terminal in real time, so that the user terminal can experience the somatosensory simulation matching the live broadcast content while watching the live video. , greatly improving the interactivity of online scenes.
- FIG. 7 is a structural block diagram of a real-time output device for multimedia data provided by an embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.
- the apparatus includes:
- the first receiving unit 71 receives the image frame sent by the live broadcast terminal in real time through the video live broadcast platform during the live broadcast of the video;
- the second receiving unit 72 if the image frame is associated with the somatosensory control data packet, receiving, by the video live broadcast platform, the somatosensory control data packet that is synchronously sent by the live broadcast terminal, where the somatosensory control data packet is used by the live broadcast terminal Generated according to the somatosensory control instruction during the live broadcast of the video;
- the output unit 73 outputs the image frame to the video playback device, and synchronously outputs the somatosensory control data packet to the somatosensory sensing device.
- the output unit 73 includes:
- a first parsing subunit parsing the somatosensory control data of the M body points from the somatosensory control data packet
- the second parsing subunit parses the control information of the N types of the somatosensory sensing devices from the parsed somatosensory control data of the i-th body point;
- the first output control subunit outputs the parsed control information to the N types of somatosensory sensing devices on the i-th body point;
- the M, the N, and the i are all integers greater than or equal to 1, and the i is less than or equal to M.
- the output unit 73 further includes:
- a second output control subunit if the control information corresponding to the jth type of the somatosensory sensing device of the i-th body point in the somatosensory control data packet is a preset character in each data bit, then While the somatosensory control data packet related to the image frame is played, the control information is stopped from being output;
- the j is an integer greater than or equal to 1, and the j is less than or equal to N.
- the preset character is 0.
- FIG. 8 is a schematic diagram of a computing node 800 according to an embodiment of the present invention. For the convenience of explanation, only the parts related to the present embodiment are shown.
- the computing node 800 may be a host server that includes computing power, or a personal computer PC, or a portable computer or terminal that can be carried.
- the specific embodiment of the present invention does not limit the specific implementation of the computing node.
- Computing node 800 includes:
- a processor 810 a communications interface 820, a memory 830, and a bus 840.
- the processor 810, the communication interface 820, and the memory 830 complete communication with each other via the bus 840.
- the communication interface 820 is configured to communicate with a network element, such as a virtual machine management center, shared storage, and the like.
- the processor 810 is configured to execute a program.
- the program can include program code, the program code including computer operating instructions.
- the processor 810 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
- CPU Central Processing Unit
- ASIC Application Specific Integrated Circuit
- the memory 830 is configured to store a program.
- the memory 830 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.
- the program may be specifically configured to execute a real-time output method of multimedia data, the method comprising:
- the image broadcast frame is received by the live broadcast platform in real time;
- the somatosensory control data packet that is synchronously sent by the live broadcast terminal is received by the video live broadcast platform, where the somatosensory control data packet is broadcasted by the live broadcast terminal. Generated according to the somatosensory control command;
- the image frame is output to the video playback device, and the somatosensory control data packet is synchronously output to the somatosensory sensing device.
- the synchronously outputting the somatosensory control data packet to the somatosensory sensing device comprises:
- the M, the N, and the i are all integers greater than or equal to 1, and the i is less than or equal to M.
- the synchronizing output of the somatosensory control data packet to the somatosensory sensing device further includes:
- the control information is a preset character in each data bit, and the control information is stopped when the image frame related to the somatosensory control data packet is played;
- the j is an integer greater than or equal to 1, and the j is less than or equal to N.
- the preset character is 0.
- each functional unit in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the integrated unit may be implemented in the form of hardware. It can also be implemented in the form of a software functional unit.
- the specific names of the respective functional units are only for the purpose of facilitating mutual differentiation, and are not intended to limit the scope of protection of the present application.
- the disclosed apparatus and apparatus may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of modules or units is only a logical function division.
- there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
- the units described as separate components may or may not be physically separated as a unit
- the displayed components may or may not be physical units, ie may be located in one place, or may be distributed over multiple network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
- An integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, can be stored in a computer readable storage medium.
- the medium includes a number of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of various embodiments of the embodiments of the present invention.
- the foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read only memory, a random access memory, a magnetic disk, or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Selective Calling Equipment (AREA)
Abstract
在视频直播过程中,对直播终端进行指令监听;当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;将所述体感控制数据包与所述图像帧进行关联;将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。使用户终端在观看视频直播的同时体验到与视频直播内容相匹配的体感模拟,极大地提高了在线场景的可互动性。
Description
本发明属于网络技术领域,尤其涉及多媒体数据的实时传输方法及装置。
多媒体是指组合两种或两种以上媒体的一种人机交互式信息交流和传播媒体,使用的媒体包括文字、图片、声音、动画和影片,以及程式所提供的互动功能。随着科学技术的发展,多媒体技术除了可以通过计算机为用户带来视觉体验和听觉体验,还可以对人所具有的其他体感进行模拟,包括对触感、力感、温感、湿感、风感、嗅觉等一切人体所具有的感知进行模拟,以为用户带来体感方面的交互体验。
体感传感的实现原理为:向附着在人体体表的体感传感装置发送体感控制信号,以控制体感传感装置向人体进行各类体感反馈。例如,在虚拟现实(Virtual Reality,VR)游戏中与虚拟人物握手时,通过力反馈控制信号对位于手部的力传感器产生的力度大小及持续时间进行控制,从而带来与游戏中的握手行为相匹配的力反馈体验。然而,技术人员在研发过程中发现,现有技术至少存在以下技术缺陷:
现有的视频文件有很大一部分是以流媒体的方式面向用户进行在线播放,且现有的视频内容提供商及各类体感设备供应商各自独立提供服务,视频内容与体感传感控制信号的格式也各不相同,因此,现有技术无法为在线播放的视频实时匹配体感数据,由此降低了在线场景的互动性。
发明内容
有鉴于此,本发明实施例提供了多媒体数据的实时传输方法及装置,以解决现有技术无法为在线播放的视频实时匹配体感数据,由此降低了在线场景的互动性的问题。
第一方面,本发明实施例提供了一种多媒体数据的实时发送方法,所述方法包括:
在视频直播过程中,对直播终端进行指令监听;
当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;
将所述体感控制数据包与所述图像帧进行关联;
将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
作为第一方面的第一种可能的实现方式,所述根据所述体感控制指令生成体感控制数据包包括:
获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;
将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;
所述M和所述N均为大于或等于1的整数。
结合第一方面的第一种可能的实现方式,在第二种可能的实现方式中,所述根据所述体感控制指令生成体感控制数据包还包括;
若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;
所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
结合第一方面的第二种可能的实现方式,在第三种可能的实现方式中,所述预设字符为0。
第二方面,一种多媒体数据的实时发送装置,所述装置包括:
监听单元,用于在视频直播过程中,对直播终端进行指令监听;
生成单元,用于当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;
关联单元,用于将所述体感控制数据包与所述图像帧进行关联;
发送单元,用于将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
作为第二方面的第一种可能的实现方式,所述生成单元包括:
获取子单元,用于获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;
第一写入子单元,用于将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;
所述M和所述N均为大于或等于1的整数。
结合第二方面的第一种可能的实现方式,在第二种可能的实现方式中,所述生成单元还包括;
第二写入子单元,用于若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;
所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
结合第二方面的第二种可能的实现方式,在第三种可能的实现方式中,所述预设字符为0。
本发明实施例可以在视频直播的同时,实时地由直播终端生成并向用户终端传输体感控制数据,以使用户终端在观看视频直播的同时体验到与视频直播内容相匹配的体感模拟,极大地提高了在线场景的可互动性。
第三方面,本发明实施例提供了一种多媒体数据的实时输出方法,所述方法包括:
在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;
若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;
输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
作为第三方面的第一种可能的实现方式,所述同步输出所述体感控制数据包至体感传感装置包括:
从所述体感控制数据包中解析出M个身体点位的体感控制数据;
从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;
将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;
所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
结合第三方面的第一种可能的实现方式,在第二种可能的实现方式中,所述同步输出所述体感控制数据包至体感传感装置还包括:
若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;
所述j为大于或等于1的整数,且所述j小于或等于N。
结合第三方面的第二种可能的实现方式,在第三种可能的实现方式中,所述预设字符为0。
第四方面,本发明实施例提供了一种多媒体数据的实时输出装置,所述装置包括:
第一接收单元,用于在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;
第二接收单元,用于若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;
输出单元,用于输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
作为第四方面的第一种可能的实现方式,所述输出单元包括:
第一解析子单元,用于从所述体感控制数据包中解析出M个身体点位的体感控制数据;
第二解析子单元,用于从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;
第一输出控制子单元,用于将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;
所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
结合第四方面的第一种可能的实现方式,在第二种可能的实现方式中,所述输出单元还包括:
第二输出控制子单元,用于若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;
所述j为大于或等于1的整数,且所述j小于或等于N。
结合第四方面的第二种可能的实现方式,在第三种可能的实现方式中,所述预设字符为0。
本发明实施例可以在观看视频直播的同时体验到与视频直播内容相匹配的体感模拟,极大地提高了在线场景的可互动性。
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的多媒体数据的实时发送方法的实现流程图;
图2是本发明实施例提供的多媒体数据的实时发送方法S102的具体实现流程图;
图3是本发明实施例提供的多媒体数据的实时发送装置的结构框图;
图4是本发明实施例提供的一种计算节点的示意图;
图5是本发明实施例提供的多媒体数据的实时输出方法的实现流程图;
图6是本发明实施例提供的多媒体数据的实时输出方法S503的具体实现流程图;
图7是本发明实施例提供的多媒体数据的实时输出装置的结构框图;
图8是本发明实施例提供的另一种计算节点的示意图。
以下描述中,为了说明而不是为了限定,提出了诸如特定***结构、技术之类的具体细节,以便透彻理解本发明实施例。然而,本领域的技术人员应当清楚,在没有这些具体细节的其它实施例中也可以实现本发明。在其它情况中,
省略对众所周知的***、装置、电路以及方法的详细说明,以免不必要的细节妨碍本发明的描述。
本发明实施例所提及的多媒体数据的实时传输方案,可以在视频直播的过程中,在直播视频流中随时***体感控制数据,并将体感控制数据随直播视频流一起通过视频直播平台传输给用户终端,以使得用户终端在观看视频直播的过程中可以同时体验到体感反馈。在本发明实施例中,所述体感控制数据用于对体感传感装置进行控制,体感控制数据以体感控制信号的形式从体感控制装置发出,并传送至与体感控制装置电连接的体感传感装置中。在实际的产品形态中,体感控制装置及一个或多个的体感传感装置可与可穿戴产品相结合,例如,将体感控制装置与多个体感传感装置附着于包裹用户全身的可穿戴本体内,通过体感控制装置向多个体感传感装置分别输出体感控制数据,从而使得这多个体感传感装置根据各自接收到的体感控制数据对用户进行体感模拟。
首先,对本发明实施例提供的多媒体数据的实时发送方法进行详细阐述,其实现流程如图1所示:
在S101中,在视频直播过程中,对直播终端进行指令监听。
在这里,视频直播主要指利用互联网及流媒体技术所进行的直播,直播终端可以指用于采集图像信号和声音信号的采集终端,可以为专业的音视频采集摄录机,还可以为智能手机、平板等具备摄像功能的智能终端设备。视频直播过程,在此主要指的是将音视频采集终端采集到的图像及声音数字信号通过流媒体编码器生成直播文件,并通过视频直播平台向用户终端进行发送。
在本发明实施例中,当直播终端处于视频直播过程时,对直播终端进行指令监听,以检测用户是否通过直播终端下达了体感控制指令。
在S102中,当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧。
在视频直播过程中,直播终端在其显示界面上提供用于下达体感控制指令的操作菜单,用户可以随时通过该操作菜单下达体感控制指令,该体感控制指
令中携带了体感控制数据,用于对体感传感装置进行控制。根据监听到的体感控制指令生成体感控制数据包,并在直播终端采集到的直播视频流中标记一图像帧,该图像帧为该体感控制指令被监听到时所采集到的图像帧。
进一步地,作为本发明的一个实施例,每个体感控制数据包中可以携带一个或多个身体点位的体感控制数据,而其中每个身体点位的体感控制数据均包含一种或多种类型的体感传感装置的控制信息。具体地,如图2所示,生成感控制数据包的过程如下:
S201,获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息。
S202,将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包。
其中,所述M和所述N均为大于或等于1的整数。
基于图2所示的实施例,当M和N均等于1时,每个体感控制数据包用于在该图像帧播放时,控制一个身体点位上的一种体感传感装置产生体感模拟;当M大于1且N等于1时,每个体感控制数据包用于在该图像帧播放时,控制多个身体点位上的同一种体感传感装置同时产生体感模拟;当M和N均大于1时,每个体感控制数据包用于控制多个身体点位上的多种体感传感装置同时产生体感模拟。
在本发明实施例中,对于每个身体点位,根据产品需求,至少可以集成如表1所列举的任意一种或多种类型的体感传感装置:
表1
序号 | 体感传感装置 | 序号 | 体感传感装置 |
1 | 按摩/振动 | 8 | 风 |
2 | 触觉 | 9 | 湿润 |
3 | 刺痛 | 10 | 外设灯 |
4 | 温暖/热 | 11 | 外设风湿 |
5 | 冷 | 12 | 外设风冷 |
6 | 气味 | 13 | 机械外骨骼 |
7 | 收缩 |
优选地,在本发明实施例中,还提出了一种体感控制数据包的帧格式,该帧格式的构成包括作为帧头的起始控制帧以及体感数据帧。
(一)起始控制帧中携带了若干字节的控制数据:
示例性地,如表2所示,该起始控制帧中包括了4个字节的控制数据,分别为控制帧字节1、控制帧字节2、控制帧字节3及控制帧字节4,每个控制帧字节可以根据体感控制的需要写入相应的数据内容,例如,可以用控制帧字节来表明体感控制数据包所覆盖的身体点位数量,或者表明该体感控制数据包是否需要被控制装置所屏蔽。
表2
(二)体感数据帧中携带了顺序写入的若干种体感传感器的体感控制数据,同时还预留了数据位,用于将来根据开发需要写入若干其他体感传感器的体感控制数据:
在体感数据帧中,每种体感传感器对应的体感控制数据可以用于表明该体感传感器相关的体感控制模式,或者用于描述该体感传感器相关的体感功能实
现。示例性地,如表3所示,每种体感传感器对应4个字节的体感控制数据,分别为体感控制模式字节1、体感功能字节1、体感功能字节2及体感功能字节3。
表3
表4示出了一个完整的体感控制数据包的帧结构示例,从表4中可以看出,该体感控制数据包的帧头为4字节的起始控制帧,体感数据帧中顺序写入了13种体感传感装置的体感控制数据,同时还包括7种功能预留,能够再写入7种体感传感装置的体感控制数据,且每种体感传感装置的体感控制数据为4个字节。
表4
示例性地,表5示出了表1提及的13种体感传感前置对应的体感控制数据帧结构:
表5
在体感传感数据的通信处理过程中,若采用上文所述的帧格式生成体感控制数据包,一方面可以将多种不同类型的体感传感装置的体感控制数据整合为统一的数据格式,有效地消除感传感装置因来自不同的供应商而导致的碎片化现象,另一方面,将多种不同类型的体感传感装置的体感控制数据写入一个数据包中进行传输,也可以在一定程度上降低数据传输过程中丢包现象的出现,提高了数据通信的可靠性。
此外,在上文所述的体感控制数据包帧格式的基础之上,作为本发明的一个实施例,在生成体感控制数据包的过程中:
若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;
所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
即,预设一个字符,若未对某个身体点位匹配某一类型的体感模拟,则在体感控制数据包中,将该身体点位对应的该类型的体感传感装置所占据的每个数据位均写入该预设字符。例如,用户下达的体感控制指令只指示了需要进行力反馈,而无需进行其他的体感模拟,则在体感控制数据包中,将所有身体点位上除力反馈装置所占据的数据位以外的所有数据位均写入该预设字符。
以上体感控制数据包的帧写入方案可以保证所有体感控制数据包的数据长度一致,这样一来,对于生成的每个体感控制数据包,数据长度均相同,有助于在数据校验过程中快速地发现丢包情况。
可选地,所述预设字符为0。
在S103中,将所述体感控制数据包与所述图像帧进行关联。
具体地,是将S102中生成的体感控制数据包与S102中标记的图像帧进行关联,该关联关系一旦确定则会一直存在,甚至是在视频直播过程结束之后,通过用户终端从视频直播平台上回放整个直播过程时,该关联关系始终存在,以保证体感模拟在直播过程中和回放过程中始终都是对应于相同的直播视频流进度。
在S104中,将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
在本发明实施例中,对于视频直播流和体感控制数据包来说,均是实时传
输至视频直播平台(服务器),再由视频直播平台广播给所有观看视频直播的用户终端的,因此,在将图像帧通过视频直播平台实时发送至用户终端的同时,也将与该图像帧关联的体感控制数据包通过同一视频直播平台同步广播给所有观看视频直播的用户终端,这样一来,用户终端在观看视频直播时,便可以在用户终端播放视频直播流的同时,同步地进行体感模拟,以真实还原出直播终端用户的体感控制意图。
对应于上文实施例所述的多媒体数据的实时发送方法,图3示出了本发明实施例提供的多媒体数据的实时发送装置的结构框图。为了便于说明,仅示出了与本实施例相关的部分。
参照图3,该装置包括:
监听单元31,在视频直播过程中,对直播终端进行指令监听;
生成单元32,当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;
关联单元33,将所述体感控制数据包与所述图像帧进行关联;
发送单元34,将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
可选地,所述生成单元31包括:
获取子单元,获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;
第一写入子单元,将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;
所述M和所述N均为大于或等于1的整数。
可选地,所述生成单元31还包括;
第二写入子单元,若未获取到第i个身体点位的第j种类型的体感传感装
置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;
所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
可选地,所述预设字符为0。
对应于上文实施例所述的多媒体数据的实时发送方法,图4示出了本发明实施例提供的一种计算节点400的示意图。为了便于说明,仅示出了与本实施例相关的部分。
其中,计算节点400可能是包含计算能力的主机服务器,或者是个人计算机PC,或者是可携带的便携式计算机或终端等等,本发明具体实施例并不对计算节点的具体实现做限定。计算节点400包括:
处理器(processor)410,通信接口(Communications Interface)420,存储器(memory)430,总线440。
处理器410,通信接口420,存储器430通过总线440完成相互间的通信。
通信接口420,用于与网元通信,比如虚拟机管理中心、共享存储等。
处理器410,用于执行程序。
具体地,程序可以包括程序代码,所述程序代码包括计算机操作指令。
处理器410可能是一个中央处理器(Central Processing Unit,CPU),或者是特定集成电路(Application Specific Integrated Circuit,ASIC),或者是被配置成实施本发明实施例的一个或多个集成电路。
存储器430,用于存放程序。存储器430可能包含高速RAM存储器,也可能还包括非易失性存储器(non-volatile memory),例如至少一个磁盘存储器。
程序具体可以用于执行一种多媒体数据的实时发送方法,所述方法包括:
在视频直播过程中,对直播终端进行指令监听;
当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;
将所述体感控制数据包与所述图像帧进行关联;
将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
可选地,所述根据所述体感控制指令生成体感控制数据包包括:
获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;
将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;
所述M和所述N均为大于或等于1的整数。
可选地,所述根据所述体感控制指令生成体感控制数据包还包括;
若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;
所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
可选地,所述预设字符为0。
本发明实施例可以在视频直播的同时,实时地由直播终端生成并向用户终端传输体感控制数据,以使用户终端在观看视频直播的同时体验到与视频直播内容相匹配的体感模拟,极大地提高了在线场景的可互动性。
接下来,对本发明实施例提供的多媒体数据的实时输出方法进行详细阐述,其实现流程如图5所示:
在S501中,在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧。
在S502中,若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由
所述直播终端在视频直播的过程中根据体感控制指令生成。
在S503中,输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
在本发明实施例中,用户终端采用现有技术中观看视频直播的方式,从视频直播平台接收直播终端实时发送的图像帧,而如前文实施例所述,若直播终端用户在某个时刻下达了体感控制指令,则会将根据该传感控制指令生成的体感控制数据包与该时刻采集到的图像帧通过视频直播平台同步发送给用户终端,那么用户终端在输出该图像帧至视频播放装置的同时,也同步输出该体感控制数据包至体感传感装置。上述体感控制数据包的生成方法、数据格式及数据内容已在上文实施例中进行了详细阐述,在此不赘述。
作为本发明的一个实施例,当每个体感控制数据包中包含了M个身体点位上的N种类型的体感传感装置的控制信息时,则如图6所示,所述同步输出所述体感控制数据包至体感传感装置包括:
S601,从所述体感控制数据包中解析出M个身体点位的体感控制数据。
S602,从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息。
S603,将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中。
所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
基于图6对应的实施例,在体感控制装置输出体感控制数据包的过程中,可以精准地将体感控制数据输出至位于一具体身体点位的一具体体感传感装置中,那么对于全身M个身体点位上总共M*N个体感传感装置,便可以在相关的图像帧播放的过程中同时实现体感模拟的输出,带来全方位的体感体验。
此外,如上文所述,在上述的体感控制数据包帧格式的基础之上预设一个字符,若未对某个身体点位匹配某一类型的体感模拟,则在该体感控制数据包
中,将该身体点位对应的该类型的体感传感装置所占据的每个数据位均写入该预设字符,例如,写入预设字符0,以保证所有体感控制数据包的数据长度一致。那么在该方案之下,所述同步输出所述体感控制数据包至体感传感装置还包括:
若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;
所述j为大于或等于1的整数,且所述j小于或等于N。
即,若体感控制装置检测到具体某个身体点位上的具体某类体感传感装置对应的控制信息在体感数据包中所占据的每个数据位上均为预设字符,则在该时刻停止对该身体点位上的该体感传感装置输出控制信息,这样一来,虽然每个图像帧相关的体感控制数据包的数据长度是相同的,但是体感控制装置仍然可以根据每个数据位上的数据内容来判断是输出控制信息还是停止输出控制信息,从而灵活地实现了对多身体点位、多个体感传感装置的输出控制。
本发明实施例可以在视频直播的同时,实时地由直播终端生成并向用户终端传输体感控制数据,这样一来,用户终端可以在观看视频直播的同时体验到与视频直播内容相匹配的体感模拟,极大地提高了在线场景的可互动性。
对应于上文实施例所述的多媒体数据的实时输出方法,图7示出了本发明实施例提供的多媒体数据的实时输出装置的结构框图。为了便于说明,仅示出了与本实施例相关的部分。
参照图7,该装置包括:
第一接收单元71,在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;
第二接收单元72,若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;
输出单元73,输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
可选地,所述输出单元73包括:
第一解析子单元,从所述体感控制数据包中解析出M个身体点位的体感控制数据;
第二解析子单元,从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;
第一输出控制子单元,将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;
所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
可选地,所述输出单元73还包括:
第二输出控制子单元,若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;
所述j为大于或等于1的整数,且所述j小于或等于N。
可选地,所述预设字符为0。
对应于上文实施例所述的多媒体数据的实时输出方法,图8示出了本发明实施例提供的一种计算节点800的示意图。为了便于说明,仅示出了与本实施例相关的部分。
其中,计算节点800可能是包含计算能力的主机服务器,或者是个人计算机PC,或者是可携带的便携式计算机或终端等等,本发明具体实施例并不对计算节点的具体实现做限定。计算节点800包括:
处理器(processor)810,通信接口(Communications Interface)820,存储器(memory)830,总线840。
处理器810,通信接口820,存储器830通过总线840完成相互间的通信。
通信接口820,用于与网元通信,比如虚拟机管理中心、共享存储等。
处理器810,用于执行程序。
具体地,程序可以包括程序代码,所述程序代码包括计算机操作指令。
处理器810可能是一个中央处理器(Central Processing Unit,CPU),或者是特定集成电路(Application Specific Integrated Circuit,ASIC),或者是被配置成实施本发明实施例的一个或多个集成电路。
存储器830,用于存放程序。存储器830可能包含高速RAM存储器,也可能还包括非易失性存储器(non-volatile memory),例如至少一个磁盘存储器。
程序具体可以用于执行一种多媒体数据的实时输出方法,所述方法包括:
在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;
若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;
输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
可选地,所述同步输出所述体感控制数据包至体感传感装置包括:
从所述体感控制数据包中解析出M个身体点位的体感控制数据;
从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;
将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;
所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
可选地,所述同步输出所述体感控制数据包至体感传感装置还包括:
若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对
应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;
所述j为大于或等于1的整数,且所述j小于或等于N。
可选地,所述预设字符为0。
所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元完成,即将装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中,上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。另外,各功能单元的具体名称也只是为了便于相互区分,并不用于限制本申请的保护范围。上述装置中单元的具体工作过程,可以参考前述装置实施例中的对应过程,在此不再赘述。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同装置来实现所描述的功能,但是这种实现不应认为超出本发明的范围。
在本发明所提供的实施例中,应该理解到,所揭露的装置和装置,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,模块或单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个***,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口,装置或单元的间接耦合或通讯连接,可以是电性,机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元
显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本发明各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)或处理器执行本发明实施例各个实施例装置的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。
以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明实施例各实施例技术方案的精神和范围。
以上仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。
Claims (16)
- 一种多媒体数据的实时发送方法,其特征在于,所述方法包括:在视频直播过程中,对直播终端进行指令监听;当监听到所述直播终端生成了体感控制指令时,根据所述体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;将所述体感控制数据包与所述图像帧进行关联;将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
- 如权利要求1所述的方法,其特征在于,所述根据所述体感控制指令生成体感控制数据包包括:获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;所述M和所述N均为大于或等于1的整数。
- 如权利要求2所述的方法,其特征在于,所述根据所述体感控制指令生成体感控制数据包还包括;若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
- 如权利要求3所述的方法,其特征在于,所述预设字符为0。
- 一种多媒体数据的实时输出方法,其特征在于,所述方法包括:在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;输出所述图像帧至视频播放装置,并同步输出所述体感控制数据包至体感传感装置。
- 如权利要求5所述的方法,其特征在于,所述同步输出所述体感控制数据包至体感传感装置包括:从所述体感控制数据包中解析出M个身体点位的体感控制数据;从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
- 如权利要求6所述的方法,其特征在于,所述同步输出所述体感控制数据包至体感传感装置还包括:若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;所述j为大于或等于1的整数,且所述j小于或等于N。
- 如权利要求7所述的方法,其特征在于,所述预设字符为0。
- 一种多媒体数据的实时发送装置,其特征在于,所述装置包括:监听单元,用于在视频直播过程中,对直播终端进行指令监听;生成单元,用于当监听到所述直播终端生成了体感控制指令时,根据所述 体感控制指令生成体感控制数据包,并标记所述直播终端当前采集到的图像帧;关联单元,用于将所述体感控制数据包与所述图像帧进行关联;发送单元,用于将所述图像帧通过视频直播平台实时发送至用户终端,同时将与所述图像帧关联的所述体感控制数据包通过所述视频直播平台同步发送至所述用户终端,以使所述用户终端在输出所述图像帧至视频播放装置的同时,同步输出所述体感控制数据包至体感传感装置。
- 如权利要求9所述的装置,其特征在于,所述生成单元包括:获取子单元,用于获取M个身体点位的体感控制数据,每个身体点位的体感控制数据均包含N种类型的体感传感装置的控制信息;第一写入子单元,用于将获取到的所述M个身体点位的体感控制数据均写入所述体感控制数据包;所述M和所述N均为大于或等于1的整数。
- 如权利要求10所述的装置,其特征在于,所述生成单元还包括;第二写入子单元,用于若未获取到第i个身体点位的第j种类型的体感传感装置的控制信息,在所述体感控制数据包中该控制信息对应的每个数据位均写入预设字符;所述i和所述j均为大于或等于1的整数,且所述i小于或等于所述M,所述j小于或等于所述N。
- 如权利要求11所述的装置,其特征在于,所述预设字符为0。
- 一种多媒体数据的实时输出装置,其特征在于,所述装置包括:第一接收单元,用于在观看视频直播的过程中,通过视频直播平台接收直播终端实时发送的图像帧;第二接收单元,用于若所述图像帧关联了体感控制数据包,通过所述视频直播平台接收所述直播终端同步发送的所述体感控制数据包,所述体感控制数据包由所述直播终端在视频直播的过程中根据体感控制指令生成;输出单元,用于输出所述图像帧至视频播放装置,并同步输出所述体感控 制数据包至体感传感装置。
- 如权利要求13所述的装置,其特征在于,所述输出单元包括:第一解析子单元,用于从所述体感控制数据包中解析出M个身体点位的体感控制数据;第二解析子单元,用于从解析出的第i个身体点位的体感控制数据中解析出N种类型的体感传感装置的控制信息;第一输出控制子单元,用于将解析出的所述控制信息分别输出至第i个身体点位上的N种类型的体感传感装置中;所述M、所述N和所述i均为大于或等于1的整数,且所述i小于或等于M。
- 如权利要求14所述的装置,其特征在于,所述输出单元还包括:第二输出控制子单元,用于若所述体感控制数据包中,第i个身体点位的第j种类型的体感传感装置对应的控制信息在每个数据位上均为预设字符,则在该体感控制数据包相关的图像帧播放的同时,停止输出该控制信息;所述j为大于或等于1的整数,且所述j小于或等于N。
- 如权利要求14所述的装置,其特征在于,所述预设字符为0。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611058946.9 | 2016-11-22 | ||
CN201611058946.9A CN106534142B (zh) | 2016-11-22 | 2016-11-22 | 多媒体数据的实时传输方法及装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018095003A1 true WO2018095003A1 (zh) | 2018-05-31 |
Family
ID=58357196
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/086495 WO2018095003A1 (zh) | 2016-11-22 | 2017-05-29 | 多媒体数据的实时传输方法及装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN106534142B (zh) |
WO (1) | WO2018095003A1 (zh) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106534142B (zh) * | 2016-11-22 | 2018-04-20 | 包磊 | 多媒体数据的实时传输方法及装置 |
CN107491172B (zh) * | 2017-08-16 | 2020-10-09 | 歌尔科技有限公司 | 体感数据获取方法、装置及电子设备 |
CN109644294B (zh) | 2017-12-29 | 2020-11-27 | 腾讯科技(深圳)有限公司 | 一种直播分享的方法、相关设备及*** |
CN109840016A (zh) * | 2018-12-14 | 2019-06-04 | 维沃移动通信有限公司 | 一种虚拟交互方法及终端设备 |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101222286A (zh) * | 2006-11-03 | 2008-07-16 | Lg电子株式会社 | 传输广播信号的装置以及发送和接收广播信号的方法 |
CN101427580A (zh) * | 2005-12-23 | 2009-05-06 | 安布克斯英国有限公司 | 使用从内容流确定的指纹进行脚本同步 |
US20100275235A1 (en) * | 2007-10-16 | 2010-10-28 | Sanghyun Joo | Sensory effect media generating and consuming method and apparatus thereof |
US20120033937A1 (en) * | 2009-04-15 | 2012-02-09 | Electronics And Telecommunications Research Institute | Method and apparatus for providing metadata for sensory effect, computer-readable recording medium on which metadata for sensory effect are recorded, and method and apparatus for sensory reproduction |
CN104423701A (zh) * | 2013-08-23 | 2015-03-18 | 意美森公司 | 体育事件的触感使能的观看 |
CN104780403A (zh) * | 2014-03-28 | 2015-07-15 | 方小祥 | 基于脚本的体感视频提供***及方法 |
US20160080682A1 (en) * | 2014-09-13 | 2016-03-17 | Vicente Diaz | System and Method for haptic integration and generation in multimedia devices |
CN106249868A (zh) * | 2015-06-12 | 2016-12-21 | 意美森公司 | 广播触觉架构 |
CN106534142A (zh) * | 2016-11-22 | 2017-03-22 | 包磊 | 多媒体数据的实时传输方法及装置 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9019087B2 (en) * | 2007-10-16 | 2015-04-28 | Immersion Corporation | Synchronization of haptic effect data in a media stream |
WO2011020003A2 (en) * | 2009-08-14 | 2011-02-17 | K12 Inc. | Systems and methods for producing, delivering and managing educational material |
US9386063B2 (en) * | 2011-09-19 | 2016-07-05 | Comcast Cable Communications, Llc | Content storage and identification |
US9261960B2 (en) * | 2013-01-24 | 2016-02-16 | Immersion Corporation | Haptic sensation recording and playback |
US9064385B2 (en) * | 2013-03-15 | 2015-06-23 | Immersion Corporation | Method and apparatus to generate haptic feedback from video content analysis |
KR20150110356A (ko) * | 2014-03-21 | 2015-10-02 | 임머숀 코퍼레이션 | 센서의 데이터를 햅틱 효과들로 변환하는 시스템들 및 방법들 |
US9812165B2 (en) * | 2014-12-19 | 2017-11-07 | Immersion Corporation | Systems and methods for recording haptic data for use with multi-media data |
-
2016
- 2016-11-22 CN CN201611058946.9A patent/CN106534142B/zh active Active
-
2017
- 2017-05-29 WO PCT/CN2017/086495 patent/WO2018095003A1/zh active Application Filing
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101427580A (zh) * | 2005-12-23 | 2009-05-06 | 安布克斯英国有限公司 | 使用从内容流确定的指纹进行脚本同步 |
CN101222286A (zh) * | 2006-11-03 | 2008-07-16 | Lg电子株式会社 | 传输广播信号的装置以及发送和接收广播信号的方法 |
US20100275235A1 (en) * | 2007-10-16 | 2010-10-28 | Sanghyun Joo | Sensory effect media generating and consuming method and apparatus thereof |
US20120033937A1 (en) * | 2009-04-15 | 2012-02-09 | Electronics And Telecommunications Research Institute | Method and apparatus for providing metadata for sensory effect, computer-readable recording medium on which metadata for sensory effect are recorded, and method and apparatus for sensory reproduction |
CN104423701A (zh) * | 2013-08-23 | 2015-03-18 | 意美森公司 | 体育事件的触感使能的观看 |
CN104780403A (zh) * | 2014-03-28 | 2015-07-15 | 方小祥 | 基于脚本的体感视频提供***及方法 |
US20160080682A1 (en) * | 2014-09-13 | 2016-03-17 | Vicente Diaz | System and Method for haptic integration and generation in multimedia devices |
CN106249868A (zh) * | 2015-06-12 | 2016-12-21 | 意美森公司 | 广播触觉架构 |
CN106534142A (zh) * | 2016-11-22 | 2017-03-22 | 包磊 | 多媒体数据的实时传输方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN106534142A (zh) | 2017-03-22 |
CN106534142B (zh) | 2018-04-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018095003A1 (zh) | 多媒体数据的实时传输方法及装置 | |
US11252444B2 (en) | Video stream processing method, computer device, and storage medium | |
WO2019205886A1 (zh) | 字幕数据推送方法、字幕展示方法、装置、设备及介质 | |
US10027731B2 (en) | Selective capture with rapid sharing of user computer or mixed reality actions, states using interactive virtual streaming | |
CN106303555A (zh) | 一种基于混合现实的直播方法、装置和*** | |
WO2017185641A1 (zh) | 一种语音弹幕的生成、播放方法及其装置、客户端 | |
CN109448709A (zh) | 一种终端投屏的控制方法和终端 | |
JP6125680B2 (ja) | メッセンジャを利用したコンテンツ提供方法、システム、および記録媒体 | |
WO2018068481A1 (zh) | 一种双目 720 度全景采集*** | |
US20130225290A1 (en) | Wearable personal mini cloud game and multimedia device | |
TWI644235B (zh) | Wearable instant interaction system | |
TW201517609A (zh) | 媒體內容分享方法和終端設備及內容分享系統 | |
TW201220070A (en) | Cloud computing system and data accessing method thereof | |
WO2023066100A1 (zh) | 文件分享方法和装置 | |
CN104244027A (zh) | 音/视频数据实时传输并共享播放进程的控制方法及*** | |
CN105721904B (zh) | 显示装置和控制显示装置的内容输出的方法 | |
CN109361954B (zh) | 视频资源的录制方法、装置、存储介质及电子装置 | |
CN107977184A (zh) | 一种基于虚拟现实技术的音乐播放方法和装置 | |
WO2018095002A1 (zh) | 多媒体数据的编码、解码方法及编码、解码装置 | |
CN104717190A (zh) | 一种无线的增强现实传输方法 | |
WO2018095001A1 (zh) | 体感传感数据的通信处理方法及装置 | |
US11165842B2 (en) | Selective capture with rapid sharing of user or mixed reality actions and states using interactive virtual streaming | |
CN111541906A (zh) | 数据发送方法、装置、计算机设备及存储介质 | |
CN110794966B (zh) | Ar显示***及方法 | |
US10084849B1 (en) | System and method for providing and interacting with coordinated presentations |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17874934 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17874934 Country of ref document: EP Kind code of ref document: A1 |