WO2023241121A1 - 数据处理方法、控制装置、电子设备、计算机可读介质 - Google Patents

数据处理方法、控制装置、电子设备、计算机可读介质 Download PDF

Info

Publication number
WO2023241121A1
WO2023241121A1 PCT/CN2023/080357 CN2023080357W WO2023241121A1 WO 2023241121 A1 WO2023241121 A1 WO 2023241121A1 CN 2023080357 W CN2023080357 W CN 2023080357W WO 2023241121 A1 WO2023241121 A1 WO 2023241121A1
Authority
WO
WIPO (PCT)
Prior art keywords
control
multimedia file
data processing
processing method
multimedia
Prior art date
Application number
PCT/CN2023/080357
Other languages
English (en)
French (fr)
Inventor
罗迤宝
魏静波
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2023241121A1 publication Critical patent/WO2023241121A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
    • G06F9/44526—Plug-ins; Add-ons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/362—Software debugging
    • G06F11/3632—Software debugging of specific synchronisation aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating

Definitions

  • the embodiments of the present disclosure relate to the field of data processing technology, and in particular to a data processing method, a control device, an electronic device, and a computer-readable medium.
  • multimedia technologies such as virtual reality (VR) and augmented reality (AR) have been widely used, and the demand for interaction based on multimedia such as video and audio continues to increase.
  • VR virtual reality
  • AR augmented reality
  • control functions for external devices that can be realized are simple and single, and the real-time response of external devices and the consistency with multimedia files are poor, which cannot meet the needs of users.
  • Embodiments of the present disclosure relate to a data processing method, a control device, an electronic device, and a computer-readable medium.
  • An embodiment of the present disclosure provides a data processing method, which includes: determining a trigger point sequence according to a first multimedia file, where the trigger point sequence includes at least one control trigger point; determining a control instruction corresponding to each of the control trigger points, so The control instructions are used to control the controlled device in cooperation with the first multimedia file; generate an accompanying program according to the trigger point sequence and the control instructions corresponding to the control trigger points.
  • An embodiment of the present disclosure provides a control device, including: a storage unit on which a companion program generated according to the data processing method described in the first aspect of the embodiment of the present disclosure is stored; a processing unit capable of executing the storage The accompanying program stored in the unit is used to control the controlled equipment.
  • An embodiment of the present disclosure provides a computer-readable medium on which a companion program generated according to the data processing method described in the first aspect of the embodiment of the present disclosure is stored.
  • Embodiments of the present disclosure provide an electronic device, including: one or more processors; and a memory on which one or more programs are stored. When the one or more programs are executed by the one or more processors, causing the one or more processors to implement the data processing method described in the first aspect of the embodiment of the present disclosure; one or more I/O interfaces, connected between the processor and the memory, configured to implement the processing information exchange between the device and the memory.
  • Embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored.
  • the program is executed by a processor, the data processing method described in the first aspect of the embodiment of the present disclosure is implemented.
  • Figure 1 is a flow chart of a data processing method in an embodiment of the present disclosure
  • Figure 2 is a flow chart of some steps in another data processing method in an embodiment of the present disclosure.
  • Figure 3 is a flow chart of some steps in yet another data processing method in an embodiment of the present disclosure.
  • Figure 4 is a flow chart of some steps in yet another data processing method in an embodiment of the present disclosure.
  • Figure 5 is a flow chart of some steps in yet another data processing method in an embodiment of the present disclosure.
  • Figure 6 is a flow chart of some steps in yet another data processing method in an embodiment of the present disclosure.
  • FIG. 7 is a block diagram of a control device in an embodiment of the present disclosure.
  • Figure 8 is a block diagram of a computer-readable medium in an embodiment of the present disclosure.
  • Figure 9 is a block diagram of an electronic device in an embodiment of the present disclosure.
  • Figure 10 is a block diagram of a computer-readable medium in an embodiment of the present disclosure.
  • an embodiment of the present disclosure provides a data processing method, including:
  • the first multimedia file and the second multimedia file are both multimedia files. “First” and “Second” are only used to distinguish different multimedia files.
  • the multimedia file may be any one of audio files, video files, VR, AR, etc.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • the controlled device is also called a peripheral device, and the embodiment of the present disclosure does not impose special limitations on the controlled device.
  • the controlled device refers to a device that can provide users with sensory stimulation, interactive responses, etc. in conjunction with the playback of multimedia files.
  • the controlled device is a vibration motor, which can provide sensory stimulation to the user through vibration in response to the content played by the multimedia file.
  • control trigger point corresponds to the characteristic point in the first multimedia file that requires the controlled device to cooperate; the control instruction corresponding to the control trigger point is used to control the controlled device in cooperation with the first multimedia file.
  • the controlled device is controlled to provide corresponding sensory stimulation to the user by executing the control instruction.
  • a companion program generated according to the trigger point sequence and the control instruction corresponding to the control trigger point is used as a plug-in program for the multimedia file.
  • a companion program is executed while playing a multimedia file.
  • the companion program encapsulates at least one control trigger point that constitutes the trigger point sequence and the control instructions corresponding to each control trigger point.
  • the accompanying program executes the control instruction of the control trigger point corresponding to the characteristic point, and controls the controlled device to perform corresponding processing.
  • a trigger point sequence including at least one control trigger point is determined based on the multimedia file, and then the control instructions corresponding to each control trigger point for controlling the controlled device in conjunction with the multimedia file are determined, and finally the accompanying Program, when used as a plug-in program for multimedia files, can control the controlled device in conjunction with the playback of multimedia files.
  • Editing the companion program in the form of a plug-in program can set more and more complex control instructions in the companion program, which is conducive to achieving more complex control of peripherals, thereby providing users with richer peripheral functions; through the edited
  • the accompanying program cooperates with the playback control peripherals of multimedia files to ensure the consistency of multimedia playback and peripheral response, which is beneficial to improving the user experience.
  • the embodiment of the present disclosure does not place any special limitations on how to determine the trigger point sequence based on the multimedia file.
  • the sequence of trigger points is determined based on the timeline of the multimedia file according to the content of the multimedia file.
  • determining a trigger point sequence according to the first multimedia file includes:
  • performing feature point analysis on the first multimedia file refers to extracting feature points in the multimedia file that require the cooperation of the controlled device.
  • the first multimedia file is a video file, and the extracted feature points that require the cooperation of the controlled device are key frames in the video file.
  • the first multimedia file is a multimedia file other than a video file, and the extracted feature points that require the cooperation of the controlled device are key points in the multimedia file.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • the extracted control trigger points correspond to the feature points of the first multimedia file that are feature analyzed.
  • multimedia analysis software can be used to analyze the first multimedia file and extract key frames or key points; artificial intelligence (AI, Artificial Intelligence) can also be used to analyze the first multimedia file. Extract keyframes or keypoints.
  • AI Artificial Intelligence
  • extract keyframes or keypoints can be used to analyze the first multimedia file.
  • determining the trigger time of each control trigger point according to the timeline of the first multimedia file means collating the control trigger point with the content of the first multimedia file, for example, through key frames or key points. Proofreading is performed, and the timing of the control trigger point for controlling the controlled device is calibrated according to the timeline of the first multimedia file, so that the timing of the control trigger point is consistent with the timeline of the first multimedia file to achieve the goal of matching the first Optimal triggering timing for multimedia file content.
  • the determined triggering time of each control trigger point constitutes the control timeline.
  • the companion program is used as a plug-in program for media files, it is executed based on the control timeline, so that when the multimedia file is played to the time point that requires the cooperation of the controlled device, the corresponding control instructions can be triggered to control the controlled device. It can ensure the consistency of multimedia playback and peripheral response, which is beneficial to improving user experience.
  • the embodiment of the present disclosure does not place any special limitations on how to determine the control instructions corresponding to each control trigger point.
  • determining the control instructions corresponding to each of the control trigger points includes:
  • editing the control instructions corresponding to each control trigger point in point-by-point editing and/or batch editing can enable the controlled device triggered by the multimedia file to have complex control actions, thereby enriching the functions implemented in conjunction with the multimedia file.
  • Sensory stimulation effect can enable the controlled device triggered by the multimedia file to have complex control actions, thereby enriching the functions implemented in conjunction with the multimedia file.
  • performing time interval conflict detection on the control instructions corresponding to each control trigger point refers to analyzing the time duration between the control action execution time of the controlled device and the time interval between two adjacent control trigger points to ensure that the controlled device executes the control action. The time when the control device performs the control action does not conflict with the time interval between two adjacent control trigger points.
  • the same time point may correspond to one control trigger point or multiple control trigger points. That is, among multiple control trigger points, the triggering times of different control trigger points may be different, or the triggering times of several control trigger points may be the same.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • corresponding control instructions can be set separately for each control trigger point, thereby triggering multiple control instructions at the same time point; each control trigger point can also be The control instructions corresponding to the control trigger point are integrated into a composite control instruction, thereby triggering a composite control instruction at the same point in time.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • editing the control instructions corresponding to each of the control trigger points includes:
  • a composite control instruction corresponding to the time point is generated according to the control instructions corresponding to the multiple control trigger points.
  • Integrating multiple control instructions corresponding to the same point in time into a composite control instruction can help avoid system complexity, ensure a unique timeline, and improve the reliability of the timeline.
  • the data processing method further includes:
  • a parameter interface for adjusting the companion program is reserved in the packaged companion program.
  • At least one of the timing of the control trigger point in the accompanying program and the control instruction corresponding to the control trigger point can be adjusted.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • adjusting the timing of the control trigger points in the companion program refers to adjusting the triggering time of each control trigger point.
  • the user may debug and calibrate the companion program through the parameter interface. For example, users can fine-tune the control timeline in the companion program based on their own feelings to achieve the best match for their personal feelings. It can also be that after generating the companion program, the user can make personalized adjustments and settings to the companion program. For example, add or modify control instructions and add peripheral functions in the companion program to meet the personalized and diverse needs of users.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • a user interface (UI, User Interface) is provided for the user to adjust the timing of the control trigger points in the accompanying program and/or the control instructions corresponding to the control trigger points.
  • a parameter interface for modifying the companion program is provided in the companion program, and the companion program can be re-edited, which is conducive to meeting the user's personalized and diversified needs for peripheral functions, and is conducive to network communication.
  • the companion program generated based on the first multimedia file can also be applied to multimedia files with the same content but different versions.
  • the data processing method further includes:
  • the first multimedia file and the second multimedia file have the same content but different versions.
  • some video files may have advertisements or different titles added to them, resulting in different versions of the video files.
  • the timelines of different versions of multimedia files are often inconsistent.
  • adjusting the timing of the control trigger points in the companion program refers to adjusting the triggering time of each control trigger point.
  • aligning the timeline of the second multimedia file with the timeline of the first multimedia file thereby aligning the time alignment of the feature films of the first multimedia file and the second multimedia file;
  • the companion program is synchronized with the second multimedia file, so that the control timeline of the companion program corresponds to the feature content of the second multimedia file, thereby achieving a precise timing state of the feature content of the second multimedia file relative to the companion program.
  • the multimedia player and the controlled device can be interconnected through wired or wireless interconnection.
  • the embodiments of the present disclosure do not impose special limitations on this.
  • the edited companion program can be stored on the multimedia player or on the controlled device.
  • the embodiments of the present disclosure do not impose special limitations on this. It should be noted that storing the companion program on the multimedia player will facilitate users to modify and adjust the companion program; storing the companion program on the controlled device will help ensure the stable operation of the controlled device.
  • the data processing method further includes:
  • adjusting the timing of the control trigger points in the companion program refers to adjusting the triggering time of each control trigger point.
  • multiple controlled devices are included, and in the companion program, different controlled devices correspond to different control instructions.
  • the delays in communication between different controlled devices and the multimedia player can be the same or different. After synchronizing the clocks of the multimedia player that plays multimedia files with the controlled device, it is necessary to adjust the triggering time of the control trigger point corresponding to each controlled device in the companion program, that is, adjust the timing of the companion program based on the clock synchronization result. .
  • the companion program when used as a plug-in program of a multimedia file to control the controlled device, synchronizing the clocks of the multimedia player and the controlled device can avoid delays caused by communication, thereby ensuring multimedia playback Consistency with peripheral response helps improve user experience.
  • the multimedia player has distance, sound, light, electricity, etc. sensors, and the trigger point sequence is determined based on the sensors.
  • determining the trigger point sequence according to the first multimedia file includes:
  • At least one control trigger point is determined according to the value of the sensor of the multimedia player that plays the multimedia file.
  • At least one is determined based on a value of a sensor of a multimedia player that plays a multimedia file.
  • the control trigger points include:
  • one control trigger point is set corresponding to each threshold value of the sensor value, wherein the sensor is set with at least one threshold value.
  • the trigger point sequence can be determined based on the content of the multimedia file and the timeline of the multimedia file; the trigger point sequence can also be determined based on the sensor; or the trigger point sequence can be determined based on the content of the multimedia file and the time of the multimedia file.
  • the trigger point sequence is determined based on the line and the trigger point sequence based on the sensor.
  • the trigger point sequence is determined based on the timeline and/or sensors, which can adapt to more complex scenarios, thereby providing a more realistic experience for users.
  • an embodiment of the present disclosure provides a control device, including:
  • a storage unit 101 on which is stored a companion program generated according to the data processing method described in the first aspect of the embodiment of the present disclosure
  • Processing unit 102 the processing unit 102 can execute the accompanying program stored in the storage unit 101 to control the controlled device.
  • an embodiment of the present disclosure provides a computer-readable medium on which a companion program generated according to the data processing method described in the first aspect of the embodiment of the present disclosure is stored.
  • an electronic device which includes:
  • processors 201 one or more processors 201;
  • the memory 202 has one or more programs stored thereon. When the one or more programs are executed by one or more processors, the one or more processors implement the data processing method described in the first aspect of the embodiment of the present disclosure;
  • One or more I/O interfaces 203 are connected between the processor and the memory, and are configured to implement information exchange between the processor and the memory.
  • the processor 201 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.
  • the memory 202 is a device with data storage capabilities, including but not limited to a random access memory (RAM, more specifically Such as SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH);
  • the I/O interface (read-write interface) 203 is connected between the processor 201 and the memory 202 , can realize information interaction between the processor 201 and the memory 202, which includes but is not limited to a data bus (Bus), etc.
  • processor 201 memory 202, and I/O interface 203 are connected to each other and, in turn, to other components of the computing device via bus 204.
  • an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored.
  • the program is executed by a processor, the data processing method described in the first aspect of the embodiment of the present disclosure is implemented.
  • This embodiment provides a peripheral control system accompanying multimedia playback, including:
  • Editing of specific control instructions for each control trigger point can be done point-by-point or classified and batch-edited. Through these editing tasks, peripherals triggered by multimedia content can have complex control actions to coordinate with the multimedia content. to achieve multi-sensory stimulation effects.
  • control program is packaged to obtain a companion program, which can be used as a multimedia playback plug-in program to cooperate with customized multimedia content.
  • Peripheral devices and programs can be interconnected in a variety of modes, either wired or wireless.
  • the companion program that has been written can be placed on the multimedia player, or it can be downloaded in advance to the cache of the controlled peripheral.
  • the selection is mainly to meet the convenience of the user.
  • the remote control of multimedia is mainly to facilitate the modification of the program. Downloading to the controlled peripheral is mainly to ensure that the peripheral works stably and is not interfered by the communication process.
  • the controlled device still needs to synchronize its clock with the multimedia control terminal to ensure that there is no timing error during operation.
  • Peripherals can be set up with timing adjustment controls to balance the delay issues caused by communication between programs and peripherals by moving the timing forward and backward.
  • This embodiment uses multiple groups of controllable vibration motors as an example, and the vibrations of the multiple groups of motors are used to respond to the multimedia content that needs to be stimulated. But this embodiment is not limited to such products.
  • the controllable motor multimedia accompanying system includes four functional module systems:
  • video media is just an example.
  • This embodiment includes multiple types of multimedia playback fluids such as video, audio, VR animation, and AR visual effects.
  • the product of this embodiment is these multimedia plug-in accompanying playback plug-ins, which are mainly responsible for Control and modification of other sensing devices.
  • the manual or AI identification of key points in this embodiment can also be used to calibrate the alignment of the subsequent video timeline and the timeline of the accompanying plug-in file, for example, setting the interception of a multimedia key frame when comparing it with other videos of the same name. It can quickly find the similarities of key frames, export the timeline of the video with the same name, and align the timeline with the standard video.
  • the key frames After the key frames are identified, they can be combined into a control timeline through the system of this embodiment, and this timeline is used as a control sequence to control the corresponding response module along with the playback of the multimedia. And set an adjustable window for the time point of the timeline on the UI surface, which is used to control the fine-tuning settings of the trigger time point.
  • This embodiment uses motors as keyframe response control, but it should actually have more applications in more complex VR and AR experience scenarios.
  • the playback system will have sensors (distance, sound, light, electricity), etc. These sensors can also trigger the implementation of the control program.
  • the timeline or key frames are not the only trigger logic. The purpose is also to create a more realistic experience for the user.
  • time key frames are the main trigger control logic, but in the scene of interactive VR game, the companion program that provides control logic according to the threshold of the sensor can be used in the form of a plug-in to help the game pass through many dimensions. realistic response.
  • This embodiment is a companion playback plug-in system that is independent of multimedia content and can trigger process control based on the multimedia content editing timeline to control a peripheral control system that is conducive to multimedia interactive experience.
  • This embodiment includes a multimedia key point or key frame identification, recording, and editing system that can turn multimedia interactive tasks into a timeline control file containing matching multimedia content, and embed complex control actions into this accompanying playback file. .
  • This embodiment includes a timeline adjustment system. Since there are many versions of multimedia content circulating on the market, some multimedia titles will have advertisements for suppressors embedded in them at the beginning, end or in the middle, resulting in inconsistent timelines among many versions of the same multimedia.
  • This embodiment Contains a keyframe matching function that can align the keyframe timelines of multiple versions of multimedia files so that the control file can be synchronized with the multimedia content.
  • This accompanying playback plug-in also provides a manual adjustment interface for timeline and control action modification. Users can also manually fine-tune the timeline according to their own feelings to achieve the best match for their personal feelings. And it can be encapsulated into plug-in files for communication on the Internet.
  • Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. removable, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or may Any other medium used to store desired information and that can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .
  • Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a general illustrative sense only and not for purpose of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone, or may be used in conjunction with other embodiments, unless expressly stated otherwise. Features and/or components are used in combination. Accordingly, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

一种数据处理方法、控制装置、一种电子设备、一种计算机可读介质，上述方法包括：根据第一多媒体文件确定触发点序列，所述触发点序列包括至少一个控制触发点（S1）；确定各个所述控制触发点对应的控制指令，所述控制指令用于配合所述第一多媒体文件控制被控设备（S2）；根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序（S3）。

Description

数据处理方法、控制装置、电子设备、计算机可读介质 技术领域
本公开实施例涉及数据处理技术领域，特别涉及一种数据处理方法、一种控制装置、一种电子设备、一种计算机可读介质。
背景技术
随着多媒体技术的发展，虚拟现实(VR，Virtual Reality)、增强现实(AR，Augmented Reality)等多媒体技术得到广泛应用，基于视频、音频等多媒体的互动需求不断提升。为了实现基于视频、音频的互动，需要在播放视频、音频等多媒体文件的同时，控制外部设备为用户模拟真实的临场感受，并提供实时的互动响应。
在一些相关技术中，能够实现的对外部设备的控制功能简单、单一，且外部设备响应的实时性、与多媒体文件的一致性较差，无法满足用户的需求。
发明内容
本公开实施例涉及一种数据处理方法、一种控制装置、一种电子设备、一种计算机可读介质。
本公开实施例提供一种数据处理方法，包括：根据第一多媒体文件确定触发点序列，所述触发点序列包括至少一个控制触发点；确定各个所述控制触发点对应的控制指令，所述控制指令用于配合所述第一多媒体文件控制被控设备；根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序。
本公开实施例提供一种控制装置，包括：存储单元，其上存储有根据本公开实施例第一方面所述的数据处理方法生成的伴随程序；处理单元，所述处理单元能够执行所述存储单元中存储的伴随程序，以对被控设备进行控制。
本公开实施例提供一种计算机可读介质，其上存储有根据本公开实施例第一方面所述的数据处理方法生成的伴随程序。
本公开实施例提供一种电子设备，包括：一个或多个处理器；存储器，其上存储有一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本公开实施例第一方面所述的数据处理方法；一个或多个I/O接口，连接在所述处理器与存储器之间，配置为实现所述处理器与存储器的信息交互。
本公开实施例提供一种计算机可读介质，其上存储有计算机程序，所述程序被处理器执行时实现本公开实施例第一方面所述的数据处理方法。
附图说明
图1是本公开实施例中一种数据处理方法的流程图；
图2是本公开实施例中另一种数据处理方法中部分步骤的流程图；
图3是本公开实施例中又一种数据处理方法中部分步骤的流程图；
图4是本公开实施例中再一种数据处理方法中部分步骤的流程图；
图5是本公开实施例中再一种数据处理方法中部分步骤的流程图；
图6是本公开实施例中再一种数据处理方法中部分步骤的流程图；
图7是本公开实施例中一种控制装置的组成框图；
图8是本公开实施例中一种计算机可读介质的组成框图；
图9是本公开实施例中一种电子设备的组成框图；
图10是本公开实施例中一种计算机可读介质的组成框图。
具体实施方式
为使本领域的技术人员更好地理解本公开的技术方案，下面结合附图对本公开实施例提供的数据处理方法、控制装置、电子设备、计算机可读介质进行详细描述。
在下文中将参考附图更充分地描述示例实施例，但是所述示例实施例可以以不同形式来体现且不应当被解释为限于本文阐述的实施例。反之，提供这些实施例的目的在于使本公开透彻和完整，并将使本领域技术人员充分理解本公开的范围。
在不冲突的情况下，本公开各实施例及实施例中的各特征可相互组合。
如本文所使用的，术语“和/或”包括一个或多个相关列举条目的任何和所有组合。
本文所使用的术语仅用于描述特定实施例，且不意欲限制本公开。如本文所使用的，单数形式“一个”和“该”也意欲包括复数形式，除非上下文另外清楚指出。还将理解的是，当本说明书中使用术语“包括”和/或“由……制成”时，指定存在所述特征、整体、步骤、操作、元件和/或组件，但不排除存在或添加一个或多个其它特征、整体、步骤、操作、元件、组件和/或其群组。
除非另外限定，否则本文所用的所有术语(包括技术和科学术语)的含义与本领域普通技术人员通常理解的含义相同。还将理解，诸如那些在常用字典中限定的那些术语应当被解释为具有与其在相关技术以及本公开的背景下的含义一致的含义，且将不解释为具有理想化或过度形式上的含义，除非本文明确如此限定。
参照图1，本公开实施例提供一种数据处理方法，包括：
S1、根据第一多媒体文件确定触发点序列，所述触发点序列包括至少一个控制触发点；
S2、确定各个所述控制触发点对应的控制指令，所述控制指令用于配合所述第一多媒体文件控制被控设备；
S3、根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序。
需要说明的是，在本公开实施例中，第一多媒体文件、第二多媒体文件都是多媒体文件。“第一”、“第二”仅为了区分不同的多媒体文件。
在本公开实施例中，多媒体文件可以是音频文件、视频文件、VR、AR等中的任意一种。本公开实施例对此不做特殊限定。
在本公开实施例中，被控设备也称外设，本公开实施例对被控设备不做特殊限定。在一些实施例中，被控设备是指能够配合多媒体文件的播放为用户提供感官刺激、互动响应等的设备。例如，被控设备为振动马达，能够通过振动呼应多媒体文件播放的内容为用户提供感官刺激。
在本公开实施例中，控制触发点与第一多媒体文件中需要被控设备配合的特征点对应；控制触发点对应的控制指令用于配合第一多媒体文件控制被控设备。例如，配合第一多媒体文件的播放，通过执行控制指令控制被控设备为用户提供相应的感官刺激。
在本公开实施例中，根据触发点序列和控制触发点对应的控制指令生成的伴随程序，作为多媒体文件的外挂程序使用。例如，播放多媒体文件的同时执行伴随程序，伴随程序中封装有组成触发点序列的至少一个控制触发点以及各个控制触发点对应的控制指令，当多媒体文件播放到需要被控设备配合的特征点时，伴随程序则执行该特征点对应的控制触发点的控制指令，控制被控设备执行相应的处理。
本公开实施例提供的数据处理方法中，根据多媒体文件确定包括至少一个控制触发点的触发点序列，然后确定各个控制触发点对应的用于配合多媒体文件控制被控设备的控制指令，最终得到伴随程序，当伴随程序作为多媒体文件的外挂程序使用时，能够配合多媒体文件的播放对被控设备进行控制。以外挂程序的形式编辑伴随程序，能够在伴随程序中设置更多、更复杂的控制指令，有利于实现对外设更加复杂的控制，从而为用户提供更加丰富的外设功能；通过已经编辑完成的伴随程序配合多媒体文件的播放控制外设，还能够确保多媒体播放与外设响应的一致性，有利于提升用户体验。
本公开实施例对如何根据多媒体文件确定触发点序列不做特殊限定。
在一些实施例中，根据多媒体文件的内容，基于多媒体文件的时间线确定触发点序列。
相应地，在一些实施例中，参照图2，根据第一多媒体文件确定触发点序列，包括：
S11、对所述第一多媒体文件进行特征点分析，提取至少一个所述控制触发点；
S12、根据所述第一多媒体文件的时间线确定各个所述控制触发点的触发时间，得到所述触发点序列。
在本公开实施例中，对第一多媒体文件进行特征点分析，是指提取多媒体文件中需要被控设备配合的特征点。在一些实施例中，第一多媒体文件为视频文件，提取的需要被控设备配合的特征点为视频文件中的关键帧。在一些实施例中，第一多媒体文件为视频文件以外的其他多媒体文件，提取的需要被控设备配合的特征点为多媒体文件中的关键点。本公开实施例对此不做特殊限定。提取的控制触发点与对第一多媒体文件进行特征分析的特征点对应。
在本公开实施例中，可以利用多媒体分析软件对第一多媒体文件进行分析，提取关键帧或关键点；也可以利用人工智能(AI，Artificial Intelligence)对第一多媒体文件进行分析，提取关键帧或关键点。本公开实施例对此不做特殊限定。
在一些实施例中，根据第一多媒体文件的时间线确定各个控制触发点的触发时间，是指将控制触发点与第一多媒体文件的内容进行校对，例如通过关键帧或关键点进行校对，将对被控设备进行控制的控制触发点按照第一多媒体文件的时间线进行时序校准，使得控制触发点的时序与第一多媒体文件的时间线一致，达到配合第一多媒体文件内容的最佳触发时序。
确定的各个控制触发点的触发时间组成控制时间轴。伴随程序作为媒体文件的外挂程序使用时，基于控制时间轴执行，从而能够在多媒体文件播放到需要被控设备配合的时间点，触发对应的控制指令对被控设备进行控制。能够确保多媒体播放与外设响应的一致性，有利于提升用户体验。
本公开实施例对于如何确定各个控制触发点对应的控制指令不做特殊限定。
在一些实施例中，参照图3，确定各个所述控制触发点对应的控制指令，包括：
S21、以逐点编辑和/或批处理编辑方式，编辑各个所述控制触发点对应的控制指令；
S22、对各个所述控制触发点对应的控制指令执行时间间隔冲突检测。
在本公开实施例中，以逐点编辑和/或批处理编辑方式编辑各个控制触发点对应的控制指令，能够使多媒体文件触发的被控设备具有复杂的控制动作，从而丰富配合多媒体文件实现的感官刺激效果。
在一些实施例中，对各个控制触发点对应的控制指令执行时间间隔冲突检测，是指分析被控设备执行控制动作的时长与相邻的两个控制触发点之间的时间间隔，以确保被控设备执行控制动作的时间与相邻的两个控制触发点之间的时间间隔不冲突。
在本公开实施例中，同一个时间点可以对应一个控制触发点，也可以对应多个控制触发点。即，多个控制触发点中，不同控制触发点的触发时间可以各不相同，或若干个控制触发点的触发时间相同。本公开实施例对此不做特殊限定。
在本公开实施例中，当同一个时间段对应多个控制触发点时，可以为每个控制触发点单独设置对应的控制指令，从而在同一个时间点触发多条控制指令；也可以将各个控制触发点对应的控制指令整合为一条复合控制指令，从而在同一个时间点触发一条复合控制指令。本公开实施例对此不做特殊限定。
相应地，在一些实施例中，编辑各个所述控制触发点对应的控制指令，包括：
当一个时间点对应多个所述控制触发点时，根据所述多个控制触发点对应的控制指令生成所述时间点对应的复合控制指令。
将同一个时间点对应的多条控制指令整合为一条复合控制指令，有利于避免***复杂度，确保时间线唯一，提高时间线的可靠性。
在一些实施例中，参照图4，所述数据处理方法还包括：
S4、响应于参数接口的修改指令，根据所述修改指令对所述伴随程序中所述控制触发点的时序、和/或所述控制触发点对应的控制指令进行调整。
在本公开实施例中，在封包好的伴随程序中预留了对伴随程序进行调整的参数接口。
在本公开实施例中，可以对伴随程序中控制触发点的时序、控制触发点对应的控制指令中的至少一者进行调整。本公开实施例对此不做特殊限定。
在本公开实施例中，对伴随程序中控制触发点的时序进行调整，是指对各个控制触发点的触发时间进行调整。
在本公开实施例中，可以是在生成伴随程序的过程中，用户通过参数接口对伴随程序进行调试、校准。例如，用户根据自身感受微调伴随程序中的控制时间轴，以达到个人感受的最佳匹配。也可以是在生成伴随程序后，用户对伴随程序进行个性化调整和设置。例如，在伴随程序中增加或修改控制指令，新增外设功能，以满足用户个性化、多样化需求。本公开实施例对此不做特殊限定。
在一些实施例中，为用户提供人机交互(UI，User Interface)界面，用于对伴随程序中控制触发点的时序、和/或控制触发点对应的控制指令进行调整。
在本公开实施例中，在伴随程序中提供对伴随程序进行修改的参数接口，能够对伴随程序进行再编辑，有利于满足用户对外设功能的个性化、多样化需求，同时有利于网络交流。
在本公开实施例中，基于第一多媒体文件生成的伴随程序还可以被应用于内容相同、版本不同的多媒体文件。
在一些实施例中，参照图5，所述数据处理方法还包括：
S51、对所述第一多媒体文件和第二多媒体文件进行关键帧对比，将所述第二多媒体文件的时间线与所述第一多媒体文件的时间线对齐，所述第一多媒体文件和所述第二多媒体文件为同一多媒体文件的不同版本；
S52、根据将所述第二多媒体文件的时间线与所述第一多媒体文件的时间线对齐的结果，对所述伴随程序的时序进行调整，以使所述伴随程序与所述第二多媒体文件同步。
在本公开实施例中，第一多媒体文件和第二多媒体文件内容相同、版本不同。例如，有些视频文件中会植入广告或增加不同的片头，从而产生不同版本的视频文件。不同版本的多媒体文件的时间线通常不一致。
在本公开实施例中，对伴随程序中控制触发点的时序进行调整，是指对各个控制触发点的触发时间进行调整。
在一些实施例中，将第二多媒体文件的时间线与第一多媒体文件的时间线对齐，从而将第一多媒体文件与第二多媒体文件的正片的时间校准一致；将伴随程序与第二多媒体文件同步，从而将伴随程序的控制时间轴与第二多媒体文件的正片内容对应，达到第二多媒体文件的正片内容相对伴随程序的精准时序状态。
在本公开实施例中，多媒体播放端与被控设备可以通过有线互联，也可以通过无线互联。本公开实施例对此不做特殊限定。
在本公开实施例中，编辑完成的伴随程序可以存储在多媒体播放端，也可以存储在被控设备。本公开实施例对此不做特殊限定。需要说明的是，将伴随程序存储在多媒体播放端，有利于方便用户对伴随程序进行修改、调整；将伴随程序存储在被控设备，有利于确保被控设备工作稳定。
在一些实施例中，参照图6，在根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序之后，所述数据处理方法还包括：
S61、将播放多媒体文件的多媒体播放端与所述被控设备进行时钟同步；
S62、根据时钟同步结果对所述伴随程序的时序进行调整。
在本公开实施例中，对伴随程序中控制触发点的时序进行调整，是指对各个控制触发点的触发时间进行调整。
在一些实施例中，包括多个被控设备，在伴随程序中，不同被控设备对应不同的控制指令。而不同的被控设备与多媒体播放端进行通信的延迟可以相同，也可以不同。在将播放多媒体文件的多媒体播放端与被控设备进行时钟同步后，需要针对伴随程序中各个被控设备对应的控制触发点的触发时间进行调整，即根据时钟同步结果对伴随程序的时序进行调整。
在本公开实施例中，在将伴随程序作为多媒体文件的外挂程序使用来控制被控设备时，将多媒体播放端与被控设备进行时钟同步，能够避免通讯带来的延迟，从而能够确保多媒体播放与外设响应的一致性，有利于提升用户体验。
在一些实施例中，多媒体播放端具有距离、声、光、电等传感器，根据传感器确定触发点序列。
在一些实施例中，根据第一多媒体文件确定触发点序列，包括：
根据播放多媒体文件的多媒体播放端的传感器的数值，确定至少一个所述控制触发点。
在一些实施例中，根据播放多媒体文件的多媒体播放端的传感器的数值，确定至少一个 所述控制触发点，包括：
针对任意一个传感器，对应于传感器的数值的每一个阈值，设置一个所述控制触发点，其中，传感器设置有至少一个阈值。
在本公开实施例中，可以只根据多媒体文件的内容，基于多媒体文件的时间线确定触发点序列；也可以只根据传感器确定触发点序列；还可以既根据多媒体文件的内容，基于多媒体文件的时间线确定触发点序列，同时也根据传感器确定触发点序列。本公开实施例对此不做特殊限定。
在本公开实施例中，基于时间线和/或传感器确定触发点序列，能够适应更多复合的场景，从而能够为用户提供更加真实的体验。
参照图7，本公开实施例提供一种控制装置，包括：
存储单元101，其上存储有根据本公开实施例第一方面所述的数据处理方法生成的伴随程序；
处理单元102，所述处理单元102能够执行所述存储单元101中存储的伴随程序，以对被控设备进行控制。
参照图8，本公开实施例提供一种计算机可读介质，其上存储有根据本公开实施例第一方面所述的数据处理方法生成的伴随程序。
参照图9，本公开实施例提供一种电子设备，其包括：
一个或多个处理器201；
存储器202，其上存储有一个或多个程序，当一个或多个程序被一个或多个处理器执行，使得一个或多个处理器实现本公开实施例第一方面所述的数据处理方法；
一个或多个I/O接口203，连接在处理器与存储器之间，配置为实现处理器与存储器的信息交互。
其中，处理器201为具有数据处理能力的器件，其包括但不限于中央处理器(CPU)等；存储器202为具有数据存储能力的器件，其包括但不限于随机存取存储器(RAM，更具体如SDRAM、DDR等)、只读存储器(ROM)、带电可擦可编程只读存储器(EEPROM)、闪存(FLASH)；I/O接口(读写接口)203连接在处理器201与存储器202间，能实现处理器201与存储器202的信息交互，其包括但不限于数据总线(Bus)等。
在一些实施例中，处理器201、存储器202和I/O接口203通过总线204相互连接，进而与计算设备的其它组件连接。
参照图10，本公开实施例提供一种计算机可读介质，其上存储有计算机程序，程序被处理器执行时实现本公开实施例第一方面所述的数据处理方法。
为了使本领域技术人员能够更清楚地理解本公开实施例提供的技术方案，下面通过具体的实施例，对本公开实施例提供的技术方案进行详细说明：
实施例一
本实施例提供一种伴随多媒体播放的外设控制***，包括：
1)控制触发点建立***
a.通过图像或音频分析软件，对多媒体的视频图像或音频特征点进行分析，把外设需要配合的控制触发点提取出来，并精确比较每个控制触发点之间的相对时间关系。
b.把提取出控制触发点和多媒体内容进行校对(通过视频关键帧的时间点)，通过时续的 修正把控制触发点的按多媒体的时间线进行时序校准，达到配合多媒体内容的最佳的触发时序，输出多媒体和控制时序的统一时间线。
2)关键点的控制内容编辑***
a.针对每个控制触发点的具体控制指令的编辑，可以逐点编辑，也可以分类批处理编辑，通过这些编辑工作，可以让多媒体内容触发的外设有复杂的控制动作，配合多媒体内容的来实现多感官刺激效果。
b.分析外设配合动作时间长度和处于控制触发点两个相邻的触发时间间隔，保证外设配合时间和间隔时间不冲突，复合的多指令多动作的外设控制可以通过单控制触发点编辑实现，不要通过一个时间点触发多个控制流程实现，避免增加***复杂度，保证时间线的唯一和可靠。
c.控制编辑和时间间隔分析完成后对控制程序进行封包，得到伴随程序，作为一个多媒体播放的外挂程序配合定制的多媒体内容使用。
d.在封包好的伴随程序中留出可调时序和控制程序的参数接口，以便用户在播放时可以针对多媒体内容对伴随程序进行个性化的调整和设置。
3)伴随程序的封包和互联网交互***
a.针对同一个多媒体内容产品，网上可能流传了各种不同的版本，有些视频会植入广告或者增加不同的片头，导致初始编辑的控制程序时间线无法一一匹配，因此需要把不同版本的视频做关键帧比对，通过前移后移操作，把正片内容的时间校准一致。
b.把伴随程序时间线和已经校对好的多媒体内容做对应，达到多媒体内容相对原始伴随程序的精准时序状态。
4)伴随程序和外设互联***
a.外设和程序互联可采用多种模式，可以是有线互联，也可以是无线互联。已经写好的伴随程序可以放在多媒体播放端，也可以提前下载到被控制外设的缓存中，选择主要为了满足用户的方便快捷，理论上在多媒体远端控制主要是为了方便程序的修改，而下载到被控外设中主要是为了外设的工作稳定，不受通讯过程的干扰，但是被控设备还是需要和多媒体控制端做时钟同步，保证在工作中没有时序误差。
b.外设可设置时序调整控制，通过前移和后移时序来平衡伴随程序和外设通讯带来的延迟问题。
实施例二
作为视频伴随的外设应用很多，主要是为多媒体观众提供多感官刺激的场景，本实施例以可控多组振动马达为实施例，通过多组马达的振动来呼应多媒体需要刺激的内容。但本实施例不仅限于此类产品。
可控马达多媒体伴随***包含四大功能模块***：
1.控制关键点建立***；
2.关键点的控制内容编辑***；
3.控制程序的封包和互联网交互***；
4.控制程序和外设互联***。
在本实施例中，视频媒体只是举例，本实施例包含视频、音频、VR动画以及AR视效等多种类型的多媒体播放流体。本实施例的产品是这些多媒体的外挂伴随播放插件，主要负责 对其他感知器件的控制和修改。
本实施例中的手动或AI识别关键点这个环节也可为后续的视频时间轴和伴随插件文件的时间轴的对齐做校准，例如设置一个多媒体关键帧的截取，在和其他同名视频比对时能够迅速找出关键帧的相似点，导出同名视频的时间轴，比对标准视频对齐时间轴。
关键帧的识别后可以通过本实施例***并且组合成一个控制时间轴，以此时间轴为控制序来伴随多媒体的播放控制相应的响应模组。并且在UI面设置时间轴的时间点的可调窗口，用于控制触发时间点的微调设置。
本实施例是以马达为关键帧的响应控制，但在更加复杂的VR，AR的体验场景中其实应该有更多的应用，比如会播放***会有传感器(距离，声，光，电)等这些传感器也能触发控制程序的实施，在这种复合的场景中时间轴或者关键帧就不是唯一的触发逻辑了，其目的也是为了用户产生更加拟真的体验感。在多媒体播放观看的场景中时间关键帧是主要的触发控制逻辑，但在交互式体验的VR游戏场景中，针对传感器的阈值来提供控制逻辑的伴随程序可以一种外挂的形式为游戏通过很多维度的拟真响应。
本实施例是一个独立于多媒体内容之外的伴随播放插件***，可以根据多媒体内容编辑时间轴控制触发流程控制有利于多媒体互动体验的外设控制***。
本实施例包含多媒体关键点或关键帧识别，记录，编辑***，可以把多媒体的互动任务变成一个包含匹配多媒体内容的时间轴控制文件，并把复杂的控制动作植入到这个伴随播放文件中。
本实施例包含时间轴调整***，由于市面上流传的多媒体内容版本较多，有些多媒体片头，片尾或者中间会植入压制者的广告等内容，导致同一多媒体的众多版本时间轴不一致，本实施例包含关键帧匹配功能，可以把众多版本的多媒体文件关键帧时间轴对齐，以便控制文件可以和多媒体内容同步。
本伴随播放插件还提供时间轴和控制动作修改的手动调整界面，用户也可以根据自身感受手动微调时间轴达到个人感受的最佳匹配。并且可以封装成插件文件用于网络上的交流.
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、***、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其它数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其它存储器技术、CD-ROM、数字多功能盘(DVD)或其它光盘存储、磁盒、磁带、磁盘存储或其它磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其它的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其它传输机制之类的调制数据信号中的其它数据，并且可包括任何信息递送介质。
本文已经公开了示例实施例，并且虽然采用了具体术语，但它们仅用于并仅应当被解释为一般说明性含义，并且不用于限制的目的。在一些实例中，对本领域技术人员显而易见的是，除非另外明确指出，否则可单独使用与特定实施例相结合描述的特征、特性和/或元素，或可与其它实施例相结合描述的特征、特性和/或元件组合使用。因此，本领域技术人员将理解，在不脱离由所附的权利要求阐明的本公开的范围的情况下，可进行各种形式和细节上的改变。

Claims (13)

  1. 一种数据处理方法，包括：
    根据第一多媒体文件确定触发点序列，所述触发点序列包括至少一个控制触发点；
    确定各个所述控制触发点对应的控制指令，所述控制指令用于配合所述第一多媒体文件控制被控设备；
    根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序。
  2. 根据权利要求1所述的数据处理方法，其中，根据第一多媒体文件确定触发点序列，包括：
    对所述第一多媒体文件进行特征点分析，提取至少一个所述控制触发点；
    根据所述第一多媒体文件的时间线确定各个所述控制触发点的触发时间，得到所述触发点序列。
  3. 根据权利要求1所述的数据处理方法，其中，确定各个所述控制触发点对应的控制指令，包括：
    以逐点编辑和/或批处理编辑方式，编辑各个所述控制触发点对应的控制指令；
    对各个所述控制触发点对应的控制指令执行时间间隔冲突检测。
  4. 根据权利要求3所述的数据处理方法，其中，编辑各个所述控制触发点对应的控制指令，包括：
    当一个时间点对应多个所述控制触发点时，根据所述多个控制触发点对应的控制指令生成所述时间点对应的复合控制指令。
  5. 根据权利要求1至4中任意一项所述的数据处理方法，其中，在根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序之后，所述数据处理方法还包括：
    响应于参数接口的修改指令，根据所述修改指令对所述伴随程序中所述控制触发点的时序、和/或所述控制触发点对应的控制指令进行调整。
  6. 根据权利要求1至4中任意一项所述的数据处理方法，其中，所述数据处理方法还包括：
    对所述第一多媒体文件和第二多媒体文件进行关键帧对比，将所述第二多媒体文件的时间线与所述第一多媒体文件的时间线对齐，所述第一多媒体文件和所述第二多媒体文件为同一多媒体文件的不同版本；
    根据将所述第二多媒体文件的时间线与所述第一多媒体文件的时间线对齐的结果，对所述伴随程序的时序进行调整，以使所述伴随程序与所述第二多媒体文件同步。
  7. 根据权利要求1至4中任意一项所述的数据处理方法，其中，在根据所述触发点序列和所述控制触发点对应的控制指令，生成伴随程序之后，所述数据处理方法还包括：
    将播放多媒体文件的多媒体播放端与所述被控设备进行时钟同步；
    根据时钟同步结果对所述伴随程序的时序进行调整。
  8. 根据权利要求1至4中任意一项所述的数据处理方法，其中，根据第一多媒体文件确定触发点序列，包括：
    根据播放多媒体文件的多媒体播放端的传感器的数值，确定至少一个所述控制触发点。
  9. 根据权利要求8所述的数据处理方法，其中，根据播放多媒体文件的多媒体播放端的传感器的数值，确定至少一个所述控制触发点，包括：
    针对任意一个传感器，对应于传感器的数值的每一个阈值，设置一个所述控制触发点，其中，传感器设置有至少一个阈值。
  10. 一种控制装置，包括：
    存储单元，其上存储有根据权利要求1至9中任意一项所述的数据处理方法生成的伴随程序；
    处理单元，所述处理单元能够执行所述存储单元中存储的伴随程序，以对被控设备进行控制。
  11. 一种计算机可读介质，其上存储有根据权利要求1至9中任意一项所述的数据处理方法生成的伴随程序。
  12. 一种电子设备，包括：
    一个或多个处理器；
    存储器，其上存储有一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现根据权利要求1至9中任意一项所述的数据处理方法；
    一个或多个I/O接口，连接在所述处理器与存储器之间，配置为实现所述处理器与存储器的信息交互。
  13. 一种计算机可读介质，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求1至9中任意一项所述的数据处理方法。
PCT/CN2023/080357 2022-06-13 2023-03-08 数据处理方法、控制装置、电子设备、计算机可读介质 WO2023241121A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210662421.5A CN117270982A (zh) 2022-06-13 2022-06-13 数据处理方法、控制装置、电子设备、计算机可读介质
CN202210662421.5 2022-06-13

Publications (1)

Publication Number Publication Date
WO2023241121A1 true WO2023241121A1 (zh) 2023-12-21

Family

ID=89193112

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/080357 WO2023241121A1 (zh) 2022-06-13 2023-03-08 数据处理方法、控制装置、电子设备、计算机可读介质

Country Status (2)

Country Link
CN (1) CN117270982A (zh)
WO (1) WO2023241121A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108319444A (zh) * 2018-01-30 2018-07-24 广州市百果园信息技术有限公司 基于音乐鼓点控制终端震动方法、存储设备及计算机设备
CN109451154A (zh) * 2018-10-31 2019-03-08 维沃移动通信有限公司 一种设置多媒体文件的方法及终端设备
CN110062272A (zh) * 2019-04-30 2019-07-26 腾讯科技（深圳）有限公司 一种视频数据处理方法和相关装置
CN113784171A (zh) * 2021-01-18 2021-12-10 北京沃东天骏信息技术有限公司 视频数据处理方法、装置、计算机***及可读存储介质
WO2022000983A1 (zh) * 2020-06-30 2022-01-06 百度在线网络技术（北京）有限公司 视频处理方法、装置、电子设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108319444A (zh) * 2018-01-30 2018-07-24 广州市百果园信息技术有限公司 基于音乐鼓点控制终端震动方法、存储设备及计算机设备
CN109451154A (zh) * 2018-10-31 2019-03-08 维沃移动通信有限公司 一种设置多媒体文件的方法及终端设备
CN110062272A (zh) * 2019-04-30 2019-07-26 腾讯科技（深圳）有限公司 一种视频数据处理方法和相关装置
WO2022000983A1 (zh) * 2020-06-30 2022-01-06 百度在线网络技术（北京）有限公司 视频处理方法、装置、电子设备和存储介质
CN113784171A (zh) * 2021-01-18 2021-12-10 北京沃东天骏信息技术有限公司 视频数据处理方法、装置、计算机***及可读存储介质

Also Published As

Publication number Publication date
CN117270982A (zh) 2023-12-22

Similar Documents

Publication Publication Date Title
US11468914B2 (en) System and method of generating video from video clips based on moments of interest within the video clips
JP7134248B2 (ja) ビデオ作成方法並びにその装置、コンピュータ機器、記憶媒体、及びコンピュータプログラム
US6240459B1 (en) Network delivery of interactive entertainment synchronized to playback of audio recordings
US8023800B2 (en) Media playback system
US7945645B2 (en) Method and system for accessing web pages based on playback of recordings
US10534525B1 (en) Media editing system optimized for distributed computing systems
US20180226101A1 (en) Methods and systems for interactive multimedia creation
EP3361738A1 (en) Method and device for stitching multimedia files
US7228280B1 (en) Finding database match for file based on file characteristics
CN108320319B (zh) 一种漫画合成方法、装置、设备及计算机可读存储介质
CN112866776B (zh) 视频生成方法和装置
US20200344549A1 (en) Synchronized multiuser audio
WO2021052130A1 (zh) 视频处理方法、装置、设备及计算机可读存储介质
JP2019520011A (ja) メディア環境駆動型コンテンツ配信プラットフォーム
WO2023241121A1 (zh) 数据处理方法、控制装置、电子设备、计算机可读介质
US11095938B2 (en) Online video editor
CN114025229A (zh) 处理音视频文件的方法、装置、计算设备及存储介质
KR101477492B1 (ko) 동영상 콘텐츠 편집 및 재생을 위한 장치 및 그 방법
US20230386522A1 (en) Computing system that applies edits model from published video to second video
US11928078B2 (en) Creating effect assets while avoiding size inflation
US11763849B1 (en) Automatic and fast generation of music audio content for videos
WO2022209648A1 (ja) 情報処理装置、情報処理方法および非一時的なコンピュータ可読記憶媒体
Kirkpatrick et al. Layout 2—SOUND & TESTING
WO2023140784A2 (en) Synchronizing multiple instances of projects
EP4315828A1 (en) Content creation based on rhythm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23822693

Country of ref document: EP

Kind code of ref document: A1