WO2022227599A1 - 实现音频与图像同步控制灯光的***及方法 - Google Patents

实现音频与图像同步控制灯光的***及方法 Download PDF

Info

Publication number
WO2022227599A1
WO2022227599A1 PCT/CN2021/137961 CN2021137961W WO2022227599A1 WO 2022227599 A1 WO2022227599 A1 WO 2022227599A1 CN 2021137961 W CN2021137961 W CN 2021137961W WO 2022227599 A1 WO2022227599 A1 WO 2022227599A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
data
image
frame
color
Prior art date
Application number
PCT/CN2021/137961
Other languages
English (en)
French (fr)
Inventor
马艺
张尚超
戴辉
谢文军
Original Assignee
深圳市凯润智能照明有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市凯润智能照明有限公司 filed Critical 深圳市凯润智能照明有限公司
Priority to US17/855,185 priority Critical patent/US11553577B2/en
Publication of WO2022227599A1 publication Critical patent/WO2022227599A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
    • H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
    • H05B47/10—Controlling the light source
    • H05B47/165—Controlling the light source following a pre-assigned programmed sequence; Logic control [LC]
    • H—ELECTRICITY
    • H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
    • H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
    • H05B47/10—Controlling the light source
    • H05B47/175—Controlling the light source by remote control
    • H05B47/19—Controlling the light source by remote control via wireless transmission
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
    • Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
    • Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection

Definitions

  • the present invention relates to the field of lighting, in particular to a system and method for realizing synchronous control of lighting with audio and images.
  • the present invention provides a system and method for realizing audio and image synchronization control of lighting.
  • the present invention proposes a system for realizing audio and image synchronous control of lighting, including: audio and video information collection and processing software in audio and video playback equipment, which is used to realize audio and screen image information collection, analyze and process according to a predetermined strategy, and package into synchronization Control instructions; cloud server, used to store user login information, lighting equipment control command information and status information reported by lighting equipment; lighting equipment, used to receive synchronous control instructions sent by audio and video information collection and processing software, and separate audio and screen.
  • the image color control command according to the predetermined control strategy, controls the color and rhythm changes of the light, and realizes the audio and image synchronization control of the light.
  • the audio and video information acquisition and processing software includes: a strategy control module, a screen acquisition module, an audio acquisition module, and a data framing module;
  • the strategy control module (11) configures information about audio, screen The sampling strategy of the image is sent and sent;
  • the screen acquisition module (12) receives the sampling strategy sent by the strategy control module (11) and collects image feature data accordingly, and the audio acquisition module (13) receives the strategy control module (11) the sent sampling strategy and collect audio feature data accordingly;
  • the data framing module (14) receives the sampling strategy sent by the strategy control module (11) and collects the images collected by the screen acquisition module (12)
  • the feature data and the audio feature data collected by the audio acquisition module are assembled into the same frame of synthetic frame data, and sent;
  • the cloud server includes an N+1 backup cloud server;
  • the lighting equipment includes: a data receiving and buffering unit, a frame data parsing unit , corresponding unit of lighting equipment state; Described data receiving and buffering unit receives the synthetic frame data sent by audio-visual information acquisition and
  • the multiple lighting devices can be controlled by one or more of WIFI LAN broadcast, WIFI WAN, WIFI MDNS networking, Bluetooth MESH and ZIGBEE MESH, Realize linkage.
  • the strategy control module is configured to receive a user-defined configuration of the sampling strategy for audio and screen images, and output the acquisition range of audio and screen images, the collection clock and clock stamp labels, and the screen images accordingly.
  • the number of blocks and the collection area; the sampling strategy of the audio includes: the input source of audio collection, the number of channels for collecting sound, the sampling frequency of the audio, and the audio data buffer; the sampling strategy of the screen image includes: the sampling of the image Frequency, number of blocks to collect color, area to collect color.
  • the screen acquisition module is configured to receive the acquisition frequency, start time stamp, number and area of each frame of image blocks outputted by the policy control module, and read a complete frame of data at a time.
  • Statistical calculation of block features to achieve dominant color extraction based on K-Means clustering algorithm obtain the RGB value with the most color in each block of data and the corresponding brightness value N, and obtain the calculation time ⁇ Tv; According to the calculation, the nth frame image is formed Feature Packet.
  • the audio acquisition module is configured to receive the acquisition frequency and start timestamp of each frame output by the strategy control module, and read a complete frame of data from the audio pulse data stream of the audio-visual playback device through an interface function each time.
  • the sampling of the time domain signal is transformed into the sampling in the discrete time Fourier transform frequency domain, and the 3 frequency values F1, F2, F3 and the amplitude Af with the strongest intensity in each sampling are obtained, and the calculation Duration ⁇ Ta; according to the calculation, add a timestamp to form the nth frame of audio feature data packet.
  • the data framing module is configured to receive the collection frequency of each frame output by the policy control module, and read the data sampled at the same time point or the adjacent time collection from the buffer according to the timestamp and the delay mark.
  • the data with the smallest delay interval is composed of the same frame of audio and image synchronous mixed data packets; if
  • the data receiving and buffering unit is used to receive the synthesized frame data sent by the audio-video information collection and processing software and buffer the data;
  • the frame data parsing unit is used to parse the synthesized frame data of each frame, Parse out the color and sound control instructions;
  • the synthetic frame data of each frame includes the lighting equipment address Nx, the area color value RGB, the area brightness value N, the sound frequency value f, the sound amplitude value Af, and the A/B/C/X information includes The color signal RGB value and the brightness value N of a certain area of the front screen, the F information includes the sound frequency value f and the amplitude value Af, Vflag represents the video signal mark, and Aflag represents the audio signal mark;
  • the predetermined control strategy is synchronously converted into the color and brightness signal values of the corresponding lighting equipment according to the parsed lighting equipment address Nx, regional color signal RGB value and brightness value N, and audio signal frequency value f and amplitude Af.
  • control of the audio and the image synchronously includes: synchronizing the screen color and sound changes by a combination of multiple lighting devices; or, two or more of the multiple lighting devices simultaneously synchronizing the screen color and sound changes; or, the independent lights in the lighting device synchronize the screen color or sound changes in two states; or, the lighting device synchronizes screen color or sound changes in two states.
  • the present invention also provides a method for realizing audio and image synchronous control of lighting, which is characterized by comprising the following steps: S1: Collect audio and screen image information in the audio and video playback device, analyze, process, and package them into a predetermined strategy according to a predetermined strategy. Synchronous control instructions, send synchronous control instructions; S2: Receive synchronous control instructions sent by audio-visual information acquisition and processing software and cache them, parse out audio and screen image color synchronous control instructions, execute synchronous control instructions according to a predetermined control strategy, and control the color of lights And rhythm changes, to achieve audio and image synchronization control lights.
  • the step S1 includes: receiving a user-defined configuration of a sampling strategy for audio and screen images, and outputting the audio and screen image acquisition ranges, acquisition clock and clock stamp labels, and screen image segmentation accordingly number and collection area;
  • the audio sampling strategy includes: the input source of audio collection, the number of channels for sound collection, the audio sampling frequency, the audio data buffer;
  • the sampling strategy of the screen image includes: the sampling frequency of the image, The number of blocks to collect color, and the area to collect color.
  • the step S1 includes: receiving the collection frequency, start time stamp, number and area of each frame of image blocks output by the policy control module, reading one complete frame of data each time, and passing the data into blocks Feature statistic calculation to realize dominant color extraction based on K-Means clustering algorithm, obtain the RGB value with the most color in each piece of data and the corresponding brightness value N, and obtain the calculation time ⁇ Tv; according to the calculation, form the nth frame of image feature data Bag.
  • the step S1 includes: receiving the acquisition frequency and start timestamp of each frame output by the strategy control module, reading a complete frame of data from the audio pulse data stream of the audio-visual playback device through an interface function each time, FFT calculation, transform the sampling of the time domain signal into the sampling in the discrete-time Fourier transform frequency domain, obtain the three frequency values F1, F2, F3 and amplitude Af with the strongest intensity in each sampling, and obtain the calculation time ⁇ Ta ; According to the calculation, add the timestamp to form the nth frame audio feature data packet.
  • the step S1 includes: receiving the collection frequency of each frame output by the policy control module, and reading the data sampled at the same time point or the data collected at the adjacent time from the cache according to the time stamp and the delay mark; According to the different delays of audio and image processing, the data with the smallest delay interval is composed of the same frame of audio and image synchronous mixed data packets; if
  • the S2 step specifically includes the following steps: S21: Receive the synthesized frame data sent by the audio-visual information collection and processing software and cache the data; S22: parse the synthesized frame data of each frame, and parse out the color and sound control Command; the synthetic frame data of each frame includes the lighting equipment address Nx, the area color value RGB, the area brightness value N, the sound frequency value f, the sound amplitude value Af, and the A/B/C/X information includes the information of a certain area of the front screen.
  • a combination of multiple lighting devices realizes synchronous screen color and sound changes; or, two or more of the multiple lighting devices simultaneously realize synchronous screen color and sound changes; or, independent lights in the lighting devices are
  • the two states synchronize the change of the screen color or the sound respectively; or, the lighting device synchronizes the change of the screen color or the sound respectively in the two states.
  • the present invention can realize the synchronous acquisition and feature extraction of multimedia audio and image content, and perform time stamp marking and delay calculation on the delay of the audio and image data processing process. , reorganize the audio and image data mixed package, eliminate the asynchrony problems such as audio and image color change lag or advance rhythm change, collect these two kinds of information synchronously at the same time, and make the background lighting in real time according to the changes of the two kinds of information. Change, so that the effect of the lighting device is displayed synchronously with the audio and image content of the multimedia, which has a better user experience.
  • Fig. 1 is the system diagram that realizes audio frequency and image synchronization control light according to the embodiment of the present invention
  • Fig. 2 is the software module diagram that realizes audio frequency and image synchronization control lighting according to the embodiment of the present invention
  • Fig. 3 is the working flow chart of the screen acquisition module that realizes the synchronous control of lighting with audio and image according to an embodiment of the present invention
  • Fig. 4 is the working flow chart of the audio acquisition module that realizes the synchronous control of light with audio and image according to an embodiment of the present invention
  • Fig. 5 is the working flow chart of the data framing module that realizes audio and image synchronization control lighting according to an embodiment of the present invention
  • FIG. 6 is a flowchart of receiving and executing a synchronization instruction for realizing audio and image synchronization control of lighting according to an embodiment of the present invention
  • FIG. 7 is a schematic diagram of lighting synchronization type 1 for realizing audio and image synchronization control of lighting according to an embodiment of the present invention
  • FIG. 8 is a schematic diagram of lighting synchronization type 2 for realizing audio and image synchronization control of lighting according to an embodiment of the present invention
  • FIG. 9 is a schematic diagram of lighting synchronization type 3 for realizing audio and image synchronization control of lighting according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a lighting synchronization type 4 for realizing audio and image synchronization control lighting according to an embodiment of the present invention
  • FIG. 11 is a work flow chart for realizing the synchronous control of light with audio and image according to an embodiment of the present invention.
  • azimuth terms such as left, right, top, bottom, top and bottom in this embodiment are only relative concepts to each other, or refer to the normal use state of the product, and should not be considered as having restrictive.
  • the problem solved by the invention is to obtain the color and brightness information of the screen image by collecting the screen of the mobile phone/PC/TV screen in real time, and simultaneously collect the current audio information related to the displayed screen image, and synthesize the synchronous control command after information processing, Through the light control device, the color of the light and the screen image are consistent; and according to the audio information, different display schemes such as patterns and frequencies are displayed synchronously.
  • the screen acquisition module 12 receives the sampling strategy sent by the strategy control module 11 and collects image feature data accordingly
  • the audio acquisition module 13 receives the sampling strategy sent by the strategy control module 11 and collects audio features accordingly data
  • the data framing module 14 receives the sampling strategy sent by the strategy control module 11 and assembles the image feature data collected by the screen capture module 12 and the audio feature data collected by the audio capture module into the same frame of synthetic frame data, and send.
  • the lighting equipment 3 includes: a data receiving and buffering unit, a frame data analysis unit, and a lighting equipment state corresponding unit; the data receiving and buffering unit receives and buffers the synthesized frame data sent by the audio-visual information collection and processing software 1; the frame data analysis The unit parses the color and sound control instructions from the synthetic frame data buffered by the data receiving and buffering unit; the corresponding unit of the lighting equipment status is synchronously converted into the corresponding color and sound control instructions according to the color and sound control instructions parsed by the frame data parsing unit. Color and brightness signal values of lighting device 3.
  • the embodiment of the present invention includes three parts:
  • the audio and video information collection and processing software 1 in the audio and video playback device is used to realize the collection of audio and screen image information, analyze and process it according to a predetermined strategy, and encapsulate it into synchronous control instructions;
  • the lighting equipment 3 is used to receive the synchronous control instructions sent by the audio and video information acquisition and processing software, separate the audio and screen image color control commands, and control the color and rhythm changes of the lights according to the predetermined control strategy to realize audio and image. Synchronized control of lights;
  • the cloud server 2 and the N+1 backup cloud server are used to store user login information, control command information of the lighting device 3 and status information reported by the lighting device 3 .
  • FIG. 2 it is a software module diagram of the first part of the content of the embodiment of the present invention, representing the software modules of audio, screen image data acquisition, processing and packaging, including four steps:
  • Step 1 the strategy control module 11 receives the user's custom configuration about the sampling strategy of audio and screen images, and outputs the collection range of audio and screen images, collection clocks and clock stamp labels, and the number of screen image blocks and collection areas accordingly;
  • the audio sampling strategy includes: the input source of audio collection, the number of channels for sound collection, the audio sampling frequency, and the audio data buffer;
  • the screen image sampling strategy includes: the sampling frequency of the image, the block of the collected color Count and collect color areas;
  • Step 3 the audio collection module 13 receives every frame collection frequency and the start time stamp that the strategy control module 11 outputs, reads a complete frame of data from the audio-video playback device audio pulse data stream through the interface function each time, and passes FFT calculation, transform the sampling of the time domain signal into the sampling in the discrete-time Fourier transform frequency domain, obtain the three frequency values F1, F2, F3 and amplitude Af with the strongest intensity in each sampling, and obtain the calculation time ⁇ Ta ;According to the calculation, add a timestamp to form the nth frame audio feature data packet;
  • the data framing module 14 receives the collection frequency of each frame output by the strategy control module 11, and reads the data sampled at the same time point or the data collected at the adjacent time from the cache according to the time stamp and the delay mark.
  • the data with the smallest delay interval is composed of the same frame of audio and image synchronous mixed data packets; if
  • FIG. 6 it is a flow chart of the second part of the embodiment of the present invention, indicating that the lighting device 3 receives the instruction and buffers it, parses the data packet and converts the command, and executes the instruction to control the function and status of the device end; the specific implementation steps are as follows:
  • Step 1 the data receiving and buffering unit receives the synthetic frame data sent by the audio-visual information acquisition and processing software 1 and buffers the data;
  • Step 2 the frame data analysis unit parses the synthetic frame data of each frame, and parses out the color and sound control instructions;
  • the synthetic frame data of each frame includes the lighting equipment address Nx, the regional color value RGB, the regional brightness value N, and the sound frequency value.
  • Sound amplitude Af, A/B/C/X information includes the color signal RGB value and luminance value N of a certain area of the front screen,
  • F information includes sound frequency value f and amplitude Af
  • Vflag represents the video signal flag,
  • Aflag represents an audio signal marker;
  • Step 3 according to the predetermined control strategy, the corresponding unit of the lighting equipment 3 state, according to the analyzed lighting equipment address Nx, the regional color signal RGB value and the brightness value N and the audio signal frequency value f and amplitude Af, are synchronously converted into the corresponding lighting equipment. Color and luminance signal values.
  • the communication mode of the multi-light linkage of the terminal lighting equipment 3 is supported as follows: It can be controlled by one or more methods such as WIFI LAN broadcast, WIFI WAN, WIFI MDNS networking, Bluetooth MESH and ZIGBEE MESH.
  • FIG. 7 , 8 , 9 , and 10 it is a diagram showing four ways of presenting the lighting effects of the terminal lighting equipment 3 with single lamp or multi-lamp linkage.
  • method 1 the combination of multiple lighting devices 3 realizes synchronous screen color and sound changes; for example, the combination of lighting device 1 + lighting device 2 realizes synchronous screen color and sound changes; the status of lighting device 1 is synchronized with the screen information, LED1 -LEDX corresponds to the screen acquisition area A1-X respectively, the color of LED1-LEDX is related to the color of the acquisition area A1-X, and the brightness of LED1-LEDX is related to the brightness of the acquisition area A1-X; the status of the lighting device 2 is synchronized with the sound information, and the color type is related to the acquisition The sound frequency is related, and the brightness is related to the collected sound amplitude;
  • mode 2 two or more of the plurality of lighting devices 3 simultaneously realize synchronous screen color and sound changes; for example, two or more of the lighting devices 1-X simultaneously realize synchronous screen color and sound changes;
  • the color of the lighting equipment 1 is related to the color of the collection area A1, and the brightness is related to the amplitude of a certain frequency F of the collected sound;
  • mode 3 the independent lights in the lighting device 3 synchronize the changes of screen color or sound in two states; for example, the lighting device synchronizes the changes of screen color or sound in two states, state one is: LED1-LEDX color It is related to the color of the collection area A1-X, and the brightness of LED1-LEDX is related to the brightness of the collection area A1-X; the second state is: the color of LED1-LEDX is related to the frequency of the collected sound, and the brightness of LED1-LEDX is related to the amplitude of the collected sound;
  • Mode 4 Lighting device 3 synchronizes screen color or sound changes in two states; for example, lighting device synchronizes screen color or sound changes in two states, state one is: LED1-LEDX color and collection area
  • the color of any area of A1-X is related to the color of any area of A1-X
  • the brightness of LED1-LEDX is related to the brightness of any area of A1-X in the acquisition area. related.
  • the workflow for realizing the synchronous control of lighting with audio and images is as follows:
  • Audio and video information collection and processing software 1 in the audio and video playback device collects audio and screen image information in the audio and video playback device, analyzes, processes, and encapsulates synchronous control instructions according to a predetermined strategy, and sends the synchronous control instructions;
  • Lighting equipment 3 receives and caches the synchronous control instructions sent by the audio-visual information acquisition and processing software 1, parses out the audio and screen image color synchronous control instructions, executes the synchronous control instructions according to the predetermined control strategy, and controls the color and rhythm changes of the lights, Realize audio and image synchronization control lights.
  • the embodiment of the present invention provides a method for realizing audio and image synchronous control of lighting, including: acquiring a screen image of a display screen by synchronously collecting audio and image in a mobile phone or PAD software 1, and simultaneously collecting the audio and video broadcast on the mobile phone/PAD speaker.
  • Another example of the present invention provides a method for realizing audio and image synchronous control of lighting, including: acquiring the screen image of the display screen and simultaneously collecting the sound broadcast on the PC speaker through the audio/video synchronization acquisition software 1 in the PC; Determine the RGB value according to the screen image, determine the audio frequency according to the collected audio signal; control the color of the backlight corresponding to the display screen according to the RGB value, and control the rhythm and pattern of the backlight according to the audio frequency feature.
  • Another example of the present invention provides a method for realizing audio and image synchronous control of lighting, including: acquiring the screen image of the display screen and simultaneously collecting the sound broadcast on the TV speaker by synchronizing the audio/video acquisition software 1 in the TV; Determine the RGB value according to the screen image, determine the audio frequency according to the collected audio signal; control the color of the backlight corresponding to the display screen according to the RGB value, and control the rhythm and pattern of the backlight according to the audio frequency feature.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Circuit Arrangement For Electric Light Sources In General (AREA)

Abstract

一种实现音频与图像同步控制灯光的***及方法，该***包括：音像播放设备中的音像信息采集及处理软件，用以实现音频及屏幕图像信息采集、按照预定策略分析、处理、并封装成同步控制指令；云服务器；灯光设备，用以接收音像信息采集及处理软件传送过来的同步控制指令，分离出音频及屏幕图像颜色控制命令，按照预定控制策略，控制灯的颜色及律动变化。实现对多媒体的音频、图像内容的同步采集、特征提取，并通过对音频、图像数据处理过程的延时进行时间戳标记及延时计算，重组音频、图像数据混合组包，消除音频、图像的不同步问题，使灯光装置效果跟随多媒体的音频、图像内容同步展现，具有更好用户体验。

Description

实现音频与图像同步控制灯光的***及方法 技术领域
本发明涉及照明领域，特别是涉及一种实现音频与图像同步控制灯光的***及方法。
背景技术
为了使普通的照明灯具有更丰富的娱乐功能，目前市面上出现了很多形式的灯控方式，主要有从PC屏和TV屏提取相应的屏幕颜色，然后控制相关的外置背景照明等；或者根据特定环境下的音乐，提取相应的音乐特征来控制外置背景灯按照一定韵律节奏而变化。但是，对于丰富的多媒体内容，往往同时包含丰富的视频图像与音频信息，市面上的照明显示方案不能实现音频、图像与灯光同步的不足，用户体验不足。
发明内容
为了弥补上述现有技术的不能实现音频、图像与灯光同步的不足，本发明提出一种实现音频与图像同步控制灯光的***及方法。
本发明的技术问题通过以下的技术方案予以解决：
本发明提出一种实现音频与图像同步控制灯光的***，包括：音像播放设备中的音像信息采集及处理软件，用以实现音频及屏幕图像信息采集、按照预定策略分析、处理、并封装成同步控制指令；云服务器，用于存储用户登录信息、灯光设备控制命令信息以及灯光设备上报的状态信息；灯光设备，用以接收音像信息采集及处理软件传送过来的同步控制指令，分离出音频及屏幕图像颜色控制命令，按照预定控制策略，控制灯的颜色及律动变化，实现音频与图像同步控制灯光。
在一些实施例中，所述音像信息采集及处理软件包括：策略控制模块、屏幕采集模块、音频采集模块、数据成帧模块；所述策略控制模块(11)根据用户自定义配置关于音频、屏幕图像的采样策略并发送；所述屏幕采集模块(12)接收所述策略控制模块(11)发送的采样策略并据此采集图像特征数据，所述音频采集模块(13)接收所述策略控制模块(11)发送的采样策略并据此采集音频特征数据；所述数据成帧模块(14)接收所述策略控制模块(11)发送的采样策略并将所述屏幕采集模块(12)采集的图像特征数据与音频采集模块采集的音频特征 数据组装成同一帧合成帧数据，并发送；所述云服务器包括N+1备份云服务器；所述灯光设备包括：数据接收及缓存单元、帧数据解析单元、灯光设备状态相应单元；所述数据接收及缓存单元接收音像信息采集及处理软件(1)发送的合成帧数据并缓存；所述帧数据解析单元从所述数据接收及缓存单元缓存的合成帧数据解析出颜色及声音控制指令；所述灯光设备状态相应单元根据所述帧数据解析单元解析出的颜色及声音控制指令，同步转换成对应的灯光设备(3)的颜色和亮度信号值。
在一些实施例中，所述灯光设备有多个，多个灯光设备之间可通过WIFI局域网广播、WIFI广域网、WIFI MDNS组网、蓝牙MESH和ZIGBEE MESH其中一种或一种以上的方式控制，实现联动。
在一些实施例中，所述策略控制模块，用以接收用户关于音频、屏幕图像的采样策略的自定义配置，据此输出音频及屏幕图像的采集范围、采集时钟及时钟戳标签、以及屏幕图像分块数量及采集区域；所述音频的采样策略包括：音频采集的输入源、采集声音的声道数、音频的采样频率、音频数据缓冲区；所述屏幕图像的采样策略包括：图像的采样频率、采集颜色的分块数、采集颜色的区域。
在一些实施例中，所述屏幕采集模块，用以接收策略控制模块输出的每帧采集频率、起始时间戳、每帧图像分块数量及区域，每次读取一个完整帧数据，经过数据分块特征统计计算以实现基于K-Means聚类算法的主色提取，获取每块数据中颜色最多的RGB值及相应的亮度值N，并获取计算时长ΔTv；根据计算，形成第n帧图像特征数据包。
在一些实施例中，所述音频采集模块，用以接收策略控制模块输出的每帧采集频率以及起始时间戳，每次通过接口函数从音像播放设备音频脉冲数据流中读取一完整帧数据，经过FFT计算，将时域信号的采样变换为在离散时间傅里叶变换频域的采样，获取每次采样中强度最强的3个频率值F1、F2、F3及幅度Af，并获取计算时长ΔTa；根据计算，加上时间戳，形成第n帧音频特征数据包。
在一些实施例中，所述数据成帧模块，用以接收策略控制模块输出的每帧采集频率，根据时间戳及延时标记，从缓存中读取同一时间点采样的数据或相 邻时间采集的数据；根据音频、图像处理的不同延时，把延时间隔最小的数据组成同一帧音频、图像同步混合数据包；若|ΔnTv-ΔnTa|<Tms，则第n帧音频和图像数据同步，组装成同一帧合成帧数据；若|ΔnTv-ΔnTa|>Tms，则第n帧音频和图像数据不同步，第n+1帧音频与第n帧图像数据组装成同一帧合成帧数据，或者第n+1帧图像与第n帧音频数据组装成同一帧合成帧数据，并发送。
在一些实施例中，所述数据接收及缓存单元，用以接收音像信息采集及处理软件发送的合成帧数据并缓存数据；所述帧数据解析单元，用以解析每一帧的合成帧数据，解析出颜色及声音控制指令；每一帧的合成帧数据包括灯光设备地址Nx、区域颜色值RGB、区域亮度值N、声音频率值f、声音幅值Af，A/B/C/X信息包含前端屏幕某一区域的颜色信号RGB值和亮度值N，F信息包含声音频率值f和幅值Af，Vflag表示视频信号标记，Aflag表示音频信号标记；所述灯光设备状态相应单元，用以按照预定控制策略，根据解析的灯光设备地址Nx、区域颜色信号RGB值和亮度值N与音频信号频率值f和幅值Af，同步转换成对应的灯光设备的颜色和亮度信号值。
在一些实施例中，所述实现音频与图像同步控制灯光，包括：多个灯光设备组合方式实现同步屏幕颜色和声音变化；或，多个灯光设备其中两个及两个以上同时实现同步屏幕颜色和声音变化；或，灯光设备内独立灯光以两种状态分别同步屏幕颜色或声音的变化；或，灯光设备以两种状态分别同步屏幕颜色或声音的变化。
本发明还提出一种实现音频与图像同步控制灯光的方法，其特征在于，包括如下步骤：S1：采集所述音像播放设备中的音频及屏幕图像信息，按照预定策略分析、处理、并封装成同步控制指令，发送同步控制指令；S2：接收音像信息采集及处理软件发送的同步控制指令并缓存，解析出音频及屏幕图像颜色同步控制指令，按照预定控制策略执行同步控制指令，控制灯的颜色及律动变化，实现音频与图像同步控制灯光。
在一些实施例中，所述S1步骤包括：接收用户关于音频、屏幕图像的采样策略的自定义配置，据此输出音频及屏幕图像的采集范围、采集时钟及时钟戳标签、以及屏幕图像分块数量及采集区域；所述音频的采样策略包括：音频采 集的输入源、采集声音的声道数、音频的采样频率、音频数据缓冲区；所述屏幕图像的采样策略包括：图像的采样频率、采集颜色的分块数、采集颜色的区域。
在一些实施例中，所述S1步骤包括：接收策略控制模块输出的每帧采集频率、起始时间戳、每帧图像分块数量及区域，每次读取一个完整帧数据，经过数据分块特征统计计算以实现基于K-Means聚类算法的主色提取，获取每块数据中颜色最多的RGB值及相应的亮度值N，并获取计算时长ΔTv；根据计算，形成第n帧图像特征数据包。
在一些实施例中，所述S1步骤包括：接收策略控制模块输出的每帧采集频率以及起始时间戳，每次通过接口函数从音像播放设备音频脉冲数据流中读取一完整帧数据，经过FFT计算，将时域信号的采样变换为在离散时间傅里叶变换频域的采样，获取每次采样中强度最强的3个频率值F1、F2、F3及幅度Af，并获取计算时长ΔTa；根据计算，加上时间戳，形成第n帧音频特征数据包。
在一些实施例中，所述S1步骤包括：接收策略控制模块输出的每帧采集频率，根据时间戳及延时标记，从缓存中读取同一时间点采样的数据或相邻时间采集的数据；根据音频、图像处理的不同延时，把延时间隔最小的数据组成同一帧音频、图像同步混合数据包；若|ΔnTv-ΔnTa|<Tms，则第n帧音频和图像数据同步，组装成同一帧合成帧数据；若|ΔnTv-ΔnTa|>Tms，则第n帧音频和图像数据不同步，第n+1帧音频与第n帧图像数据组装成同一帧合成帧数据，或者第n+1帧图像与第n帧音频数据组装成同一帧合成帧数据，并发送。
在一些实施例中，所述S2步骤具体包括如下步骤：S21：接收音像信息采集及处理软件发送的合成帧数据并缓存数据；S22：解析每一帧的合成帧数据，解析出颜色及声音控制指令；每一帧的合成帧数据包括灯光设备地址Nx、区域颜色值RGB、区域亮度值N、声音频率值f、声音幅值Af，A/B/C/X信息包含前端屏幕某一区域的颜色信号RGB值和亮度值N，F信息包含声音频率值f和幅值Af，Vflag表示视频信号标记，Aflag表示音频信号标记；S23：按照预定控制策略，根据解析的灯光设备地址Nx、区域颜色信号RGB值和亮度值N与音频信号频率值f和幅值Af，同步转换成对应的灯光设备的颜色和亮度信号值。
在一些实施例中，多个灯光设备组合方式实现同步屏幕颜色和声音变化； 或，多个灯光设备其中两个及两个以上同时实现同步屏幕颜色和声音变化；或，灯光设备内独立灯光以两种状态分别同步屏幕颜色或声音的变化；或，灯光设备以两种状态分别同步屏幕颜色或声音的变化。
本发明与现有技术对比的有益效果包括：本发明可以实现对多媒体的音频、图像内容的同步采集、特征提取，并通过对音频、图像数据处理过程的延时进行时间戳标记及延时计算，重组音频、图像数据混合组包，消除音频、图像的颜色变滞后或超前音律变化等不同步问题，把这两种信息同时同步采集出来，实时使背景照明等根据这两者信息的变化而变化，使灯光装置效果跟随多媒体的音频、图像内容同步展现，具有更好用户体验。
附图说明
图1是本发明实施例的实现音频与图像同步控制灯光的***图；
图2是本发明实施例的实现音频与图像同步控制灯光的软件模块图；
图3是本发明实施例的实现音频与图像同步控制灯光的屏幕采集模块工作流程图；
图4是本发明实施例的实现音频与图像同步控制灯光的音频采集模块工作流程图；
图5是本发明实施例的实现音频与图像同步控制灯光的数据成帧模块工作流程图；
图6是本发明实施例的实现音频与图像同步控制灯光的同步指令接收及执行流程图；
图7是本发明实施例的实现音频与图像同步控制灯光的灯光同步类型一的示意图；
图8是本发明实施例的实现音频与图像同步控制灯光的灯光同步类型二的示意图；
图9是本发明实施例的实现音频与图像同步控制灯光的灯光同步类型三的示意图；
图10是本发明实施例的实现音频与图像同步控制灯光的灯光同步类型四的示意图；
图11是本发明实施例的实现音频与图像同步控制灯光的工作流程图。
具体实施方式
下面对照附图并结合优选的实施方式对本发明作进一步说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
需要说明的是，本实施例中的左、右、上、下、顶、底等方位用语，仅是互为相对概念，或是以产品的正常使用状态为参考的，而不应该认为是具有限制性的。
本发明解决的问题是通过实时采集手机/PC/电视屏的屏幕获取屏幕图像的颜色及亮度信息，并同时同步采集当前与所显示屏幕图像相关的音频信息，进行信息处理后合成同步控制命令，通过灯控装置，实现灯光与屏幕图像颜色一致；并根据音频信息，同步展现不同的图案、频率等显示方案。
参考图1，是实现音频、图像同步控制灯光的***图。整个***由前端音像播放设备(比如TV/PC/PAD/Phone)等中的音频、屏幕图像采集及处理软件1，简称为“音像信息采集及处理软件”1；中间的云服务器2及N+1备份云服务器；后端的灯光设备3；本地局域网利用WiFi无线方式，外网利用以太网通讯方式。音像信息采集及处理软件1包括：策略控制模块11、屏幕采集模块12、音频采集模块13、数据成帧模块14；所述策略控制模块11根据用户自定义配置关于音频、屏幕图像的采样策略并发送；所述屏幕采集模块12接收所述策略控制模块11发送的采样策略并据此采集图像特征数据，所述音频采集模块13接收所述策略控制模块11发送的采样策略并据此采集音频特征数据；所述数据成帧模块14接收所述策略控制模块11发送的采样策略并将所述屏幕采集模块12采集的图像特征数据与音频采集模块采集的音频特征数据组装成同一帧合成帧数据，并发送。
灯光设备3包括：数据接收及缓存单元、帧数据解析单元、灯光设备状态相应单元；所述数据接收及缓存单元接收音像信息采集及处理软件1发送的合成帧数据并缓存；所述帧数据解析单元从所述数据接收及缓存单元缓存的合成帧数据解析出颜色及声音控制指令；所述灯光设备状态相应单元根据所述帧数据解析单元解析出的颜色及声音控制指令，同步转换成对应的灯光设备3的颜色和亮度信号值。
灯光设备3有多个，多个灯光设备3之间可通过WIFI局域网广播、WIFI 广域网、WIFI MDNS组网、蓝牙MESH和ZIGBEE MESH其中一种或一种以上的方式控制，实现联动。本发明实施例包含三部分内容：
其一，音像播放设备中的音像信息采集及处理软件1，用以实现音频及屏幕图像信息采集、按照预定策略分析、处理、并封装成同步控制指令；
其二，灯光设备3，用以接收音像信息采集及处理软件传送过来的同步控制指令，分离出音频及屏幕图像颜色控制命令，按照预定控制策略，控制灯的颜色及律动变化，实现音频与图像同步控制灯光；
其三，终端灯光设备3单灯或多灯联动的通信方式和灯光效果呈现方式。
云服务器2及N+1备份云服务器，用于存储用户登录信息、灯光设备3控制命令信息以及灯光设备3上报的状态信息。
参考图2，是本发明实施例第一部分内容的软件模块图，表示音频、屏幕图像数据采集、处理及封装的软件模块，包括四个步骤：
步骤①，策略控制模块11接收用户关于音频、屏幕图像的采样策略的自定义配置，据此输出音频及屏幕图像的采集范围、采集时钟及时钟戳标签、以及屏幕图像分块数量及采集区域；所述音频的采样策略包括：音频采集的输入源、采集声音的声道数、音频的采样频率、音频数据缓冲区；所述屏幕图像的采样策略包括：图像的采样频率、采集颜色的分块数、采集颜色的区域；
步骤②，参考图3，屏幕采集模块12接收策略控制模块11输出的每帧采集频率、起始时间戳、每帧图像分块数量及区域，每次读取一个完整帧数据，经过数据分块特征统计计算以实现基于K-Means聚类算法的主色提取，获取每块数据中颜色最多的RGB值及相应的亮度值N，并获取计算时长ΔTv；根据计算，形成第n帧图像特征数据包；
步骤③，参考图4，音频采集模块13接收策略控制模块11输出的每帧采集频率以及起始时间戳，每次通过接口函数从音像播放设备音频脉冲数据流中读取一完整帧数据，经过FFT计算，将时域信号的采样变换为在离散时间傅里叶变换频域的采样，获取每次采样中强度最强的3个频率值F1、F2、F3及幅度Af，并获取计算时长ΔTa；根据计算，加上时间戳，形成第n帧音频特征数据包；
步骤④，参考图5，数据成帧模块14接收策略控制模块11输出的每帧采集 频率，根据时间戳及延时标记，从缓存中读取同一时间点采样的数据或相邻时间采集的数据；根据音频、图像处理的不同延时，把延时间隔最小的数据组成同一帧音频、图像同步混合数据包；若|ΔnTv-ΔnTa|<Tms，则第n帧音频和图像数据同步，组装成同一帧合成帧数据；若|ΔnTv-ΔnTa|>Tms，则第n帧音频和图像数据不同步，第n+1帧音频与第n帧图像数据组装成同一帧合成帧数据，或者第n+1帧图像与第n帧音频数据组装成同一帧合成帧数据，并发送。
参考图6，为本发明实施例第二部分内容的流程图，表示灯光设备3端接收指令并缓存、数据包解析并转换命令、执行指令控制设备端功能及状态；具体实施步骤如下：
步骤①，数据接收及缓存单元接收音像信息采集及处理软件1发送的合成帧数据并缓存数据；
步骤②，帧数据解析单元解析每一帧的合成帧数据，解析出颜色及声音控制指令；每一帧的合成帧数据包括灯光设备地址Nx、区域颜色值RGB、区域亮度值N、声音频率值f、声音幅值Af，A/B/C/X信息包含前端屏幕某一区域的颜色信号RGB值和亮度值N，F信息包含声音频率值f和幅值Af，Vflag表示视频信号标记，Aflag表示音频信号标记；
步骤③，灯光设备3状态相应单元按照预定控制策略，根据解析的灯光设备地址Nx、区域颜色信号RGB值和亮度值N与音频信号频率值f和幅值Af，同步转换成对应的灯光设备的颜色和亮度信号值。
终端灯光设备3多灯联动的通信方式支持如下：可通过WIFI局域网广播、WIFI广域网、WIFI MDNS组网、蓝牙MESH和ZIGBEE MESH等其中一种及一种以上的方式控制。
参考图7、图8、图9、图10，是终端灯光设备3单灯或多灯联动的四种灯光效果呈现方式图。
终端灯光设备3单灯或多灯联动的灯光效果呈现方式如下：
参考图7，方式一：多个灯光设备3组合方式实现同步屏幕颜色和声音变化；比如灯光设备1+灯光设备2组合方式实现同步屏幕颜色和声音变化；灯光设备1状态与屏幕信息同步，LED1-LEDX分别对应屏幕采集区域A1-X，LED1-LEDX颜色与采集区域A1-X颜色相关，LED1-LEDX亮度与采集区域 A1-X亮度相关；灯光设备2状态与声音信息同步，颜色种类与采集声音频率相关，亮度大小与采集声音幅值相关；
参考图8，方式二：多个灯光设备3其中两个及两个以上同时实现同步屏幕颜色和声音变化；比如灯光设备1-X其中两个及两个以上同时实现同步屏幕颜色和声音变化；灯光设备1颜色与采集区域A1颜色相关，亮度与采集声音某一频率F的幅度相关；
参考图9，方式三：灯光设备3内独立灯光以两种状态分别同步屏幕颜色或声音的变化；比如灯光设备以两种状态分别同步屏幕颜色或声音的变化，状态一为：LED1-LEDX颜色与采集区域A1-X颜色相关，LED1-LEDX亮度与采集区域A1-X亮度相关；状态二为：LED1-LEDX颜色与采集声音频率相关，LED1-LEDX亮度与采集声音幅值相关；
参考图10，方式四：灯光设备3以两种状态分别同步屏幕颜色或声音的变化；比如灯光设备以两种状态分别同步屏幕颜色或声音的变化，状态一为：LED1-LEDX颜色与采集区域A1-X任意区域的颜色相关，LED1-LEDX亮度与采集区域A1-X任意区域的亮度相关；状态二为：LED1-LEDX颜色与采集声音某一频率相关，LED1-LEDX亮度与采集声音幅值相关。
参考图11，一种实施例的实现音频与图像同步控制灯光的工作流程如示：
S1：音像播放设备中的音像信息采集及处理软件1，采集所述音像播放设备中的音频及屏幕图像信息，按照预定策略分析、处理、并封装成同步控制指令，发送同步控制指令；
S2：灯光设备3，接收音像信息采集及处理软件1发送的同步控制指令并缓存，解析出音频及屏幕图像颜色同步控制指令，按照预定控制策略执行同步控制指令，控制灯的颜色及律动变化，实现音频与图像同步控制灯光。
本发明实施例提供一种实现音频与图像同步控制灯光的方法，包括：通过对手机或PAD中的音频、图像同步采集软件1获取显示屏的屏幕图像以及同时采集在手机/PAD扬声器播出的声音；根据所述屏幕图像确定RGB值，根据采集音频信号确定音频频率；根据所述RGB值控制与所述显示屏对应的背景灯的颜色，根据音频频率特征控制背景灯的韵律及图案；实现了显示屏背景灯与屏幕画面颜色的同步；且同时采集对应屏幕音频信号，通过音频、图像同步算法，避免音频、 图像出现超前或滞后的，从而获取与原始内容音频、图像匹配的灯控效果。
本发明另一种实例提供一种实现音频与图像同步控制灯光的方法，包括：通过对PC中的音/视频同步采集软件1获取显示屏的屏幕图像以及同时采集在PC扬声器播出的声音；根据所述屏幕图像确定RGB值，根据采集音频信号确定音频频率；根据所述RGB值控制与所述显示屏对应的背景灯的颜色，根据音频频率特征控制背景灯的韵律及图案。
本发明另一种实例提供一种实现音频与图像同步控制灯光的方法，包括：通过对TV中的音/视频同步采集软件1获取显示屏的屏幕图像以及同时采集在TV扬声器播出的声音；根据所述屏幕图像确定RGB值，根据采集音频信号确定音频频率；根据所述RGB值控制与所述显示屏对应的背景灯的颜色，根据音频频率特征控制背景灯的韵律及图案。
本发明实施例根据前端采集软件的指令，实现背景灯的音频、图像两种因素结合下的同步显示效果。
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的技术人员来说，在不脱离本发明构思的前提下，还可以做出若干等同替代或明显变型，而且性能或用途相同，都应当视为属于本发明的保护范围。

Claims (16)

  1. 一种实现音频与图像同步控制灯光的***，其特征在于，包括：
    音像播放设备中的音像信息采集及处理软件(1)，用以实现音频及屏幕图像信息采集、按照预定策略分析、处理、并封装成同步控制指令；
    云服务器(2)，用于存储用户登录信息、灯光设备(3)控制命令信息以及灯光设备(3)上报的状态信息；
    灯光设备(3)，用以接收音像信息采集及处理软件(1)传送过来的同步控制指令，分离出音频及屏幕图像颜色控制命令，按照预定控制策略，控制灯的颜色及律动变化，实现音频与图像同步控制灯光。
  2. 如权利要求1所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述音像信息采集及处理软件(1)包括：策略控制模块(11)、屏幕采集模块(12)、音频采集模块(13)、数据成帧模块(14)；所述策略控制模块(11)根据用户自定义配置关于音频、屏幕图像的采样策略并发送；所述屏幕采集模块(12)接收所述策略控制模块(11)发送的采样策略并据此采集图像特征数据，所述音频采集模块(13)接收所述策略控制模块(11)发送的采样策略并据此采集音频特征数据；所述数据成帧模块(14)接收所述策略控制模块(11)发送的采样策略并将所述屏幕采集模块(12)采集的图像特征数据与音频采集模块采集的音频特征数据组装成同一帧合成帧数据，并发送；
    所述云服务器(2)包括N+1备份云服务器；
    所述灯光设备(3)包括：数据接收及缓存单元、帧数据解析单元、灯光设备状态相应单元；所述数据接收及缓存单元接收音像信息采集及处理软件(1)发送的合成帧数据并缓存；所述帧数据解析单元从所述数据接收及缓存单元缓存的合成帧数据解析出颜色及声音控制指令；所述灯光设备状态相应单元根据所述帧数据解析单元解析出的颜色及声音控制指令，同步转换成对应的灯光设备(3)的颜色和亮度信号值。
  3. 如权利要求1所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述灯光设备(3)有多个，所述多个灯光设备(3)之间可通过WIFI局域网广播、WIFI广域网、WIFI MDNS组网、蓝牙MESH和ZIGBEE MESH其中一种或一种以上的方式控制，实现联动。
  4. 如权利要求2所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述策略控制模块(11)，用以接收用户关于音频、屏幕图像的采样策略的自定义配置，据此输出音频及屏幕图像的采集范围、采集时钟及时钟戳标签、以及屏幕图像分块数量及采集区域；所述音频的采样策略包括：音频采集的输入源、采集声音的声道数、音频的采样频率、音频数据缓冲区；所述屏幕图像的采样策略包括：图像的采样频率、采集颜色的分块数、采集颜色的区域。
  5. 如权利要求2所述的实现音频与图像同步控制灯光的***，其特征在于，所述屏幕采集模块(12)，用以接收策略控制模块(11)输出的每帧采集频率、起始时间戳、每帧图像分块数量及区域，每次读取一个完整帧数据，经过数据分块特征统计计算以实现基于K-Means聚类算法的主色提取，获取每块数据中颜色最多的RGB值及相应的亮度值N，并获取计算时长ΔTv；根据计算，形成第n帧图像特征数据包。
  6. 如权利要求2所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述音频采集模块(13)，用以接收策略控制模块(11)输出的每帧采集频率以及起始时间戳，每次通过接口函数从音像播放设备音频脉冲数据流中读取一完整帧数据，经过FFT计算，将时域信号的采样变换为在离散时间傅里叶变换频域的采样，获取每次采样中强度最强的3个频率值F1、F2、F3及幅度Af，并获取计算时长ΔTa；根据计算，加上时间戳，形成第n帧音频特征数据包。
  7. 如权利要求2所述的实现音频与图像同步控制灯光的***，其特征在于，所述数据成帧模块(14)，用以接收策略控制模块(11)输出的每帧采集频率，根据时间戳及延时标记，从缓存中读取同一时间点采样的数据或相邻时间采集的数据；根据音频、图像处理的不同延时，把延时间隔最小的数据组成同一帧音频、图像同步混合数据包；若|ΔnTv-ΔnTa|<Tms，则第n帧音频和图像数据同步，组装成同一帧合成帧数据；若|ΔnTv-ΔnTa|>Tms，则第n帧音频和图像数据不同步，第n+1帧音频与第n帧图像数据组装成同一帧合成帧数据，或者第n+1帧图像与第n帧音频数据组装成同一帧合成帧数据，并发送。
  8. 如权利要求2所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述数据接收及缓存单元，用以接收音像信息采集及处理软件发送的合成帧数据并缓存数据；
    所述帧数据解析单元，用以解析每一帧的合成帧数据，解析出颜色及声音 控制指令；每一帧的合成帧数据包括灯光设备地址Nx、区域颜色值RGB、区域亮度值N、声音频率值f、声音幅值Af，A/B/C/X信息包含前端屏幕某一区域的颜色信号RGB值和亮度值N，F信息包含声音频率值f和幅值Af，Vflag表示视频信号标记，Aflag表示音频信号标记；
    所述灯光设备状态相应单元，用以按照预定控制策略，根据解析的灯光设备地址Nx、区域颜色信号RGB值和亮度值N与音频信号频率值f和幅值Af，同步转换成对应的灯光设备的颜色和亮度信号值。
  9. 如权利要求1所述的实现音频与图像同步控制灯光的***，其特征在于，
    所述实现音频与图像同步控制灯光，包括：
    多个灯光设备(3)组合方式实现同步屏幕颜色和声音变化；
    或，多个灯光设备(3)其中两个及两个以上同时实现同步屏幕颜色和声音变化；
    或，灯光设备(3)内独立灯光以两种状态分别同步屏幕颜色或声音的变化；
    或，灯光设备(3)以两种状态分别同步屏幕颜色或声音的变化。
  10. 一种实现音频与图像同步控制灯光的方法，其特征在于，包括如下步骤：
    S1：采集所述音像播放设备中的音频及屏幕图像信息，按照预定策略分析、处理、并封装成同步控制指令，发送同步控制指令；
    S2：接收音像信息采集及处理软件发送的同步控制指令并缓存，解析出音频及屏幕图像颜色同步控制指令，按照预定控制策略执行同步控制指令，控制灯的颜色及律动变化，实现音频与图像同步控制灯光。
  11. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在于，所述S1步骤包括：接收用户关于音频、屏幕图像的采样策略的自定义配置，据此输出音频及屏幕图像的采集范围、采集时钟及时钟戳标签、以及屏幕图像分块数量及采集区域；所述音频的采样策略包括：音频采集的输入源、采集声音的声道数、音频的采样频率、音频数据缓冲区；所述屏幕图像的采样策略包括：图像的采样频率、采集颜色的分块数、采集颜色的区域。
  12. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在于，所述S1步骤包括：接收策略控制模块(11)输出的每帧采集频率、起始时间戳、每帧图像分块数量及区域，每次读取一个完整帧数据，经过数据分块特 征统计计算以实现基于K-Means聚类算法的主色提取，获取每块数据中颜色最多的RGB值及相应的亮度值N，并获取计算时长ΔTv；根据计算，形成第n帧图像特征数据包。
  13. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在于，所述S1步骤包括：接收策略控制模块(11)输出的每帧采集频率以及起始时间戳，每次通过接口函数从音像播放设备音频脉冲数据流中读取一完整帧数据，经过FFT计算，将时域信号的采样变换为在离散时间傅里叶变换频域的采样，获取每次采样中强度最强的3个频率值F1、F2、F3及幅度Af，并获取计算时长ΔTa；根据计算，加上时间戳，形成第n帧音频特征数据包。
  14. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在于，所述S1步骤包括：接收策略控制模块(11)输出的每帧采集频率，根据时间戳及延时标记，从缓存中读取同一时间点采样的数据或相邻时间采集的数据；根据音频、图像处理的不同延时，把延时间隔最小的数据组成同一帧音频、图像同步混合数据包；若|ΔnTv-ΔnTa|<Tms，则第n帧音频和图像数据同步，组装成同一帧合成帧数据；若|ΔnTv-ΔnTa|>Tms，则第n帧音频和图像数据不同步，第n+1帧音频与第n帧图像数据组装成同一帧合成帧数据，或者第n+1帧图像与第n帧音频数据组装成同一帧合成帧数据，并发送。
  15. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在于，所述S2步骤具体包括如下步骤：
    S21：接收音像信息采集及处理软件发送的合成帧数据并缓存数据；
    S22：解析每一帧的合成帧数据，解析出颜色及声音控制指令；每一帧的合成帧数据包括灯光设备地址Nx、区域颜色值RGB、区域亮度值N、声音频率值f、声音幅值Af，A/B/C/X信息包含前端屏幕某一区域的颜色信号RGB值和亮度值N，F信息包含声音频率值f和幅值Af，Vflag表示视频信号标记，Aflag表示音频信号标记；
    S23：按照预定控制策略，根据解析的灯光设备地址Nx、区域颜色信号RGB值和亮度值N与音频信号频率值f和幅值Af，同步转换成对应的灯光设备的颜色和亮度信号值。
  16. 如权利要求10所述的实现音频与图像同步控制灯光的方法，其特征在 于，
    多个灯光设备(3)组合方式实现同步屏幕颜色和声音变化；
    或，多个灯光设备(3)其中两个及两个以上同时实现同步屏幕颜色和声音变化；
    或，灯光设备(3)内独立灯光以两种状态分别同步屏幕颜色或声音的变化；
    或，灯光设备(3)以两种状态分别同步屏幕颜色或声音的变化。
PCT/CN2021/137961 2021-04-30 2021-12-14 实现音频与图像同步控制灯光的***及方法 WO2022227599A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/855,185 US11553577B2 (en) 2021-04-30 2022-06-30 System and method for achieving synchronized audio and image control of lighting

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110482642.X 2021-04-30
CN202110482642.XA CN113286405B (zh) 2021-04-30 2021-04-30 实现音频与图像同步控制灯光的***及方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/855,185 Continuation-In-Part US11553577B2 (en) 2021-04-30 2022-06-30 System and method for achieving synchronized audio and image control of lighting

Publications (1)

Publication Number Publication Date
WO2022227599A1 true WO2022227599A1 (zh) 2022-11-03

Family

ID=77278093

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137961 WO2022227599A1 (zh) 2021-04-30 2021-12-14 实现音频与图像同步控制灯光的***及方法

Country Status (2)

Country Link
CN (1) CN113286405B (zh)
WO (1) WO2022227599A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113286405B (zh) * 2021-04-30 2022-06-28 深圳市凯润智能照明有限公司 实现音频与图像同步控制灯光的***及方法
CN113613370B (zh) * 2021-08-30 2024-03-19 江苏惠通集团有限责任公司 氛围灯控制方法及装置、计算机可读存储介质、终端
CN113965661A (zh) * 2021-10-19 2022-01-21 谢文军 一种从视频信号采集用于同步控制灯光的数字信号的方法
CN114286194B (zh) * 2021-12-08 2024-05-03 北京金山云网络技术有限公司 即时通信视频的处理方法、装置、电子设备及存储介质
CN114554655A (zh) * 2022-02-18 2022-05-27 青岛易来智能科技股份有限公司 灯光效果的执行方法和装置、存储介质及电子装置
CN115334099B (zh) * 2022-07-20 2024-02-27 榜威电子科技（上海）有限公司 流媒体音/视频数据与灯具的联动***、方法及存储介质
CN115623645A (zh) * 2022-09-05 2023-01-17 深圳市集贤科技有限公司 灯光设备联动控制***及其灯光设备联动控制方法
CN117560828B (zh) * 2024-01-11 2024-03-15 深圳市智岩科技有限公司 氛围灯设备及多灯效混播控制方法和相应的装置、介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101252801A (zh) * 2008-03-21 2008-08-27 中兴通讯股份有限公司 一种灯光控制方法及装置
CN105657901A (zh) * 2016-02-29 2016-06-08 浙江凯耀照明股份有限公司 音视频信号及灯光控制***
CN106332393A (zh) * 2015-06-30 2017-01-11 芋头科技（杭州）有限公司 一种音乐灯光律动***及方法
CN106713699A (zh) * 2016-12-20 2017-05-24 广州明浩电子科技有限公司 一种控制***
CN110602852A (zh) * 2019-09-05 2019-12-20 佛山市云米电器科技有限公司 一种控制家庭影院灯光效果的***
US20200275152A1 (en) * 2017-09-01 2020-08-27 Signify Holding B.V. Rendering a dynamic light scene based on audio-visual content
CN112512184A (zh) * 2020-12-02 2021-03-16 深圳市智岩科技有限公司 一种取色照明控制方法、装置、***及存储介质
CN113286405A (zh) * 2021-04-30 2021-08-20 深圳市凯润智能照明有限公司 实现音频与图像同步控制灯光的***及方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2124508A4 (en) * 2006-12-28 2011-03-23 Sharp Kk DEVICE, SYSTEM AND METHOD FOR CONTROLLING AN AUDIOVISUAL ENVIRONMENT
CN106878787B (zh) * 2017-03-08 2020-02-14 深圳创维－Rgb电子有限公司 电视影院模式实现方法及装置
US11510300B2 (en) * 2018-11-01 2022-11-22 Signify Holding B.V. Determinning light effects based on video and audio information in dependence on video and audio weights

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101252801A (zh) * 2008-03-21 2008-08-27 中兴通讯股份有限公司 一种灯光控制方法及装置
CN106332393A (zh) * 2015-06-30 2017-01-11 芋头科技（杭州）有限公司 一种音乐灯光律动***及方法
CN105657901A (zh) * 2016-02-29 2016-06-08 浙江凯耀照明股份有限公司 音视频信号及灯光控制***
CN106713699A (zh) * 2016-12-20 2017-05-24 广州明浩电子科技有限公司 一种控制***
US20200275152A1 (en) * 2017-09-01 2020-08-27 Signify Holding B.V. Rendering a dynamic light scene based on audio-visual content
CN110602852A (zh) * 2019-09-05 2019-12-20 佛山市云米电器科技有限公司 一种控制家庭影院灯光效果的***
CN112512184A (zh) * 2020-12-02 2021-03-16 深圳市智岩科技有限公司 一种取色照明控制方法、装置、***及存储介质
CN113286405A (zh) * 2021-04-30 2021-08-20 深圳市凯润智能照明有限公司 实现音频与图像同步控制灯光的***及方法

Also Published As

Publication number Publication date
CN113286405B (zh) 2022-06-28
CN113286405A (zh) 2021-08-20

Similar Documents

Publication Publication Date Title
WO2022227599A1 (zh) 实现音频与图像同步控制灯光的***及方法
TWI595786B (zh) 基於時間戳記的音訊與視訊處理方法及其系統
US9451313B2 (en) Network media adapter
CN103546662A (zh) 一种网络监控***中音视频同步方法
WO2016095369A1 (zh) 屏幕录制的方法及装置
US9088818B2 (en) Adaptive media delay matching
WO2015000328A1 (zh) 一种音频同步输出方法及***
CN109565466A (zh) 多设备间唇音同步方法及设备
CN102932676A (zh) 基于音视频同步的自适应带宽传输和播放方法
US11895352B2 (en) System and method for operating a transmission network
CN106791271B (zh) 一种音视频同步方法
CN104284098A (zh) 视频数据的处理方法及***
CN213342512U (zh) 一种播放盒及led显示装置
WO2016147176A1 (en) A method of controlling mobile devices in concert during a mass spectators event
US20130166769A1 (en) Receiving device, screen frame transmission system and method
US11553577B2 (en) System and method for achieving synchronized audio and image control of lighting
CN107835449A (zh) Android平台上基于网络实现多终端同步联屏的方法
CN106782598A (zh) 电视画面和外设声音同步控制方法和装置
WO2022199491A1 (zh) 一种立体声组网方法、***及相关装置
CN101977286B (zh) 数字电视节目替换装置
CN106131374B (zh) 一种机器人文件使用、存储方法及***
JP2000165368A (ja) マルチレート伝送システムにおけるフレーム同期方法及びこれに用いる送信装置，送受信装置
TWI482470B (zh) 數位看板播放系統、即時監看系統及其即時監看之方法
CN205566482U (zh) 一种多传感器多通道视频数据输入装置
CN204836359U (zh) 一种多格式音视频处理器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21939058

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE