WO2020164277A1 - Monitoring method and apparatus based on audio and video linkage, and terminal device and medium - Google Patents

Monitoring method and apparatus based on audio and video linkage, and terminal device and medium Download PDF

Info

Publication number
WO2020164277A1
WO2020164277A1 PCT/CN2019/118158 CN2019118158W WO2020164277A1 WO 2020164277 A1 WO2020164277 A1 WO 2020164277A1 CN 2019118158 W CN2019118158 W CN 2019118158W WO 2020164277 A1 WO2020164277 A1 WO 2020164277A1
Authority
WO
WIPO (PCT)
Prior art keywords
monitoring
area
audio
hot spot
surveillance
Prior art date
Application number
PCT/CN2019/118158
Other languages
French (fr)
Chinese (zh)
Inventor
王路生
陆进
陈斌
宋晨
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2020164277A1 publication Critical patent/WO2020164277A1/en

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D3/00—Control of position or direction
    • G05D3/12—Control of position or direction using feedback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This application belongs to the technical field of security monitoring, and in particular relates to a monitoring method, device, terminal equipment, and media based on audio and video linkage.
  • Video surveillance is an important part of the security system. Video surveillance in traditional technology focuses on the collection of image information in the monitored area, and usually does not collect sound. However, there are a small number of monitoring terminal equipment equipped with sound collection equipment on the market. Because the monitoring area has a certain range, the sound collection direction is difficult to align with the sound source direction, resulting in low quality of the collected sound, which cannot play its due role in audio monitoring.
  • this application provides a monitoring method, device, terminal equipment, and media based on audio and video linkage to solve the problem of low sound quality of audio monitoring in the prior art and high hardware cost for large-scale video monitoring.
  • the problem is a monitoring method, device, terminal equipment, and media based on audio and video linkage to solve the problem of low sound quality of audio monitoring in the prior art and high hardware cost for large-scale video monitoring. The problem.
  • the first aspect of the embodiments of the present application provides a monitoring method based on audio and video linkage, including:
  • the direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  • the second aspect of the embodiments of the present application provides a monitoring device based on audio and video linkage, including:
  • Image acquisition unit for acquiring surveillance images of the surveillance area
  • the first hot spot analysis unit is configured to perform hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area;
  • a first position calculation unit configured to calculate the first position of the first monitoring hot spot in the monitoring area if the first monitoring hot spot exists in the monitoring area;
  • the video monitoring unit is configured to determine the direction in which the first azimuth is located as the central direction of video monitoring for video monitoring;
  • the audio monitoring unit is used to determine the direction in which the first azimuth is located as the central direction of audio monitoring for audio monitoring.
  • a third aspect of the embodiments of the present application provides a terminal device, including a memory and a processor.
  • the memory stores computer-readable instructions that can run on the processor.
  • the processor executes the computer The following steps are implemented when reading instructions:
  • the direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  • a fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores computer-readable instructions, wherein the computer-readable instructions are implemented when executed by at least one processor The following steps:
  • the direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  • This application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance hotspot exists in the surveillance area, calculate the The first location of the first surveillance hotspot in the surveillance area; the direction in which the first location is located is determined as the central direction of video surveillance for video surveillance; that is, the surveillance area is discovered by performing hot spot analysis on surveillance images
  • the monitoring hotspot is positioned in the center of the monitoring field of view, so that there is no need to arrange multiple monitoring equipment when performing a large-scale video monitoring, which can reduce the hardware cost of video monitoring equipment;
  • the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, which is beneficial to improve the sound during audio monitoring. quality.
  • FIG. 1 is a flowchart of an embodiment of a monitoring method based on audio and video linkage in an embodiment of the application
  • FIG. 2 is a flowchart of another embodiment of a monitoring method based on audio and video linkage in an embodiment of the application;
  • Figure 3 is a structural diagram of an embodiment of a monitoring device based on audio and video linkage in an embodiment of the application
  • Fig. 4 is a schematic block diagram of a terminal device in an embodiment of the application.
  • an embodiment of the monitoring method based on audio-video linkage in the embodiment of the present application may include:
  • step S101 a surveillance image of a surveillance area is acquired.
  • the monitoring image of the monitoring area captured by the monitoring terminal device is acquired.
  • the monitoring terminal device may be equipped with a camera component, through which the monitoring area is video monitored and real-time screen images of the monitoring area are collected.
  • the camera component may be provided with a pan-tilt, and the direction of shooting the video image of the camera component can be adjusted through the pan-tilt.
  • the PTZ when the range of the monitoring area is larger than the monitoring field of view of the camera component, the PTZ can be used to control the shooting direction of the camera component to cyclically move within the range of the monitoring area to realize the monitoring of the monitoring field of view larger than the camera component Video surveillance of the area.
  • step S102 a hot spot analysis is performed on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area.
  • the surveillance image of the surveillance area can be acquired in real time or at specified time intervals, and hot spot analysis can be performed on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area.
  • the first monitoring hotspot refers to the monitoring focus in the monitoring area.
  • the monitoring hotspot may be a target person appearing in the monitoring area or a conflict event appearing in the monitoring area.
  • it is possible to determine whether there is a surveillance hot spot in the surveillance area by specifying the image characteristics of the surveillance hot spot in advance.
  • the facial features of the target person A can be stored in advance.
  • the surveillance images are acquired and the surveillance images are recognized by facial features.
  • the face feature of the target person A appears on the surveillance image, it is determined that the first surveillance hot spot appears in the surveillance area, and the location of the target person A in the surveillance area is also the location of the first surveillance hot spot.
  • step S102 may include:
  • the image area exists, it is determined that there is a first surveillance hot spot in the surveillance area;
  • a sliding window of a specified size can be preset, and the sliding window can be traversed through the monitoring image.
  • the population density of the sliding window is counted, and it is determined whether the population density of the sliding window is greater than a preset density threshold .
  • the image area corresponding to the position of the sliding window determines the area where the first monitoring hotspot is located.
  • the surveillance hotspot with the largest crowd density can be selected as the first surveillance hotspot.
  • the crowd density can be determined by identifying the number of heads in the sliding window.
  • the crowd density is high, it may be a group event, so it can be used as a monitoring terminal device.
  • step S103 if the first monitoring hot spot exists in the monitoring area, the first position of the first monitoring hot spot in the monitoring area is calculated.
  • the first location of the first surveillance hotspot in the surveillance area can be determined according to the position of the image area corresponding to the first surveillance hotspot on the surveillance image.
  • step S103 may include:
  • the corresponding relationship between the monitored image and the monitored area is related to the actual parameters of the camera component.
  • the coordinate position correspondence relationship can be established in advance, and the coordinate position correspondence relationship stores the pixels on the monitoring image.
  • Correspondence between point coordinates and actual monitoring area orientation Therefore, for the image area corresponding to the first surveillance hot spot on the surveillance image, the central pixel point of the image area can be selected, and the orientation corresponding to the coordinates of the central pixel point can be found according to the coordinate azimuth correspondence relationship. First position.
  • step S104 the direction in which the first azimuth is located is determined as the central direction of video surveillance for video surveillance.
  • step S105 the direction in which the first azimuth is located is determined as the center direction of audio monitoring for audio monitoring.
  • the central direction of the camera component monitoring can be adjusted to the first direction by adjusting the pan/tilt, that is, the first surveillance hot spot is located in the camera component The central position of the monitoring field of view, so that when an event corresponding to a monitoring hot spot occurs in the monitoring area, more comprehensive monitoring video information can be obtained.
  • the audio collection direction of the audio monitoring device can be adjusted to make the audio collection direction of the audio monitoring device consistent with the first direction. That is, the audio collection direction of the audio monitoring device is aligned with the first monitoring hot spot, so that when an event corresponding to the monitoring hot spot occurs in the monitoring area, higher quality monitoring audio information can be obtained.
  • this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance.
  • the hardware cost of the device on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
  • another embodiment of the monitoring method based on audio and video linkage in the embodiment of the present application may include:
  • Step S201 Obtain monitoring audio of the monitoring area.
  • the monitoring terminal device may be configured with an audio collection component, for example, the audio collection component may be a microphone.
  • the audio information of the monitoring area is collected through the audio collection component, and the monitoring audio of the monitoring area collected by the audio collection component is obtained.
  • the audio collection direction of the audio collection component can be adjusted at a certain speed, so that the audio collection direction of the audio collection component can traverse the monitoring area, so as to collect more audio information.
  • Step S202 Perform a hot spot analysis on the monitoring audio to determine whether a second monitoring hot spot exists in the monitoring area.
  • hot spot analysis can be performed on the collected monitoring audio in real time to determine whether there is a second monitoring hot spot in the monitoring area.
  • the second monitoring hotspot refers to the monitoring focus of the monitoring area recognized by sound.
  • the second monitoring hotspot can be a designated sound (recognized by voiceprint features), or a designated voice (such as help or "Help", through speech to text for speech recognition), it can also be a sound source whose sound intensity exceeds a certain value.
  • step S202 may include:
  • the target sound source exists, it is determined that there is a second monitoring hot spot in the monitoring area;
  • the target sound source does not exist, it is determined that there is no second monitoring hot spot in the monitoring area.
  • the presence of louder sound intensity in the monitoring audio indicates that there may be an event that needs to be monitored in the monitoring area.
  • the sound intensity of the collected sound information of different sound sources can be detected.
  • a target sound source whose sound intensity is greater than a preset intensity threshold is detected, it is determined that there is a second monitoring hot spot in the monitoring area.
  • Step S203 If the second monitoring hot spot exists in the monitoring area, calculate the second orientation of the second monitoring hot spot in the monitoring area.
  • the sound propagation time and the sound propagation direction information can be used to calculate the position of the target sound source.
  • the audio collection device may be a microphone array, and the audio information of the monitored area is collected through the microphone array.
  • the foregoing step S203 may include:
  • the position arrangement of the microphones in the microphone array and the time difference between the microphones in the microphone array to collect the sound from the target sound source calculate the target sound source at the second monitoring hotspot at the The second position in the monitoring area.
  • Step S204 Determine the direction in which the second azimuth is located as the central direction of the video surveillance for video surveillance.
  • Step S205 Determine the direction in which the second azimuth is located as the center direction of audio monitoring to perform audio monitoring.
  • the central direction of the camera component monitoring can be adjusted to the second direction by adjusting the pan/tilt, that is, the second surveillance hot spot is located in the camera component The central position of the monitoring field of view, so that when an event corresponding to a monitoring hot spot occurs in the monitoring area, more comprehensive monitoring video information can be obtained.
  • the audio collection direction of the audio monitoring device can be adjusted to make the audio collection direction of the audio monitoring device consistent with the second direction. That is, the audio collection direction of the audio monitoring device is aligned with the second monitoring hotspot, so that when an event corresponding to the monitoring hotspot occurs in the monitoring area, higher-quality monitoring audio information can be obtained.
  • the embodiment shown in FIG. 1 and the embodiment shown in FIG. 2 can be implemented together to realize the linkage of audio monitoring and video monitoring. It can enable the camera component to obtain more video picture information, and enable the audio collection device to obtain higher-quality sound information.
  • the hotspot scores of the first surveillance hotspot and the second surveillance hotspot can be calculated separately; for example, different crowd density values Corresponding to a certain hotspot score, different sound intensities correspond to a certain hotspot score, and the orientation corresponding to the monitoring hotspot with a higher hotspot score is determined as the target orientation;
  • the direction in which the target azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  • the more important monitoring hotspot is selected from the first monitoring hotspot and the second monitoring hotspot as the monitoring center point to obtain the target orientation, and the direction of the target orientation is determined as
  • the central direction of video surveillance is for video surveillance
  • the central direction of audio surveillance is for audio surveillance, so as to realize key monitoring of the most important events in the surveillance area.
  • this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance.
  • the hardware cost of the device on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
  • FIG. 3 shows a structural diagram of an embodiment of a monitoring device based on audio and video linkage provided in an embodiment of the present application.
  • the monitoring device 3 based on audio and video linkage may include: an image acquisition unit 31, a first hot spot analysis unit 32, a first orientation calculation unit 33, a video monitoring unit 34, and an audio monitoring unit 35.
  • the image acquisition unit 31 is used to acquire the surveillance image of the surveillance area
  • the first hot spot analysis unit 32 is configured to perform hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area;
  • the first position calculating unit 33 is configured to calculate the first position of the first monitoring hot spot in the monitoring area if the first monitoring hot spot exists in the monitoring area;
  • the video monitoring unit 34 is configured to determine the direction in which the first azimuth is located as the center direction of video monitoring for video monitoring;
  • the audio monitoring unit 35 is configured to determine the direction in which the first orientation is located as the center direction of audio monitoring for audio monitoring.
  • the monitoring device 3 based on audio and video linkage may also include:
  • a density judgment unit configured to judge whether there is an image area on the monitoring image with a crowd density greater than a preset density threshold
  • the first hot spot analysis unit 32 is specifically configured to: if the density determination unit determines that the image area exists, determine that the first monitoring hot spot exists in the monitoring area; and, if the density determination unit determines that the image area does not exist, determine There is no first monitoring hot spot in the monitoring area.
  • the monitoring device 3 based on audio and video linkage may also include:
  • a coordinate obtaining unit configured to obtain the first coordinate according to the position of the central pixel of the image area on the monitoring image
  • the first orientation calculation unit 33 is specifically configured to determine the first orientation of the first coordinate in the monitoring area according to a preset coordinate orientation correspondence.
  • the monitoring device 3 based on audio and video linkage may also include:
  • An audio acquisition unit for acquiring monitoring audio of the monitoring area
  • the second hot spot analysis unit is configured to perform hot spot analysis on the monitoring audio to determine whether there is a second monitoring hot spot in the monitoring area;
  • a second position calculation unit configured to calculate a second position of the second monitoring hot spot in the monitoring area if the second monitoring hot spot exists in the monitoring area;
  • the video monitoring unit 34 is further configured to determine the direction in which the second azimuth is located as the center direction of video monitoring for video monitoring;
  • the audio monitoring unit 35 is further configured to determine the direction in which the second orientation is located as the center direction of audio monitoring for audio monitoring.
  • the monitoring device 3 based on audio and video linkage may also include:
  • An intensity judging unit for judging whether there is a target sound source with a sound intensity greater than a preset intensity threshold in the monitored audio
  • the second hot spot analysis unit is specifically configured to: if the intensity determination unit determines that the target sound source exists, determine that there is a second monitoring hot spot in the monitoring area; and, if the intensity determination unit determines that the target sound source does not exist, It is determined that there is no second monitoring hot spot in the monitoring area.
  • the audio acquiring unit is specifically configured to acquire the monitored audio of the monitoring area by using a microphone array
  • the second orientation calculation unit is specifically configured to calculate the time difference between the microphones in the microphone array and the sound emitted by the target sound source according to the positional arrangement of the microphones in the microphone array.
  • the target sound source is in a second orientation of the second monitoring hotspot in the monitoring area.
  • the monitoring device 3 based on audio and video linkage may also include:
  • the hotspot degree calculation unit is configured to, if the first monitoring hotspot and the second monitoring hotspot exist in the monitoring area at the same time, and the first direction and the second monitoring hotspot corresponding to the first monitoring hotspot and the second monitoring hotspot If the second orientation is inconsistent, the hotspot scores of the first monitoring hotspot and the second monitoring hotspot are calculated respectively.
  • the target orientation determining unit is used to determine the orientation corresponding to the monitoring hotspot with a higher hotspot score as the target orientation
  • the video monitoring unit 34 is also used to determine the direction in which the target azimuth is located as the central direction of video monitoring for video monitoring; the audio monitoring unit 35 is also used to determine the direction in which the target azimuth is located as the audio monitoring Perform audio monitoring in the center direction.
  • this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance.
  • the hardware cost of the device on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
  • FIG. 4 shows a schematic block diagram of a terminal device provided by an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.
  • the terminal device 4 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the terminal device 4 may include: a processor 40, a memory 41, and computer readable instructions 42 stored in the memory 41 and running on the processor 40, such as those that execute the aforementioned monitoring method based on audio and video linkage.
  • Computer readable instructions When the processor 40 executes the computer-readable instructions 42, the steps in the above-mentioned embodiments of the monitoring method based on audio and video linkage are implemented, such as steps S101 to S105 shown in FIG. 1.
  • the processor 40 implements the functions of the units in the foregoing device embodiments when executing the computer-readable instructions 42, for example, the functions of the units 31 to 35 shown in FIG. 3.
  • the computer-readable instructions 42 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 41 and executed by the processor 40, To complete this application.
  • the one or more modules/units may be a series of computer-readable instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer-readable instructions 42 in the terminal device 4.
  • the processor 40 may be a central processing unit (Central Processing Unit, CPU), it can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4.
  • the memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk equipped on the terminal device 4, a smart memory card (Smart Media Card, SMC), or a secure digital (Secure Digital, SD). Card, Flash Card, etc.
  • the memory 41 may also include both an internal storage unit of the terminal device 4 and an external storage device.
  • the memory 41 is used to store the computer-readable instructions and other instructions and data required by the terminal device 4.
  • the memory 41 can also be used to temporarily store data that has been output or will be output.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • ROM read-only memory
  • PROM programmable ROM
  • EPROM electrically programmable ROM
  • EEPROM electrically erasable programmable ROM
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The present application relates to the technical field of security and protection monitoring, and in particular, to a monitoring method and apparatus based on audio and video linkage, and a terminal device and a medium. The monitoring method comprises: obtaining a monitoring image of a monitoring area; performing hot spot analysis on the monitoring image to determine whether a first monitoring hot spot exists in the monitoring area; if the first monitoring hot spot exists in the monitoring area, calculating a first orientation of the first monitoring hot spot in the monitoring area; determining a direction in which the first orientation is located as a central direction of video monitoring for video monitoring; and determining the direction in which the first orientation is located as a central direction of audio monitoring for audio monitoring. According to the present application, hardware costs of a video monitoring device can be reduced when large-scale video monitoring is performed, and the sound quality is facilitated to be improved when audio monitoring is performed.

Description

一种基于音视频联动的监控方法、装置、终端设备及介质Monitoring method, device, terminal equipment and medium based on audio and video linkage
本申请要求于2019年02月12日提交中国专利局、申请号为201910111104.2 、发明名称为“一种基于音视频联动的监控方法、装置、终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on February 12, 2019, the application number is 201910111104.2, and the invention title is "A monitoring method, device, terminal and storage medium based on audio and video linkage". The entire content is incorporated into this application by reference.
技术领域Technical field
本申请属于安防监控技术领域，尤其涉及一种基于音视频联动的监控方法、装置、终端设备及介质。This application belongs to the technical field of security monitoring, and in particular relates to a monitoring method, device, terminal equipment, and media based on audio and video linkage.
背景技术Background technique
视频监控是安全防范***的重要组成部分，传统技术中的视频监控侧重于对监控区域的图像信息进行采集，通常不进行声音采集，而市场上存在的少量配置有声音采集设备的监控终端设备，由于监控区域具备一定的范围，声音采集方向难以对准声源方向，导致采集的声音质量较低，不能起到音频监控的应有作用。Video surveillance is an important part of the security system. Video surveillance in traditional technology focuses on the collection of image information in the monitored area, and usually does not collect sound. However, there are a small number of monitoring terminal equipment equipped with sound collection equipment on the market. Because the monitoring area has a certain range, the sound collection direction is difficult to align with the sound source direction, resulting in low quality of the collected sound, which cannot play its due role in audio monitoring.
另外，由于视频监控的摄像设备存在一定的监控视野范围，导致在监控视范围边缘附近发生的事件难以被摄像设备完整记录，传统技术中通常采用配备多个摄像设备的方式进行多摄像头组合监控，这样提高了监控的硬件成本。In addition, because the camera equipment for video surveillance has a certain monitoring field of view, it is difficult for the event that occurs near the edge of the surveillance field to be fully recorded by the camera equipment. Traditional technology usually adopts multiple camera equipment for multi-camera combined monitoring. This increases the hardware cost of monitoring.
技术问题technical problem
有鉴于此，本申请提供了一种基于音视频联动的监控方法、装置、终端设备及介质，以解决现有技术中音频监控的声音质量不高以及在进行较大范围视频监控时硬件成本高的问题。In view of this, this application provides a monitoring method, device, terminal equipment, and media based on audio and video linkage to solve the problem of low sound quality of audio monitoring in the prior art and high hardware cost for large-scale video monitoring. The problem.
技术解决方案Technical solutions
本申请实施例的第一方面提供了一种基于音视频联动的监控方法，包括：The first aspect of the embodiments of the present application provides a monitoring method based on audio and video linkage, including:
获取监控区域的监控图像；Obtain surveillance images of the surveillance area;
对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
本申请实施例的第二方面提供了一种基于音视频联动的监控装置，包括：The second aspect of the embodiments of the present application provides a monitoring device based on audio and video linkage, including:
图像获取单元，用于获取监控区域的监控图像；Image acquisition unit for acquiring surveillance images of the surveillance area;
第一热点分析单元，用于对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；The first hot spot analysis unit is configured to perform hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area;
第一方位计算单元，用于若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；A first position calculation unit, configured to calculate the first position of the first monitoring hot spot in the monitoring area if the first monitoring hot spot exists in the monitoring area;
视频监控单元，用于将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；The video monitoring unit is configured to determine the direction in which the first azimuth is located as the central direction of video monitoring for video monitoring;
音频监控单元，用于将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The audio monitoring unit is used to determine the direction in which the first azimuth is located as the central direction of audio monitoring for audio monitoring.
本申请实施例的第三方面提供了一种终端设备，包括存储器、处理器，所述存储器上存储有可在所述处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现如下步骤：A third aspect of the embodiments of the present application provides a terminal device, including a memory and a processor. The memory stores computer-readable instructions that can run on the processor. The processor executes the computer The following steps are implemented when reading instructions:
对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可读指令，其特征在于，所述计算机可读指令被至少一个处理器执行时实现如下步骤：A fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores computer-readable instructions, wherein the computer-readable instructions are implemented when executed by at least one processor The following steps:
对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
有益效果Beneficial effect
本申请通过获取监控区域的监控图像；对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；也即，通过对监控图像进行热点分析来发现监控区域的监控热点，并通过调整监控视野的中心方向使监控热点处于监控视野中心，如此一来在进行较大范围的视频监控时可以无需额外布置多个监控设备，能够降低视频监控设备的硬件成本；另一方面，通过将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控，可以将音频监控设备的声音采集方向对准监控热点方向，从而在进行音频监控时有利于提高声音质量。This application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance hotspot exists in the surveillance area, calculate the The first location of the first surveillance hotspot in the surveillance area; the direction in which the first location is located is determined as the central direction of video surveillance for video surveillance; that is, the surveillance area is discovered by performing hot spot analysis on surveillance images By adjusting the center direction of the monitoring field of view, the monitoring hotspot is positioned in the center of the monitoring field of view, so that there is no need to arrange multiple monitoring equipment when performing a large-scale video monitoring, which can reduce the hardware cost of video monitoring equipment; On the other hand, by determining the direction in which the first azimuth is located as the central direction of audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, which is beneficial to improve the sound during audio monitoring. quality.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.
图1为本申请实施例中基于音视频联动的监控方法的一个实施例流程图；FIG. 1 is a flowchart of an embodiment of a monitoring method based on audio and video linkage in an embodiment of the application;
图2为本申请实施例中基于音视频联动的监控方法的另一个实施例流程图；2 is a flowchart of another embodiment of a monitoring method based on audio and video linkage in an embodiment of the application;
图3为本申请实施例中基于音视频联动的监控装置的一个实施例结构图；Figure 3 is a structural diagram of an embodiment of a monitoring device based on audio and video linkage in an embodiment of the application;
图4为本申请实施例中一种终端设备的示意框图。Fig. 4 is a schematic block diagram of a terminal device in an embodiment of the application.
本发明的实施方式Embodiments of the invention
以下描述中，为了说明而不是为了限定，提出了诸如特定***结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的***、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.
为了说明本申请所述的技术方案，下面通过具体实施例来进行说明。In order to illustrate the technical solutions described in the present application, specific embodiments are used for description below.
请参阅图1，本申请实施例中基于音视频联动的监控方法的一个实施例可以包括：Referring to FIG. 1, an embodiment of the monitoring method based on audio-video linkage in the embodiment of the present application may include:
在步骤S101中、获取监控区域的监控图像。In step S101, a surveillance image of a surveillance area is acquired.
在本申请实施例中，获取监控终端设备拍摄的监控区域的监控图像，具体的，该监控终端设备可以配置有摄像组件，通过摄像组件对监控区域进行视频监控，采集监控区域的实时画面图像。In the embodiment of the present application, the monitoring image of the monitoring area captured by the monitoring terminal device is acquired. Specifically, the monitoring terminal device may be equipped with a camera component, through which the monitoring area is video monitored and real-time screen images of the monitoring area are collected.
在一种实现方式中，摄像组件可以设置有云台，通过云台可以调整摄像组件的视频画面拍摄方向。In an implementation manner, the camera component may be provided with a pan-tilt, and the direction of shooting the video image of the camera component can be adjusted through the pan-tilt.
在本申请实施例中，当监控区域的范围大于摄像组件的监控视野时，可以通过云台来控制摄像组件的拍摄方向在监控区域范围内循环移动，实现对范围大于摄像组件的监控视野的监控区域的视频监控。In the embodiments of the present application, when the range of the monitoring area is larger than the monitoring field of view of the camera component, the PTZ can be used to control the shooting direction of the camera component to cyclically move within the range of the monitoring area to realize the monitoring of the monitoring field of view larger than the camera component Video surveillance of the area.
在步骤S102中、对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点。In step S102, a hot spot analysis is performed on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area.
在视频监控过程中，可以实时或每隔指定的时间间隔获取监控区域的监控图像，并对该监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点。During the video surveillance process, the surveillance image of the surveillance area can be acquired in real time or at specified time intervals, and hot spot analysis can be performed on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area.
在本申请实施例中，第一监控热点是指监控区域中的监控重点，例如，监控热点可以是监控区域中出现的一个目标人物，也可以是监控区域中出现的一个冲突事件。在实际应用中，可以通过预先指定该监控热点的图像特征是来实现对监控区域中是否出现监控热点的判断。In the embodiment of the present application, the first monitoring hotspot refers to the monitoring focus in the monitoring area. For example, the monitoring hotspot may be a target person appearing in the monitoring area or a conflict event appearing in the monitoring area. In practical applications, it is possible to determine whether there is a surveillance hot spot in the surveillance area by specifying the image characteristics of the surveillance hot spot in advance.
例如，以某目标人物A为第一监控热点时，可以预先存储该目标人物A的人脸特征，在视频监控过程中，通过获取监控图像，并对监控图像进行人脸特征识别，当识别出监控图像上出现该目标人物A的人脸特征时，判定监控区域中出现第一监控热点，监控区域中目标人物A的所在位置也即第一监控热点的位置。For example, when a target person A is the first surveillance hotspot, the facial features of the target person A can be stored in advance. During the video surveillance process, the surveillance images are acquired and the surveillance images are recognized by facial features. When the face feature of the target person A appears on the surveillance image, it is determined that the first surveillance hot spot appears in the surveillance area, and the location of the target person A in the surveillance area is also the location of the first surveillance hot spot.
在一种实现方式中，上述步骤S102可以包括：In an implementation manner, the foregoing step S102 may include:
判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；Judging whether there is an image area with a crowd density greater than a preset density threshold on the monitoring image;
若存在所述图像区域，则判定所述监控区域中存在第一监控热点；If the image area exists, it is determined that there is a first surveillance hot spot in the surveillance area;
若不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。If the image area does not exist, it is determined that there is no first surveillance hot spot in the surveillance area.
在本实施例中，可以预先设置指定大小的滑动窗口，并将该滑动窗口遍历监控图像，在遍历过程中，统计滑动窗口的人群密度，并判断滑动窗口的人群密度是否大于预设的密度阈值，将滑动窗口的人群密度大于预设的密度阈值时，该滑动窗口所在位置对应的图像区域确定第一监控热点的所在区域。In this embodiment, a sliding window of a specified size can be preset, and the sliding window can be traversed through the monitoring image. During the traversal, the population density of the sliding window is counted, and it is determined whether the population density of the sliding window is greater than a preset density threshold , When the crowd density of the sliding window is greater than the preset density threshold, the image area corresponding to the position of the sliding window determines the area where the first monitoring hotspot is located.
需要说明的是，当监控图像中存在两个以上大于预设密度阈值的图像区域时，可以选取其中人群密度最大的一个监控热点作为第一监控热点。It should be noted that, when there are more than two image areas larger than the preset density threshold in the surveillance image, the surveillance hotspot with the largest crowd density can be selected as the first surveillance hotspot.
在本实施例中，可以通过识别滑动窗口中的人头数量，来确定人群密度。人群密度较大时，表示可能为一个***，故可将其作为监控终端设备。In this embodiment, the crowd density can be determined by identifying the number of heads in the sliding window. When the crowd density is high, it may be a group event, so it can be used as a monitoring terminal device.
在步骤S103中、若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位。In step S103, if the first monitoring hot spot exists in the monitoring area, the first position of the first monitoring hot spot in the monitoring area is calculated.
在本申请实施例中，由于监控图像和监控区域存在一定的对应关系，可以根据第一监控热点对应的图像区域在监控图像上的位置来确定第一监控热点在监控区域中的第一方位。In the embodiments of the present application, since there is a certain correspondence between the surveillance image and the surveillance area, the first location of the first surveillance hotspot in the surveillance area can be determined according to the position of the image area corresponding to the first surveillance hotspot on the surveillance image.
在一种实现方式中，上述步骤S103可以包括：In an implementation manner, the foregoing step S103 may include:
根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；Obtaining the first coordinate according to the position of the central pixel of the image area on the monitoring image;
根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。Determine the first position of the first coordinate in the monitoring area according to the preset coordinate position correspondence.
在本实施例中，监控图像和监控区域存在的对应关系与摄像组件的实际参数相关，基于摄像组件的实际参数，可以预先建立坐标方位对应关系，该坐标方位对应关系存储有监控图像上的像素点坐标与实际监控区域方位的对应关系。由此，对于监控图像上与第一监控热点对应的图像区域，可以选取该图像区域的中心像素点，根据坐标方位对应关系，查找与该中心像素点的坐标对应的方位，即可得到所述第一方位。In this embodiment, the corresponding relationship between the monitored image and the monitored area is related to the actual parameters of the camera component. Based on the actual parameters of the camera component, the coordinate position correspondence relationship can be established in advance, and the coordinate position correspondence relationship stores the pixels on the monitoring image. Correspondence between point coordinates and actual monitoring area orientation. Therefore, for the image area corresponding to the first surveillance hot spot on the surveillance image, the central pixel point of the image area can be selected, and the orientation corresponding to the coordinates of the central pixel point can be found according to the coordinate azimuth correspondence relationship. First position.
在步骤S104中、将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控。In step S104, the direction in which the first azimuth is located is determined as the central direction of video surveillance for video surveillance.
在步骤S105中、将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。In step S105, the direction in which the first azimuth is located is determined as the center direction of audio monitoring for audio monitoring.
在本申请实施例中，由于第一方位为监控区域中的第一监控热点，可以通过调整云台将摄像组件监控的中心方向调整为该第一方向，也即使得第一监控热点位于摄像组件监控视野的中心位置，这样可以使得监控区域中发生监控热点对应的事件时，能够获得较为全面的监控视频信息。In the embodiment of the present application, since the first orientation is the first surveillance hot spot in the surveillance area, the central direction of the camera component monitoring can be adjusted to the first direction by adjusting the pan/tilt, that is, the first surveillance hot spot is located in the camera component The central position of the monitoring field of view, so that when an event corresponding to a monitoring hot spot occurs in the monitoring area, more comprehensive monitoring video information can be obtained.
同理，在本申请实施例中，由于第一方位为监控区域中的第一监控热点，可以通过调整音频监控设备的音频采集方向，使得音频监控设备的音频采集方向与该第一方向一致，也即使得音频监控设备的音频采集方向对准了第一监控热点，这样可以使得监控区域中发生监控热点对应的事件时，能够获得较高质量的监控音频信息。Similarly, in the embodiment of the present application, since the first orientation is the first monitoring hotspot in the monitoring area, the audio collection direction of the audio monitoring device can be adjusted to make the audio collection direction of the audio monitoring device consistent with the first direction. That is, the audio collection direction of the audio monitoring device is aligned with the first monitoring hot spot, so that when an event corresponding to the monitoring hot spot occurs in the monitoring area, higher quality monitoring audio information can be obtained.
综上所述，本申请通过获取监控区域的监控图像；对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；也即，通过对监控图像进行热点分析来发现监控区域的监控热点，并通过调整监控视野的中心方向使监控热点处于监控视野中心，如此一来在进行较大范围的视频监控时可以无需额外布置多个监控设备，能够降低视频监控设备的硬件成本；另一方面，通过将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控，可以将音频监控设备的声音采集方向对准监控热点方向，从而在进行音频监控时有利于提高声音质量。In summary, this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance. The hardware cost of the device; on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
请参阅图2，本申请实施例中基于音视频联动的监控方法的另一个实施例可以包括：Referring to FIG. 2, another embodiment of the monitoring method based on audio and video linkage in the embodiment of the present application may include:
步骤S201、获取所述监控区域的监控音频。Step S201: Obtain monitoring audio of the monitoring area.
在本申请实施例中，监控终端设备可以配置有音频采集组件，例如该音频采集组件可以为麦克风。通过音频采集组件采集监控区域的音频信息，获取音频采集组件采集到的监控区域的监控音频。In the embodiment of the present application, the monitoring terminal device may be configured with an audio collection component, for example, the audio collection component may be a microphone. The audio information of the monitoring area is collected through the audio collection component, and the monitoring audio of the monitoring area collected by the audio collection component is obtained.
在一种实现方式中，可以以一定的速度调整音频采集组件的音频采集方向，使得该音频采集组件的音频采集方向可以遍历监控区域，以便采集到更多的音频信息。In an implementation manner, the audio collection direction of the audio collection component can be adjusted at a certain speed, so that the audio collection direction of the audio collection component can traverse the monitoring area, so as to collect more audio information.
步骤S202、对所述监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点。Step S202: Perform a hot spot analysis on the monitoring audio to determine whether a second monitoring hot spot exists in the monitoring area.
在音频监控过程中，可以实时对采集的监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点。在这里，第二监控热点是指通过声音识别到的监控区域的监控重点，例如，第二监控热点可以为指定的声音（通过声纹特征进行识别），也可以为指定的语音（例如救命或者“help”，通过语音转文字进行语音识别），还可以为声音强度超过一定值的声源。In the audio monitoring process, hot spot analysis can be performed on the collected monitoring audio in real time to determine whether there is a second monitoring hot spot in the monitoring area. Here, the second monitoring hotspot refers to the monitoring focus of the monitoring area recognized by sound. For example, the second monitoring hotspot can be a designated sound (recognized by voiceprint features), or a designated voice (such as help or "Help", through speech to text for speech recognition), it can also be a sound source whose sound intensity exceeds a certain value.
在一种实现方式中，上述步骤S202可以包括：In an implementation manner, the foregoing step S202 may include:
判断所述监控音频中是否存在声音强度大于预设强度阈值的目标声源；Determining whether there is a target sound source with a sound intensity greater than a preset intensity threshold in the monitored audio;
若存在所述目标声源，则判定所述监控区域中存在第二监控热点；If the target sound source exists, it is determined that there is a second monitoring hot spot in the monitoring area;
若不存在所述目标声源，则判定所述监控区域中不存在第二监控热点。If the target sound source does not exist, it is determined that there is no second monitoring hot spot in the monitoring area.
监控音频中出现较大声音强度的声音，表示该监控区域中可能出现需重点监控的事件。在本实施例中，可以对采集的不同声源的声音信息进行声音强度的检测，当检测到声音强度大于预设强度阈值的目标声源时，判定监控区域中存在第二监控热点。The presence of louder sound intensity in the monitoring audio indicates that there may be an event that needs to be monitored in the monitoring area. In this embodiment, the sound intensity of the collected sound information of different sound sources can be detected. When a target sound source whose sound intensity is greater than a preset intensity threshold is detected, it is determined that there is a second monitoring hot spot in the monitoring area.
步骤S203、若所述监控区域中存在所述第二监控热点，则计算所述第二监控热点在所述监控区域中的第二方位。Step S203: If the second monitoring hot spot exists in the monitoring area, calculate the second orientation of the second monitoring hot spot in the monitoring area.
在本申请实施例中，对于监控区域中的第二监控热点，可以通过声音传播时间和声音传播方向的信息进行计算，获得目标声源的方位。In the embodiment of the present application, for the second monitoring hotspot in the monitoring area, the sound propagation time and the sound propagation direction information can be used to calculate the position of the target sound source.
在一种实现方式中，音频采集设备可以为麦克风阵列，通过麦克风阵列来采集监控区域的音频信息，上述步骤S203可以包括：In an implementation manner, the audio collection device may be a microphone array, and the audio information of the monitored area is collected through the microphone array. The foregoing step S203 may include:
根据所述麦克风阵列中各麦克风的位置排布，以及，所述麦克风阵列中各麦克风采集到所述目标声源发出的声音的时间差，计算所述目标声源在所述第二监控热点在所述监控区域中的第二方位。According to the position arrangement of the microphones in the microphone array and the time difference between the microphones in the microphone array to collect the sound from the target sound source, calculate the target sound source at the second monitoring hotspot at the The second position in the monitoring area.
步骤S204、将所述第二方位所在的方向确定为视频监控的中心方向进行视频监控。Step S204: Determine the direction in which the second azimuth is located as the central direction of the video surveillance for video surveillance.
步骤S205、将所述第二方位所在的方向确定为音频监控的中心方向进行音频监控。Step S205: Determine the direction in which the second azimuth is located as the center direction of audio monitoring to perform audio monitoring.
在本申请实施例中，由于第二方位为监控区域中的第二监控热点，可以通过调整云台将摄像组件监控的中心方向调整为该第二方向，也即使得第二监控热点位于摄像组件监控视野的中心位置，这样可以使得监控区域中发生监控热点对应的事件时，能够获得较为全面的监控视频信息。In the embodiment of the present application, since the second orientation is the second surveillance hot spot in the surveillance area, the central direction of the camera component monitoring can be adjusted to the second direction by adjusting the pan/tilt, that is, the second surveillance hot spot is located in the camera component The central position of the monitoring field of view, so that when an event corresponding to a monitoring hot spot occurs in the monitoring area, more comprehensive monitoring video information can be obtained.
同理，在本申请实施例中，由于第二方位为监控区域中的第二监控热点，可以通过调整音频监控设备的音频采集方向，使得音频监控设备的音频采集方向与该第二方向一致，也即使得音频监控设备的音频采集方向对准了第二监控热点，这样可以使得监控区域中发生监控热点对应的事件时，能够获得较高质量的监控音频信息。Similarly, in the embodiment of the present application, since the second orientation is the second monitoring hot spot in the monitoring area, the audio collection direction of the audio monitoring device can be adjusted to make the audio collection direction of the audio monitoring device consistent with the second direction. That is, the audio collection direction of the audio monitoring device is aligned with the second monitoring hotspot, so that when an event corresponding to the monitoring hotspot occurs in the monitoring area, higher-quality monitoring audio information can be obtained.
在一种实现方式中，上述图1所示实施例和图2所示实施例可以共同实施，以实现音频监控和视频监控的联动。可以使得摄像组件能够获取到较多的视频画面信息，并且，使得音频采集设备获取到较高质量的声音信息。In an implementation manner, the embodiment shown in FIG. 1 and the embodiment shown in FIG. 2 can be implemented together to realize the linkage of audio monitoring and video monitoring. It can enable the camera component to obtain more video picture information, and enable the audio collection device to obtain higher-quality sound information.
可选的，在上述图1所示实施例和图2所示实施例合并实施的方案中，若所述监控区域中同时存在所述第一监控热点与所述第二监控热点，且，所述第一监控热点与所述第二监控热点对应的第一方位和第二方位不一致，则可以分别计算所述第一监控热点和第二监控热点的热点度得分；例如，不同的人群密度值对应于一定的热点度分值，不同的声音强度对应于一定的热点度分值，通过将热点度得分较高的监控热点对应的方位确定为目标方位；Optionally, in the solution implemented by combining the embodiment shown in FIG. 1 and the embodiment shown in FIG. 2, if the first monitoring hot spot and the second monitoring hot spot simultaneously exist in the monitoring area, and If the first azimuth and the second azimuth corresponding to the first surveillance hotspot and the second surveillance hotspot are inconsistent, the hotspot scores of the first surveillance hotspot and the second surveillance hotspot can be calculated separately; for example, different crowd density values Corresponding to a certain hotspot score, different sound intensities correspond to a certain hotspot score, and the orientation corresponding to the monitoring hotspot with a higher hotspot score is determined as the target orientation;
将所述目标方位所在的方向确定为视频监控的中心方向进行视频监控；Determine the direction in which the target azimuth is located as the central direction of video surveillance for video surveillance;
将所述目标方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the target azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
在本申请实施例中，通过热点度得分的比较，从第一监控热点和第二监控热点中选取更重要的监控热点作为监控的中心点，得到目标方位，并将目标方位所在的方向确定为视频监控的中心方向进行视频监控，以及音频监控的中心方向进行音频监控，实现对监控区域中最重要的事件进行重点监控。In the embodiment of the present application, by comparing the hotspot scores, the more important monitoring hotspot is selected from the first monitoring hotspot and the second monitoring hotspot as the monitoring center point to obtain the target orientation, and the direction of the target orientation is determined as The central direction of video surveillance is for video surveillance, and the central direction of audio surveillance is for audio surveillance, so as to realize key monitoring of the most important events in the surveillance area.
综上所述，本申请通过获取监控区域的监控图像；对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；也即，通过对监控图像进行热点分析来发现监控区域的监控热点，并通过调整监控视野的中心方向使监控热点处于监控视野中心，如此一来在进行较大范围的视频监控时可以无需额外布置多个监控设备，能够降低视频监控设备的硬件成本；另一方面，通过将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控，可以将音频监控设备的声音采集方向对准监控热点方向，从而在进行音频监控时有利于提高声音质量。In summary, this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance. The hardware cost of the device; on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
对应于上文实施例所述的基于音视频联动的监控方法，图3示出了本申请实施例提供的基于音视频联动的监控装置的一个实施例结构图。Corresponding to the monitoring method based on audio and video linkage described in the above embodiment, FIG. 3 shows a structural diagram of an embodiment of a monitoring device based on audio and video linkage provided in an embodiment of the present application.
本实施例中，基于音视频联动的监控装置3可以包括：图像获取单元31，第一热点分析单元32，第一方位计算单元33，视频监控单元34和音频监控单元35。In this embodiment, the monitoring device 3 based on audio and video linkage may include: an image acquisition unit 31, a first hot spot analysis unit 32, a first orientation calculation unit 33, a video monitoring unit 34, and an audio monitoring unit 35.
图像获取单元31，用于获取监控区域的监控图像；The image acquisition unit 31 is used to acquire the surveillance image of the surveillance area;
第一热点分析单元32，用于对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；The first hot spot analysis unit 32 is configured to perform hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area;
第一方位计算单元33，用于若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；The first position calculating unit 33 is configured to calculate the first position of the first monitoring hot spot in the monitoring area if the first monitoring hot spot exists in the monitoring area;
视频监控单元34，用于将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；The video monitoring unit 34 is configured to determine the direction in which the first azimuth is located as the center direction of video monitoring for video monitoring;
音频监控单元35，用于将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The audio monitoring unit 35 is configured to determine the direction in which the first orientation is located as the center direction of audio monitoring for audio monitoring.
可选的，基于音视频联动的监控装置3还可以包括：Optionally, the monitoring device 3 based on audio and video linkage may also include:
密度判断单元，用于判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；A density judgment unit, configured to judge whether there is an image area on the monitoring image with a crowd density greater than a preset density threshold;
第一热点分析单元32具体用于，若密度判断单元判定存在所述图像区域，则判定所述监控区域中存在第一监控热点；以及，若密度判断单元判定不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。The first hot spot analysis unit 32 is specifically configured to: if the density determination unit determines that the image area exists, determine that the first monitoring hot spot exists in the monitoring area; and, if the density determination unit determines that the image area does not exist, determine There is no first monitoring hot spot in the monitoring area.
可选的，基于音视频联动的监控装置3还可以包括：Optionally, the monitoring device 3 based on audio and video linkage may also include:
坐标获取单元，用于根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；A coordinate obtaining unit, configured to obtain the first coordinate according to the position of the central pixel of the image area on the monitoring image;
第一方位计算单元33具体用于，根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。The first orientation calculation unit 33 is specifically configured to determine the first orientation of the first coordinate in the monitoring area according to a preset coordinate orientation correspondence.
可选的，基于音视频联动的监控装置3还可以包括：Optionally, the monitoring device 3 based on audio and video linkage may also include:
音频获取单元，用于获取所述监控区域的监控音频；An audio acquisition unit for acquiring monitoring audio of the monitoring area;
第二热点分析单元，用于对所述监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点；The second hot spot analysis unit is configured to perform hot spot analysis on the monitoring audio to determine whether there is a second monitoring hot spot in the monitoring area;
第二方位计算单元，用于若所述监控区域中存在所述第二监控热点，则计算所述第二监控热点在所述监控区域中的第二方位；A second position calculation unit, configured to calculate a second position of the second monitoring hot spot in the monitoring area if the second monitoring hot spot exists in the monitoring area;
视频监控单元34还用于，将所述第二方位所在的方向确定为视频监控的中心方向进行视频监控；The video monitoring unit 34 is further configured to determine the direction in which the second azimuth is located as the center direction of video monitoring for video monitoring;
音频监控单元35还用于，将所述第二方位所在的方向确定为音频监控的中心方向进行音频监控。The audio monitoring unit 35 is further configured to determine the direction in which the second orientation is located as the center direction of audio monitoring for audio monitoring.
可选的，基于音视频联动的监控装置3还可以包括：Optionally, the monitoring device 3 based on audio and video linkage may also include:
强度判断单元，用于判断所述监控音频中是否存在声音强度大于预设强度阈值的目标声源；An intensity judging unit for judging whether there is a target sound source with a sound intensity greater than a preset intensity threshold in the monitored audio;
第二热点分析单元具体用于，若强度判断单元判定存在所述目标声源，则判定所述监控区域中存在第二监控热点；以及，若强度判断单元判定不存在所述目标声源，则判定所述监控区域中不存在第二监控热点。The second hot spot analysis unit is specifically configured to: if the intensity determination unit determines that the target sound source exists, determine that there is a second monitoring hot spot in the monitoring area; and, if the intensity determination unit determines that the target sound source does not exist, It is determined that there is no second monitoring hot spot in the monitoring area.
可选的，音频获取单元具体用于，利用麦克风阵列获取所述监控区域的监控音频；Optionally, the audio acquiring unit is specifically configured to acquire the monitored audio of the monitoring area by using a microphone array;
相应的，第二方位计算单元具体用于，根据所述麦克风阵列中各麦克风的位置排布，以及，所述麦克风阵列中各麦克风采集到所述目标声源发出的声音的时间差，计算所述目标声源在所述第二监控热点在所述监控区域中的第二方位。Correspondingly, the second orientation calculation unit is specifically configured to calculate the time difference between the microphones in the microphone array and the sound emitted by the target sound source according to the positional arrangement of the microphones in the microphone array. The target sound source is in a second orientation of the second monitoring hotspot in the monitoring area.
可选的，基于音视频联动的监控装置3还可以包括：Optionally, the monitoring device 3 based on audio and video linkage may also include:
热点度计算单元，用于若所述监控区域中同时存在所述第一监控热点与所述第二监控热点，且，所述第一监控热点与所述第二监控热点对应的第一方位和第二方位不一致，则分别计算所述第一监控热点和第二监控热点的热点度得分。The hotspot degree calculation unit is configured to, if the first monitoring hotspot and the second monitoring hotspot exist in the monitoring area at the same time, and the first direction and the second monitoring hotspot corresponding to the first monitoring hotspot and the second monitoring hotspot If the second orientation is inconsistent, the hotspot scores of the first monitoring hotspot and the second monitoring hotspot are calculated respectively.
目标方位确定单元，用于将热点度得分较高的监控热点对应的方位确定为目标方位；The target orientation determining unit is used to determine the orientation corresponding to the monitoring hotspot with a higher hotspot score as the target orientation;
相应的，视频监控单元34还用于，将所述目标方位所在的方向确定为视频监控的中心方向进行视频监控；音频监控单元35还用于，将所述目标方位所在的方向确定为音频监控的中心方向进行音频监控。Correspondingly, the video monitoring unit 34 is also used to determine the direction in which the target azimuth is located as the central direction of video monitoring for video monitoring; the audio monitoring unit 35 is also used to determine the direction in which the target azimuth is located as the audio monitoring Perform audio monitoring in the center direction.
综上所述，本申请通过获取监控区域的监控图像；对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；也即，通过对监控图像进行热点分析来发现监控区域的监控热点，并通过调整监控视野的中心方向使监控热点处于监控视野中心，如此一来在进行较大范围的视频监控时可以无需额外布置多个监控设备，能够降低视频监控设备的硬件成本；另一方面，通过将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控，可以将音频监控设备的声音采集方向对准监控热点方向，从而在进行音频监控时有利于提高声音质量。In summary, this application obtains surveillance images of the surveillance area; performs hot spot analysis on the surveillance images to determine whether there is a first surveillance hotspot in the surveillance area; if the first surveillance exists in the surveillance area Hotspot, calculate the first orientation of the first surveillance hotspot in the surveillance area; determine the direction in which the first orientation is located as the central direction of video surveillance for video surveillance; that is, perform hotspots on surveillance images Analyze to find the monitoring hotspots in the monitoring area, and adjust the center direction of the monitoring field to make the monitoring hotspots in the center of the monitoring field of view. In this way, there is no need to arrange multiple monitoring equipment for large-scale video monitoring, which can reduce video surveillance. The hardware cost of the device; on the other hand, by determining the direction in which the first orientation is located as the central direction of the audio monitoring for audio monitoring, the sound collection direction of the audio monitoring device can be aligned with the direction of the monitoring hotspot, thereby performing audio monitoring Time helps to improve the sound quality.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的装置，模块和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working processes of the above described devices, modules, and units can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.
图4示出了本申请实施例提供的一种终端设备的示意框图，为了便于说明，仅示出了与本申请实施例相关的部分。FIG. 4 shows a schematic block diagram of a terminal device provided by an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.
在本实施例中，所述终端设备4可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。该终端设备4可包括：处理器40、存储器41以及存储在所述存储器41中并可在所述处理器40上运行的计算机可读指令42，例如执行上述的基于音视频联动的监控方法的计算机可读指令。所述处理器40执行所述计算机可读指令42时实现上述各个基于音视频联动的监控方法实施例中的步骤，例如图1所示的步骤S101至步骤S105。或者，所述处理器40执行所述计算机可读指令42时实现上述各装置实施例中各单元的功能，例如图3所示单元31至35的功能。In this embodiment, the terminal device 4 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device 4 may include: a processor 40, a memory 41, and computer readable instructions 42 stored in the memory 41 and running on the processor 40, such as those that execute the aforementioned monitoring method based on audio and video linkage. Computer readable instructions. When the processor 40 executes the computer-readable instructions 42, the steps in the above-mentioned embodiments of the monitoring method based on audio and video linkage are implemented, such as steps S101 to S105 shown in FIG. 1. Alternatively, the processor 40 implements the functions of the units in the foregoing device embodiments when executing the computer-readable instructions 42, for example, the functions of the units 31 to 35 shown in FIG. 3.
示例性的，所述计算机可读指令42可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器41中，并由所述处理器40执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机可读指令段，该指令段用于描述所述计算机可读指令42在所述终端设备4中的执行过程。Exemplarily, the computer-readable instructions 42 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 41 and executed by the processor 40, To complete this application. The one or more modules/units may be a series of computer-readable instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer-readable instructions 42 in the terminal device 4.
所述处理器40可以是中央处理单元（Central Processing Unit，CPU），还可以是其它通用处理器、数字信号处理器（Digital Signal Processor，DSP）、专用集成电路（Application Specific Integrated Circuit，ASIC）、现场可编程门阵列（Field-Programmable Gate Array，FPGA）或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 40 may be a central processing unit (Central Processing Unit, CPU), it can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
所述存储器41可以是所述终端设备4的内部存储单元，例如终端设备4的硬盘或内存。所述存储器41也可以是所述终端设备4的外部存储设备，例如所述终端设备4上配备的插接式硬盘，智能存储卡（Smart Media Card， SMC），安全数字（Secure Digital， SD）卡，闪存卡（Flash Card）等。进一步地，所述存储器41还可以既包括所述终端设备4的内部存储单元也包括外部存储设备。所述存储器41用于存储所述计算机可读指令以及所述终端设备4所需的其它指令和数据。所述存储器41还可以用于暂时地存储已经输出或者将要输出的数据。The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk equipped on the terminal device 4, a smart memory card (Smart Media Card, SMC), or a secure digital (Secure Digital, SD). Card, Flash Card, etc. Further, the memory 41 may also include both an internal storage unit of the terminal device 4 and an external storage device. The memory 41 is used to store the computer-readable instructions and other instructions and data required by the terminal device 4. The memory 41 can also be used to temporarily store data that has been output or will be output.
在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。The functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器（ROM）、可编程ROM（PROM）、电可编程ROM（EPROM）、电可擦除可编程ROM（EEPROM）或闪存。易失性存储器可包括随机存取存储器（RAM）或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM（SRAM）、动态RAM（DRAM）、同步DRAM（SDRAM）、双数据率SDRAM（DDRSDRAM）、增强型SDRAM（ESDRAM）、同步链路（Synchlink） DRAM（SLDRAM）、存储器总线（Rambus）直接RAM（RDRAM）、直接存储器总线动态RAM（DRDRAM）、以及存储器总线动态RAM（RDRAM）等。A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions, which can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (20)

  1. 一种基于音视频联动的监控方法，其特征在于，包括：A monitoring method based on audio and video linkage, which is characterized in that it includes:
    获取监控区域的监控图像；Obtain surveillance images of the surveillance area;
    对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
    若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
    将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
    将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  2. 根据权利要求1所述的基于音视频联动的监控方法，其特征在于，所述对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点包括：The monitoring method based on audio and video linkage according to claim 1, wherein the performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area comprises:
    判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；Judging whether there is an image area with a crowd density greater than a preset density threshold on the monitoring image;
    若存在所述图像区域，则判定所述监控区域中存在第一监控热点；If the image area exists, it is determined that there is a first surveillance hot spot in the surveillance area;
    若不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。If the image area does not exist, it is determined that there is no first surveillance hot spot in the surveillance area.
  3. 根据权利要求2所述的基于音视频联动的监控方法，其特征在于，所述若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位，包括：The monitoring method based on audio and video linkage according to claim 2, wherein if the first monitoring hotspot exists in the monitoring area, then calculating the first monitoring hotspot in the monitoring area The first position includes:
    根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；Obtaining the first coordinate according to the position of the central pixel of the image area on the monitoring image;
    根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。Determine the first position of the first coordinate in the monitoring area according to the preset coordinate position correspondence.
  4. 根据权利要求1至3任一项所述的基于音视频联动的监控方法，其特征在于，所述监控方法还包括：The monitoring method based on audio and video linkage according to any one of claims 1 to 3, wherein the monitoring method further comprises:
    获取所述监控区域的监控音频；Acquiring monitoring audio of the monitoring area;
    对所述监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点；Performing hot spot analysis on the monitoring audio to determine whether there is a second monitoring hot spot in the monitoring area;
    若所述监控区域中存在所述第二监控热点，则计算所述第二监控热点在所述监控区域中的第二方位；If the second monitoring hotspot exists in the monitoring area, calculating the second position of the second monitoring hotspot in the monitoring area;
    将所述第二方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the second azimuth is located as the central direction of video surveillance for video surveillance;
    将所述第二方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the second azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  5. 根据权利要求4所述的基于音视频联动的监控方法，其特征在于，所述对所述监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点，包括：The monitoring method based on audio-video linkage according to claim 4, characterized in that the performing hot spot analysis on the monitoring audio to determine whether there is a second monitoring hot spot in the monitoring area comprises:
    判断所述监控音频中是否存在声音强度大于预设强度阈值的目标声源；Determining whether there is a target sound source with a sound intensity greater than a preset intensity threshold in the monitored audio;
    若存在所述目标声源，则判定所述监控区域中存在第二监控热点；If the target sound source exists, it is determined that there is a second monitoring hot spot in the monitoring area;
    若不存在所述目标声源，则判定所述监控区域中不存在第二监控热点。If the target sound source does not exist, it is determined that there is no second monitoring hot spot in the monitoring area.
  6. 根据权利要求5所述的基于音视频联动的监控方法，其特征在于，所述获取所述监控区域的监控音频包括：The monitoring method based on audio and video linkage according to claim 5, wherein said acquiring the monitoring audio of the monitoring area comprises:
    利用麦克风阵列获取所述监控区域的监控音频；Acquiring the monitoring audio of the monitoring area by using a microphone array;
    相应的，所述若所述监控区域中存在所述第二监控热点，则计算所述第二监控热点在所述监控区域中的第二方位包括：Correspondingly, if the second monitoring hot spot exists in the monitoring area, calculating the second position of the second monitoring hot spot in the monitoring area includes:
    根据所述麦克风阵列中各麦克风的位置排布，以及，所述麦克风阵列中各麦克风采集到所述目标声源发出的声音的时间差，计算所述目标声源在所述第二监控热点在所述监控区域中的第二方位。According to the position arrangement of the microphones in the microphone array and the time difference between the microphones in the microphone array to collect the sound from the target sound source, calculate the target sound source at the second monitoring hotspot at the The second position in the monitoring area.
  7. 根据权利要求4所述的基于音视频联动的监控方法，其特征在于，所述监控方法还包括：The monitoring method based on audio and video linkage according to claim 4, wherein the monitoring method further comprises:
    若所述监控区域中同时存在所述第一监控热点与所述第二监控热点，且，所述第一监控热点与所述第二监控热点对应的第一方位和第二方位不一致，则分别计算所述第一监控热点和第二监控热点的热点度得分；If the first monitoring hotspot and the second monitoring hotspot exist in the monitoring area at the same time, and the first and second positions corresponding to the first monitoring hotspot and the second monitoring hotspot are inconsistent, respectively Calculating the hotspot scores of the first monitoring hotspot and the second monitoring hotspot;
    将热点度得分较高的监控热点对应的方位确定为目标方位；Determine the orientation corresponding to the surveillance hotspot with a higher hotspot score as the target orientation;
    将所述目标方位所在的方向确定为视频监控的中心方向进行视频监控；Determine the direction in which the target azimuth is located as the central direction of video surveillance for video surveillance;
    将所述目标方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the target azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  8. 一种基于音视频联动的监控装置，其特征在于，包括：A monitoring device based on audio and video linkage, which is characterized in that it includes:
    图像获取单元，用于获取监控区域的监控图像；Image acquisition unit for acquiring surveillance images of the surveillance area;
    第一热点分析单元，用于对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；The first hot spot analysis unit is configured to perform hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area;
    第一方位计算单元，用于若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；A first position calculation unit, configured to calculate the first position of the first monitoring hot spot in the monitoring area if the first monitoring hot spot exists in the monitoring area;
    视频监控单元，用于将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；The video monitoring unit is configured to determine the direction in which the first azimuth is located as the central direction of video monitoring for video monitoring;
    音频监控单元，用于将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The audio monitoring unit is used to determine the direction in which the first azimuth is located as the central direction of audio monitoring for audio monitoring.
  9. 如权利要求8所述的基于音视频联动的监控装置，其特征在于，第一热点分析单元，包括：The monitoring device based on audio and video linkage according to claim 8, wherein the first hot spot analysis unit comprises:
    密度判断单元，用于判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；A density judgment unit, configured to judge whether there is an image area on the monitoring image with a crowd density greater than a preset density threshold;
    判断分析单元，用于若密度判断单元判定存在所述图像区域，则判定所述监控区域中存在第一监控热点；以及，若密度判断单元判定不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。The judgment analysis unit is configured to determine that the first monitoring hot spot exists in the monitoring area if the density determination unit determines that the image area exists; and, if the density determination unit determines that the image area does not exist, determine the monitoring area There is no first monitoring hotspot in.
  10. 如权利要求9所述的基于音视频联动的监控装置，其特征在于，第一方位计算单元，包括：The monitoring device based on audio and video linkage according to claim 9, wherein the first position calculation unit comprises:
    坐标获取单元，用于根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；A coordinate obtaining unit, configured to obtain the first coordinate according to the position of the central pixel of the image area on the monitoring image;
    坐标计算单元，用于根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。The coordinate calculation unit is configured to determine the first position of the first coordinate in the monitoring area according to a preset coordinate position correspondence.
  11. 如权利要求8至10任意一项所述的基于音视频联动的监控装置，其特征在于，还包括：The monitoring device based on audio and video linkage according to any one of claims 8 to 10, further comprising:
    音频获取单元，用于获取所述监控区域的监控音频；An audio acquisition unit for acquiring monitoring audio of the monitoring area;
    第二热点分析单元，用于对所述监控音频进行热点分析，以判断所述监控区域中是否存在第二监控热点；The second hot spot analysis unit is configured to perform hot spot analysis on the monitoring audio to determine whether there is a second monitoring hot spot in the monitoring area;
    第二方位计算单元，用于若所述监控区域中存在所述第二监控热点，则计算所述第二监控热点在所述监控区域中的第二方位；A second position calculation unit, configured to calculate a second position of the second monitoring hot spot in the monitoring area if the second monitoring hot spot exists in the monitoring area;
    所述视频监控单元，还用于将所述第二方位所在的方向确定为视频监控的中心方向进行视频监控；The video monitoring unit is further configured to determine the direction in which the second azimuth is located as the center direction of video monitoring for video monitoring;
    所述音频监控单元，还用于将所述第二方位所在的方向确定为音频监控的中心方向进行音频监控。The audio monitoring unit is further configured to determine the direction in which the second orientation is located as the center direction of audio monitoring for audio monitoring.
  12. 如权利要求11所述的基于音视频联动的监控装置，其特征在于，所述第二热点分析单元，包括：The monitoring device based on audio and video linkage according to claim 11, wherein the second hot spot analysis unit comprises:
    强度判断单元，用于判断所述监控音频中是否存在声音强度大于预设强度阈值的目标声源；An intensity judging unit for judging whether there is a target sound source with a sound intensity greater than a preset intensity threshold in the monitored audio;
    强度分析单元，用于若强度判断单元判定存在所述目标声源，则判定所述监控区域中存在第二监控热点；以及，若强度判断单元判定不存在所述目标声源，则判定所述监控区域中不存在第二监控热点。An intensity analysis unit, configured to determine that a second monitoring hot spot exists in the monitoring area if the intensity determination unit determines that the target sound source exists; and, if the intensity determination unit determines that the target sound source does not exist, determine the There is no second monitoring hot spot in the monitoring area.
  13. 如权利要求12所述的基于音视频联动的监控装置，其特征在于，音频获取单元，具体用于：The monitoring device based on audio and video linkage according to claim 12, wherein the audio acquisition unit is specifically configured to:
    利用麦克风阵列获取所述监控区域的监控音频；Acquiring the monitoring audio of the monitoring area by using a microphone array;
    相应的，所述第二方位计算单元具体用于，根据所述麦克风阵列中各麦克风的位置排布，以及，所述麦克风阵列中各麦克风采集到所述目标声源发出的声音的时间差，计算所述目标声源在所述第二监控热点在所述监控区域中的第二方位。Correspondingly, the second orientation calculation unit is specifically configured to calculate according to the position arrangement of the microphones in the microphone array and the time difference between each microphone in the microphone array to collect the sound emitted by the target sound source The target sound source is in a second orientation of the second monitoring hot spot in the monitoring area.
  14. 如权利要求11所述的基于音视频联动的监控装置，其特征在于，The monitoring device based on audio and video linkage according to claim 11, characterized in that:
    热点度计算单元，用于若所述监控区域中同时存在所述第一监控热点与所述第二监控热点，且，所述第一监控热点与所述第二监控热点对应的第一方位和第二方位不一致，则分别计算所述第一监控热点和第二监控热点的热点度得分；The hotspot degree calculation unit is configured to, if the first monitoring hotspot and the second monitoring hotspot exist in the monitoring area at the same time, and the first direction and the second monitoring hotspot corresponding to the first monitoring hotspot and the second monitoring hotspot If the second azimuths are inconsistent, calculate the hotspot scores of the first monitoring hot spot and the second monitoring hot spot respectively;
    目标方位确定单元，用于将热点度得分较高的监控热点对应的方位确定为目标方位；The target orientation determining unit is used to determine the orientation corresponding to the monitoring hotspot with a higher hotspot score as the target orientation;
    相应的，视频监控单元还用于，将所述目标方位所在的方向确定为视频监控的中心方向进行视频监控；音频监控单元还用于，将所述目标方位所在的方向确定为音频监控的中心方向进行音频监控。Correspondingly, the video monitoring unit is also used to determine the direction in which the target position is located as the center direction of video monitoring for video monitoring; the audio monitoring unit is also used to determine the direction in which the target position is located as the center of audio monitoring Direction for audio monitoring.
  15. 一种终端设备，其特征在于，所述终端设备包括存储器、处理器，所述存储器上存储有可在所述处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现如下步骤：A terminal device, characterized in that, the terminal device includes a memory and a processor, the memory stores computer-readable instructions that can run on the processor, and the processor executes the computer-readable instructions When implementing the following steps:
    获取监控区域的监控图像；Obtain surveillance images of the surveillance area;
    对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
    若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
    将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
    将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  16. 根据权利要求15所述的终端设备，其特征在于，所述对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点包括：The terminal device according to claim 15, wherein the performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area comprises:
    判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；Judging whether there is an image area with a crowd density greater than a preset density threshold on the monitoring image;
    若存在所述图像区域，则判定所述监控区域中存在第一监控热点；If the image area exists, it is determined that there is a first surveillance hot spot in the surveillance area;
    若不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。If the image area does not exist, it is determined that there is no first surveillance hot spot in the surveillance area.
  17. 根据权利要求16所述的终端设备，其特征在于，所述若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位，包括：The terminal device according to claim 16, wherein said if the first monitoring hot spot exists in the monitoring area, calculating the first position of the first monitoring hot spot in the monitoring area comprises :
    根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；Obtaining the first coordinate according to the position of the central pixel of the image area on the monitoring image;
    根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。Determine the first position of the first coordinate in the monitoring area according to the preset coordinate position correspondence.
  18. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可读指令，其特征在于，所述计算机可读指令被至少一个处理器执行时实现如下步骤：A computer-readable storage medium that stores computer-readable instructions, wherein the computer-readable instructions are executed by at least one processor to implement the following steps:
    获取监控区域的监控图像；Obtain surveillance images of the surveillance area;
    对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点；Performing hot spot analysis on the monitoring image to determine whether there is a first monitoring hot spot in the monitoring area;
    若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位；If the first monitoring hotspot exists in the monitoring area, calculating the first position of the first monitoring hotspot in the monitoring area;
    将所述第一方位所在的方向确定为视频监控的中心方向进行视频监控；Determining the direction in which the first azimuth is located as the central direction of video surveillance for video surveillance;
    将所述第一方位所在的方向确定为音频监控的中心方向进行音频监控。The direction in which the first azimuth is located is determined as the central direction of audio monitoring for audio monitoring.
  19. 根据权利要求18所述的计算机可读存储介质，其特征在于，所述对所述监控图像进行热点分析，以判断所述监控区域中是否存在第一监控热点包括：The computer-readable storage medium according to claim 18, wherein the performing hot spot analysis on the surveillance image to determine whether there is a first surveillance hot spot in the surveillance area comprises:
    判断所述监控图像上是否存在人群密度大于预设密度阈值的图像区域；Judging whether there is an image area with a crowd density greater than a preset density threshold on the monitoring image;
    若存在所述图像区域，则判定所述监控区域中存在第一监控热点；If the image area exists, it is determined that there is a first surveillance hot spot in the surveillance area;
    若不存在所述图像区域，则判定所述监控区域中不存在第一监控热点。If the image area does not exist, it is determined that there is no first surveillance hot spot in the surveillance area.
  20. 根据权利要求19所述的计算机可读存储介质，其特征在于，所述若所述监控区域中存在所述第一监控热点，则计算所述第一监控热点在所述监控区域中的第一方位，包括：The computer-readable storage medium according to claim 19, wherein if the first monitoring hot spot exists in the monitoring area, the first monitoring hot spot in the monitoring area is calculated. Location, including:
    根据所述图像区域的中心像素点在所述监控图像上的位置，得到第一坐标；Obtaining the first coordinate according to the position of the central pixel of the image area on the monitoring image;
    根据预设的坐标方位对应关系，确定所述第一坐标在所述监控区域中的第一方位。Determine the first position of the first coordinate in the monitoring area according to the preset coordinate position correspondence.
PCT/CN2019/118158 2019-02-12 2019-11-13 Monitoring method and apparatus based on audio and video linkage, and terminal device and medium WO2020164277A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910111104.2 2019-02-12
CN201910111104.2A CN109922311B (en) 2019-02-12 2019-02-12 Monitoring method, device, terminal and storage medium based on audio and video linkage

Publications (1)

Publication Number Publication Date
WO2020164277A1 true WO2020164277A1 (en) 2020-08-20

Family

ID=66961420

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/118158 WO2020164277A1 (en) 2019-02-12 2019-11-13 Monitoring method and apparatus based on audio and video linkage, and terminal device and medium

Country Status (2)

Country Link
CN (1) CN109922311B (en)
WO (1) WO2020164277A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109922311B (en) * 2019-02-12 2022-01-28 平安科技（深圳）有限公司 Monitoring method, device, terminal and storage medium based on audio and video linkage
CN111010546A (en) * 2019-12-20 2020-04-14 浙江大华技术股份有限公司 Method and device for adjusting monitoring preset point and storage medium
CN113676593B (en) * 2021-08-06 2022-12-06 Oppo广东移动通信有限公司 Video recording method, video recording device, electronic equipment and storage medium
CN113840087B (en) * 2021-09-09 2023-06-16 Oppo广东移动通信有限公司 Sound processing method, sound processing device, electronic equipment and computer readable storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080252722A1 (en) * 2007-04-11 2008-10-16 Yuan-Kai Wang System And Method Of Intelligent Surveillance And Analysis
CN101511000A (en) * 2009-02-27 2009-08-19 中山大学 Intelligent monitoring pick-up head device using acoustic location
CN102833476A (en) * 2012-08-17 2012-12-19 歌尔声学股份有限公司 Camera for terminal equipment and implementation method of camera for terminal equipment
CN106303211A (en) * 2015-10-27 2017-01-04 小米科技有限责任公司 Control the method for shooting, Apparatus and system
CN106341665A (en) * 2016-09-30 2017-01-18 浙江宇视科技有限公司 Tracking monitoring method and device
CN109922311A (en) * 2019-02-12 2019-06-21 平安科技（深圳）有限公司 Monitoring method, device, terminal and storage medium based on audio/video linkage

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4316541B2 (en) * 2005-06-27 2009-08-19 パナソニック株式会社 Monitoring recording apparatus and monitoring recording method
US9111147B2 (en) * 2011-11-14 2015-08-18 Massachusetts Institute Of Technology Assisted video surveillance of persons-of-interest
CN204046720U (en) * 2014-03-04 2014-12-24 深圳信息职业技术学院 A kind of safety monitoring system
WO2016013298A1 (en) * 2014-07-25 2016-01-28 日本電気株式会社 Image processing apparatus, monitor system, image processing method, and program
CN104135653A (en) * 2014-08-15 2014-11-05 魏鹏 Sound localization and video linkage platform
CN107889054A (en) * 2017-12-08 2018-04-06 沈阳首视科技有限责任公司 Crowd's station acquisition device, in real time monitoring guidance system and interaction monitoring method
CN109241933A (en) * 2018-09-21 2019-01-18 深圳市九洲电器有限公司 Video linkage monitoring method, monitoring server, video linkage monitoring system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080252722A1 (en) * 2007-04-11 2008-10-16 Yuan-Kai Wang System And Method Of Intelligent Surveillance And Analysis
CN101511000A (en) * 2009-02-27 2009-08-19 中山大学 Intelligent monitoring pick-up head device using acoustic location
CN102833476A (en) * 2012-08-17 2012-12-19 歌尔声学股份有限公司 Camera for terminal equipment and implementation method of camera for terminal equipment
CN106303211A (en) * 2015-10-27 2017-01-04 小米科技有限责任公司 Control the method for shooting, Apparatus and system
CN106341665A (en) * 2016-09-30 2017-01-18 浙江宇视科技有限公司 Tracking monitoring method and device
CN109922311A (en) * 2019-02-12 2019-06-21 平安科技（深圳）有限公司 Monitoring method, device, terminal and storage medium based on audio/video linkage

Also Published As

Publication number Publication date
CN109922311B (en) 2022-01-28
CN109922311A (en) 2019-06-21

Similar Documents

Publication Publication Date Title
WO2020164277A1 (en) Monitoring method and apparatus based on audio and video linkage, and terminal device and medium
WO2019218824A1 (en) Method for acquiring motion track and device thereof, storage medium, and terminal
JP6511149B2 (en) Method of calculating area of fingerprint overlap area, electronic device for performing the same, computer program, and recording medium
WO2020073505A1 (en) Image processing method, apparatus and device based on image recognition, and storage medium
WO2020083111A1 (en) Liveness detection method and device, electronic apparatus, storage medium and related system using the liveness detection method
WO2020094088A1 (en) Image capturing method, monitoring camera, and monitoring system
US9971933B1 (en) Facial image screening method and face recognition system thereof
WO2020184207A1 (en) Object tracking device and object tracking method
CN106529406B (en) Method and device for acquiring video abstract image
WO2020233397A1 (en) Method and apparatus for detecting target in video, and computing device and storage medium
TW202026948A (en) Methods and devices for biological testing and storage medium thereof
WO2017107345A1 (en) Image processing method and apparatus
WO2020001219A1 (en) Image processing method and apparatus, storage medium, and electronic device
CN111432115A (en) Face tracking method based on voice auxiliary positioning, terminal and storage device
CN107103299B (en) People counting method in monitoring video
TWI798815B (en) Target re-identification method, device, and computer readable storage medium
WO2021120190A1 (en) Data processing method and apparatus, electronic device, and storage medium
TW202022706A (en) People-flow analysis system and people-flow analysis method
US11775834B2 (en) Joint upper-body and face detection using multi-task cascaded convolutional networks
WO2018058573A1 (en) Object detection method, object detection apparatus and electronic device
EP4068151B1 (en) Linear processing method and apparatus in face image, terminal device, and storage medium
WO2022002214A1 (en) Video editing method and apparatus, computer readable storage medium, and camera
WO2019041447A1 (en) 3d video frame feature point extraction method and system
EP4135314A1 (en) Camera-view acoustic fence
CN113794814B (en) Method, device and storage medium for controlling video image output

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19914854

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 07.10.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19914854

Country of ref document: EP

Kind code of ref document: A1