TW202242803A - Positioning method and apparatus, electronic device and storage medium - Google Patents

Positioning method and apparatus, electronic device and storage medium Download PDF

Info

Publication number
TW202242803A
TW202242803A TW111105018A TW111105018A TW202242803A TW 202242803 A TW202242803 A TW 202242803A TW 111105018 A TW111105018 A TW 111105018A TW 111105018 A TW111105018 A TW 111105018A TW 202242803 A TW202242803 A TW 202242803A
Authority
TW
Taiwan
Prior art keywords
target
position coordinates
target object
initial position
coordinates
Prior art date
Application number
TW111105018A
Other languages
Chinese (zh)
Inventor
關英妲
劉文韜
錢晨
Original Assignee
大陸商北京市商湯科技開發有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 大陸商北京市商湯科技開發有限公司 filed Critical 大陸商北京市商湯科技開發有限公司
Publication of TW202242803A publication Critical patent/TW202242803A/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Studio Devices (AREA)

Abstract

In this disclosure, a positioning method and apparatus, an electronic device, and a storage medium are provided. The positioning method includes: acquiring video images captured at a same time by a plurality of acquisition devices arranged in a target site; where, different acquisition devices have different acquisition perspectives in the target site, and the video images involves a target object; determining initial position coordinates of the target object in the target site based on the video images captured by multiple acquisition devices at the same time; and obtaining, by fusing initial position coordinates of a same target object, target position coordinates of the target object in the target site.

Description

定位方法、裝置、電子設備及儲存媒體Positioning method, device, electronic device and storage medium

本公開涉及計算機視覺技術領域,具體而言,涉及一種定位方法、裝置、電子設備及儲存媒體。The present disclosure relates to the technical field of computer vision, in particular, to a positioning method, device, electronic equipment and storage media.

人工智能技術在打造智能教育,文教娛樂及生活上發揮著越來越重要的作用,其中計算機視覺作為關鍵的技術之一,應用廣泛。例如基於計算機視覺的定位技術,可以對不同場景下的目標場所內的目標對象進行定位,確定目標場所內目標對象的位置。Artificial intelligence technology is playing an increasingly important role in creating intelligent education, culture, education, entertainment and life. Among them, computer vision, as one of the key technologies, is widely used. For example, a positioning technology based on computer vision can locate a target object in a target place in different scenarios, and determine the position of the target object in the target place.

在基於計算機視覺進行定位的過程中,可以通過相機採集的目標場所圖像,確定目標場所圖像中的目標對象的位置,進一步確定目標對象在目標場所內的位置,完成對目標場所內的目標對象的追蹤。In the process of positioning based on computer vision, the target location image collected by the camera can be used to determine the position of the target object in the target location image, further determine the location of the target object in the target location, and complete the target location in the target location. object tracking.

本公開實施例至少提供一種定位方案。The embodiments of the present disclosure provide at least one positioning solution.

第一方面,本公開實施例提供了一種定位方法,包括:In a first aspect, an embodiment of the present disclosure provides a positioning method, including:

獲取目標場所內設置的多個採集設備在同一時刻採集的多張視頻畫面;其中,不同的採集設備在所述目標場所中的採集視角不同,所述多張視頻畫面中包括目標對象;其中,所述目標對象為所述目標場所待進行定位的對象;Acquiring multiple video images collected at the same time by a plurality of acquisition devices set in the target place; wherein different acquisition devices have different acquisition angles of view in the target place, and the plurality of video images include the target object; wherein, The target object is an object to be positioned in the target place;

基於所述多張視頻畫面,分別確定所述目標對象在所述目標場所中的初始位置坐標;Based on the plurality of video images, respectively determine the initial position coordinates of the target object in the target place;

對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標。The initial position coordinates of the same target object among the target objects are fused to obtain the target position coordinates of the target object in the target place.

本公開實施例中,可以通過目標場所內設置的多個採集視角不同的採集設備在同一時刻採集的視頻畫面,確定不同視頻畫面中的目標對象的初始位置坐標,進一步對不同視頻畫面中的同一目標對象的初始位置坐標進行融合,確定目標對象在目標場所中的目標位置坐標。通過該方式一方面可以完成對空間較大和/或場地複雜的目標場所中的目標對象的全面定位,另一方面可以得到同一目標對象準確度較高的目標位置坐標。In the embodiment of the present disclosure, the initial position coordinates of the target object in different video pictures can be determined through the video pictures collected at the same time by multiple collection devices with different collection angles of view set in the target place, and further the same image in different video pictures can be determined. The initial position coordinates of the target object are fused to determine the target position coordinates of the target object in the target place. In this way, on the one hand, the overall positioning of the target object in a target place with a large space and/or a complex site can be completed, and on the other hand, the target position coordinates of the same target object can be obtained with high accuracy.

在一種可能的實施方式中,基於所述多張視頻畫面,分別確定所述目標對象在所述目標場所中的初始位置坐標,包括:In a possible implementation manner, determining the initial position coordinates of the target object in the target place based on the plurality of video images respectively includes:

獲取所述多張視頻畫面中所述目標對象的像素坐標;Obtaining the pixel coordinates of the target object in the plurality of video frames;

針對所述多個採集設備裡的每一個,基於該採集設備採集的視頻畫面中所述目標對象中至少一者的像素坐標和該採集設備的參數資訊,確定該採集設備採集的所述目標對象中至少一者在所述目標場所對應的世界坐標系下的初始位置坐標。For each of the plurality of collection devices, based on the pixel coordinates of at least one of the target objects in the video frame collected by the collection device and the parameter information of the collection device, determine the target object collected by the collection device The initial position coordinates of at least one of them in the world coordinate system corresponding to the target location.

本公開實施例中,可以先確定目標對象在視頻畫面中的像素坐標,然後再根據採集設備的參數資訊,得到目標對象在目標場所中的初始位置坐標,為後續確定目標對象在目標場所中的目標位置坐標提供準備。In the embodiment of the present disclosure, the pixel coordinates of the target object in the video screen can be determined first, and then the initial position coordinates of the target object in the target place can be obtained according to the parameter information of the acquisition device, so as to determine the position of the target object in the target place subsequently. Target location coordinates are provided for preparation.

在一種可能的實施方式中,獲取多張視頻畫面中的所述目標對象的像素坐標,包括:In a possible implementation manner, obtaining the pixel coordinates of the target object in multiple video frames includes:

將所述多張視頻畫面輸入預先訓練的神經網路,針對所述多張視頻畫面中的每一張,得到該視頻畫面中的目標對象的檢測框;其中,所述神經網路包含多個用於檢測不同尺寸的目標對象的目標檢測子網路;The multiple video frames are input into a pre-trained neural network, and for each of the multiple video frames, the detection frame of the target object in the video frame is obtained; wherein the neural network includes multiple Object detection subnetworks for detecting objects of different sizes;

提取該視頻畫面中目標對象的檢測框上的目標位置點在該視頻畫面中的像素坐標,得到該視頻畫面中目標對象的像素坐標。Extracting the pixel coordinates of the target position point on the detection frame of the target object in the video frame in the video frame to obtain the pixel coordinates of the target object in the video frame.

本公開實施例中,神經網路中包含多個用於檢測不同尺寸的目標對象的目標檢測子網路,這樣在通過神經網路對視頻畫面中目標對象進行目標檢測時,可以準確地檢測出同一視頻畫面中不同尺寸的目標對象。In the embodiment of the present disclosure, the neural network includes multiple target detection sub-networks for detecting target objects of different sizes, so that when the target object in the video screen is detected through the neural network, it can be accurately detected Objects of different sizes in the same video frame.

在一種可能的實施方式中,基於該採集設備採集的所述視頻畫面中的所述目標對象中至少一者的像素坐標和該採集設備的參數資訊,確定該採集設備採集的所述目標對象中至少一者在所述目標場所對應的世界坐標系下的初始位置坐標,包括:In a possible implementation manner, based on the pixel coordinates of at least one of the target objects in the video frame collected by the collection device and the parameter information of the collection device, it is determined that one of the target objects collected by the collection device is The initial position coordinates of at least one of them in the world coordinate system corresponding to the target site, including:

基於預先確定的該採集設備的內參矩陣和畸變參數,對該採集設備採集的視頻畫面中的所述目標對象中至少一者的像素坐標進行修正,得到該視頻畫面中的所述目標對象中至少一者的修正像素坐標;Based on the predetermined internal reference matrix and distortion parameters of the acquisition device, the pixel coordinates of at least one of the target objects in the video picture collected by the acquisition device are corrected to obtain at least one of the target objects in the video picture. the corrected pixel coordinates of one;

基於預先確定的該採集設備的單應性矩陣和該採集設備採集的視頻畫面中的所述目標對象中至少一者的修正像素坐標,確定該視頻畫面中的所述目標對象中至少一者的初始位置坐標。Based on the predetermined homography matrix of the acquisition device and the corrected pixel coordinates of at least one of the target objects in the video picture captured by the acquisition device, determine the at least one of the target objects in the video picture Initial position coordinates.

本公開實施例中,在得到視頻畫面中目標對象的像素坐標後,先基於採集該視頻畫面的採集設備的內參矩陣和畸變係數對像素坐標進行修正,從而可以得到準確度較高的修正像素坐標,進一步得到目標對象在目標場所中準確度較高的初始位置坐標。In the embodiment of the present disclosure, after obtaining the pixel coordinates of the target object in the video picture, the pixel coordinates are first corrected based on the internal reference matrix and the distortion coefficient of the acquisition device that captures the video picture, so that the corrected pixel coordinates with high accuracy can be obtained , and further obtain the initial position coordinates of the target object in the target place with high accuracy.

在一種可能的實施方式中,對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標,包括:In a possible implementation manner, the initial position coordinates of the same target object among the target objects are fused to obtain the target position coordinates of the target object in the target place, including:

基於所述目標對象在所述目標場所中的所述初始位置坐標,確定與所述目標對象中同一目標對象關聯的多個初始位置坐標;determining a plurality of initial position coordinates associated with the same one of the target objects based on the initial position coordinates of the target objects in the target location;

將與該目標對象關聯的所述多個所述初始位置坐標進行依次融合,得到該目標對象在所述目標場所中的目標位置坐標。The multiple initial position coordinates associated with the target object are sequentially fused to obtain the target position coordinates of the target object in the target location.

本公開實施例中,考慮到基於不同採集設備採集視頻畫面確定的同一目標對象的初始位置坐標會存在一些誤差,因此可以通過對多個採集設備採集的同一目標對象的初始位置坐標進行融合,從而可以得到該同一目標對象準確度較高的目標位置坐標。In the embodiment of the present disclosure, considering that there may be some errors in the initial position coordinates of the same target object determined based on video images collected by different acquisition devices, it is possible to fuse the initial position coordinates of the same target object collected by multiple acquisition devices, so that The target position coordinates of the same target object with high accuracy can be obtained.

在一種可能的實施方式中,將與該目標對象關聯的所述多個初始位置坐標進行依次融合,得到該目標對象在所述目標場所中的目標位置坐標,包括:In a possible implementation manner, the multiple initial position coordinates associated with the target object are sequentially fused to obtain the target position coordinates of the target object in the target place, including:

從該目標對象關聯的所述多個初始位置坐標中選取任一初始位置坐標,將選取的任一初始位置坐標作為第一中間融合位置坐標;Select any initial position coordinate from the plurality of initial position coordinates associated with the target object, and use the selected initial position coordinate as the first intermediate fusion position coordinate;

將所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標,將所述第二中間融合位置坐標作為更新後的所述第一中間融合位置坐標,並返回生成所述第二中間融合位置坐標的步驟,直到所述多個初始位置坐標中不存在待融合的初始位置坐標。Fusing the first intermediate fusion position coordinates with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates, and using the second intermediate fusion position coordinates as the updated The coordinates of the first intermediate fusion position, and return to the step of generating the second intermediate fusion position coordinates until there is no initial position coordinate to be fused in the plurality of initial position coordinates.

在一種可能的實施方式中,將所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標,包括:In a possible implementation manner, the first intermediate fusion position coordinates are fused with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates, including:

確定所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標的中點坐標,將該中點坐標作為所述第二中間融合位置坐標。Determine the midpoint coordinates of the first intermediate fusion position coordinates and any other initial position coordinates to be fused among the plurality of initial position coordinates, and use the midpoint coordinates as the second intermediate fusion position coordinates.

本公開實施例中,針對與同一目標對象關聯的多個初始位置坐標,可以按照依次取中點的方式融合,從而得到準確度較高的目標位置坐標。In the embodiment of the present disclosure, multiple initial position coordinates associated with the same target object may be fused in a manner of sequentially taking midpoints, so as to obtain target position coordinates with high accuracy.

在一種可能的實施方式中,基於所述目標對象在所述目標場所中的所述初始位置坐標,確定與所述目標對象中同一目標對象關聯的多個初始位置坐標,包括:In a possible implementation manner, based on the initial position coordinates of the target object in the target location, determining a plurality of initial position coordinates associated with the same target object in the target object includes:

針對所述多張視頻畫面中的任意兩張視頻畫面,將所述任意兩張視頻畫面中第一視頻畫面中的目標對象確定為第一目標對象,所述任意兩張視頻畫面中第二視頻畫面中的目標對象確定為第二目標對對象;在所述第一視頻畫面中確定每個所述第一目標對象的初始位置坐標;在所述第二視頻畫面中確定每個所述第二目標對象的初始位置坐標;針對每個所述第一目標對象的初始位置坐標,確定該第一目標對象的初始位置坐標與每個所述第二目標對象的初始位置坐標之間的距離;For any two video frames in the plurality of video frames, the target object in the first video frame in the any two video frames is determined as the first target object, and the second video frame in the any two video frames is determined as the first target object. The target object in the picture is determined as the second target object; the initial position coordinates of each of the first target objects are determined in the first video picture; and the initial position coordinates of each of the first target objects are determined in the second video picture. The initial position coordinates of the target object; for the initial position coordinates of each of the first target objects, determine the distance between the initial position coordinates of the first target object and the initial position coordinates of each of the second target objects;

確定與該第一目標對象具有最小距離的第二目標對象與該第一目標對象為同一目標對象,其中,所述最小距離小於預設融合距離閾值;將該第一目標對象的初始位置坐標和與該第一目標對象具有最小距離的第二目標對象的初始位置坐標,作為與所述目標對象中同一目標對象關聯的多個初始位置坐標。Determining that a second target object having a minimum distance from the first target object is the same target object as the first target object, wherein the minimum distance is smaller than a preset fusion distance threshold; combining the initial position coordinates of the first target object with The initial position coordinates of the second target object having the smallest distance from the first target object are used as a plurality of initial position coordinates associated with the same target object among the target objects.

本公開實施例中,根據任意兩張視頻畫面中不同目標對象的初始位置坐標,和預設融合距離閾值,可以快速確定出與同一目標對象關聯的初始位置坐標,從而為後續確定各目標對象的目標位置坐標提供依據。In the embodiment of the present disclosure, according to the initial position coordinates of different target objects in any two video frames and the preset fusion distance threshold, the initial position coordinates associated with the same target object can be quickly determined, so as to determine the location of each target object in the future. The target position coordinates provide the basis.

在一種可能的實施方式中,在得到該目標對象在所述目標場所中的目標位置坐標之後,所述定位方法還包括:In a possible implementation manner, after obtaining the target position coordinates of the target object in the target place, the positioning method further includes:

基於所述目標場所中的各目標對象分別對應的目標位置坐標,以及預先設定的目標區域,確定是否存在進入所述目標區域的目標對象;determining whether there is a target object entering the target area based on the target position coordinates corresponding to each target object in the target location and the preset target area;

在確定存在進入所述目標區域的目標對象的情況下,進行預警提示。When it is determined that there is a target object entering the target area, an early warning prompt is given.

本公開實施例中,在得到目標場所中的各目標對象準確度較高的目標位置坐標後,可以基於預先設定的目標區域,例如預先設定的危險區域,判斷目標場所中的目標對象是否進入目標區域,以便及時預警提示,提高目標場所的安全性。In the embodiment of the present disclosure, after obtaining the target position coordinates of each target object in the target place with high accuracy, it can be judged whether the target object in the target place enters the target area based on the preset target area, such as the preset dangerous area. area, so as to timely warn and prompt and improve the safety of the target site.

第二方面,本公開實施例提供了一種定位裝置,包括:In a second aspect, an embodiment of the present disclosure provides a positioning device, including:

獲取模組,用於獲取目標場所內設置的多個採集設備在同一時刻採集的多張視頻畫面;其中,不同的採集設備在所述目標場所中的採集視角不同,所述多張視頻畫面中包括目標對象;其中,所述目標對象為所述目標場所內待進行定位的對象;The acquisition module is used to acquire multiple video images collected at the same time by a plurality of acquisition devices set in the target place; where different acquisition devices have different acquisition angles of view in the target place, and among the plurality of video images Including a target object; wherein, the target object is an object to be positioned in the target place;

確定模組,用於基於所述多張視頻畫面,分別確定所述目標對象在目標場所中的初始位置坐標;A determining module, configured to respectively determine the initial position coordinates of the target object in the target place based on the multiple video images;

融合模組,用於對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標。The fusion module is used to fuse the initial position coordinates of the same target object among the target objects to obtain the target position coordinates of the target object in the target place.

第三方面,本公開實施例提供了一種電子設備,包括:處理器、儲存器和匯流排,所述儲存器儲存有所述處理器可執行的機器可讀指令,當電子設備運行時,所述處理器與所述儲存器之間通過匯流排通訊,所述機器可讀指令被所述處理器執行時執行如第一方面所述的定位方法的步驟。In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a bus bar. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the The processor communicates with the storage through a bus, and when the machine-readable instructions are executed by the processor, the steps of the positioning method described in the first aspect are executed.

第四方面,本公開實施例提供了一種計算機可讀儲存媒體,該計算機可讀儲存媒體上儲存有計算機程式,該計算機程式被處理器運行時執行如第一方面所述的定位方法的步驟。In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the positioning method described in the first aspect are executed.

第五方面,本公開實施例提供了一種計算機程式產品,該計算機程式產品包括計算機程式並儲存於儲存媒體上,該計算機程式被處理器執行時執行如第一方面所述的定位方法的步驟。In a fifth aspect, an embodiment of the present disclosure provides a computer program product. The computer program product includes a computer program and is stored on a storage medium. When the computer program is executed by a processor, the steps of the positioning method described in the first aspect are executed.

為使本公開的上述目的、特徵和優點能更明顯易懂,下文特舉較佳實施例,並配合所附附圖,作詳細說明如下。In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

為了更清楚地說明本公開實施例的技術方案,下面將對實施例中所需要使用的附圖作簡單地介紹,此處的附圖被併入說明書中並構成本說明書中的一部分,這些附圖示出了符合本公開的實施例,並與說明書一起用於說明本公開的技術方案。應當理解,以下附圖僅示出了本公開的某些實施例,因此不應被看作是對範圍的限定,對於本領域普通技術人員來講,在不付出創造性勞動的前提下,還可以根據這些附圖獲得其他相關的附圖。In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

為使本公開實施例的目的、技術方案和優點更加清楚,下面將結合本公開實施例中附圖,對本公開實施例中的技術方案進行清楚、完整地描述,顯然,所描述的實施例僅僅是本公開一部分實施例,而不是全部的實施例。通常在此處附圖中描述和繪示的本公開實施例的組件可以以各種不同的配置來佈置和設計。因此,以下對在附圖中提供的本公開的實施例的詳細描述並非旨在限制要求保護的本公開的範圍,而是僅僅表示本公開的選定實施例。基於本公開的實施例,本領域技術人員在沒有做出創造性勞動的前提下所獲得的所有其他實施例,都屬本公開保護的範圍。In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present disclosure.

應注意到:相似的標號和字母在下面的附圖中表示類似項,因此,一旦某一項在一個附圖中被定義,則在隨後的附圖中不需要對其進行進一步定義和解釋。It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

本文中術語“和/或”,僅僅是描述一種關聯關係,表示可以存在三種關係,例如,A和/或B,可以表示:單獨存在A,同時存在A和B,單獨存在B這三種情況。另外,本文中術語“至少一種”表示多種中的任意一種或多種中的至少兩種的任意組合,例如,包括A、B、C中的至少一種,可以表示包括從A、B和C構成的集合中選擇的任意一個或多個元素。The term "and/or" in this article only describes an association relationship, which means that there may be three relationships, for example, A and/or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including the composition consisting of A, B, and C Any one or more elements selected in the collection.

在很多應用場景中,通常需要對一個場所內的目標對象進行定位。例如在工廠內,需要檢測員工是否在指定位置工作,或者是否進入危險區域。在商場中,可以通過對顧客的定位,檢測商場中的人流分佈。在對場所內的目標對象進行定位的過程中,可以通過多個相機採集的圖像,確定目標對象的位置。然而針對一些場地複雜面積較大的目標場所,在基於多個相機對目標對象進行定位的過程中,可能無法捕捉到全部的目標對象,存在目標對象定位不全面的問題;還可能存在一些遮擋區域,針對這些遮擋區域的目標對象則無法完成定位。In many application scenarios, it is usually necessary to locate a target object in a place. For example, in a factory, it is necessary to detect whether an employee is working at a designated location, or whether he has entered a dangerous area. In shopping malls, the distribution of people in the shopping mall can be detected by locating customers. In the process of locating the target object in the place, the position of the target object can be determined through images collected by multiple cameras. However, for some target sites with complex and large areas, in the process of locating target objects based on multiple cameras, it may not be possible to capture all the target objects, and there is a problem of incomplete target object positioning; there may also be some occluded areas , the target objects in these occluded areas cannot be located.

基於上述研究,本公開提供了一種定位方法,可以通過目標場所內設置的多個採集視角不同的採集設備在同一時刻採集的視頻畫面,確定不同視頻畫面中的目標對象的初始位置坐標,進一步對不同視頻畫面中的同一目標對象的初始位置坐標進行融合,確定目標對象在目標場所中的目標位置坐標。通過該方式一方面可以完成對空間較大和/或場地複雜的目標場所中的目標對象的全面定位,另一方面可以得到同一目標對象準確度較高的目標位置坐標。Based on the above research, the present disclosure provides a positioning method, which can determine the initial position coordinates of the target object in different video pictures through the video pictures collected at the same time by multiple collection devices with different collection angles of view set in the target place. The initial position coordinates of the same target object in different video frames are fused to determine the target position coordinates of the target object in the target place. In this way, on the one hand, the overall positioning of the target object in a target place with a large space and/or a complex site can be completed, and on the other hand, the target position coordinates of the same target object can be obtained with high accuracy.

為便於對本實施例進行理解,首先對本公開實施例所公開的一種定位方法進行詳細介紹,本公開實施例所提供的定位方法的執行主體為具有計算能力的計算機設備,該計算機設備例如包括:伺服器或其它處理設備。在一些可能的實現方式中,該定位方法可以通過處理器調用儲存器中儲存的計算機可讀指令的方式來實現。In order to facilitate the understanding of this embodiment, a positioning method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the positioning method provided in the embodiment of the present disclosure is a computer device with computing capabilities, and the computer device includes, for example: a servo processor or other processing device. In some possible implementation manners, the positioning method may be implemented by a processor invoking computer-readable instructions stored in a memory.

參見圖1所示,為本公開實施例提供的定位方法的流程圖,該定位方法包括以下S101~S103:Referring to Figure 1, which is a flow chart of a positioning method provided by an embodiment of the present disclosure, the positioning method includes the following S101~S103:

S101,獲取目標場所內設置的多個採集設備在同一時刻採集的視頻畫面;其中,不同的採集設備在目標場所中的採集視角不同,視頻畫面中包括目標對象。S101. Obtain video images collected at the same time by multiple acquisition devices set in the target place; where different acquisition devices have different collection perspectives in the target place, and the video images include target objects.

示例性地,針對不同的應用場景,目標場所可以為與該應用場景對應的場所,例如需要對工廠內的員工進行定位的情況下,目標場所可以為工廠;需要對商場內的顧客進行定位的情況下,目標場所可以為商場;需要對體育館內的運動員進行定位的情況下,目標場所可以為體育館。Exemplarily, for different application scenarios, the target location may be a location corresponding to the application scenario. For example, if employees in a factory need to be located, the target location may be a factory; In this case, the target place may be a shopping mall; in the case where athletes in the stadium need to be positioned, the target place may be a gymnasium.

示例性地,目標對象為目標場所內待進行定位的對象,例如上述提到的員工、顧客和運動員。Exemplarily, the target object is an object to be located in the target place, such as the aforementioned employees, customers, and athletes.

示例性地,採集設備可以為單鏡頭攝像機或者雙鏡頭攝像機,目標場所內可以設置多個採集設備。針對不同的目標場所,可以根據目標場所的實際場地確定多個採集設備的安裝位置。例如可以使得採集設備在目標場所中的採集視角不同,以覆蓋目標場所的全部區域,不留死角。另外,考慮到採集設備過多會導致同一時刻採集的視頻畫面過多,因此會影響視頻畫面的處理速度。故在目標場所中安裝採集設備時,需要同時考慮到採集設備的安裝角度、以及數量。例如可以使得進入目標場所中的每個目標對象可以同時被兩個採集設備採集到,這樣目標場所內設置的多個採集設備可以完整的採集到目標場所整個區域的視頻畫面。Exemplarily, the collection device may be a single-lens camera or a dual-lens camera, and multiple collection devices may be set in the target place. For different target sites, the installation locations of multiple acquisition devices can be determined according to the actual site of the target site. For example, the acquisition angles of the acquisition equipment in the target place may be different, so as to cover the entire area of the target place without leaving any dead spots. In addition, considering that too many capture devices will lead to too many video frames collected at the same time, it will affect the processing speed of the video frames. Therefore, when installing the collection equipment in the target place, it is necessary to consider the installation angle and quantity of the collection equipment at the same time. For example, each target object entering the target place can be captured by two capture devices at the same time, so that multiple capture devices set in the target place can completely capture video images of the entire area of the target place.

S102,基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標。S102. Determine initial position coordinates of the target object in the target place based on video images collected by multiple collection devices at the same time.

示例性地,在獲取到多個採集設備在同一時刻採集的視頻畫面後,可以進一步對多個採集設備在同一時刻採集的視頻畫面進行目標檢測,確定不同視頻畫面中的目標對象在目標場所對應的世界坐標系下的初始位置坐標。具體地,可以基於檢測出的視頻畫面中的目標對象在視頻畫面中的像素坐標,以及採集該視頻畫面的採集設備的參數資訊,確定該視頻畫面中的目標對象的初始位置坐標。Exemplarily, after acquiring the video images collected by multiple acquisition devices at the same time, target detection can be further performed on the video images collected by multiple acquisition devices at the same time, to determine that the target objects in different video images correspond to The initial position coordinates in the world coordinate system of . Specifically, the initial position coordinates of the target object in the video frame can be determined based on the detected pixel coordinates of the target object in the video frame in the video frame and parameter information of the acquisition device that captures the video frame.

示例性地,目標場所對應的世界坐標系可以預先確定。例如以目標場所地面的中心點位置為世界坐標系的原點,以經過原點垂直於地面的方向為Z軸方向,以目標場所的地面上經過原點的一個方向為X軸方向,以目標場所的地面上經過原點且垂直於X軸的方向為Y軸方向。Exemplarily, the world coordinate system corresponding to the target location may be predetermined. For example, take the center point of the ground of the target site as the origin of the world coordinate system, take the direction passing through the origin perpendicular to the ground as the Z-axis direction, take a direction passing the origin on the ground of the target site as the X-axis direction, and take the direction of the target site as the X-axis direction. The direction passing through the origin and perpendicular to the X-axis on the ground of the site is the Y-axis direction.

S103,對目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在目標場所中的目標位置坐標。S103. Fusing the initial position coordinates of the same target object in the target objects to obtain the target position coordinates of the target object in the target place.

示例性地,考慮到不同採集設備的參數資訊之間存在一些誤差,因此基於不同採集設備採集到的視頻畫面確定的同一目標對象的初始位置坐標會有一些差距。可以對同一目標對象的初始位置坐標進行融合,得到同一目標對象在目標場所對應的世界坐標系下的目標位置坐標。For example, considering that there are some errors between the parameter information of different collection devices, there may be some gaps in the initial position coordinates of the same target object determined based on the video images collected by different collection devices. The initial position coordinates of the same target object can be fused to obtain the target position coordinates of the same target object in the world coordinate system corresponding to the target location.

本公開實施例中,可以通過目標場所內設置的多個採集視角不同的採集設備在同一時刻採集的視頻畫面,確定不同視頻畫面中的目標對象的初始位置坐標,進一步對不同視頻畫面中的同一目標對象的初始位置坐標進行融合,確定目標對象在目標場所中的目標位置坐標。通過該方式一方面可以完成對空間較大和/或場地複雜的目標場所中的目標對象的全面定位,另一方面可以得到同一目標對象準確度較高的目標位置坐標。In the embodiment of the present disclosure, the initial position coordinates of the target object in different video pictures can be determined through the video pictures collected at the same time by multiple collection devices with different collection angles of view set in the target place, and further the same image in different video pictures can be determined. The initial position coordinates of the target object are fused to determine the target position coordinates of the target object in the target place. In this way, on the one hand, the overall positioning of the target object in a target place with a large space and/or a complex site can be completed, and on the other hand, the target position coordinates of the same target object can be obtained with high accuracy.

針對上述S102,在基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標時,如圖2所示,可以包括以下S201~S202:For the above S102, when the initial position coordinates of the target object in the target place are respectively determined based on the video images collected by multiple collection devices at the same time, as shown in Figure 2, the following S201~S202 may be included:

S201,獲取多個採集設備在同一時刻分別採集的視頻畫面中的目標對象的像素坐標。S201. Acquire pixel coordinates of a target object in video images respectively collected by multiple collection devices at the same time.

示例性地,可以基於預先訓練的用於進行目標檢測的神經網路來識別視頻畫面中的目標對象,進一步可以讀取目標對象中設定位置點在視頻畫面對應的圖像坐標系中的像素坐標,將該設定位置點對應的像素坐標作為目標對象的像素坐標。Exemplarily, the target object in the video picture can be identified based on the pre-trained neural network for target detection, and the pixel coordinates of the set position point in the target object in the image coordinate system corresponding to the video picture can be read , and use the pixel coordinates corresponding to the set location point as the pixel coordinates of the target object.

具體地,在獲取多個採集設備在同一時刻分別採集的視頻畫面中包含的目標對象的像素坐標時,可以包括以下S2011~S2012:Specifically, when obtaining the pixel coordinates of the target object included in the video images collected by multiple collection devices at the same time, the following S2011~S2012 may be included:

S2011,將多張視頻畫面輸入預先訓練的神經網路,得到每張視頻畫面中的目標對象的檢測框;其中,神經網路包含多個用於檢測不同尺寸的目標對象的目標檢測子網路;S2011, input multiple video frames into the pre-trained neural network to obtain the detection frame of the target object in each video frame; wherein, the neural network includes multiple target detection sub-networks for detecting target objects of different sizes ;

S2012,提取每張視頻畫面中的目標對象的檢測框上的目標位置點在該視頻畫面中的像素坐標,得到該視頻畫面中的目標對象的像素坐標。S2012. Extract the pixel coordinates of the target position point on the detection frame of the target object in each video frame in the video frame to obtain the pixel coordinates of the target object in the video frame.

示例性地,神經網路可以檢測出視頻畫面中包含的每個目標對象,並標記出每個目標對象的檢測框。如圖3所示,為視頻畫面中的包含的目標對象的檢測框的示意圖。該視頻畫面中包含兩個目標對象對應的檢測框,分別包括目標對象1的檢測框A1B1C1D1和目標對象2的檢測框A2B2C2D2。可以在每個目標對象的檢測框上提取一個位置點作為目標位置點,例如提取檢測框底邊的中點作為目標位置點。如圖3中通過檢測框A1B1C1D1底邊D1C1的中點K1的像素坐標表示目標對象1的像素坐標,通過檢測框A2B2C2D2底邊D2C2的中點位置點K2的像素坐標表示目標對象2的像素坐標。Exemplarily, the neural network can detect each target object included in the video frame, and mark the detection frame of each target object. As shown in FIG. 3 , it is a schematic diagram of a detection frame of a target object contained in a video frame. The video frame contains two detection frames corresponding to the target object, including the detection frame A1B1C1D1 of the target object 1 and the detection frame A2B2C2D2 of the target object 2 respectively. A position point may be extracted on the detection frame of each target object as the target position point, for example, the midpoint of the bottom edge of the detection frame may be extracted as the target position point. As shown in Figure 3, the pixel coordinates of the target object 1 are represented by the pixel coordinates of the midpoint K1 of the bottom edge D1C1 of the detection frame A1B1C1D1, and the pixel coordinates of the target object 2 are represented by the pixel coordinates of the midpoint position point K2 of the bottom edge D2C2 of the detection frame A2B2C2D2.

示例性地,考慮到目標對象在目標場所中的位置是變化的,且目標場所中設置的多個採集設備在目標場所中的採集視角不同,因此,不同採集設備在同一時刻採集的視頻畫面中包含的目標對象的尺寸可能不同。為了能夠準確地標記出不同尺寸的目標對象的檢測框,本公開實施例使用的神經網路可以包含多個用於檢測不同尺寸的目標對象的目標檢測子網路。例如可以是特徵金字塔網路。該特徵金字塔網路中的每個目標檢測子網路用於檢測識別出視頻畫面中與該目標檢測子網路對應尺寸的目標對象,通過該神經網路,可以準確地檢測出同一視頻畫面中不同尺寸的目標對象。Exemplarily, considering that the position of the target object in the target place changes, and the collection angles of multiple collection devices installed in the target place are different in the target place, therefore, different collection devices in the video frames collected at the same time Included objects may vary in size. In order to accurately mark the detection frames of target objects of different sizes, the neural network used in the embodiments of the present disclosure may include multiple target detection sub-networks for detecting target objects of different sizes. For example, it can be a feature pyramid network. Each target detection sub-network in the feature pyramid network is used to detect and identify the target object in the video frame corresponding to the size of the target detection sub-network, through the neural network, it can accurately detect the target object in the same video frame Objects of different sizes.

本公開實施例中,神經網路中包含多個用於檢測不同尺寸的目標對象的目標檢測子網路。這樣在通過神經網路對視頻畫面中的目標對象進行目標檢測時,可以準確地檢測出同一視頻畫面中不同尺寸的目標對象。In the embodiment of the present disclosure, the neural network includes multiple target detection sub-networks for detecting target objects of different sizes. In this way, when the target object in the video frame is detected by the neural network, target objects of different sizes in the same video frame can be accurately detected.

S202,基於每個採集設備採集的視頻畫面中的目標對象的像素坐標和該採集設備的參數資訊,確定該採集設備採集的目標對象在目標場所對應的世界坐標系下的初始位置坐標。S202. Based on the pixel coordinates of the target object in the video frame captured by each capture device and the parameter information of the capture device, determine the initial position coordinates of the target object captured by the capture device in the world coordinate system corresponding to the target location.

示例性地,每個採集設備的參數資訊可以包含採集設備的單應性矩陣,其中,單應性矩陣可以表示採集設備採集的視頻畫面對應的圖像坐標系和採集設備所處的目標場所對應的世界坐標系之間的轉換關係。這樣,在得到目標對象在視頻畫面對應的圖像坐標系中的像素坐標後,可以根據採集設備的參數資訊,確定目標對象在目標場所對應的世界坐標系中的初始位置坐標。Exemplarily, the parameter information of each collection device may include a homography matrix of the collection device, wherein the homography matrix may indicate that the image coordinate system corresponding to the video picture collected by the collection device corresponds to the target location where the collection device is located. The conversion relationship between the world coordinate systems. In this way, after obtaining the pixel coordinates of the target object in the image coordinate system corresponding to the video frame, the initial position coordinates of the target object in the world coordinate system corresponding to the target location can be determined according to the parameter information of the acquisition device.

示例性地,目標場所對應的世界坐標系可以以目標場所中一固定位置為坐標原點,建立唯一世界坐標系。例如可以以目標場所地面中心點為坐標系原點,在地面上設定一個方向作為世界坐標系X軸的正方向,在地面上設定與X軸垂直的方向作為世界坐標系Y軸的正方向,將垂直與地面向上的方向作為世界坐標系Z軸的正方向。Exemplarily, the world coordinate system corresponding to the target location may use a fixed position in the target location as the coordinate origin to establish a unique world coordinate system. For example, you can take the center point of the ground of the target site as the origin of the coordinate system, set a direction on the ground as the positive direction of the X-axis of the world coordinate system, and set a direction perpendicular to the X-axis on the ground as the positive direction of the Y-axis of the world coordinate system. Take the vertical and ground-up direction as the positive direction of the Z-axis of the world coordinate system.

本公開實施例中,可以先確定目標對象在視頻畫面中的像素坐標,然後再根據採集設備的參數資訊,得到目標對象在目標場所中的初始位置坐標,為後續確定目標對象在目標場所中的目標位置坐標提供準備。In the embodiment of the present disclosure, the pixel coordinates of the target object in the video screen can be determined first, and then the initial position coordinates of the target object in the target place can be obtained according to the parameter information of the acquisition device, so as to determine the position of the target object in the target place subsequently. Target location coordinates are provided for preparation.

在一種實施方式中,針對上述S202,在基於每個採集設備採集的視頻畫面中的目標對象的像素坐標和該採集設備的參數資訊,確定該採集設備採集的目標對象在目標場所對應的世界坐標系下的初始位置坐標時,包括以下S2021~S2022:In one embodiment, for the above S202, based on the pixel coordinates of the target object in the video images collected by each collection device and the parameter information of the collection device, determine the world coordinates of the target object collected by the collection device corresponding to the target location When the initial position coordinates under the system, include the following S2021~S2022:

S2021,基於預先確定的每個採集設備的內參矩陣和畸變參數,對該採集設備採集的視頻畫面中的目標對象的像素坐標進行修正,得到該視頻畫面中的目標對象的修正像素坐標。S2021. Based on the predetermined internal reference matrix and distortion parameters of each acquisition device, correct the pixel coordinates of the target object in the video frame captured by the capture device to obtain the corrected pixel coordinates of the target object in the video frame.

示例性地,採集設備的內參矩陣包含

Figure 02_image001
Figure 02_image003
表示採集設備的焦距,
Figure 02_image005
表示採集設備採集的視頻畫面的中心點在圖像坐標系中的像素坐標。採集設備的畸變參數包含徑向畸變參數和切向畸變係數。在預先得到每個採集設備的內參矩陣和畸變係數後,可以根據該採集設備的內參矩陣和畸變係數對該採集設備採集的視頻畫面中的目標對象的像素坐標進行去畸變處理。例如可以通過Opencv軟體中的去畸變函數,得到該採集設備採集的視頻畫面中的目標對象的修正像素坐標。 Exemplarily, the internal parameter matrix of the acquisition device contains
Figure 02_image001
,
Figure 02_image003
Indicates the focal length of the acquisition device,
Figure 02_image005
Indicates the pixel coordinates of the center point of the video frame captured by the capture device in the image coordinate system. The distortion parameters of the acquisition device include radial distortion parameters and tangential distortion coefficients. After obtaining the internal reference matrix and distortion coefficient of each acquisition device in advance, de-distortion processing can be performed on the pixel coordinates of the target object in the video image captured by the acquisition device according to the internal reference matrix and distortion coefficient of the acquisition device. For example, the corrected pixel coordinates of the target object in the video picture captured by the capture device can be obtained through the de-distortion function in the Opencv software.

示例性地,可以按照張正友棋盤標定的方式預先確定每個採集設備的內參矩陣和畸變參數。例如可以從不同角度拍攝多張棋盤格圖像,檢測出圖像中的特徵點。根據這些特徵點在棋盤格圖像中的像素坐標,求解出採集設備的內參矩陣和畸變參數,然後不斷對內參矩陣和畸變參數進行優化。在優化過程中,可以根據相鄰兩次得到的內參矩陣和畸變參數對同一像素坐標進行修正處理。通過前後兩次的修正像素坐標的差異確定是否結束優化,例如可以在該差異不再降低後,結束優化得到採集設備的內參矩陣和畸變參數。Exemplarily, the internal parameter matrix and distortion parameters of each acquisition device can be determined in advance according to Zhang Zhengyou chessboard calibration. For example, multiple checkerboard images can be taken from different angles to detect feature points in the image. According to the pixel coordinates of these feature points in the checkerboard image, the internal reference matrix and distortion parameters of the acquisition device are solved, and then the internal reference matrix and distortion parameters are continuously optimized. During the optimization process, the coordinates of the same pixel can be corrected according to the internal reference matrix and distortion parameters obtained twice adjacently. Whether to end the optimization is determined by the difference between the two corrected pixel coordinates. For example, after the difference is no longer reduced, the optimization can be ended to obtain the internal reference matrix and distortion parameters of the acquisition device.

S2022,基於預先確定的該採集設備的單應性矩陣和該採集設備採集的視頻畫面中的目標對象的修正像素坐標,確定該視頻畫面中的目標對象的初始位置坐標。S2022. Determine the initial position coordinates of the target object in the video frame based on the predetermined homography matrix of the capture device and the corrected pixel coordinates of the target object in the video frame captured by the capture device.

示例性地,單應性矩陣可以表示採集設備採集的視頻畫面對應的圖像坐標系和採集設備位於的目標場所對應的世界坐標系之間的轉換關係。該單應性矩陣可以在預先對採集設備進行標定時確定。例如可以通過採集設備採集帶有多個標誌物的樣本視頻畫面,預先確定多個標誌物與地面(世界坐標系X軸和Y軸所在的平面)的交點在目標場所對應的世界坐標系中的世界坐標,然後根據上述方式確定多個標誌物與地面的交點在樣本視頻畫面中對應的修正像素坐標,進一步可以基於多個標誌物分別對應的修正像素坐標和世界坐標,確定該採集設備的單應性矩陣。Exemplarily, the homography matrix may represent a conversion relationship between the image coordinate system corresponding to the video frame captured by the capture device and the world coordinate system corresponding to the target location where the capture device is located. The homography matrix can be determined when the acquisition device is calibrated in advance. For example, the sample video screen with multiple markers can be collected through the acquisition device, and the intersection point between the multiple markers and the ground (the plane where the X-axis and Y-axis of the world coordinate system are located) in the world coordinate system corresponding to the target site can be determined in advance. World coordinates, and then determine the corrected pixel coordinates corresponding to the intersection points of multiple markers and the ground in the sample video screen according to the above method, and further determine the unit of the acquisition device based on the corrected pixel coordinates and world coordinates corresponding to multiple markers. Responsive matrix.

示例性地,在確定視頻畫面中的目標對象的初始位置坐標時,可以根據視頻畫面中的目標對象的修正像素坐標和採集該視頻畫面的採集設備的單應性矩陣,得到視頻畫面中的目標對象在目標場所對應的世界坐標系中的初始位置坐標。For example, when determining the initial position coordinates of the target object in the video frame, the target object in the video frame can be obtained according to the corrected pixel coordinates of the target object in the video frame and the homography matrix of the acquisition device that captures the video frame The initial position coordinates of the object in the world coordinate system corresponding to the target location.

本公開實施例中,在得到視頻畫面中目標對象的像素坐標後,先基於採集該視頻畫面的採集設備的內參矩陣和畸變係數對像素坐標進行修正,從而可以得到準確度較高的修正像素坐標,進一步得到目標對象在目標場所中準確度較高的初始位置坐標。In the embodiment of the present disclosure, after obtaining the pixel coordinates of the target object in the video picture, the pixel coordinates are first corrected based on the internal reference matrix and the distortion coefficient of the acquisition device that captures the video picture, so that the corrected pixel coordinates with high accuracy can be obtained , and further obtain the initial position coordinates of the target object in the target place with high accuracy.

在一種實施方式中,針對上述S103,在對同一目標對象的初始位置坐標進行融合,得到目標對象在目標場所中的目標位置坐標時,如圖4所示,可以包括以下S301~S302:In one embodiment, for the above S103, when the initial position coordinates of the same target object are fused to obtain the target position coordinates of the target object in the target place, as shown in FIG. 4 , the following S301~S302 may be included:

S301,基於多張視頻畫面確定的目標對象的初始位置坐標,確定與同一目標對象關聯的多個初始位置坐標。S301. Determine multiple initial position coordinates associated with the same target object based on the initial position coordinates of the target object determined in multiple video frames.

示例性地,根據上文提到的目標場所中的每個目標對象至少被兩個採集設備同時採集到,針對每個目標對象,在同一時刻被不同的採集設備拍攝到的情況下,採集設備的參數資訊存在一定的誤差,且不同的採集設備的參數資訊之間的誤差不同。因此,基於不同的視頻畫面確定的同一目標對象的初始位置坐標可能不同。在對同一目標對象的初始位置坐標進行融合之前,需要先確定與同一目標對象關聯的多個初始位置坐標。Exemplarily, according to the above-mentioned target location in which each target object is captured by at least two capture devices at the same time, for each target object, if it is photographed by different capture devices at the same time, the capture device There is a certain error in the parameter information of different acquisition devices, and the error between the parameter information of different acquisition devices is different. Therefore, the initial position coordinates of the same target object determined based on different video frames may be different. Before fusing the initial position coordinates of the same target object, it is necessary to determine multiple initial position coordinates associated with the same target object.

S302,將與同一目標對象關聯的多個初始位置坐標進行依次融合,得到同一目標對象在目標場所中的目標位置坐標。S302, sequentially fuse multiple initial position coordinates associated with the same target object to obtain target position coordinates of the same target object in the target place.

示例性地,假設與同一目標對象關聯的多個初始位置坐標包含N個,可以先將前兩個進行融合,得到融合後的初始位置坐標。然後將融合後的初始位置坐標與第三個初始位置坐標進行融合,直至與最後一個初始位置坐標進行融合後,將最終融合得到的位置坐標作為同一目標對象的目標位置坐標。Exemplarily, assuming that there are N initial position coordinates associated with the same target object, the first two may be fused first to obtain the fused initial position coordinates. Then, the fused initial position coordinates are fused with the third initial position coordinates until they are fused with the last initial position coordinates, and the final fused position coordinates are used as the target position coordinates of the same target object.

本公開實施例中,考慮到基於不同採集設備採集視頻畫面確定的同一目標對象的初始位置坐標會存在一些誤差,因此可以通過對多個採集設備採集的同一目標對象的初始位置坐標進行融合,從而可以得到該同一目標對象準確度較高的目標位置坐標。In the embodiment of the present disclosure, considering that there may be some errors in the initial position coordinates of the same target object determined based on video images collected by different acquisition devices, it is possible to fuse the initial position coordinates of the same target object collected by multiple acquisition devices, so that The target position coordinates of the same target object with high accuracy can be obtained.

在一種實施方式中,針對上述S301,在基於多張視頻畫面確定的目標對象的初始位置坐標,確定與同一目標對象關聯的多個初始位置坐標時,包括以下S3011~S3012:In one embodiment, for the above S301, when determining multiple initial position coordinates associated with the same target object based on the initial position coordinates of the target object determined based on multiple video images, the following S3011~S3012 are included:

S3011,針對所述多張視頻畫面中的任意兩張視頻畫面,確定任意兩張視頻畫面中第一視頻畫面中的目標對象為第一目標對象,任意兩張視頻畫面中第二視頻畫面中的目標對象為第二目標對象,針對每個所述第一目標對象的初始位置坐標,確定該第一目標對象的初始位置坐標與任意兩張視頻畫面中第二視頻畫面中各個第二目標對象的初始位置坐標之間的距離;S3011. For any two video frames in the plurality of video frames, determine that the target object in the first video frame in any two video frames is the first target object, and determine that the target object in the second video frame in any two video frames is The target object is the second target object, and for the initial position coordinates of each of the first target objects, determine the initial position coordinates of the first target object and the coordinates of each second target object in the second video frame in any two video frames. The distance between the initial position coordinates;

S3012,確定與該第一目標對象具有最小距離的第二目標對象與該第一目標對象為同一目標對象,其中,所述最小距離小於預設融合距離閾值;將該第一目標對象的初始位置坐標,作為與目標對象中同一目標對象關聯的多個初始位置坐標。S3012. Determine that the second target object having the minimum distance from the first target object is the same target object as the first target object, wherein the minimum distance is smaller than the preset fusion distance threshold; the initial position of the first target object Coordinates as multiple initial position coordinates associated with the same target object in the target object.

示例性地,例如目標場所中設置A個採集設備,假設在同一時刻A個採集設備採集的視頻畫面中均包含至少一個目標對象,在該時刻可以得到A組初始位置坐標,A組初始位置坐標構成初始坐標集合

Figure 02_image007
。其中,
Figure 02_image009
依次表示為A個採集設備中的第一個採集設備、第二個採集設備、第三個採集設備至第A個採集設備拍攝的視頻畫面中的目標對象的初始位置坐標。下面將以下任意兩個視頻畫面為第一個採集設備和第二個採集設備在同一時刻採集的視頻畫面為例,說明如何確定與同一目標對象關聯的多個初始位置坐標: Exemplarily, for example, A collection devices are set up in the target place, assuming that at the same time, the video images collected by the A collection devices all contain at least one target object, at this time, A group of initial position coordinates can be obtained, A group of initial position coordinates form the initial set of coordinates
Figure 02_image007
. in,
Figure 02_image009
The initial position coordinates of the target object in the video frame captured by the first acquisition device, the second acquisition device, the third acquisition device to the A-th acquisition device in the A acquisition devices are sequentially represented. The following takes any two of the following video frames as the video frames captured by the first capture device and the second capture device at the same time as an example to illustrate how to determine multiple initial position coordinates associated with the same target object:

示例性地,S1中包含a個第一目標對象的初始位置坐標(也稱作第一初始位置坐標),S2中包含b個第二目標對象的初始位置坐標(也稱作第二初始位置坐標),可以確定每個第一初始位置坐標和每個第二初始位置坐標之間的歐式距離,得到距離矩陣:

Figure 02_image011
Exemplarily, S1 contains the initial position coordinates of a first target object (also referred to as the first initial position coordinates), and S2 contains the initial position coordinates of b second target objects (also referred to as the second initial position coordinates ), the Euclidean distance between each first initial position coordinate and each second initial position coordinate can be determined to obtain a distance matrix:
Figure 02_image011

其中,

Figure 02_image013
表示S1中第1個第一初始位置坐標和S2中第1個第二初始位置坐標之間的距離;
Figure 02_image015
表示S1中第1個第一初始位置坐標和S2中第b個第二初始位置坐標之間的距離;
Figure 02_image017
表示S1中第i個第一初始位置坐標和S2中第j個第二初始位置坐標之間的距離;
Figure 02_image019
表示S1中第a個第一初始位置坐標和S2中第1個第二初始位置坐標之間的距離;
Figure 02_image021
表示S1中第a個第一初始位置坐標和S2中第b個第二初始位置坐標之間的距離。 in,
Figure 02_image013
Indicates the distance between the first first initial position coordinate in S1 and the first second initial position coordinate in S2;
Figure 02_image015
Indicates the distance between the first first initial position coordinate in S1 and the second b second initial position coordinate in S2;
Figure 02_image017
Indicates the distance between the i-th first initial position coordinate in S1 and the j-th second initial position coordinate in S2;
Figure 02_image019
Indicates the distance between the a-th first initial position coordinate in S1 and the first second initial position coordinate in S2;
Figure 02_image021
Indicates the distance between the a-th first initial position coordinate in S1 and the b-th second initial position coordinate in S2.

示例性地,具體在操作時,可以按照以下方式確定S1和S2中與同一目標對象關聯的多個初始位置坐標,包括S30121~S30124:Exemplarily, during operation, multiple initial position coordinates associated with the same target object in S1 and S2 may be determined in the following manner, including S30121~S30124:

S30121,在當前距離矩陣中的元素中查找當前最小距離;S30121. Find the current minimum distance among the elements in the current distance matrix;

示例性地,在首次查找最小距離的情況下,當前距離矩陣中的元素包含S1中每個第一初始位置坐標和S2中每個第二初始位置坐標之間的歐式距離。Exemplarily, in the case of finding the minimum distance for the first time, the elements in the current distance matrix include the Euclidean distance between each first initial position coordinate in S1 and each second initial position coordinate in S2.

S30122,判斷當前最小距離是否小於預設融合距離閾值。S30122. Determine whether the current minimum distance is smaller than a preset fusion distance threshold.

示例性地,預設融合距離可以根據經驗設定。例如預先通過不同採集設備針對同一目標對象進行拍攝,然後根據不同採集設備採集的視頻畫面分別確定出該同一目標對象在目標場所中的多個位置坐標。根據多個位置坐標之間的距離來確定該預設融合距離閾值。Exemplarily, the preset fusion distance can be set according to experience. For example, the same target object is photographed in advance by different acquisition devices, and then multiple position coordinates of the same target object in the target place are respectively determined according to the video images collected by different acquisition devices. The preset fusion distance threshold is determined according to the distance between the multiple position coordinates.

S30123,在確定該當前最小距離小於預設融合距離閾值的情況下,確定該當前最小距離關聯的兩個初始位置坐標為同一目標對象關聯的初始位置坐標。S30123. When it is determined that the current minimum distance is less than the preset fusion distance threshold, determine that the two initial position coordinates associated with the current minimum distance are initial position coordinates associated with the same target object.

示例性地,例如確定出

Figure 02_image019
為當前最小距離,且
Figure 02_image019
小於預設融合距離閾值,可以將S1中第a個第一初始位置坐標和S2中第1個第二初始位置坐標作為與同一目標對象關聯的初始位置坐標。 Exemplarily, for example, it is determined that
Figure 02_image019
is the current minimum distance, and
Figure 02_image019
If it is smaller than the preset fusion distance threshold, the a-th first initial position coordinate in S1 and the first second initial position coordinate in S2 may be used as the initial position coordinates associated with the same target object.

S30124,將當前距離矩陣中的當前最小距離,以及與當前最小距離關聯的兩個初始位置坐標中任一初始位置坐標之間的所有其它距離設置為預設融合距離閾值後,返回執行S30121,直至當前距離矩陣中的當前最小距離大於或等於預設融合距離閾值的情況下,得到S1和S2中所有與同一目標對象關聯的初始位置坐標。S30124. After setting the current minimum distance in the current distance matrix and all other distances between any of the two initial position coordinates associated with the current minimum distance as the preset fusion distance threshold, return to execute S30121 until When the current minimum distance in the current distance matrix is greater than or equal to the preset fusion distance threshold, all initial position coordinates associated with the same target object in S1 and S2 are obtained.

示例性地,假設當前距離矩陣由S1和S2中初始位置坐標計算得到,具體一個為3×3矩陣:

Figure 02_image023
, As an example, assume that the current distance matrix is calculated from the initial position coordinates in S1 and S2, specifically a 3×3 matrix:
Figure 02_image023
,

預設融合閾值為

Figure 02_image025
;假設
Figure 02_image027
為當前矩陣中最小距離且小於
Figure 02_image025
,那麼S1中的第1個第一初始位置坐標以及S2中的第1個第二初始位置坐標,為同一目標對象的關聯初始位置坐標。則在當前距離矩陣中,與這兩個初始位置坐標中任一初始位置坐標計算出的所有其它距離為
Figure 02_image029
Figure 02_image031
Figure 02_image033
Figure 02_image035
。所以,根據S30124,在當前矩陣中,需要把
Figure 02_image027
Figure 02_image029
Figure 02_image031
Figure 02_image033
Figure 02_image035
均設置為
Figure 02_image025
;設置後的矩陣為:
Figure 02_image037
, The preset fusion threshold is
Figure 02_image025
; suppose
Figure 02_image027
is the minimum distance in the current matrix and is less than
Figure 02_image025
, then the first first initial position coordinates in S1 and the first second initial position coordinates in S2 are the associated initial position coordinates of the same target object. Then in the current distance matrix, all other distances calculated from any of the two initial position coordinates are
Figure 02_image029
,
Figure 02_image031
,
Figure 02_image033
,
Figure 02_image035
. Therefore, according to S30124, in the current matrix, it is necessary to put
Figure 02_image027
,
Figure 02_image029
,
Figure 02_image031
,
Figure 02_image033
,
Figure 02_image035
are set to
Figure 02_image025
;The matrix after setting is:
Figure 02_image037
,

之後返回執行S30121。Then return to execute S30121.

示例性地,在將當前距離矩陣中的當前最小距離,以及與當前最小距離關聯的兩個初始位置坐標中任一初始位置坐標之間的所有其它距離設置為預設融合距離閾值後,在繼續查找當前最小距離的過程中,可以排除被設置為預設融合距離閾值的元素,從而提高搜索效率。Exemplarily, after setting the current minimum distance in the current distance matrix and all other distances between any one of the two initial position coordinates associated with the current minimum distance as the preset fusion distance threshold, continue In the process of finding the current minimum distance, the elements set as the preset fusion distance threshold can be excluded, thereby improving the search efficiency.

示例性地,在一種實施方式中,在得到S1和S2中與同一目標對象關聯的多個初始位置坐標後,可以繼續基於其它任意兩張視頻畫面確定是否存在與同一目標對象關聯的初始位置坐標,直至判斷完A個採集設備在同一時刻採集的視頻畫面後,可以得到A個採集設備在同一時刻採集的視頻畫面中的各目標對象的不同初始位置坐標。然後對與同一目標對象關聯的初始位置坐標進行融合,得到A個採集設備在同一時刻拍攝的A張視頻畫面中的各目標對象在目標場所中的目標位置坐標。Exemplarily, in an implementation manner, after obtaining multiple initial position coordinates associated with the same target object in S1 and S2, it may continue to determine whether there are initial position coordinates associated with the same target object based on any other two video frames , until the video frames collected by the A collection devices at the same time are judged, the different initial position coordinates of the target objects in the video frames collected by the A collection devices at the same time can be obtained. Then, the initial position coordinates associated with the same target object are fused to obtain the target position coordinates of each target object in the target place in A video frames captured by A collection devices at the same time.

示例性地,在另一種實施方式中,在得到S1和S2中與同一目標對象關聯的多個初始位置坐標後,可以對多個初始位置坐標進行坐標融合,得到該同一目標對象更新後的初始位置坐標。針對S1和S2中未參與融合的初始位置坐標,可以與更新後的初始位置坐標構成S2’。進一步通過S2’和S3中的初始位置坐標構成新的當前距離矩陣重複執行S30121至S30124的步驟,得到S2’和S3中與同一目標對象關聯的多個初始位置坐標,按照相同的方式得到S3’。進一步通過S3’和S4中的初始位置坐標構成新的當前距離矩陣,重複執行S30121至S30124的步驟,直至完成與初始坐標集合中的最後一個元素的初始位置坐標的融合後,得到A個採集設備在同一時刻拍攝的A張視頻畫面中的各目標對象在目標場所中的目標位置坐標。Exemplarily, in another implementation manner, after obtaining multiple initial position coordinates associated with the same target object in S1 and S2, coordinate fusion may be performed on the multiple initial position coordinates to obtain the updated initial position coordinates of the same target object. Position coordinates. For the initial position coordinates in S1 and S2 that are not involved in the fusion, S2' can be formed with the updated initial position coordinates. Further, a new current distance matrix is formed by the initial position coordinates in S2' and S3, and the steps from S30121 to S30124 are repeated to obtain multiple initial position coordinates associated with the same target object in S2' and S3, and obtain S3' in the same way . Further construct a new current distance matrix through the initial position coordinates in S3' and S4, and repeat the steps from S30121 to S30124 until the fusion with the initial position coordinates of the last element in the initial coordinate set is completed, and A acquisition devices are obtained The target position coordinates of each target object in the target place in the A video frames shot at the same moment.

特別地,在直至完成與初始坐標集合中的最後一個元素的初始位置坐標的融合後,若檢測到存在任一初始位置坐標從開始到結束均為參與融合,考慮到目標場所中的每個目標對象至少被兩個採集設備同時採集到,因此可以將該任一初始位置坐標作為誤差初始位置坐標進行過濾。In particular, until the fusion with the initial position coordinates of the last element in the initial coordinate set is completed, if any initial position coordinates are detected to be involved in the fusion from the beginning to the end, considering that each target in the target place The object is simultaneously collected by at least two collection devices, so any initial position coordinate can be filtered as the error initial position coordinate.

本公開實施例中,根據任意兩張視頻畫面中不同目標對象的初始位置坐標,和預設融合距離閾值,可以快速確定出與同一目標對象關聯的初始位置坐標,從而為後續確定各目標對象的目標位置坐標提供依據。In the embodiment of the present disclosure, according to the initial position coordinates of different target objects in any two video frames and the preset fusion distance threshold, the initial position coordinates associated with the same target object can be quickly determined, so as to determine the location of each target object in the future. The target position coordinates provide the basis.

具體地,針對上述S302,在將與同一目標對象關聯的多個初始位置坐標進行依次融合,得到同一目標對象在目標場所中的目標位置坐標時,可以包括以下S3021~S3022:Specifically, for the above S302, when the multiple initial position coordinates associated with the same target object are sequentially fused to obtain the target position coordinates of the same target object in the target place, the following S3021~S3022 may be included:

S3021,從同一目標對象關聯的多個初始位置坐標中選取任一初始位置坐標,將該初始位置坐標作為第一中間融合位置坐標。S3021. Select any initial position coordinate from multiple initial position coordinates associated with the same target object, and use the initial position coordinate as the first intermediate fusion position coordinate.

S3022,將第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標;將第二中間融合位置坐標作為更新後的第一中間融合位置坐標,並返回生成第二中間融合位置坐標的步驟,直到不存在待融合的初始位置坐標。S3022. Fuse the first intermediate fusion position coordinates with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates; use the second intermediate fusion position coordinates as the updated first an intermediate fused position coordinate, and return to the step of generating the second intermediate fused position coordinate until there is no initial position coordinate to be fused.

其中,待融合的初始位置坐標是指未參與融合的初始位置坐標。Wherein, the initial position coordinates to be fused refer to the initial position coordinates not involved in the fusion.

示例性地,在將第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標時,包括:確定第一中間融合位置坐標與其它任一待融合的初始位置坐標的中點坐標,將該中點坐標作為生成的第二中間融合位置坐標。Exemplarily, when the coordinates of the first intermediate fusion position are fused with any other initial position coordinates to be fused in the plurality of initial position coordinates to generate the coordinates of the second intermediate fusion position, it includes: determining the first intermediate fusion position The midpoint coordinates of the coordinates and any other initial position coordinates to be fused, and the midpoint coordinates are used as the generated second intermediate fusion position coordinates.

示例性地,結合上述實施例若確定出與目標對象A關聯的多個初始位置坐標包含N個,可以將任一初始位置坐標作為第一中間融合位置坐標,確定該第一中間融合位置坐標與其它任一待融合的初始位置坐標的中點坐標。然後將該中點坐標作為更新後的第一中間融合位置坐標,繼續與其它任一待融合的初始位置坐標進行融合。直到N個初始位置坐標中不存在待融合的初始位置坐標後,得到目標對象A的目標位置坐標。Exemplarily, in combination with the above-mentioned embodiment, if it is determined that the multiple initial position coordinates associated with the target object A include N, any initial position coordinate can be used as the first intermediate fusion position coordinate, and the first intermediate fusion position coordinate and The midpoint coordinates of any other initial position coordinates to be fused. Then use the coordinates of the midpoint as the updated coordinates of the first intermediate fusion position, and continue to perform fusion with any other initial position coordinates to be fused. Until there is no initial position coordinate to be fused in the N initial position coordinates, the target position coordinate of the target object A is obtained.

本公開實施例中,提出針對與同一目標對象關聯的多個初始位置坐標,可以按照依次取中點的方式融合,從而得到準確度較高的目標位置坐標。In the embodiments of the present disclosure, it is proposed that multiple initial position coordinates associated with the same target object can be fused in a manner of sequentially taking midpoints, so as to obtain target position coordinates with high accuracy.

本公開實施例提出的定位方法可以準確地確定目標場所中各目標對象的目標位置坐標,該方式可以應用於多種應用場景。以應用於工廠為例,在得到目標對象在目標場所中的目標位置坐標之後,如圖5所示,本公開實施例提供的定位方法還包括以下S401~S402:The positioning method proposed in the embodiment of the present disclosure can accurately determine the target position coordinates of each target object in the target place, and this method can be applied to various application scenarios. Taking the application in a factory as an example, after obtaining the target position coordinates of the target object in the target place, as shown in Figure 5, the positioning method provided by the embodiment of the present disclosure also includes the following S401~S402:

S401,基於目標場所中的各目標對象分別對應的目標位置坐標,以及預先設定的目標區域,確定是否存在進入目標區域的目標對象;S401. Determine whether there is a target object entering the target area based on the target position coordinates corresponding to each target object in the target place and the preset target area;

S402,在確定存在進入目標區域的目標對象的情況下,進行預警提示。S402. When it is determined that there is a target object entering the target area, an early warning prompt is given.

示例性地,目標場所為工廠的情況下,可以預先在目標場所對應的世界坐標下中設定工廠記憶體在危險的目標區域對應的坐標範圍。然後根據確定的目標場所中的各目標對象分別對應的目標位置坐標以及目標位於對應的坐標範圍,確定是否存在進入目標區域的目標對象。進一步在確定存在進入目標區域的目標對象的情況下,進行預警提示。Exemplarily, when the target site is a factory, the coordinate range corresponding to the dangerous target area in the factory memory may be set in advance in the world coordinates corresponding to the target site. Then, according to the target position coordinates corresponding to each target object in the determined target location and the corresponding coordinate range where the target is located, it is determined whether there is a target object entering the target area. Further, when it is determined that there is a target object entering the target area, an early warning prompt is provided.

示例性地,預警提示可以包括但不限於聲光報警提示、語音報警提示等。通過預警提示,可以保障目標場所中員工的安全,提高目標場所的安全性。Exemplarily, the warning prompt may include, but not limited to, an audible and visual warning prompt, a voice warning prompt, and the like. Through the early warning prompt, the safety of employees in the target place can be guaranteed and the safety of the target place can be improved.

本公開實施例中,在得到目標場所中的各目標對象準確度較高的目標位置坐標後,可以基於預先設定的目標區域,例如預先設定的危險區域,判斷目標場所中的目標對象是否進入目標區域,以便及時預警提示,提高目標場所的安全性。In the embodiment of the present disclosure, after obtaining the target position coordinates of each target object in the target place with high accuracy, it can be judged whether the target object in the target place enters the target area based on the preset target area, such as the preset dangerous area. area, so as to timely warn and prompt and improve the safety of the target site.

下面以目標場所為工廠為例,結合具體實施例對本公開提供的定位方法進行整體介紹:Taking the target place as a factory as an example, the positioning method provided by the present disclosure will be introduced as a whole in combination with specific embodiments:

1)針對工廠進行採集設備安裝,例如在工廠安裝多個相機。為了實現對場景內目標的精準定位,並保證演算法通用性及魯棒性,使得不同的採集設備在工廠中的採集視角不同,確保進入工廠中的每個員工至少被兩個採集設備同時採集到。1) Install acquisition equipment for the factory, such as installing multiple cameras in the factory. In order to achieve precise positioning of objects in the scene and ensure the versatility and robustness of the algorithm, different collection devices have different collection perspectives in the factory, ensuring that each employee entering the factory is collected by at least two collection devices at the same time arrive.

2)使用張正友標定方式確定每個相機的內參矩陣和畸變係數。2) Use the Zhang Zhengyou calibration method to determine the internal reference matrix and distortion coefficient of each camera.

3)在工廠內設置多個標誌物,確定標誌物與地面的交點在工廠對應的世界坐標系中的位置坐標。並根據相機的內參矩陣和畸變係數確定標誌物與地面的交點在樣本視頻畫面中的修正像素坐標。並根據交點在世界坐標系中的位置坐標和在樣本視頻畫面中的修正像素坐標,確定每個相機的單應性矩陣。3) Set multiple markers in the factory, and determine the position coordinates of the intersection point of the markers and the ground in the world coordinate system corresponding to the factory. And according to the internal reference matrix of the camera and the distortion coefficient, the corrected pixel coordinates of the intersection point of the marker and the ground in the sample video frame are determined. And according to the position coordinates of the intersection point in the world coordinate system and the corrected pixel coordinates in the sample video frame, the homography matrix of each camera is determined.

4)使用加入特徵金字塔的神經網路對工廠中的相機採集的視頻畫面進行目標檢測,得到每張視頻畫面中包含的員工的像素坐標。4) Use the neural network added to the feature pyramid to perform target detection on the video pictures collected by the cameras in the factory, and obtain the pixel coordinates of the employees contained in each video picture.

5)根據採集該張視頻畫面的相機的內參矩陣和畸變係數,對該張視頻畫面中包含的員工的像素坐標進行修正,得到該張視頻畫面中包含的員工的修正像素坐標。5) According to the internal reference matrix and distortion coefficient of the camera that collected the video frame, correct the pixel coordinates of the employees contained in the video frame to obtain the corrected pixel coordinates of the employees contained in the video frame.

6)根據採集該張視頻畫面的相機的單應性矩陣和該張視頻畫面中包含的員工的修正像素坐標,確定該張視頻畫面中包含的員工在工廠中的初始位置坐標。6) According to the homography matrix of the camera that captures the video frame and the corrected pixel coordinates of the employee included in the video frame, determine the initial position coordinates of the employee contained in the video frame in the factory.

7)對同一時刻採集到的視頻畫面中的同一員工的初始位置坐標進行融合,得到工廠中的員工在該時刻的目標位置坐標。7) The initial position coordinates of the same employee in the video images collected at the same time are fused to obtain the target position coordinates of the employees in the factory at that moment.

8)根據工廠中的員工在該時刻的目標位置坐標和工廠中預設的危險區域,確定是否存在員工進入危險區域。在確定存在員工進入危險區域的情況下,進行預警提示。8) According to the target position coordinates of the employees in the factory at this moment and the dangerous areas preset in the factory, determine whether there are employees entering the dangerous area. When it is determined that there are employees entering the dangerous area, an early warning prompt will be given.

本領域技術人員可以理解,在具體實施方式的上述方法中,各步驟的撰寫順序並不意味著嚴格的執行順序而對實施過程構成任何限定,各步驟的具體執行順序應當以其功能和可能的內在邏輯確定。Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

基於同一技術構思,本公開實施例中還提供了與定位方法對應的定位裝置,由於本公開實施例中的裝置解決問題的原理與本公開實施例上述定位方法相似,因此裝置的實施可以參見方法的實施,重複之處不再贅述。Based on the same technical concept, the embodiment of the present disclosure also provides a positioning device corresponding to the positioning method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned positioning method in the embodiment of the present disclosure, the implementation of the device can refer to the method The implementation of this method will not be repeated here.

參照圖6所示,為本公開實施例提供的一種定位裝置500的示意圖,該定位裝置包括:Referring to FIG. 6 , it is a schematic diagram of a positioning device 500 provided by an embodiment of the present disclosure. The positioning device includes:

獲取模組501,用於獲取目標場所內設置的多個採集設備在同一時刻採集的視頻畫面;其中,不同的採集設備在目標場所中的採集視角不同,視頻畫面中包括目標對象;The acquiring module 501 is used to acquire the video images collected at the same time by a plurality of acquisition devices set in the target place; where different acquisition devices have different acquisition angles of view in the target place, and the video images include the target object;

確定模組502,用於基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標;The determination module 502 is used to determine the initial position coordinates of the target object in the target place based on the video images collected by multiple acquisition devices at the same time;

融合模組503,用於對同一目標對象的初始位置坐標進行融合,得到該目標對象在目標場所中的目標位置坐標。The fusion module 503 is configured to fuse the initial position coordinates of the same target object to obtain the target position coordinates of the target object in the target place.

在一種實施方式中,確定模組502在用於基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標時,包括:In one embodiment, when the determination module 502 is used to determine the initial position coordinates of the target object in the target place based on the video images captured by multiple capture devices at the same time, it includes:

獲取多個採集設備在同一時刻分別採集的視頻畫面中的目標對象的像素坐標;Obtaining the pixel coordinates of the target object in the video images respectively collected by multiple collection devices at the same time;

基於每個採集設備採集的視頻畫面中的目標對象的像素坐標和該採集設備的參數資訊,確定該採集設備採集的目標對象在目標場所對應的世界坐標系下的初始位置坐標。Based on the pixel coordinates of the target object in the video frame collected by each collection device and the parameter information of the collection device, determine the initial position coordinates of the target object collected by the collection device in the world coordinate system corresponding to the target location.

在一種可能的實施方式中,確定模組502在用於獲取多個採集設備在同一時刻分別採集的視頻畫面中的目標對象的像素坐標時,包括:In a possible implementation manner, when the determination module 502 is used to obtain the pixel coordinates of the target object in the video images collected by multiple collection devices at the same time, it includes:

將多張視頻畫面輸入預先訓練的神經網路,得到每張視頻畫面中的目標對象的檢測框;其中,神經網路包含多個用於檢測不同尺寸的目標對象的目標檢測子網路;Input multiple video frames into the pre-trained neural network to obtain the detection frame of the target object in each video frame; wherein, the neural network includes multiple target detection sub-networks for detecting target objects of different sizes;

提取每張視頻畫面中的目標對象的檢測框上的目標位置點在該視頻畫面中的像素坐標,得到該視頻畫面中的目標對象的像素坐標。Extracting the pixel coordinates of the target position point on the detection frame of the target object in each video frame in the video frame to obtain the pixel coordinates of the target object in the video frame.

在一種可能的實施方式中,確定模組502在用於基於每個採集設備採集的視頻畫面中的目標對象的像素坐標和該採集設備的參數資訊,確定該採集設備採集的目標對象在目標場所對應的世界坐標系下的初始位置坐標時,包括:In a possible implementation, the determination module 502 is used to determine that the target object collected by the collection device is in the target location based on the pixel coordinates of the target object in the video frame collected by each collection device and the parameter information of the collection device. The initial position coordinates in the corresponding world coordinate system include:

基於預先確定的每個採集設備的內參矩陣和畸變參數,對該採集設備採集的視頻畫面中的目標對象的像素坐標進行修正,得到該視頻畫面中的目標對象的修正像素坐標;Based on the predetermined internal reference matrix and distortion parameters of each acquisition device, the pixel coordinates of the target object in the video image collected by the acquisition device are corrected to obtain the corrected pixel coordinates of the target object in the video image;

基於預先確定的該採集設備的單應性矩陣和該採集設備採集的視頻畫面中的目標對象的修正像素坐標,確定該視頻畫面中的目標對象的初始位置坐標。The initial position coordinates of the target object in the video frame are determined based on the predetermined homography matrix of the capture device and the corrected pixel coordinates of the target object in the video frame captured by the capture device.

在一種可能的實施方式中,融合模組503在用於對同一目標對象的初始位置坐標進行融合,得到目標對象在目標場所中的目標位置坐標,包括:In a possible implementation manner, the fusion module 503 is used to fuse the initial position coordinates of the same target object to obtain the target position coordinates of the target object in the target place, including:

基於多張視頻畫面確定的目標對象的初始位置坐標,確定與同一目標對象關聯的多個初始位置坐標;Determining a plurality of initial position coordinates associated with the same target object based on the initial position coordinates of the target object determined from multiple video frames;

將與同一目標對象關聯的多個初始位置坐標進行依次融合,得到同一目標對象在目標場所中的目標位置坐標。Multiple initial position coordinates associated with the same target object are sequentially fused to obtain the target position coordinates of the same target object in the target place.

在一種可能的實施方式中,融合模組503在用於將與同一目標對象關聯的多個初始位置坐標進行依次融合,得到同一目標對象在目標場所中的目標位置坐標時,包括:In a possible implementation manner, when the fusion module 503 is used to sequentially fuse multiple initial position coordinates associated with the same target object to obtain the target position coordinates of the same target object in the target place, it includes:

從同一目標對象關聯的多個初始位置坐標中選取任一初始位置坐標,將該初始位置坐標作為第一中間融合位置坐標;Select any initial position coordinate from a plurality of initial position coordinates associated with the same target object, and use the initial position coordinate as the first intermediate fusion position coordinate;

將第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標,將第二中間融合位置坐標作為更新後的第一中間融合位置坐標,並返回生成第二中間融合位置坐標的步驟,直到不存在待融合的初始位置坐標。Fusing the first intermediate fusion position coordinates with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates, using the second intermediate fusion position coordinates as the updated first intermediate position coordinates Fuse the position coordinates, and return to the step of generating the second intermediate fused position coordinates until there is no initial position coordinates to be fused.

在一種可能的實施方式中,融合模組503在用於將第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標時,包括:In a possible implementation manner, the fusion module 503 is used to fuse the coordinates of the first intermediate fusion position with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate the coordinates of the second intermediate fusion position , including:

確定第一中間融合位置坐標與其它任一待融合的初始位置坐標的中點坐標,將該中點坐標作為生成的第二中間融合位置坐標。Determine the midpoint coordinates of the first intermediate fusion position coordinates and any other initial position coordinates to be fused, and use the midpoint coordinates as the generated second intermediate fusion position coordinates.

在一種可能的實施方式中,融合模組503在用於基於多張視頻畫面確定的目標對象的初始位置坐標,確定與同一目標對象關聯的多個初始位置坐標時,包括:In a possible implementation manner, when the fusion module 503 is used to determine multiple initial position coordinates associated with the same target object based on the initial position coordinates of the target object determined based on multiple video frames, it includes:

針對所述多張視頻畫面中的任意兩張視頻畫面,確定任意兩張視頻畫面中第一視頻畫面中的目標對象為第一目標對象,任意兩張視頻畫面中第二視頻畫面中的目標對象為第二目標對象,針對每個所述第一目標對象的初始位置坐標,確定該第一目標對象的初始位置坐標與任意兩張視頻畫面中第二視頻畫面中各個第二目標對象的初始位置坐標之間的距離;For any two video frames in the plurality of video frames, determine that the target object in the first video frame in any two video frames is the first target object, and the target object in the second video frame in any two video frames For the second target object, for the initial position coordinates of each of the first target objects, determine the initial position coordinates of the first target object and the initial position of each second target object in the second video frame in any two video frames distance between coordinates;

確定與該第一目標對象具有最小距離的第二目標對象與該第一目標對象為同一目標對象,其中,所述最小距離小於預設融合距離閾值;將該第一目標對象的初始位置坐標,作為與目標對象中同一目標對象關聯的多個初始位置坐標。Determining that a second target object having a minimum distance from the first target object is the same target object as the first target object, wherein the minimum distance is smaller than a preset fusion distance threshold; the initial position coordinates of the first target object, As multiple initial position coordinates associated with the same target object in the target object.

在一種可能的實施方式中,在融合模組503得到目標對象在目標場所中的目標位置坐標之後,確定模組502還用於:In a possible implementation manner, after the fusion module 503 obtains the target position coordinates of the target object in the target place, the determination module 502 is also used to:

基於目標場所中的各目標對象分別對應的目標位置坐標,以及預先設定的目標區域,確定是否存在進入目標區域的目標對象;Based on the target position coordinates corresponding to each target object in the target place and the preset target area, determine whether there is a target object entering the target area;

在確定存在進入目標區域的目標對象的情況下,進行預警提示。When it is determined that there is a target object entering the target area, an early warning prompt is given.

關於裝置中的各模組的處理流程、以及各模組之間的交互流程的描述可以參照上述方法實施例中的相關說明,這裡不再詳述。For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

對應於圖1中的定位方法,本公開實施例還提供了一種電子設備600,如圖7所示,為本公開實施例提供的電子設備600結構示意圖,包括:Corresponding to the positioning method in FIG. 1, the embodiment of the present disclosure also provides an electronic device 600, as shown in FIG. 7, which is a schematic structural diagram of the electronic device 600 provided in the embodiment of the present disclosure, including:

處理器610、儲存器620、和匯流排630;儲存器620用於儲存執行指令,包括記憶體621和外部儲存器622;這裡的記憶體621也稱記憶體儲存器,用於暫時存放處理器610中的運算數據,以及與硬碟等外部儲存器622交換的數據,處理器610通過記憶體621與外部儲存器622進行數據交換,當所述電子設備600運行時,所述處理器610與所述儲存器620之間通過匯流排630通訊,使得所述處理器610執行以下指令:獲取目標場所內設置的多個採集設備在同一時刻採集的視頻畫面;其中,不同的採集設備在目標場所中的採集視角不同,視頻畫面中包含目標對象;基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對在目標場所中的初始位置坐標;對同一目標對象的初始位置坐標進行融合,得到目標對象在目標場所中的目標位置坐標。Processor 610, storage 620, and bus 630; storage 620 is used to store execution instructions, including memory 621 and external storage 622; memory 621 here is also called memory storage, and is used for temporarily storing the processor The computing data in 610 and the data exchanged with external storage 622 such as hard disk, the processor 610 exchanges data with the external storage 622 through the memory 621, when the electronic device 600 is running, the processor 610 and the external storage 622 exchange data. The storage 620 communicates with each other through the bus 630, so that the processor 610 executes the following instructions: acquire the video images collected at the same time by multiple collection devices set in the target place; where different collection devices are in the target place The acquisition angles of view are different, and the video picture contains the target object; based on the video pictures collected by multiple acquisition devices at the same time, the initial position coordinates of the target pair in the target place are respectively determined; the initial position coordinates of the same target object are fused, The target position coordinates of the target object in the target place are obtained.

本公開實施例還提供一種計算機可讀儲存媒體,該計算機可讀儲存媒體上儲存有計算機程式,該計算機程式被處理器運行時執行上述方法實施例中所述的定位方法的步驟。其中,該儲存媒體可以是揮發性或非揮發性的計算機可讀取儲存媒體。Embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the positioning method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

本公開實施例還提供一種計算機程式產品,該計算機程式產品承載有程式代碼並儲存於儲存媒體中,所述程式代碼包括的指令可用於執行上述方法實施例中所述的定位方法的步驟,具體可參見上述方法實施例,在此不再贅述。An embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code and is stored in a storage medium, and the instructions included in the program code can be used to execute the steps of the positioning method described in the above method embodiments, specifically Reference may be made to the foregoing method embodiments, and details are not repeated here.

其中,上述計算機程式產品可以具體通過硬體、軟體或其結合的方式實現。在一個可選實施例中,所述計算機程式產品具體體現為計算機儲存媒體,在另一個可選實施例中,計算機程式產品具體體現為軟體產品,例如軟體開發包(Software Development Kit,SDK)等等。Wherein, the above-mentioned computer program product can be realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

所屬領域的技術人員可以清楚地瞭解到,為描述的方便和簡潔,上述描述的系統和裝置的具體工作過程,可以參考前述方法實施例中的對應過程,在此不再贅述。在本公開所提供的幾個實施例中,應該理解到,所揭露的系統、裝置和方法,可以通過其它的方式實現。以上所描述的裝置實施例僅僅是示意性的,例如,所述單元的劃分,僅僅為一種邏輯功能劃分,實際實現時可以有另外的劃分方式,又例如,多個單元或組件可以結合或者可以集成到另一個系統,或一些特徵可以忽略,或不執行。另一點,所顯示或討論的相互之間的耦合或直接耦合或通訊連接可以是通過一些通訊介面,裝置或單元的間接耦合或通訊連接,可以是電性,機械或其它的形式。Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

所述作為分離部件說明的單元可以是或者也可以不是物理上分開的,作為單元顯示的部件可以是或者也可以不是物理單元,即可以位於一個地方,或者也可以分佈到多個網路單元上。可以根據實際的需要選擇其中的部分或者全部單元來實現本實施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

另外,在本公開各個實施例中的各功能單元可以整合在一個處理單元中,也可以是各個單元單獨物理存在,也可以兩個或兩個以上單元整合在一個單元中。In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

所述功能如果以軟體功能單元的形式實現並作為獨立的產品銷售或使用時,可以儲存在一個處理器可執行的非揮發性的計算機可讀取儲存媒體中。基於這樣的理解,本公開的技術方案本質上或者說對現有技術做出貢獻的部分或者該技術方案的部分可以以軟體產品的形式體現出來,該計算機軟體產品儲存在一個儲存媒體中,包括若干指令用以使得一台計算機設備(可以是個人計算機,伺服器,或者網路設備等)執行本公開各個實施例所述方法的全部或部分步驟。而前述的儲存媒體包括:隨身碟、移動硬碟、唯讀記憶體(Read-Only Memory,ROM)、隨機存取記憶體(Random Access Memory,RAM)、磁碟或者光碟等各種可以儲存程式代碼的媒體。If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium, including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: flash drive, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. media.

最後應說明的是:以上所述實施例,僅為本公開的具體實施方式,用以說明本公開的技術方案,而非對其限制,本公開的保護範圍並不局限於此,儘管參照前述實施例對本公開進行了詳細的說明,本領域的普通技術人員應當理解:任何熟悉本技術領域的技術人員在本公開揭露的技術範圍內,其依然可以對前述實施例所記載的技術方案進行修改或可輕易想到變化,或者對其中部分技術特徵進行等同替換;而這些修改、變化或者替換,並不使相應技術方案的本質脫離本公開實施例技術方案的精神和範圍,都應涵蓋在本公開的保護範圍之內。因此,本公開的保護範圍應所述以申請專利範圍的保護範圍為准。Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the patent application.

S101:獲取目標場所內設置的多個採集設備在同一時刻採集的視頻畫面;其中,不同的採集設備在目標場所中的採集視角不同,視頻畫面中包括目標對象 S102:基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標 S103:對目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在目標場所中的目標位置坐標 S201:獲取多個採集設備在同一時刻分別採集的視頻畫面中的目標對象的像素坐標 S202:基於每個採集設備採集的視頻畫面中的目標對象的像素坐標和該採集設備的參數資訊,確定該採集設備採集的目標對象在目標場所對應的世界坐標系下的初始位置坐標 S301:基於多張視頻畫面確定的目標對象的初始位置坐標,確定與同一目標對象關聯的多個初始位置坐標 S302:將與同一目標對象關聯的多個初始位置坐標進行依次融合,得到同一目標對象在目標場所中的目標位置坐標 S401:基於目標場所中的各目標對象分別對應的目標位置坐標,以及預先設定的目標區域,確定是否存在進入目標區域的目標對象 S402:在確定存在進入目標區域的目標對象的情況下,進行預警提示 500:定位裝置500 501:獲取模組 502:確定模組 503:融合模組 600:電子設備 610:處理器 620:儲存器 621:記憶體 622:外部儲存器 630:匯流排 S101: Obtain the video images collected at the same time by a plurality of acquisition devices set in the target place; wherein, different acquisition devices have different acquisition angles of view in the target place, and the video images include the target object S102: Determine the initial position coordinates of the target object in the target place based on the video images collected by multiple acquisition devices at the same time S103: Fusing the initial position coordinates of the same target object in the target object to obtain the target position coordinates of the target object in the target place S201: Obtain the pixel coordinates of the target object in the video images collected by multiple acquisition devices respectively at the same time S202: Based on the pixel coordinates of the target object in the video frame collected by each collection device and the parameter information of the collection device, determine the initial position coordinates of the target object collected by the collection device in the world coordinate system corresponding to the target site S301: Determine a plurality of initial position coordinates associated with the same target object based on the initial position coordinates of the target object determined based on multiple video frames S302: sequentially fuse multiple initial position coordinates associated with the same target object to obtain the target position coordinates of the same target object in the target place S401: Determine whether there is a target object entering the target area based on the target position coordinates corresponding to each target object in the target place and the preset target area S402: When it is determined that there is a target object entering the target area, give an early warning prompt 500: positioning device 500 501: get module 502: Determine the module 503: fusion module 600: Electronic equipment 610: Processor 620: storage 621: memory 622: External storage 630: busbar

圖1示出了本公開實施例所提供的一種定位方法的流程圖。 圖2示出了本公開實施例所提供的一種確定目標對象的初始位置坐標的方法流程圖。 圖3示出了本公開實施例所提供的一種針對視頻畫面中檢測到的目標對象的示意圖。 圖4示出了本公開實施例所提供的一種確定目標對象的目標位置坐標的方法流程圖。 圖5示出了本公開實施例所提供的一種預警提示的方法流程圖。 圖6示出了本公開實施例所提供的一種定位裝置的結構示意圖。 圖7示出了本公開實施例所提供的一種電子設備的示意圖。 Fig. 1 shows a flow chart of a positioning method provided by an embodiment of the present disclosure. Fig. 2 shows a flow chart of a method for determining initial position coordinates of a target object provided by an embodiment of the present disclosure. Fig. 3 shows a schematic diagram of a target object detected in a video frame provided by an embodiment of the present disclosure. Fig. 4 shows a flowchart of a method for determining target position coordinates of a target object provided by an embodiment of the present disclosure. Fig. 5 shows a flow chart of a method for early warning provided by an embodiment of the present disclosure. Fig. 6 shows a schematic structural diagram of a positioning device provided by an embodiment of the present disclosure. Fig. 7 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

S101:獲取目標場所內設置的多個採集設備在同一時刻採集的視頻畫面;其中,不同的採集設備在目標場所中的採集視角不同,視頻畫面中包括目標對象 S101: Obtain the video images collected at the same time by a plurality of acquisition devices set in the target place; wherein, different acquisition devices have different acquisition angles of view in the target place, and the video images include the target object

S102:基於多個採集設備在同一時刻採集的視頻畫面,分別確定目標對象在目標場所中的初始位置坐標 S102: Determine the initial position coordinates of the target object in the target place based on the video images collected by multiple acquisition devices at the same time

S103:對目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在目標場所中的目標位置坐標 S103: Fusing the initial position coordinates of the same target object in the target object to obtain the target position coordinates of the target object in the target place

Claims (12)

一種定位方法,包括: 獲取目標場所內設置的多個採集設備在同一時刻採集的多張視頻畫面;其中,不同的採集設備在所述目標場所中的採集視角不同,所述多張視頻畫面中包括目標對象;其中,所述目標對象為所述目標場所內待進行定位的對象; 基於所述多張視頻畫面,分別確定所述目標對象在所述目標場所中的初始位置坐標; 對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標。 A positioning method, comprising: Acquiring multiple video images collected at the same time by a plurality of acquisition devices set in the target place; wherein different acquisition devices have different acquisition angles of view in the target place, and the plurality of video images include the target object; wherein, The target object is an object to be positioned in the target place; Based on the plurality of video images, respectively determine the initial position coordinates of the target object in the target place; The initial position coordinates of the same target object among the target objects are fused to obtain the target position coordinates of the target object in the target place. 如請求項1所述的定位方法,其特徵在於,基於所述多張視頻畫面,分別確定所述目標對象在所述目標場所中的初始位置坐標,包括: 獲取所述多張視頻畫面中所述目標對象的像素坐標; 針對所述多個採集設備裡的每一個,基於該採集設備採集的視頻畫面中所述目標對象中至少一者的像素坐標和該採集設備的參數資訊,確定該採集設備採集的所述目標對象中至少一者在所述目標場所對應的世界坐標系下的初始位置坐標。 The positioning method according to claim 1, wherein the initial position coordinates of the target object in the target place are respectively determined based on the plurality of video images, including: Obtaining the pixel coordinates of the target object in the plurality of video frames; For each of the plurality of collection devices, based on the pixel coordinates of at least one of the target objects in the video frame collected by the collection device and the parameter information of the collection device, determine the target object collected by the collection device The initial position coordinates of at least one of them in the world coordinate system corresponding to the target location. 如請求項2所述的定位方法,其特徵在於,獲取所述多張視頻畫面中的所述目標對象的像素坐標,包括: 將所述多張視頻畫面輸入預先訓練的神經網路, 針對所述多張視頻畫面中的每一張, 得到該視頻畫面中目標對象的檢測框; 提取該視頻畫面中目標對象的檢測框上的目標位置點在該視頻畫面中的像素坐標,得到該視頻畫面中目標對象的像素坐標。 The positioning method as described in claim 2, wherein obtaining the pixel coordinates of the target object in the plurality of video images includes: Inputting the multiple video frames into a pre-trained neural network, For each of the plurality of video frames, Obtain the detection frame of the target object in the video frame; Extracting the pixel coordinates of the target position point on the detection frame of the target object in the video frame in the video frame to obtain the pixel coordinates of the target object in the video frame. 如請求項2或3所述的定位方法,其特徵在於,基於該採集設備採集的所述視頻畫面中的所述目標對象中至少一者的像素坐標和該採集設備的參數資訊,確定該採集設備採集的所述目標對象中至少一者在所述目標場所對應的世界坐標系下的初始位置坐標,包括: 基於預先確定的該採集設備的內參矩陣和畸變參數,對該採集設備採集的視頻畫面中的所述目標對象中至少一者的像素坐標進行修正,得到該視頻畫面中的所述目標對象中至少一者的修正像素坐標; 基於預先確定的該採集設備的單應性矩陣和該採集設備採集的視頻畫面中的所述目標對象中至少一者的修正像素坐標,確定該視頻畫面中的所述目標對象中至少一者的初始位置坐標。 The positioning method as described in claim 2 or 3, wherein the acquisition is determined based on the pixel coordinates of at least one of the target objects in the video picture captured by the acquisition device and the parameter information of the acquisition device. The initial position coordinates of at least one of the target objects collected by the device in the world coordinate system corresponding to the target site include: Based on the predetermined internal reference matrix and distortion parameters of the acquisition device, the pixel coordinates of at least one of the target objects in the video picture collected by the acquisition device are corrected to obtain at least one of the target objects in the video picture. the corrected pixel coordinates of one; Based on the predetermined homography matrix of the acquisition device and the corrected pixel coordinates of at least one of the target objects in the video picture captured by the acquisition device, determine the at least one of the target objects in the video picture Initial position coordinates. 如請求項1至4任一所述的定位方法,其特徵在於,對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標,包括: 基於所述目標對象在所述目標場所中的所述初始位置坐標,確定與所述目標對象中同一目標對象關聯的多個初始位置坐標; 將與該目標對象關聯的所述多個初始位置坐標進行依次融合,得到該目標對象在所述目標場所中的目標位置坐標。 The positioning method according to any one of claim items 1 to 4, wherein the initial position coordinates of the same target object in the target objects are fused to obtain the target position coordinates of the target object in the target place, including : determining a plurality of initial position coordinates associated with the same one of the target objects based on the initial position coordinates of the target objects in the target location; The plurality of initial position coordinates associated with the target object are sequentially fused to obtain the target position coordinates of the target object in the target location. 如請求項5所述的定位方法,其特徵在於,將與該目標對象關聯的所述多個初始位置坐標進行依次融合,得到該目標對象在所述目標場所中的目標位置坐標,包括: 從該目標對象關聯的所述多個初始位置坐標中選取任一初始位置坐標,將選取的任一初始位置坐標作為第一中間融合位置坐標; 將所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標,將所述第二中間融合位置坐標作為更新後的所述第一中間融合位置坐標,並返回生成所述第二中間融合位置坐標的步驟,直到所述多個初始位置坐標中不存在待融合的初始位置坐標。 The positioning method as described in claim 5, wherein the multiple initial position coordinates associated with the target object are sequentially fused to obtain the target position coordinates of the target object in the target place, including: Select any initial position coordinate from the plurality of initial position coordinates associated with the target object, and use the selected initial position coordinate as the first intermediate fusion position coordinate; Fusing the first intermediate fusion position coordinates with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates, and using the second intermediate fusion position coordinates as the updated The coordinates of the first intermediate fusion position, and return to the step of generating the second intermediate fusion position coordinates until there is no initial position coordinate to be fused in the plurality of initial position coordinates. 如請求項6所述的定位方法,其特徵在於,將所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標進行融合,生成第二中間融合位置坐標,包括: 確定所述第一中間融合位置坐標與所述多個初始位置坐標中其它任一待融合的初始位置坐標的中點坐標,將該中點坐標作為所述第二中間融合位置坐標。 The positioning method according to claim 6, wherein the coordinates of the first intermediate fusion position are fused with any other initial position coordinates to be fused among the plurality of initial position coordinates to generate a second intermediate fusion position coordinates, including: Determine the midpoint coordinates of the first intermediate fusion position coordinates and any other initial position coordinates to be fused among the plurality of initial position coordinates, and use the midpoint coordinates as the second intermediate fusion position coordinates. 如請求項5至7任一所述的定位方法,其特徵在於,基於所述目標對象在所述目標場所中的所述初始位置坐標,確定與所述目標對象中同一目標對象關聯的多個初始位置坐標,包括: 針對所述多張視頻畫面中的任意兩張視頻畫面,將所述任意兩張視頻畫面中第一視頻畫面中的目標對象確定為第一目標對象,所述任意兩張視頻畫面中第二視頻畫面中的目標對象確定為第二目標對象; 在所述第一視頻畫面中確定每個所述第一目標對象的初始位置坐標; 在所述第二視頻畫面中確定每個所述第二目標對象的初始位置坐標; 針對每個所述第一目標對象的初始位置坐標, 確定該第一目標對象的初始位置坐標與每個所述第二目標對象的初始位置坐標之間的距離; 確定與該第一目標對象具有最小距離的第二目標對象與該第一目標對象為同一目標對象,其中,所述最小距離小於預設融合距離閾值; 將該第一目標對象的初始位置坐標和與該第一目標對象具有最小距離的第二目標對象的初始位置坐標,作為與所述目標對象中同一目標對象關聯的多個初始位置坐標。 The positioning method according to any one of claims 5 to 7, characterized in that, based on the initial position coordinates of the target object in the target place, multiple Initial location coordinates, including: For any two video frames in the plurality of video frames, the target object in the first video frame in the any two video frames is determined as the first target object, and the second video frame in the any two video frames is determined as the first target object. The target object in the picture is determined as the second target object; determining the initial position coordinates of each of the first target objects in the first video frame; determining the initial position coordinates of each of the second target objects in the second video frame; For the initial position coordinates of each of the first target objects, determining the distance between the initial position coordinates of the first target object and the initial position coordinates of each of the second target objects; determining that a second target object having a minimum distance from the first target object is the same target object as the first target object, wherein the minimum distance is smaller than a preset fusion distance threshold; The initial position coordinates of the first target object and the initial position coordinates of the second target object having the smallest distance from the first target object are used as a plurality of initial position coordinates associated with the same target object among the target objects. 如請求項1至8任一所述的定位方法,其特徵在於,在得到該目標對象在所述目標場所中的目標位置坐標之後,所述定位方法還包括: 基於所述目標場所中的各所述目標對象分別對應的目標位置坐標,以及預先設定的目標區域,確定是否存在進入所述目標區域的目標對象; 在確定存在進入所述目標區域的目標對象的情況下,進行預警提示。 The positioning method according to any one of claim items 1 to 8, wherein, after obtaining the target position coordinates of the target object in the target place, the positioning method further includes: determining whether there is a target object entering the target area based on the target position coordinates corresponding to each of the target objects in the target place and the preset target area; When it is determined that there is a target object entering the target area, an early warning prompt is given. 一種定位裝置,包括: 獲取模組,用於獲取目標場所內設置的多個採集設備在同一時刻採集的多張視頻畫面;其中,不同的採集設備在所述目標場所中的採集視角不同,所述多張視頻畫面中包含目標對象;其中,所述目標對象為所述目標場所內待進行定位的對象; 確定模組,用於基於所述多張視頻畫面,分別確定所述目標對象在所述目標場所中的初始位置坐標; 融合模組,用於對所述目標對象中同一目標對象的初始位置坐標進行融合,得到該目標對象在所述目標場所中的目標位置坐標。 A positioning device, comprising: The acquisition module is used to acquire multiple video images collected at the same time by a plurality of acquisition devices set in the target place; where different acquisition devices have different acquisition angles of view in the target place, and among the plurality of video images Including a target object; wherein, the target object is an object to be positioned in the target place; A determining module, configured to determine the initial position coordinates of the target object in the target place based on the plurality of video images; The fusion module is used to fuse the initial position coordinates of the same target object among the target objects to obtain the target position coordinates of the target object in the target place. 一種電子設備,包括:處理器、儲存器和匯流排,所述儲存器儲存有所述處理器可執行的機器可讀指令,當電子設備運行時,所述處理器與所述儲存器之間通過匯流排通訊,所述機器可讀指令被所述處理器執行時執行如請求項1至9任一所述的定位方法的步驟。An electronic device, comprising: a processor, a memory, and a bus bar, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the connection between the processor and the memory Through bus communication, when the machine-readable instructions are executed by the processor, the steps of the positioning method described in any one of claims 1 to 9 are executed. 一種計算機可讀儲存媒體,該計算機可讀儲存媒體上儲存有計算機程式,該計算機程式被處理器運行時執行如請求項1至9任一所述的定位方法的步驟。A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the positioning method described in any one of claims 1 to 9 are executed.
TW111105018A 2021-04-28 2022-02-11 Positioning method and apparatus, electronic device and storage medium TW202242803A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110467657.9A CN113129378A (en) 2021-04-28 2021-04-28 Positioning method, positioning device, electronic equipment and storage medium
CN202110467657.9 2021-04-28

Publications (1)

Publication Number Publication Date
TW202242803A true TW202242803A (en) 2022-11-01

Family

ID=76781086

Family Applications (1)

Application Number Title Priority Date Filing Date
TW111105018A TW202242803A (en) 2021-04-28 2022-02-11 Positioning method and apparatus, electronic device and storage medium

Country Status (3)

Country Link
CN (1) CN113129378A (en)
TW (1) TW202242803A (en)
WO (1) WO2022227462A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113129378A (en) * 2021-04-28 2021-07-16 北京市商汤科技开发有限公司 Positioning method, positioning device, electronic equipment and storage medium
CN117541910A (en) * 2023-10-27 2024-02-09 北京市城市规划设计研究院 Fusion method and device for urban road multi-radar data

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10310054B2 (en) * 2014-03-21 2019-06-04 The Boeing Company Relative object localization process for local positioning system
CN107015193B (en) * 2017-04-18 2019-10-11 中国矿业大学(北京) A kind of binocular CCD vision mine movable object localization method and system
CN110163914B (en) * 2018-08-01 2021-05-25 京东方科技集团股份有限公司 Vision-based positioning
CN110210446A (en) * 2019-06-12 2019-09-06 广东工业大学 A kind of sitting posture condition detection method, device, equipment and the medium of target object
CN110363179B (en) * 2019-07-23 2022-03-25 联想(北京)有限公司 Map acquisition method, map acquisition device, electronic equipment and storage medium
CN111380502B (en) * 2020-03-13 2022-05-24 商汤集团有限公司 Calibration method, position determination method, device, electronic equipment and storage medium
CN112653848B (en) * 2020-12-23 2023-03-24 北京市商汤科技开发有限公司 Display method and device in augmented reality scene, electronic equipment and storage medium
CN113129378A (en) * 2021-04-28 2021-07-16 北京市商汤科技开发有限公司 Positioning method, positioning device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN113129378A (en) 2021-07-16
WO2022227462A1 (en) 2022-11-03

Similar Documents

Publication Publication Date Title
CN108629791B (en) Pedestrian tracking method and device and cross-camera pedestrian tracking method and device
KR20200040665A (en) Systems and methods for detecting a point of interest change using a convolutional neural network
CN102833478B (en) Fault-tolerant background model
CN109416744A (en) Improved camera calibration system, target and process
Chen et al. Indoor camera pose estimation via style‐transfer 3D models
CN113129339B (en) Target tracking method and device, electronic equipment and storage medium
Tang et al. ESTHER: Joint camera self-calibration and automatic radial distortion correction from tracking of walking humans
WO2010101227A1 (en) Device for creating information for positional estimation of matter, method for creating information for positional estimation of matter, and program
TW202242803A (en) Positioning method and apparatus, electronic device and storage medium
CN103246044A (en) Automatic focusing method, automatic focusing system, and camera and camcorder provided with automatic focusing system
JP2023015989A (en) Item identification and tracking system
CN109313805A (en) Image processing apparatus, image processing system, image processing method and program
CN109313806A (en) Image processing apparatus, image processing system, image processing method and program
CN115376109B (en) Obstacle detection method, obstacle detection device, and storage medium
Nousias et al. Large-scale, metric structure from motion for unordered light fields
WO2023284358A1 (en) Camera calibration method and apparatus, electronic device, and storage medium
JP2009245207A (en) Probability distribution constructing method, probability distribution constructing device, probability distribution constructing program, photographic subject detecting method, photographic subject detecting device and photographic subject detecting program
Angladon et al. The toulouse vanishing points dataset
CN113763419B (en) Target tracking method, device and computer readable storage medium
CN109816628A (en) Face evaluation method and Related product
US20190095754A1 (en) Method and system for object tracking in multiple non-linear distortion lenses
CN103903269B (en) The description method and system of ball machine monitor video
US11166005B2 (en) Three-dimensional information acquisition system using pitching practice, and method for calculating camera parameters
WO2022174603A1 (en) Pose prediction method, pose prediction apparatus, and robot
JP2005031044A (en) Three-dimensional error measuring device