WO2020024845A1 - 定位方法和装置 - Google Patents
定位方法和装置 Download PDFInfo
- Publication number
- WO2020024845A1 WO2020024845A1 PCT/CN2019/097248 CN2019097248W WO2020024845A1 WO 2020024845 A1 WO2020024845 A1 WO 2020024845A1 CN 2019097248 W CN2019097248 W CN 2019097248W WO 2020024845 A1 WO2020024845 A1 WO 2020024845A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera
- coordinate system
- coordinates
- world coordinate
- image
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Definitions
- the present disclosure relates to the field of positioning technology, and in particular, to a positioning method, apparatus, and device, and a computer-readable storage medium.
- the methods used for indoor positioning mainly include vision-based positioning methods, and vision-based positioning methods mainly include SLAM (simultaneous localization and mapping) positioning methods and deep learning-based positioning methods.
- vision-based positioning methods mainly include SLAM (simultaneous localization and mapping) positioning methods and deep learning-based positioning methods.
- the vision odometer-based positioning method is a SLAM-based positioning method.
- the initial coordinates of the camera in the world coordinate system need to be determined in advance, and then positioning is performed by frame matching based on the initial coordinates and a continuous image sequence.
- the visual odometer-based positioning method also needs to eliminate the cumulative error every interval of a preset time.
- the scene-based end-to-end positioning method is a deep learning-based positioning method.
- the scene-based positioning method directly obtains the three-dimensional pose of the camera in the world coordinate system based on the scene of the image captured by the camera.
- a positioning method including:
- real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
- the method further includes:
- obtaining the initial coordinates of the camera in the world coordinate system based on the first coordinates includes:
- the first coordinate is used as an initial coordinate of the camera in a world coordinate system.
- the method further includes:
- obtaining the initial coordinates of the camera in the world coordinate system based on the first coordinates includes:
- the second coordinate is used as an initial coordinate of the camera in a world coordinate system.
- the obtaining the second coordinate of the camera in the world coordinate system based on the marker and the first coordinate includes:
- a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
- the detecting whether a marker is included in the image includes:
- Whether a marker is included in the image is detected based on a single-shot multi-frame detector algorithm.
- the method further includes:
- the real-time coordinates of the camera in the world coordinate system are determined through frame matching.
- the updating the initial coordinates of the camera in the world coordinate system includes:
- the initial coordinates of the camera in the world coordinate system are obtained.
- the deep learning model includes a deep learning model based on a PoseNet network.
- a positioning device including:
- An image acquirer configured to acquire an image captured by a camera
- An initial coordinate determiner coupled to the image acquirer and configured to obtain a first coordinate of the camera in a world coordinate system from the image based on a deep learning model, and obtain the first coordinate based on the first coordinate The camera's initial coordinates in the world coordinate system;
- the real-time position determiner is coupled with the image acquirer and the initial coordinate determiner, and is configured to determine the camera in the world coordinate system through frame matching based on the initial coordinates of the camera in the world coordinate system. Real-time coordinates.
- the initial coordinate determiner is further configured to detect whether a landmark is included in the image, wherein the landmark has at least 3 vertices;
- the first coordinate is used as an initial coordinate of the camera in a world coordinate system.
- the initial coordinate determiner is further configured to: detect whether the image includes a marker, wherein the marker has at least 3 vertices;
- the second coordinate is used as an initial coordinate of the camera in a world coordinate system.
- the initial coordinate determiner is further configured to:
- a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
- the initial coordinate determiner is further configured to detect whether a marker is included in the image based on a single-shot multi-frame detector algorithm.
- the initial coordinate determiner is further configured to:
- the real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
- the initial coordinate determiner is further configured to:
- An initial coordinate of the camera in a world coordinate system is obtained based on at least one of a scene and a landmark included in an image acquired by the image acquirer in real time.
- the deep learning model includes a deep learning model based on a PoseNet network.
- a positioning device including:
- One or more processors are One or more processors.
- a memory coupled to the one or more processors and storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors Perform the method.
- a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by one or more processors, cause the one or more Processors execute the method.
- FIG. 1 shows a schematic diagram of a positioning device according to an embodiment of the present disclosure
- FIG. 2 is a schematic diagram showing a flow of a positioning method according to an embodiment of the present disclosure
- FIG. 3 is a schematic diagram showing a flow of a positioning method according to an embodiment of the present disclosure.
- FIG. 4 shows a schematic diagram of a positioning device according to an embodiment of the present disclosure.
- Visual odometer-based positioning methods are virtually unaffected by environmental changes.
- the positioning method based on the visual odometer needs to obtain the initial coordinates of the camera in the world coordinate system, and eliminate the cumulative error every interval.
- the acquisition of the camera's initial coordinates in the world coordinate system and the elimination of accumulated errors require the use of a hardware device such as a WIFI module or Bluetooth, which adds additional hardware costs.
- the scene-based end-to-end positioning method is robust to environmental changes, but its positioning accuracy is relatively low.
- One of the objectives of the embodiments of the present disclosure is to provide a positioning method that satisfies the required positioning accuracy.
- FIG. 1 shows a schematic diagram of a positioning device 110 according to an embodiment of the present disclosure.
- the device 110 and the camera 120 may be coupled to each other. They communicate via wireless technologies such as WIFI, Bluetooth, and so on.
- wireless technologies such as WIFI, Bluetooth, and so on.
- the positioning device 110 and the camera 120 are shown as separate devices in FIG. 1, according to an embodiment of the present disclosure, the positioning device 110 and the camera 120 may be integrated together.
- the camera 120 is configured to capture an image and send the captured image to the positioning device 110.
- the camera 120 may capture and send images in real time or at preset time intervals.
- the camera 120 may be any device capable of capturing an image. In a practical application, for example, the camera 120 may be fixed on a shopping cart and move with the shopping cart to capture multiple images in a real scene.
- the location of the camera 120 is essentially the location of the shopping cart or the user.
- the positioning device 110 is coupled to the camera 120 and is configured to acquire or receive images from the camera 120.
- the positioning device 110 may be any positioning device such as a mobile phone, a computer, a server, and a netbook.
- the positioning device 110 may include an image acquirer 101, an existence determiner 102, an initial coordinate determiner 103, and a real-time position determiner 104.
- the image acquirer 101 may be configured to acquire or receive an image in real time from the camera 120, for example, and send the acquired or received image to the presence determiner 102 and / or the initial coordinate determiner 103 and / or the real-time position determiner 104.
- the presence determiner 102 may be coupled with the image acquirer 101, and may be configured to determine, for example, whether the initial coordinates of the camera 120 in the world coordinate system are included in the memory based on the image captured by the camera 120, and send the determined result Gives the initial coordinate determiner 103.
- the initial coordinate determiner 103 may be coupled with the image acquirer 101 and the presence determiner 102.
- the initial coordinate determiner 103 may obtain or receive an image from the image acquirer 101 and obtain or receive a result of the determination from the existence determiner 102.
- the initial coordinate determiner 103 may be configured to obtain the current coordinates of the camera 120 in the world coordinate system from the acquired current image based on the deep learning model in response to determining that the initial coordinates of the camera in the world coordinate system are not included in the memory, The initial coordinates of the camera 120 in the world coordinate system are obtained based on the current coordinates.
- the current coordinates may be used as the initial coordinates of the camera 120 in the world coordinate system, and may also be used as the current position of the camera 120.
- the deep learning model may include a deep learning model based on a PoseNet network.
- the input of the PoseNet network can be RGB images, and the output can be the 3D pose of the camera.
- a real scene can be continuously collected for learning as needed.
- the initial coordinate determiner 103 may be configured to save the initial coordinates of the camera 120 in the world coordinate system to a memory and the initial coordinates Send to real-time position determiner 104.
- the real-time position determiner 104 may be coupled with the image acquirer 101 and the initial coordinate determiner 103.
- the real-time position determiner 104 may obtain or receive an image from the image acquirer 101 and obtain or receive the saved initial coordinates from the initial coordinate determiner 103.
- the real-time position determiner 104 may determine the real-time coordinates of the camera in the world coordinate system through inter-frame matching based on the current initial coordinates of the camera in the world coordinate system obtained or received from the initial coordinate determiner 103.
- the real-time position determiner 104 may be configured to continuously receive an image sequence from the image acquirer 101, and determine that the camera is under world coordinates through inter-frame matching based on the initial coordinates obtained or received from the initial coordinate determiner 103.
- the initial coordinates of the camera and the coordinates obtained in real time may be fused by Kalman filtering, and the result of the fusion may be better than the initial coordinates and the coordinates obtained in real time.
- the implementation of obtaining the coordinates of the camera in the world coordinate system from the image based on the deep learning model and determining the real-time coordinates of the camera in the world coordinate system through frame matching based on the initial coordinates of the camera in the world coordinate system may be implemented. It is known from the related art that it will not be described in detail here.
- the positioning method according to an embodiment of the present disclosure is completely based on vision without requiring any additional hardware devices such as WiFi and Bluetooth, thereby saving hardware costs.
- a portion according to an embodiment of the present disclosure is based on a visual odometer, and thus a relatively high positioning accuracy can be satisfied.
- the initial coordinate determiner 103 may be further configured to detect whether the image includes a landmark, wherein the landmark has at least three vertices. In such a case, the initial coordinate determiner 103 may be further configured to, in response to the landmark not being included in the image, use the current coordinates of the camera obtained from the scene of the image in the world coordinate system as Initial coordinates of the camera in the world coordinate system; and in response to detecting that the image includes the landmark, obtaining the camera's world coordinates based on the landmark and the current coordinates of the camera obtained from the scene of the image in the world coordinate system Another coordinate in the system; and using the other coordinate as an initial coordinate of the camera in the world coordinate system.
- obtaining another coordinate of the camera in the world coordinate system based on the landmark and the current coordinates of the camera obtained from the scene of the image in the world coordinate system may include: obtaining the vertex of the landmark on the camera based on the image Coordinates in the coordinate system; based on the current coordinates of the camera in the world coordinate system obtained from the scene of the image, the landmarks included in the image are determined from the coordinates of the vertices of multiple landmarks stored in the world coordinate system in advance The coordinates of the vertex of the object in the world coordinate system; based on the coordinates of the vertex of the marker included in the image in the camera coordinate system and the coordinates of the vertex of the marker included in the image in the world coordinate system The coordinates obtain a rotation matrix and a translation matrix of the camera coordinate system relative to the world coordinate system; and determine the coordinates of the camera in the world coordinate system based on the rotation matrix and the translation matrix.
- the markers can be arranged around the mall at certain space intervals so that the camera can capture the markers from time to time during the journey.
- the marker may be any shape having at least three vertices, such as a triangle, rectangle, trapezoid, diamond, pentagon, and so on. Because the markers are arranged in advance, the coordinates of each vertex of the marker in the world coordinate system can be determined in advance, and these coordinates are stored in the memory to determine the real-time position of the camera in the world coordinate system.
- R is the rotation matrix of the camera relative to the world coordinate system
- ⁇ is the yaw angle of the camera rotating around the Z axis of the world coordinate system
- ⁇ is the pitch angle of the camera rotating around the Y axis of the world coordinate system
- ⁇ is The roll angle of the camera around the X axis of the world coordinate system
- t is the translation of the camera relative to the world coordinate system
- ⁇ Z is the coordinate of the camera on the Z axis of the world coordinate system
- ⁇ X is on the X axis of the world coordinate system
- Y is the coordinate on the Y axis of the world coordinate system.
- SSD Single Shot MultiBox Detector
- the background is removed from the image, the detected marker is cropped, and line features are extracted to obtain each vertex of the marker, so that the pixel coordinates of each vertex of the marker can be obtained.
- the corresponding depth data of each vertex can be directly obtained by the camera (such as a binocular camera or a depth camera) itself (internal parameters of the camera), and then the coordinates of each vertex of the detected landmark in the camera coordinates are determined according to the pinhole camera model.
- the coordinates Pc (Xc i , Yc i , Zc i ), i ⁇ 3 and so on.
- a coordinate set of each vertex of the marker in the camera coordinate system can be obtained (wherein the marker includes at least three vertices, the coordinate set includes at least 3 coordinates).
- each of the vertices of the detected marker coordinates corresponding to the world coordinate system (P (X i, Y i, Z i), i ⁇ 3) previously stored in the memory, so they Is known.
- the heights ⁇ Z, ⁇ elevation angles, and roll angles ⁇ of the cameras are also known. In this case, only the three parameters of the yaw angle ⁇ , ⁇ X and ⁇ Y in the rotation matrix R and the translation matrix T are unknown.
- the coordinates of each vertex of the marker in the camera coordinate system and its corresponding coordinates in the world coordinate system have been obtained, and the number of vertices is at least three, so the bias can be obtained according to formula (1) Heading angles ⁇ , ⁇ X and ⁇ Y.
- a rotation matrix R and a translation matrix T can be obtained, thereby obtaining the current coordinates of the camera in the world coordinate system.
- the coordinates of a plurality of markers and their corresponding vertices are stored in the memory in advance. Therefore, when a marker is detected from the current image, it is necessary to find the detected marker from a plurality of markers stored in the memory, and then find the coordinates of its corresponding vertex in the world coordinate system.
- the current coordinates (Xs, Ys, Zs) of the camera in the world coordinate system may be obtained from the current image based on the deep learning model (such as the PoseNet network) by scene-based positioning. Then based on the coordinates (Xs, Ys, Zs) of the camera in the world coordinate system, a marker closest to the coordinates is searched from the memory.
- the distribution positions of the markers are relatively scattered, such as arranging one at several tens of meters, at the camera position where the marker is detected, the closest marker to the camera is unique (that is, no marker is detected and there are two equal Markers of distance), and then find the coordinates of each vertex of the marker in the world coordinate system stored in advance. Since there is a certain error based on scene positioning (assuming the maximum positioning error is d), the coordinate range of the camera obtained based on scene positioning may be (Xs ⁇ d, Ys ⁇ d, Zs ⁇ d).
- positioning is performed based on the marker to determine the initial position of the camera in the world coordinate system. This can further improve the accuracy of positioning.
- the initial coordinate determiner 103 may be further configured to update the initial coordinates of the camera 120 in the world coordinate system at preset time intervals.
- the real-time position determiner 104 may be configured to determine the real-time coordinates of the camera in the world coordinate system through inter-frame matching based on the initial coordinates of the updated camera in the world coordinate system.
- updating the initial coordinates of the camera in the world coordinate system may include the initial coordinate determiner 103 re-obtaining the camera based on at least one of a scene and a landmark included in the image obtained in real time from the image acquirer 101 The initial coordinates in the world coordinate system.
- a timer may be set for the initial coordinates.
- the timer (preset time interval) may be, for example, 30 seconds, or any other suitable time.
- the initial coordinates of the stored camera in the world coordinate system are kept unchanged, and continue to be obtained from the image sequence through inter-frame matching based on the stored initial coordinates.
- the real-time coordinates of the camera in the world coordinate system When the initial coordinates have passed the time period set by the timer, the acquisition of the initial coordinates of the camera in the world coordinate system according to the embodiment of the present disclosure may be triggered again.
- acquisition of a current image, detection of a marker in a captured current image may be triggered, and then a camera is determined based on a scene of the current image or a marker of the current image as described above.
- the current coordinates in the world coordinate system, and the current coordinates of the previously stored camera in the world coordinate system are updated or replaced with the current coordinates of the determined camera in the world coordinate system, so that the determined camera is in the world coordinate system.
- the current coordinates are used as the new initial coordinates of the camera in the world coordinate system, and then the real-time coordinates of the camera in the world coordinate system are determined through frame matching based on the updated initial coordinates of the camera in the world coordinate system. While using the determined current coordinates of the camera in the world coordinate system as the new initial coordinates of the camera in the world coordinate system, a timer is set again, and so on.
- the stored camera's initial coordinates in the world coordinate system are updated at preset time intervals, the accumulated positioning error can be eliminated, thereby further improving the positioning accuracy.
- FIG. 2 is a schematic diagram illustrating a flow of a positioning method 200 according to an embodiment of the present disclosure.
- the method 200 may be performed, for example, by a positioning device 110 as shown in FIG. 1.
- an image captured by the camera may be acquired, for example, a current image captured by the camera 120 is received from the camera 120.
- a first coordinate (ie, current coordinate) of the camera in the world coordinate system is obtained from the image (current image) based on the deep learning model.
- the deep learning model includes a deep learning model based on a PoseNet network.
- the initial coordinates of the camera in the world coordinate system are obtained based on the first coordinates (current coordinates) of the camera in the world coordinate system.
- the first coordinate (current coordinate) of the camera in the world coordinate system can be used as the initial coordinate of the camera in the world coordinate system.
- step 204 the current (real-time) coordinates of the camera in the world coordinate system are determined through inter-frame matching based on the initial coordinates of the camera in the world coordinate system. Since then, the image has been repeatedly received, and the current (real-time) coordinates of the camera in the world coordinate system are determined by inter-frame matching based on the stored camera's initial coordinates in the world coordinate system.
- the coordinates of the camera in the world coordinate system obtained through inter-frame matching can be obtained from the visual odometer-based positioning method, which will not be described in detail here.
- the method 200 in FIG. 2 may further include a step of determining whether the initial coordinates of the camera in the world coordinate system are included in the memory. In response to determining that the initial coordinates of the camera in the world coordinate system are already included in the memory, proceed directly to step 203.
- FIG. 3 illustrates a flowchart of a positioning method 300 according to an embodiment of the present disclosure.
- the method 300 may be performed, for example, by a positioning device 110 as shown in FIG. 1.
- an image captured by the camera may be acquired, for example, a current image captured by the camera 120 is received from the camera 120.
- a first coordinate (ie, current coordinate) of the camera in the world coordinate system is obtained from the image (current image) based on the deep learning model.
- the deep learning model includes a deep learning model based on a PoseNet network.
- step 303 it can be detected whether the acquired image includes a marker.
- the marker has at least 3 vertices.
- step 304 is performed in response to detecting that the image includes the marker.
- the second coordinate of the camera in the world coordinate system is obtained based on the marker and the first coordinate (current coordinate) of the camera in the world coordinate system determined in step 302, and the camera is in the world coordinate system.
- the initial coordinates In one embodiment, coordinates of a vertex of the landmark in a camera coordinate system are obtained based on the image. The coordinates of the vertices of the landmarks included in the image in the world coordinate system are determined based on the coordinates of the vertices of the plurality of landmarks stored in advance in the world coordinate system.
- a second coordinate of the camera in a world coordinate system is determined based on the rotation matrix and the translation matrix.
- step 305 in response to detecting that the image does not include the marker, step 305 is performed.
- the first coordinate (current coordinate) of the camera in the world coordinate system obtained from the acquired image is directly used as the initial coordinate of the camera in the world coordinate system.
- step 306 based on the initial coordinates of the camera in the world coordinate system, real-time coordinates of the camera in the world coordinate system are determined through inter-frame matching.
- the method 300 may further include, in response to determining that the initial coordinates of the camera in the world coordinate system are included in the memory, determining whether the initial coordinates have passed a preset period of time. Further, in response to determining that the initial coordinates have not experienced the preset time period, the real-time coordinates of the camera are determined from the camera based on the current initial coordinates through inter-frame matching. In response to determining that the initial coordinates have passed the preset time period, step 301 is performed.
- the initial coordinates of the camera in the world coordinate system are updated based on the scene of the image or the landmarks of the image by determining whether a landmark is included in the image, so that based on the updated initial coordinates of the camera in the world coordinate system, Inter-frame matching determines the real-time coordinates of the camera in the world coordinate system.
- the initial coordinates of the camera in the world coordinate system are continuously updated at preset time intervals, and the real-time coordinates of the camera in the world coordinate system are determined through frame matching based on the updated initial coordinates of the camera in the world coordinate system. Updating the initial coordinates of the camera in the world coordinate system may include executing method 300 once.
- FIG. 4 shows a schematic diagram of a positioning device 400 according to an embodiment of the present disclosure.
- the positioning device 400 may be used to implement the method of FIG. 2 or FIG. 3.
- the positioning device 400 is only one example of a suitable positioning device and is not intended to suggest any limitation as to the scope of use or functionality of the presently disclosed subject matter.
- the components of the positioning device 400 may include, but are not limited to, a processor 401, a memory 402, and a bus 403 that couples various system components including the memory to the processor 401.
- the bus 403 may be any one of several types of bus structures including a memory bus or a memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- bus architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and also known as mezzanine bus Peripheral Component Interconnect (PCI) bus.
- the positioning device 400 may include a variety of computer-readable media.
- Computer-readable media can be any available media that can be accessed by the positioning device 400 and includes both volatile and nonvolatile media, removable and non-removable media.
- Computer-readable media may include computer-readable storage media and communication media.
- Computer-readable storage media includes volatile and nonvolatile and removable and non-volatile implementations using any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data Removable media.
- Computer-readable storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, cartridges Tape, magnetic tape, magnetic disk storage, or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by the positioning device 400.
- Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and includes any information delivery media.
- modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner that information is encoded in the signal.
- communication media includes: wired media such as a wired network or direct-wired connection; and wireless media such as RF and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.
- the memory 402 may include computer storage media in the form of volatile and / or non-volatile memory such as ROM and RAM.
- a basic input / output system (BIOS) containing basic routines such as helping to transfer information between elements within the positioning device 400 during startup is typically stored in ROM.
- the RAM typically contains data and / or program modules that are immediately accessible to the processor 401 and / or are currently being manipulated by the processor 401.
- the data that can be stored in the memory 402 shown in FIG. 4 may include a BIOS, an operating system, an application program, other program modules, and program data.
- the positioning device 400 may also include other removable / non-removable, volatile / nonvolatile computer storage media.
- removable / non-removable, volatile / non-volatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, cartridges, flash cards, DVDs, digital video tapes, solid state RAM, Solid-state ROM, etc.
- the computer storage medium discussed above provides the positioning device 400 with storage for computer-executable instructions, data structures, program modules, and other data.
- the user can input commands and information into the pointing device 400 through input devices such as a keyboard and a pointing device commonly referred to as a mouse, trackball, or touchpad.
- Other input devices may include a microphone, a joystick, a game pad, a satellite dish, a scanner, and the like.
- I / O user input-output
- a monitor or other type of display device may be connected to the bus 403 via a user input-output (I / O) interface 14 such as a video interface.
- the positioning device 400 may also be connected to other peripheral output devices such as speakers and printers through a user input-output (I / O) interface 14.
- the positioning device 400 may be connected to one or more remote computers via the network interface 15.
- the remote computer may be a personal computer, server, router, network PC, peer-to-peer device, or other common network node, and typically includes many or all of the elements described above with respect to the positioning device 400.
- An embodiment of the present disclosure also provides a computer-readable storage medium on which computer-executable instructions are stored.
- the computer instructions are executed on one or more processors, the one or more processors execute the instructions according to the present invention.
- the methods and functions of the embodiments are disclosed.
- the computer-readable medium may include any one of the computer-readable media described above.
- An embodiment of the present disclosure also provides a computer program product.
- the instructions in the computer program product are executed by a processor, the method according to the embodiment of the present disclosure can be implemented.
- first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as “first” and “second” may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality” is at least two, for example, two, three, etc., unless it is specifically and specifically defined otherwise.
- Any process or method description in a flowchart or otherwise described herein can be understood as representing a module, fragment, or portion of code that includes one or more executable instructions for implementing steps of a custom logic function or process
- the scope of the embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed (including in a substantially simultaneous manner or in the reverse order according to the functions involved), which should be performed by It is understood by those skilled in the art to which the embodiments of the present disclosure belong.
- Logic and / or steps represented in a flowchart or otherwise described herein, for example, a sequenced list of executable instructions that may be considered to implement a logical function, may be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (such as a computer-based system, a system that includes a processor, or another system that can fetch and execute instructions from an instruction execution system, device, or device) Or equipment.
- a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
- Computer-readable media may include, for example, the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memories (Random Access Memory), Read-only memory (Read Only Memory), erasable and editable read-only memory (Erasable, Programmable, Read Only Memory) or flash memory, fiber optic devices, and portable optical disc read-only memory (Compact Disc Read Only Memory).
- the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.
- portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
- multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
- a suitable instruction execution system For example, if it is implemented in hardware, it may be implemented in any one of the following technologies or a combination thereof known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a suitable combination Application-specific integrated circuits for logic gate circuits, Programmable Gate Arrays, Field Programmable Gate Arrays, etc.
- each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist separately physically, or two or more units may be integrated into one module.
- the above integrated modules may be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (18)
- 一种定位方法,包括:获取相机所捕获的图像;基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标,基于所述第一坐标获得所述相机在世界坐标系下的初始坐标;以及基于所述相机在世界坐标系下的初始坐标,通过帧间匹配确定所述相机在世界坐标系下的实时坐标。
- 根据权利要求1所述的方法,还包括:检测所述图像中是否包括标志物,其中所述标志物具有至少3个顶点,其中,所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括:响应于所述图像中不包括所述标志物,将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
- 根据权利要求1所述的方法,还包括:检测所述图像是否包括标志物,其中所述标志物具有至少3个顶点;其中,所述基于所述第一坐标获得所述相机在世界坐标系下的初始坐标包括:响应于所述图像中包括所述标志物,基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标;以及将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
- 根据权利要求3所述的方法,其中,所述基于所述标志物和所述第一坐标获得相机在世界坐标系下的第二坐标包括:基于所述图像获得所述标志物的顶点在相机坐标系下的坐标;基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标,确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标;基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标,获得相机坐标系相对于世界坐标系的旋转矩阵和 平移矩阵;以及基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
- 根据权利要求2或3所述的方法,其中,所述检测所述图像中是否包括标志物包括:基于单次多框检测器算法检测所述图像中是否包括标志物。
- 根据权利要求2或3所述的方法,还包括,以预设的时间间隔更新所述相机在世界坐标系下的初始坐标;以及基于更新的相机在世界坐标系下的初始坐标,通过帧间匹配确定相机在世界坐标系下的实时坐标。
- 根据权利要求6所述的方法,其中所述更新所述相机在世界坐标系下的初始坐标包括:基于实时获取的图像中包括的场景和标志物中的至少一个,获得相机在世界坐标系下的初始坐标。
- 根据权利要求1所述的方法,其中,所述深度学习模型包括基于PoseNet网络的深度学习模型。
- 一种基于定位装置,包括:图像获取器,被配置成获取相机所捕获的图像;初始坐标确定器,与所述图像获取器相耦接且被配置成:基于深度学习模型从所述图像获得所述相机在世界坐标系下的第一坐标,基于所述第一坐标获得所述相机在世界坐标系下的初始坐标:以及实时位置确定器,与图像获取器和所述初始坐标确定器相耦接,且被配置成:基于所述相机在世界坐标系下的初始坐标,通过帧间匹配确定所述相机在世界坐标系下的实时坐标。
- 根据权利要求9所述的装置,其中,所述初始坐标确定器还被配置成:检测所述图像中是否包括标志物,其中所述标志物具有至少3个顶点;以及响应于所述图像中不包括所述标志物,将所述第一坐标用作所述相机在世界坐标系下的初始坐标。
- 根据权利要求9所述的装置,其中,所述初始坐标确定器还被配置成:检测所述图像是否包括标志物,其中所述标志物具有至少3个顶点;响应于所述图像中包括所述标志物,基于所述标志物和所述第一坐标获得所述相机在世界坐标系下的第二坐标,以及将所述第二坐标用作所述相机在世界坐标系下的初始坐标。
- 根据权利要求11所述的装置,其中,所述初始坐标确定器还被配置成:基于所述图像获得所述标志物的顶点在相机坐标系下的坐标;基于所述第一坐标从预先存储的多个标志物的顶点在世界坐标系下的坐标,确定所述图像中包括的所述标志物的顶点在世界坐标系下的坐标;基于所述图像中包括的所述标志物的顶点在相机坐标系下的坐标以及所述图像中包括的所述标志物的顶点在世界坐标系下的坐标,获得相机坐标系相对于世界坐标系的旋转矩阵和平移矩阵;以及基于所述旋转矩阵和所述平移矩阵确定所述相机在世界坐标系下的第二坐标。
- 根据权利要求10或11所述的装置,其中,所述初始坐标确定器还被配置成:基于单次多框检测器算法检测所述图像中是否包括标志物。
- 根据权利要求10或11所述的装置,其中,所述初始坐标确定器还被配置成:以预设的时间间隔更新所述相机在世界坐标系下的初始坐标;以及基于更新的相机在世界坐标系下的初始坐标,通过帧间匹配确定相机在世界坐标系下的实时坐标。
- 根据权利要求14所述的装置,其中所述初始坐标确定器还被配置成:基于由图像获取器实时获取的图像中包括的场景和标志物中的至少一个,获得所述相机在世界坐标系下的初始坐标。
- 根据权利要求9所述的装置,其中,所述深度学习模型包括基于PoseNet网络的深度学习模型。
- 一种定位设备,包括:一个或多个处理器;以及存储器,与所述一个或多个处理器相耦接,并存储有计算机可执行指令,所述计算机可执行指令在由所述一个或多个处理器执行时使得所述一个或多个处理器执行如权利要求1-8中任一项所述的方法。
- 一种计算机可读存储介质,其上存储了计算机可执行指令,所述计算机可执行指令在由一个或多个处理器执行时,使所述一个或多个处理器执行如权利要求1-8中任一项所述的方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/635,036 US11100670B2 (en) | 2018-08-01 | 2019-07-23 | Positioning method, positioning device and nonvolatile computer-readable storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810862721.1A CN110163914B (zh) | 2018-08-01 | 2018-08-01 | 基于视觉的定位 |
CN201810862721.1 | 2018-08-01 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020024845A1 true WO2020024845A1 (zh) | 2020-02-06 |
Family
ID=67645162
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/097248 WO2020024845A1 (zh) | 2018-08-01 | 2019-07-23 | 定位方法和装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US11100670B2 (zh) |
CN (1) | CN110163914B (zh) |
WO (1) | WO2020024845A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113283496A (zh) * | 2021-05-21 | 2021-08-20 | 国网宁夏电力有限公司中卫供电公司 | 一种应用于配网不停电作业危险动作智能识别的方法 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111445531B (zh) * | 2020-03-24 | 2022-08-30 | 云南电网有限责任公司楚雄供电局 | 一种多目相机导航方法、装置、设备及存储介质 |
CN113129378A (zh) * | 2021-04-28 | 2021-07-16 | 北京市商汤科技开发有限公司 | 一种定位方法、装置、电子设备及存储介质 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140300637A1 (en) * | 2013-04-05 | 2014-10-09 | Nokia Corporation | Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system |
CN106485753A (zh) * | 2016-09-09 | 2017-03-08 | 奇瑞汽车股份有限公司 | 用于无人驾驶汽车的摄像机标定的方法和装置 |
CN106780699A (zh) * | 2017-01-09 | 2017-05-31 | 东南大学 | 一种基于sins/gps和里程计辅助的视觉slam方法 |
CN107438752A (zh) * | 2016-12-23 | 2017-12-05 | 深圳前海达闼云端智能科技有限公司 | 定位方法、终端和服务器 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9031809B1 (en) * | 2010-07-14 | 2015-05-12 | Sri International | Method and apparatus for generating three-dimensional pose using multi-modal sensor fusion |
CN102506830B (zh) * | 2011-11-21 | 2014-03-12 | 奇瑞汽车股份有限公司 | 视觉定位方法及装置 |
CN104881860B (zh) * | 2014-02-28 | 2019-01-08 | 国际商业机器公司 | 基于照片进行定位的方法和装置 |
ES2584554B2 (es) * | 2015-11-30 | 2017-06-13 | Defensya Ingeniería Internacional, S.L. | Sistema de detección de punta de la pértiga y boca de receptáculo, automatización progresiva del repostaje aéreo con botalón y procedimiento de repostaje |
CN106780609B (zh) * | 2016-11-28 | 2019-06-11 | 中国电子科技集团公司第三研究所 | 视觉定位方法和视觉定位装置 |
CN107742311B (zh) * | 2017-09-29 | 2020-02-18 | 北京易达图灵科技有限公司 | 一种视觉定位的方法及装置 |
CN107833236B (zh) * | 2017-10-31 | 2020-06-26 | 中国科学院电子学研究所 | 一种动态环境下结合语义的视觉定位***和方法 |
US11487351B2 (en) * | 2018-11-23 | 2022-11-01 | International Business Machines Corporation | Intelligent directing system in an internet of things (IoT) computing environment |
US10885666B2 (en) * | 2019-02-06 | 2021-01-05 | Ford Global Technologies, Llc | Hybrid metric-topological camera-based localization |
US20210082149A1 (en) * | 2020-11-24 | 2021-03-18 | Intel Corporation | Real-time calibration of wide-baseline outside-in multi-camera systems |
-
2018
- 2018-08-01 CN CN201810862721.1A patent/CN110163914B/zh active Active
-
2019
- 2019-07-23 WO PCT/CN2019/097248 patent/WO2020024845A1/zh active Application Filing
- 2019-07-23 US US16/635,036 patent/US11100670B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140300637A1 (en) * | 2013-04-05 | 2014-10-09 | Nokia Corporation | Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system |
CN106485753A (zh) * | 2016-09-09 | 2017-03-08 | 奇瑞汽车股份有限公司 | 用于无人驾驶汽车的摄像机标定的方法和装置 |
CN107438752A (zh) * | 2016-12-23 | 2017-12-05 | 深圳前海达闼云端智能科技有限公司 | 定位方法、终端和服务器 |
CN106780699A (zh) * | 2017-01-09 | 2017-05-31 | 东南大学 | 一种基于sins/gps和里程计辅助的视觉slam方法 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113283496A (zh) * | 2021-05-21 | 2021-08-20 | 国网宁夏电力有限公司中卫供电公司 | 一种应用于配网不停电作业危险动作智能识别的方法 |
CN113283496B (zh) * | 2021-05-21 | 2022-07-29 | 国网宁夏电力有限公司中卫供电公司 | 一种应用于配网不停电作业危险动作智能识别的方法 |
Also Published As
Publication number | Publication date |
---|---|
CN110163914A (zh) | 2019-08-23 |
CN110163914B (zh) | 2021-05-25 |
US20200380717A1 (en) | 2020-12-03 |
US11100670B2 (en) | 2021-08-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5950973B2 (ja) | フレームを選択する方法、装置、及びシステム | |
US10810734B2 (en) | Computer aided rebar measurement and inspection system | |
US9542745B2 (en) | Apparatus and method for estimating orientation of camera | |
US7554575B2 (en) | Fast imaging system calibration | |
WO2020024845A1 (zh) | 定位方法和装置 | |
US20150103183A1 (en) | Method and apparatus for device orientation tracking using a visual gyroscope | |
WO2021136386A1 (zh) | 数据处理方法、终端和服务器 | |
US9747516B2 (en) | Keypoint detection with trackability measurements | |
WO2021129305A1 (zh) | 光学动捕***中标定杆检测方法、装置、设备和存储介质 | |
US20180060682A1 (en) | Parallax minimization stitching method and apparatus using control points in overlapping region | |
US20140348380A1 (en) | Method and appratus for tracking objects | |
KR20130115332A (ko) | 증강 현실 표현을 위한 2차원 이미지 캡쳐 | |
CN110648363A (zh) | 相机姿态确定方法、装置、存储介质及电子设备 | |
US20170085656A1 (en) | Automatic absolute orientation and position | |
US10878577B2 (en) | Method, system and apparatus for segmenting an image of a scene | |
CN109754034A (zh) | 一种基于二维码的终端设备定位方法及装置 | |
WO2020114234A1 (zh) | 一种确定目标gps的方法及摄像机 | |
US10930068B2 (en) | Estimation apparatus, estimation method, and non-transitory computer-readable storage medium for storing estimation program | |
WO2022016909A1 (zh) | 获取Wi-Fi指纹空间分布的方法、装置和电子设备 | |
US9800773B2 (en) | Digital camera apparatus with dynamically customized focus reticle and automatic focus reticle positioning | |
US20210133995A1 (en) | Electronic devices, methods, and computer program products for controlling 3d modeling operations based on pose metrics | |
JP2006113832A (ja) | ステレオ画像処理装置およびプログラム | |
JP6591594B2 (ja) | 情報提供システム、サーバ装置、及び情報提供方法 | |
JP2014102805A (ja) | 情報処理装置、情報処理方法及びプログラム | |
WO2022205841A1 (zh) | 机器人导航方法、装置、终端设备及计算机可读存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19843938 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19843938 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15.10.2021) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19843938 Country of ref document: EP Kind code of ref document: A1 |