WO2018228353A1 - Control method and apparatus for synchronous exposure of multi-camera system, and terminal device - Google Patents

Control method and apparatus for synchronous exposure of multi-camera system, and terminal device Download PDF

Info

Publication number
WO2018228353A1
WO2018228353A1 PCT/CN2018/090744 CN2018090744W WO2018228353A1 WO 2018228353 A1 WO2018228353 A1 WO 2018228353A1 CN 2018090744 W CN2018090744 W CN 2018090744W WO 2018228353 A1 WO2018228353 A1 WO 2018228353A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
hardware time
time code
initial
camera system
Prior art date
Application number
PCT/CN2018/090744
Other languages
French (fr)
Chinese (zh)
Inventor
崔永太
Original Assignee
深圳市瑞立视多媒体科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市瑞立视多媒体科技有限公司 filed Critical 深圳市瑞立视多媒体科技有限公司
Publication of WO2018228353A1 publication Critical patent/WO2018228353A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • H04N5/06—Generation of synchronising signals
    • H04N5/067—Arrangements or circuits at the transmitter end
    • H04N5/073—Arrangements or circuits at the transmitter end for mutually locking plural sources of synchronising signals, e.g. studios or relay stations
    • H04N5/0733—Arrangements or circuits at the transmitter end for mutually locking plural sources of synchronising signals, e.g. studios or relay stations for distributing synchronisation pulses to different TV cameras

Definitions

  • the invention belongs to the technical field of multi-camera systems, and in particular relates to a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system.
  • the multi-camera system is a system based on the principle of computer vision, which combines multiple cameras, light sources, storage devices, etc., and is commonly used in 3D reconstruction, motion capture, multi-view video, and the like.
  • optical motion capture is a technique for capturing motion by monitoring and tracking target feature points from different angles by multiple high-speed cameras based on the principle of computer vision. For any point in space, as long as it is seen by both cameras at the same time, the position of the point in space at this moment can be determined. When the camera continuously shoots at a high enough rate, the image sequence can be obtained. The trajectory of the point, if a plurality of points are marked on one object, and the object is photographed simultaneously by multiple cameras, the motion trajectory of the object can be obtained.
  • the existing method of synchronizing multiple cameras usually uses a synchronization signal to trigger multiple cameras, and transmits a synchronization trigger signal to each camera through a wired or wireless manner, but in reality, network transmission delay, network instability, External environmental factors such as temperature and humidity can cause multiple cameras to capture images without being completely synchronized.
  • the present invention provides a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system, which can improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in a complex scene.
  • a first aspect of the present invention provides a method for controlling simultaneous exposure of a multi-camera system, the control method comprising:
  • a control device for synchronous exposure of a multi-camera system comprising:
  • An acquiring module configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes
  • a determining module configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera value;
  • a sending module configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system
  • the exposure time of all cameras is synchronized.
  • a terminal device includes a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor executing the computer
  • the steps of the method provided by the first aspect above are implemented at the time of the program.
  • a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method provided by the first aspect above.
  • the technical solution provided by the present invention after receiving an instruction of exposure synchronization, acquiring an initial hardware time code of each camera, and determining a synchronization reference value according to the plurality of initial hardware time codes, and according to the synchronization reference value and each An initial hardware time code of the camera, determining a frame compensation value corresponding to each camera in the multi-camera system, and transmitting the frame compensation value to each camera corresponding to the multi-camera system, thereby enabling each camera in the multi-camera system to
  • the determined frame compensation value corresponds to the frame length of the adjusted image, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image, which improves the accuracy of exposure synchronization of multiple cameras in the multi-camera system.
  • FIG. 1 is a schematic flow chart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention
  • FIG. 2 is a schematic flow chart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention
  • FIG. 3 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 4 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 5 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 6 is a schematic flow chart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • FIG. 7 is a schematic block diagram of a first embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention.
  • FIG. 8 is a schematic block diagram of a second embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention.
  • FIG. 9 is a schematic block diagram of an embodiment of a terminal device provided by the present invention.
  • Figure 10 is a diagram showing the relationship between the control device and the multi-camera system provided by the present invention.
  • the term “if” can be interpreted as “when” or “on” or “in response to determining” or “in response to detecting” depending on the context. .
  • the phrase “if determined” or “if detected [condition or event described]” may be interpreted in context to mean “once determined” or “in response to determining” or “once detected [condition or event described] ] or “in response to detecting [conditions or events described]”.
  • the hardware time code is the minimum timing unit of the camera sensor input clock (MCLK) crystal oscillation period, that is, every crystal clock cycle
  • MCLK camera sensor input clock
  • the hardware time codes passed by all the cameras to capture one frame of image are the same.
  • the image sensor models of all cameras are different (and other hardware and software configurations are the same)
  • the image resolution of the camera is different at this time, but the hardware time code of all the cameras capturing one frame of image is still the same.
  • the oscillation period of the actual crystal oscillator of the camera is constantly changing due to some factors such as temperature and humidity.
  • the crystal oscillation period of each camera is not necessarily equal, so the time elapsed when each camera captures one frame of images in a multi-camera system is not necessarily the same, which is the root cause of camera exposure unsynchronization in a multi-camera system.
  • FIG. 1 is a schematic flowchart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 1 may include the following steps:
  • Step S101 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Embodiments of the present invention are for controlling simultaneous exposure photography of multiple cameras in a multi-camera system.
  • the method can be applied to a control device for controlling the simultaneous exposure of each camera in a multi-camera system.
  • the instruction for exposure synchronization may be issued by a camera in a multi-camera system or may be issued by the control device itself.
  • the exposure synchronization command may be a timer inside the camera, and an exposure synchronization command is issued each time the timer count ends.
  • the clock phase will drift for one exposure period (1 ms) in about 30 seconds, so it is necessary to perform one exposure synchronization within 30 seconds, and then it can be set every 20 seconds. Issue an instruction to synchronize the exposure.
  • control device If the instruction for the exposure synchronization is issued by the control device itself, the control device also has a timer internally, which functions the same as the timer described above. As another embodiment, it is also possible to directly set an instruction that the control device itself issues an exposure synchronization every time a predetermined time elapses.
  • the exposure time of each camera in the multi-camera system is theoretically the same.
  • the oscillation period of the actual crystal oscillator is also changing, and is not completely constant, so that the problem of unsynchronized exposure occurs.
  • the hardware time code of each camera in the multi-camera system needs to be acquired, so that the exposure time of each camera of the multi-camera system can be adjusted according to the acquired hardware time code, so that The exposure time of all cameras is synchronized.
  • each camera has a time code register inside, which is used to read and write the hardware time code of the current camera.
  • the hardware timer module starts timing.
  • the exposure start time and exposure duration of each frame of the camera are timed by the hardware time code recorded by the hardware timer.
  • the process of the control device specifically acquiring the initial hardware time code of each camera in the multi-camera system may be: assuming that there are a total of N cameras in the multi-camera system, the control device respectively sends a hardware time code acquisition request to each of the multi-camera systems, Each camera receives the request, reads its own hardware time code through its own time code register and returns it to the control device.
  • the control device receives the hardware time code sent by each camera in the multi-camera system and counts as Initial hardware time code T 1 , T 2 , T 3 ... T N . At this time, initial hardware time codes T 1 , T 2 , T 3 ... T N corresponding to each camera in the multi-camera system are obtained.
  • Step S102 determining a synchronization reference value according to the plurality of initial hardware time codes, and determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera.
  • the synchronization reference value may be determined according to the plurality of initial hardware time codes.
  • any one of the plurality of initial hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of initial hardware time codes may be selected as the synchronization reference value, of course,
  • An average of the plurality of initial hardware time codes is calculated and used as a synchronization reference value.
  • a frame compensation value for each camera is then determined based on the synchronization reference value and the plurality of initial hardware time codes obtained.
  • the frame extension register calculates a corresponding frame compensation value for each camera based on the deviation of the synchronization reference value from the camera's initial hardware time code.
  • Step S103 transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems
  • the camera's exposure time is synchronized.
  • the control device calculates the frame compensation value corresponding to each camera and transmits the frame compensation value to the corresponding camera in the multi-camera system. After each camera in the multi-camera system receives the corresponding frame compensation value, the frame length of the image stored in the frame extension register inside the camera is updated so that each camera in the multi-camera system can align the exposure time when acquiring the image. It should be noted that if the determined synchronization reference value does not deviate from the initial hardware time code of the camera, it means that the camera does not need to perform frame compensation during the current exposure synchronization.
  • the frame length of the image when the frame length of the image is adjusted according to the frame compensation value, the frame length of the current image frame can be adjusted, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is to say, The next frame image is the image to be synchronized.
  • the camera can also adjust the frame length of the next frame image according to the frame compensation value, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is, the next frame image is to be synchronized. Image, and so on.
  • the method for synchronously exposing a multi-camera system acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and determines a synchronization reference value according to the plurality of initial hardware time codes, and according to each Deviation between the initial hardware time code of the camera and the synchronization reference value, determining a frame compensation value of the image, and transmitting the determined frame compensation value to a corresponding camera in the multi-camera system so that each camera in the multi-camera system can
  • the frame length of the image is adjusted correspondingly according to the received frame compensation value, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image.
  • the reason for causing the multiple camera exposure times in the multi-camera system to be out of sync is that there are other factors besides the crystal oscillation period is not equal.
  • the network delay when acquiring the initial time code of each camera in the multi-camera system the operation delay when acquiring the initial time code of each camera, and the hardware cumulative delay value of each camera itself in the multi-camera system. Since the obtained initial hardware time code includes the above delay values, in order to further improve the synchronization accuracy of the multi-camera system, after the initial hardware time code is acquired, before the frame compensation value is determined, the acquired initial hardware time code may also be acquired. Perform delay correction processing.
  • the delay correction processing may include at least one of a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing.
  • the corrected hardware time code is obtained when the synchronization reference value is obtained, and the corrected hardware time code is also used when determining the frame compensation value of each camera.
  • the delay correction processing including the network delay correction processing, the operation delay correction processing, and the hardware cumulative correction processing will be described in detail as an example.
  • FIG. 2 is a schematic flowchart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 2 may include the following steps:
  • Step S201 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Step S202 performing hardware accumulation correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes in sequence.
  • the hardware timer starts to count after the camera sensor starts to work. After a certain period of operation, the crystal oscillator generates a certain cumulative error. To eliminate this error, it can be used in each frame of the image.
  • the preset position (such as the position of the 100th pixel) is set to trigger the CPU hardware interrupt. When the interrupt is triggered, the hardware timer will automatically latch the time code of the current time. Each time an interrupt is triggered, a time code is latched.
  • step 202 when hardware initial correction processing is performed on a plurality of initial hardware time codes, the operation flow may be performed according to the flow shown in FIG.
  • step 202 when the flowchart of the hardware cumulative correction processing in step 202 is performed, the following steps may be included:
  • Step 301 Corresponding to acquiring an interrupt time code of the last interrupt trigger time of each camera.
  • Step 302 Perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtain a corrected hardware time code of each camera.
  • the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, by taking the difference between the obtained initial hardware time code and the interrupt time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
  • the hardware cumulative correction processing operation for each camera in the multi-camera system can be performed by the control device, or of course, by each camera in the multi-camera system.
  • each camera needs to send it after reading its own initial hardware time code and the interruption time code of its own last interruption time.
  • the control device compares the initial hardware time code of each camera in the multi-camera system with the interrupt time code of each camera, and obtains the corrected hardware time code of each camera after the hardware cumulative correction processing.
  • the hardware cumulative correction processing operation is performed.
  • the corrected hardware time code K i T i - T i0 of the following camera , where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the most recent interruption of camera i, i ranges from 1 to N.
  • network delay correction processing is performed on the initial hardware time code.
  • the camera in the multi-camera system When acquiring the initial hardware time code of the camera, it is necessary to send a request for acquiring a hardware time code to each camera, and the camera in the multi-camera system passes its own time after receiving the acquisition request of the hardware time code sent by the control device.
  • the code register reads its own hardware time code and sends it to the control device.
  • the time delay from the time the control device sends a request to the time the camera in the multi-camera system receives the request is the network delay in the embodiment of the present invention.
  • the operation may be specifically performed according to the flow shown in FIG. 4.
  • FIG. 4 it is a schematic flowchart of an embodiment of performing network delay correction processing on an initial hardware time code, and the specific method may include:
  • Step 401 Acquire a network delay value between each camera in the local and multi-camera system.
  • Step 402 Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera in the multi-camera system.
  • the specific operation method may be, for example:
  • the network delay between each camera in the local and multi-camera systems can be measured by the IEEE 1588 Precision Clock Synchronization Protocol and converted in units of the sensor clock crystal period. Network latency between each camera in the local and multi-camera systems.
  • the hardware time code after the hardware cumulative correction processing should be used in the difference calculation in this step. Where i ranges from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the network delay values between each of the acquired local and multi-camera systems are Y 1 , Y 2 , Y 3 ... Y N , respectively .
  • the operation delay correction processing is performed on the initial hardware time code.
  • FIG. 5 it is a schematic flowchart of an embodiment of performing an operation delay correction process on an initial hardware time code, and includes the following steps:
  • Step 501 When acquiring the initial hardware time code of each camera, reading the local instantaneous hardware time code.
  • Step 502 Perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when the initial hardware time code of the first camera is obtained, to obtain a local and each camera Operation delay value.
  • Step 503 Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained operation delay value between the local and each camera.
  • step 501 the hardware time code of the control device itself is read once before the initial hardware time code acquisition request is sent to each camera, and is recorded as the instantaneous hardware time code. This also requires that the control device itself also contains a time code register for reading and writing the hardware time code of the current control device. Then, an initial hardware time code acquisition request is sent to each camera in the multi-camera system, and the camera in the multi-camera system reads the hardware time code of the camera and returns the hardware time code of the camera to the control after receiving the request. Device.
  • the operation delay value C i T 1i - T 11 .
  • T 1i is the instantaneous hardware time code local to the control device read before the initial hardware time code of the i-th camera is acquired
  • T 11 is the instantaneous hardware local to the control device read when the initial hardware time code of the first camera is read.
  • the camera hardware time corrected code N i T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N.
  • the corrected hardware time code N i T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
  • the hardware cumulative correction processing and the network delay correction processing have been performed on the initial hardware delay code in step 202, the hardware cumulative correction processing and network extension should be used in the difference calculation in this step.
  • the hardware time code after processing is corrected, that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the operational delay values obtained between each of the cameras in the multi-camera system are C 1 , C 2 , C 3 ... C N , respectively .
  • the above embodiment adopts three delay correction processings of network delay correction processing, operation delay correction processing, and hardware cumulative correction processing, in practical applications, only one of them may be selected or Two delay correction processing. Which delay correction processing is selected in the process of correcting the hardware time code by performing the delay correction processing on the initial hardware time code, and correspondingly, which delay correction result obtained by the delay correction processing is calculated together with the initial hardware time code Obtaining the modified hardware time code, the embodiment of selecting only one of the delay correction processing in the above three delay correction processing and the embodiment for selecting any two of the delay correction processing are not illustrated herein, and can be referred to The calculation process of the above embodiment is obtained.
  • Step 203 Determine the synchronization reference value according to the plurality of modified hardware time codes, and determine frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and the corrected hardware time code of each camera. value.
  • the synchronization reference value when the synchronization reference value is determined, it is necessary to determine the synchronization reference value based on the corrected plurality of corrected time codes. Specifically, when determining the synchronization reference value, any one of the plurality of modified hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of modified hardware time codes may be selected as the synchronization reference value. It is of course also possible to calculate an average of the plurality of modified hardware time codes and use the average as a synchronization reference value.
  • the synchronization reference value needs to be determined according to the hardware time code after the hardware cumulative correction processing, and so on.
  • the hardware cumulative correction processing, the network delay correction processing, and the operation delay correction processing are performed on the initial hardware time code, then the hardware cumulative correction processing, the network delay correction processing, and the operation delay should be performed according to the hardware.
  • the corrected hardware time code of each camera obtained after processing determines the synchronization reference value. After selecting the sync reference value, all cameras are aligned with the camera corresponding to the selected sync reference value.
  • Step S204 sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems
  • the camera's exposure time is synchronized.
  • the corrected hardware time code of the camera is greater than the synchronization reference value, the hardware time code of the camera is relatively faster than the synchronization reference value, and the current camera needs to be The frame image or the next frame image is extended for a certain period of time, so that the time code of the camera corresponding to the synchronization reference value can be caught up, so that the exposure time is synchronized when the lower frame or the lower frame image is acquired; if the corrected hardware time code of the camera is smaller than the synchronization
  • the reference value indicates that the time code of the camera is slow, and it is necessary to catch up with the time code of the camera corresponding to the synchronization reference value.
  • the camera's time code can be caught up so that the exposure is synchronized when the next frame or the next frame is captured.
  • the method for synchronously exposing a multi-camera system acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes to obtain a plurality of Correcting a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera And transmitting a frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value so that all of the multi-camera system
  • the camera's exposure time is synchronized.
  • Embodiments of the present invention may apply the method to a control device for controlling simultaneous synchronization of each camera in a multi-camera system, and the method may also be applied to one camera in a multi-camera system, that is, the control may be
  • the device is integrated into one of the cameras in the multi-camera system.
  • One camera in the multi-camera system can be selected as the main camera, the other is named as the slave camera, the control device is integrated into the main camera, or the method is directly applied to the main camera.
  • the acquired control device and the delay value of each camera are replaced by the delay values of the main camera and each camera.
  • the delay value of the main camera and itself can be replaced by the value 0.
  • the main camera is used as a control device to realize synchronous exposure of each camera in the multi-camera system, which can be obtained by the above content, which is not exemplified herein.
  • FIG. 6 is a schematic flowchart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 6 may include the following steps:
  • Step S601 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Step S602 sequentially performing hardware cumulative correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes to obtain a plurality of modified hardware time codes.
  • Step S603 determining a synchronization reference value according to the plurality of modified hardware time codes.
  • Step S605 sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system.
  • step S604 When determining the frame compensation value, the formula in step S604 is specifically employed.
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents one of each camera in the multi-camera system.
  • the size of the frame image in units of oscillation periods, and % indicates the modulo operation.
  • the specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code.
  • the modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
  • the determined synchronization reference value may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value.
  • the minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
  • the camera in the multi-camera system can adjust the length of the current frame or the next frame image according to the frame compensation value, so that the camera can realize when acquiring the lower frame or the lower frame image. Exposure sync.
  • the size of the serial number of each step does not mean the order of execution sequence, and the order of execution of each step should be determined by its function and internal logic, and should not constitute any implementation process of the embodiment of the present invention. limited.
  • FIG. 1 to FIG. 6 describe in detail the control method of the synchronous exposure of the multi-camera system.
  • the control decoration of the multi-camera system synchronous exposure, the terminal device and the computer-readable storage medium will be described in detail below with reference to the accompanying drawings. To avoid redundancy, the terms already described above may not be repeatedly described below.
  • FIG. 7 is a structural block diagram of a control apparatus 700 provided by the present invention.
  • the control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant.
  • the synchronous exposure apparatus 700 includes:
  • the obtaining module 701 is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes.
  • a determining module 702 configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera Compensation value
  • the sending module 703 is configured to send the frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system The exposure time of all cameras is synchronized.
  • FIG. 8 it is a structural block diagram of a control apparatus 700 for simultaneous exposure of a multi-camera system provided by the present invention, and for convenience of explanation, only parts related to the embodiment of the present invention are shown.
  • the control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant.
  • the control device 700 includes an acquisition module 701, a determination module 702, and a transmission module 703.
  • the difference between the embodiment of the present invention and the embodiment shown in FIG. 7 is that the correction module 704 is further included.
  • the correction module 704 is configured to sequentially perform hardware cumulative correction processing, network delay processing, and operation delay processing on the plurality of initial hardware time codes obtained by the obtaining module 701.
  • the correction module 704 may specifically include: an obtaining unit, and a correcting unit. Specifically, when the correction module 704 performs hardware cumulative correction processing on a plurality of initial hardware time codes, the specific operation method may be:
  • the acquiring unit is configured to obtain an interrupt time code of the last interrupt trigger time of each camera, and the correcting unit performs a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtains a corrected hardware time of each camera. code.
  • the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, the read initial hardware time code and the interrupt time code are sent to the acquiring unit, and the acquiring unit transmits the initial hardware time code and the interrupt time code acquired from the camera to the correcting unit process. By correcting the initial hardware time code of the correction unit and the interruption time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
  • the correction module 704 corrects the initial hardware time code of the camera
  • the initial hardware time code of the camera is T i
  • the acquired interrupt time code of the latest interrupt of the camera is T i0
  • the hardware is accumulated.
  • Correct the corrected hardware time code K i T i - T i0 of the camera after the processing operation, where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the last interrupt of camera i, and the value range of i is 1 to N.
  • the specific operation method may be:
  • the obtaining unit acquires a network delay value between each camera in the local and multi-camera system, and the correcting unit associates the plurality of initial hardware time codes with a network delay value between the local and each camera acquired by the acquiring unit The difference operation corresponds to the corrected hardware time code of each camera.
  • the operation method may be: measuring the network delay between each camera and the camera through the IEEE 1588 precision clock synchronization protocol, and performing conversion in units of the sensor clock crystal period, thereby obtaining At this point, the network delay between the control device itself and each camera.
  • the correction unit performs the difference correction after the network delay correction processing is performed on the network delay value between the plurality of initial hardware time codes and the acquired control device and each camera.
  • the hardware time code M i T i - Y i , where T i is the initial hardware time code of camera i, Y i is the network delay value of camera i, and i ranges from 1 to N.
  • the hardware time code after the hardware cumulative correction processing should be used when performing the difference calculation. Values range from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the specific operation method may be:
  • the obtaining unit When acquiring the initial hardware time code of each camera, the obtaining unit reads the instantaneous hardware time code local to the control device, and the correction unit will respectively read the instantaneous hardware time code local to the control device and acquire the first camera.
  • the instantaneous hardware time code local to the control device read at the initial hardware time code is subjected to a difference operation to obtain an operation delay value between the control device and each camera, and according to the plurality of initial hardware time codes and acquired locations.
  • the operation delay value is calculated to obtain the corrected hardware time code of each camera.
  • the acquisition unit Before the initial hardware time code acquisition request is sent to each camera, the acquisition unit first reads the hardware time code local to the control device, which is counted as the instantaneous hardware time code. An acquisition request for the initial hardware time code is then sent to each camera, and each camera receives the request and reads the hardware time code of each camera itself and sends it to the control device.
  • T 1i is the instantaneous hardware time code of the control device itself read before acquiring the initial hardware time code of the i-th camera
  • T 11 is the instantaneous hardware time of the control device itself read when reading the hardware time code of the first camera code.
  • i is in the range of 1 to N, and when i is equal to 1, it represents the operational delay value between the control device and the first camera.
  • the camera hardware time corrected code N i T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N.
  • the corrected hardware time code N i T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
  • the hardware cumulative correction processing and the network delay correction processing should be used when performing the difference calculation.
  • the hardware time code that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the control device for synchronous exposure of the multi-camera system of the embodiment of the present invention obtains the initial hardware time code of each camera after receiving the instruction of the exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes. Determining a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining, corresponding to each camera in the multi-camera system, according to the synchronization reference value and an initial hardware time code of each camera Frame compensation value; finally transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value such that the multi-camera system
  • the exposure time of all cameras is synchronized.
  • the determining module 702 of the embodiment of the present invention specifically includes: a synchronization reference value unit and a frame compensation value determining unit.
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents a frame image of each camera in the multi-camera system.
  • the size in units of oscillation periods, % represents the modulo operation.
  • the specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code.
  • the modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
  • the synchronization reference value determined by the synchronization reference value determining unit may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value.
  • the minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
  • each functional module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit or module, and the integrated unit or module may adopt hardware.
  • the form is implemented in the form of a software functional unit.
  • the specific names of the respective functional modules are only for the purpose of distinguishing from each other, and are not intended to limit the scope of protection of the present application.
  • FIG. 9 is a schematic block diagram of a terminal device according to an embodiment of the present invention.
  • the terminal device can also be a camera in a multi-camera system, such as a main camera.
  • the terminal device 9 of this embodiment includes: one or more processors 90, a memory 91, and a computer program 92 stored in the memory 91 and operable on the processor 90.
  • the processor 90 implements the functions of the modules/units in the terminal device embodiment when the computer program 92 is executed, such as the functions of the modules 701 to 703 shown in FIG.
  • the computer program 92 can be partitioned into one or more modules/units that are stored in the memory 91 and executed by the processor 90 to complete this invention.
  • the one or more modules/units may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program 92 in the terminal device 9.
  • the computer program 92 can be partitioned into an acquisition module, a determination module, and a transmission module.
  • the acquiring module is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes;
  • the determining module is configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine, according to the synchronization reference value and an initial hardware time code of each camera, each camera in the multi-camera system Frame compensation value;
  • the sending module is configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value to make the multi-camera
  • the exposure time of all cameras in the system is synchronized.
  • it may also include:
  • a correction module configured to perform delay correction processing on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes
  • the delay correction processing includes at least one of the following: a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing;
  • the determining module is specifically configured to:
  • the modification module includes:
  • An obtaining unit configured to acquire a network delay value between the local and each camera in the multi-camera system
  • a correction unit configured to obtain a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera.
  • the acquiring unit is further configured to: when acquiring an initial hardware time code of each camera, read a local instantaneous hardware time code;
  • the correcting unit is further configured to perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when acquiring the initial hardware time code of the first camera, to obtain local and per Operation delay value between cameras;
  • a corrected hardware time code for each camera is obtained based on the plurality of initial hardware time codes and the resulting operational delay value between the local and each camera.
  • the acquiring unit is further configured to: correspondingly acquire an interrupt time code of a last interrupt trigger time of each camera;
  • the modifying unit is further configured to perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and correspondingly obtain a corrected hardware time code of each camera.
  • the determining module is specifically configured to:
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents a frame image of each camera in the multi-camera system.
  • the size in units of oscillation periods, % represents the modulo operation.
  • the terminal device includes, but is not limited to, a processor 90 and a memory 91. It will be understood by those skilled in the art that FIG. 9 is only an example of the terminal device 9, and does not constitute a limitation on the terminal device 9, which may include more or less components than those illustrated, or combine some components, or Different components, such as the terminal device, may also include input devices, output devices, network access devices, buses, and the like.
  • the memory 91 is configured to store software programs, modules, units, and data information required in the terminal device, and the processor 90 executes various functional applications by running software programs, modules, and units stored in the memory 91. And data processing to improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in complex scenes.
  • the memory 91 can include read only memory and random access memory and provides instructions and data to the processor 90.
  • a portion of the memory 91 may also include a non-volatile random access memory.
  • the memory 91 can also store information of the device type.
  • the processor 90 can be a central processing unit (Central) Processing Unit (CPU), which can also be other general-purpose processors, digital signal processors (DSPs), and application specific integrated circuits (Application). Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the processor is a control center of the terminal device, and connects various parts of the entire terminal device by using various interfaces and lines.
  • the processor 90 and the memory 91 described in the embodiments of the present invention may implement the implementation manner described in the embodiment of the method for controlling the synchronous exposure of the multi-camera system provided by the embodiment of the present invention, and may also execute the multi-camera system.
  • the implementation described in the embodiment of the synchronous exposure control device will not be described herein.
  • control device 7 is used to control a multi-camera system, and the multi-camera system includes a plurality of cameras 101.
  • the control device 7 can represent the control device described in any of the above control devices, and can also represent any of the above embodiments.
  • an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the application provided by the embodiment of the present invention is implemented.
  • the steps of the control method for simultaneous exposure of a multi-camera system are not limited to:
  • control method control device, and terminal device may be implemented in other manners.
  • the embodiments described above are merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the medium includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), and a random access memory (RAM, Random Access).
  • ROM read-only memory
  • RAM random access memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

A control method and apparatus for the synchronous exposure of a multi-camera system, and a terminal device thereof, the control method comprising: after receiving a synchronous exposure instruction, acquiring initial hardware time codes of cameras in the multi-camera system so as to obtain a plurality of initial hardware time codes; determining a synchronous reference value according to the plurality of initial hardware time codes; determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronous reference value and the initial hardware time code of each camera; and sending the frame compensation value corresponding to each camera to each corresponding camera in the multi-camera system, so that the each camera adjusts the length of an image according to the corresponding frame compensation value thereof so that the exposure time of all cameras in the multi-camera system is synchronous. Employing the described solution may improve the precision of synchronous exposure of a plurality of cameras in the multi-camera system in a complex scene.

Description

一种多相机***同步曝光的控制方法、装置及终端设备Control method, device and terminal device for synchronous exposure of multi-camera system 技术领域Technical field
本发明属于多相机***技术领域，尤其涉及一种多相机***同步曝光的控制方法、装置及终端设备。The invention belongs to the technical field of multi-camera systems, and in particular relates to a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system.
背景技术Background technique
多相机***是基于计算机视觉原理，将多个相机、光源、存储设备等组合在一起组建的***，常应用于3D重建、运动捕捉、多视点视频等。例如光学式动作捕捉就是基于计算机视觉原理，由多个高速相机从不同角度对目标特征点的监视和跟踪来进行动作捕捉的技术。对于空间中的任意一点，只要它同时被两部相机所见，就可以确定这一时刻该点在空间中的位置，当相机以足够高的速率连续拍摄时，从图像序列中就可以得到该点的运动轨迹，若是在一个物体标记多个点，通过多台相机同时拍摄这个物体，就可以得到这个物体的运动轨迹。The multi-camera system is a system based on the principle of computer vision, which combines multiple cameras, light sources, storage devices, etc., and is commonly used in 3D reconstruction, motion capture, multi-view video, and the like. For example, optical motion capture is a technique for capturing motion by monitoring and tracking target feature points from different angles by multiple high-speed cameras based on the principle of computer vision. For any point in space, as long as it is seen by both cameras at the same time, the position of the point in space at this moment can be determined. When the camera continuously shoots at a high enough rate, the image sequence can be obtained. The trajectory of the point, if a plurality of points are marked on one object, and the object is photographed simultaneously by multiple cameras, the motion trajectory of the object can be obtained.
但是，这也就需要参与拍摄的多个相机采集每一帧图像时曝光对齐，否则由多个相机得到的标记点运动轨迹与真实运动轨迹就会有差异，甚至出现扭曲现象。现有的将多个相机同步的方法通常是采用同步信号对多个相机进行触发，通过有线或者无线的方式将同步触发信号传输至各相机，但是在实际中发现网络传输延迟、网络不稳定、温度、湿度等外部环境因素会造成多个相机采集图像时并不会完全同步。However, this also requires multiple cameras participating in the shooting to capture the alignment of each frame of the image, otherwise the mark motion track obtained by multiple cameras will be different from the real motion track, and even distortion. The existing method of synchronizing multiple cameras usually uses a synchronization signal to trigger multiple cameras, and transmits a synchronization trigger signal to each camera through a wired or wireless manner, but in reality, network transmission delay, network instability, External environmental factors such as temperature and humidity can cause multiple cameras to capture images without being completely synchronized.
技术问题technical problem
鉴于此，本发明提供一种多相机***同步曝光的控制方法、装置及终端设备，能够提高复杂场景下多相机***中多个相机同步曝光的精度。In view of this, the present invention provides a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system, which can improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in a complex scene.
技术解决方案Technical solution
本发明的第一方面，提供一种多相机***同步曝光的控制方法，所述控制方法包括：A first aspect of the present invention provides a method for controlling simultaneous exposure of a multi-camera system, the control method comprising:
在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码；After receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes;
根据所述多个初始硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；Determining a synchronization reference value according to the plurality of initial hardware time codes, and determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera;
将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。Transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value such that the exposure time of all the cameras in the multi-camera system Synchronize.
本发明的第二方面，提供一种多相机***同步曝光的控制装置，所述控制装置包括：According to a second aspect of the present invention, a control device for synchronous exposure of a multi-camera system is provided, the control device comprising:
获取模块，用于在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码；An acquiring module, configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes;
确定模块，用于根据所述多个初始硬件时间码确定同步基准值，以及根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；a determining module, configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera value;
发送模块，用于将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。a sending module, configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system The exposure time of all cameras is synchronized.
本发明的第三方面，提供一种终端设备，所述终端设备包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述第一方面提供的所述方法的步骤。According to a third aspect of the present invention, a terminal device includes a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor executing the computer The steps of the method provided by the first aspect above are implemented at the time of the program.
本发明的第四方面，提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述第一方面提供的所述方法的步骤。According to a fourth aspect of the invention, there is provided a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method provided by the first aspect above.
有益效果Beneficial effect
本发明与现有技术相比存在的有益效果是：The beneficial effects of the present invention over the prior art are:
本发明提供的技术方案，在接收到曝光同步的指令之后，获取每个相机的初始硬件时间码，并根据该多个初始硬件时间码确定同步基准值，以及根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值，并将帧补偿值发送至多相机***中对应的每个相机，从而使得多相机***中每个相机能够根据该确定出来的帧补偿值对应调整图像的帧长，进而使得多相机***中的每个相机在采集图像时曝光时间同步，提高了多相机***中多个相机曝光同步的精度。The technical solution provided by the present invention, after receiving an instruction of exposure synchronization, acquiring an initial hardware time code of each camera, and determining a synchronization reference value according to the plurality of initial hardware time codes, and according to the synchronization reference value and each An initial hardware time code of the camera, determining a frame compensation value corresponding to each camera in the multi-camera system, and transmitting the frame compensation value to each camera corresponding to the multi-camera system, thereby enabling each camera in the multi-camera system to The determined frame compensation value corresponds to the frame length of the adjusted image, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image, which improves the accuracy of exposure synchronization of multiple cameras in the multi-camera system.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below. It is obvious that the drawings in the following description are only the present invention. For some embodiments, other drawings may be obtained from those of ordinary skill in the art in light of the inventive workability.
图1是本发明提供的多相机***同步曝光的控制方法的第一实施例的流程示意图；1 is a schematic flow chart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention;
图2是本发明提供的多相机***同步曝光的控制方法的第二实施例的流程示意图；2 is a schematic flow chart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention;
图3是图2中步骤202的实施例的流程示意图；3 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
图4是图2中步骤202的实施例的流程示意图；4 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
图5是图2中步骤202的实施例的流程示意图；5 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
图6是本发明提供的多相机***同步曝光的控制方法的第三实施例的流程示意图；6 is a schematic flow chart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention;
图7是本发明提供的多相机***同步曝光的控制装置的第一实施例的示意框图；7 is a schematic block diagram of a first embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention;
图8是本发明提供的多相机***同步曝光的控制装置的第二实施例的示意框图；8 is a schematic block diagram of a second embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention;
图9是本发明提供的终端设备的实施例的示意框图；9 is a schematic block diagram of an embodiment of a terminal device provided by the present invention;
图10是本发明提供的控制装置与多相机***的关系图。Figure 10 is a diagram showing the relationship between the control device and the multi-camera system provided by the present invention.
本发明的实施方式Embodiments of the invention
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。The term "comprising", when used in the specification and the claims of the claims The existence or addition of , whole, steps, operations, elements, components, and/or collections thereof.
还应当理解，在此本发明说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本发明。如在本发明说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。It is also to be understood that the terminology of the present invention is to be construed as a The singular forms "", ",",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
还应当进一步理解，在本发明说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。It is further understood that the term "and/or" used in the description of the invention and the appended claims means any combination and all possible combinations of one or more of the associated listed items, .
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "on" or "in response to determining" or "in response to detecting" depending on the context. . Similarly, the phrase "if determined" or "if detected [condition or event described]" may be interpreted in context to mean "once determined" or "in response to determining" or "once detected [condition or event described] ] or "in response to detecting [conditions or events described]".
在描述本发明实施例的具体实施方式之前，首先分析造成多相机***中各个相机曝光不同步的原因。Before describing a specific embodiment of an embodiment of the present invention, first analyze the cause of the unsynchronized exposure of each camera in a multi-camera system.
通常地，相机内部有一个FPGA构建的64位的硬件计时器，即硬件时间码，该硬件时间码以相机传感器的输入时钟(MCLK)晶振振荡周期为最小计时单位，即每经过一个晶振时钟周期硬件时间码的值自动累加1。Generally, there is a 64-bit hardware timer built by the FPGA inside the camera, that is, hardware time code. The hardware time code is the minimum timing unit of the camera sensor input clock (MCLK) crystal oscillation period, that is, every crystal clock cycle The value of the hardware time code is automatically incremented by 1.
一种情况，当所有相机初始设置相同（即软硬件配置均相同），所有相机采集一帧图像经过的硬件时间码都是相同的。另一种情况，当所有相机的图像传感器型号不同时（且其他软硬件配置相同时），此时相机的图像分辨率虽不相同，但所有相机采集一帧图像经过的硬件时间码仍然是相同的。但在实际中，往往由于温度、湿度等一些因素，相机实际晶振的振荡周期是在变化的。即是说，每个相机的晶振振荡周期不一定相等，因此多相机***中每个相机采集一帧图像时经过的时间不一定相同，这也是多相机***中出现相机曝光不同步的根本原因。In one case, when all cameras have the same initial settings (that is, the hardware and software configurations are the same), the hardware time codes passed by all the cameras to capture one frame of image are the same. In another case, when the image sensor models of all cameras are different (and other hardware and software configurations are the same), the image resolution of the camera is different at this time, but the hardware time code of all the cameras capturing one frame of image is still the same. of. However, in practice, the oscillation period of the actual crystal oscillator of the camera is constantly changing due to some factors such as temperature and humidity. That is to say, the crystal oscillation period of each camera is not necessarily equal, so the time elapsed when each camera captures one frame of images in a multi-camera system is not necessarily the same, which is the root cause of camera exposure unsynchronization in a multi-camera system.
当然，实际使用中还发现，网络传输延迟、网络不稳定等外部环境因素也会造成多个相机采集图像时并不会完全同步。Of course, in actual use, it is also found that external environmental factors such as network transmission delay and network instability may cause multiple cameras to acquire images without being completely synchronized.
为解决多相机***中相机曝光不同步的问题，提出了本发明的多相机***同步曝光的控制方法、装置及终端设备。下面，将通过具体的实施例进行详细描述。In order to solve the problem that the camera exposure is not synchronized in the multi-camera system, the control method, device and terminal device of the multi-camera system synchronous exposure of the present invention are proposed. Hereinafter, a detailed description will be made by way of specific examples.
请参见图1，图1是本发明提供的多相机***同步曝光的控制方法的第一实施例的流程示意图，如图1所示该多相机***同步曝光的控制方法可以包括以下步骤：Referring to FIG. 1, FIG. 1 is a schematic flowchart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention. The control method for synchronous exposure of the multi-camera system shown in FIG. 1 may include the following steps:
步骤S101，在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码。Step S101, after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
本发明实施例是用于控制多相机***中的多个相机同步曝光拍照。该方法可以应用于一个控制装置，该控制装置用于控制多相机***中每个相机同步曝光。Embodiments of the present invention are for controlling simultaneous exposure photography of multiple cameras in a multi-camera system. The method can be applied to a control device for controlling the simultaneous exposure of each camera in a multi-camera system.
本发明实施例中，曝光同步的指令可以由多相机***中的相机发出，也可以由控制装置自身发出。当曝光同步的指令由多相机***中的相机发出时，曝光同步的指令可以是相机内部一个定时器，每当定时器计数结束就发一次曝光同步命令。例如，在实际应用中，当相机传感器的输入时钟为40MHZ时，大约30秒左右时钟相位会漂移一个曝光周期（1ms），因此需要在30s之内进行一次曝光同步，这时可以设定每20s发出一次曝光同步的指令。如果所述曝光同步的指令是由控制装置本身发出的，则控制装置内部也有一个定时器，作用与上述的定时器作用相同。作为另一个实施例，还可以直接设置控制装置自身每经过预定时间就发出一个曝光同步的指令。In the embodiment of the present invention, the instruction for exposure synchronization may be issued by a camera in a multi-camera system or may be issued by the control device itself. When the exposure synchronization command is issued by a camera in a multi-camera system, the exposure synchronization command may be a timer inside the camera, and an exposure synchronization command is issued each time the timer count ends. For example, in practical applications, when the input clock of the camera sensor is 40 MHz, the clock phase will drift for one exposure period (1 ms) in about 30 seconds, so it is necessary to perform one exposure synchronization within 30 seconds, and then it can be set every 20 seconds. Issue an instruction to synchronize the exposure. If the instruction for the exposure synchronization is issued by the control device itself, the control device also has a timer internally, which functions the same as the timer described above. As another embodiment, it is also possible to directly set an instruction that the control device itself issues an exposure synchronization every time a predetermined time elapses.
本发明实施例中，理论上来说多相机***中每个相机的曝光时间是相同的。根据前文的描述可知，在实际中由于温度、湿度等一些因素，实际晶振的振荡周期也是在变化的，并不是完全不变的，因此出现了曝光不同步的问题。为解决曝光不同步的问题，本步骤中需要获取多相机***中每个相机的硬件时间码，以便于可以根据获取的硬件时间码对多相机***的每个相机的曝光时间进行调整，以使得所有相机的曝光时间同步。In the embodiment of the present invention, the exposure time of each camera in the multi-camera system is theoretically the same. According to the foregoing description, in practice, due to some factors such as temperature and humidity, the oscillation period of the actual crystal oscillator is also changing, and is not completely constant, so that the problem of unsynchronized exposure occurs. In order to solve the problem of unsynchronized exposure, in this step, the hardware time code of each camera in the multi-camera system needs to be acquired, so that the exposure time of each camera of the multi-camera system can be adjusted according to the acquired hardware time code, so that The exposure time of all cameras is synchronized.
其中，每个相机的内部都有时间码寄存器，其用于读写当前相机的硬件时间码。相机传感器开始工作后，硬件计时器模块开始计时。相机每一帧图像的曝光开始时刻和曝光持续时间都是以硬件计时器记录的硬件时间码进行计时的。控制装置具体获取多相机***中每个相机的初始硬件时间码的过程可以是：假设多相机***中共有N个相机，控制装置分别向多相机***中的每一个相机发送硬件时间码获取请求，每个相机接收到该请求时，通过自身的时间码寄存器读取自身的硬件时间码并将其返回控制装置，控制装置接收多相机***中每个相机发来的硬件时间码，并分别计为初始硬件时间码T 1、T 2、T 3……T N。此时，便得到了多相机***中每个相机对应的初始硬件时间码T 1、T 2、T 3……T N。 Among them, each camera has a time code register inside, which is used to read and write the hardware time code of the current camera. After the camera sensor starts working, the hardware timer module starts timing. The exposure start time and exposure duration of each frame of the camera are timed by the hardware time code recorded by the hardware timer. The process of the control device specifically acquiring the initial hardware time code of each camera in the multi-camera system may be: assuming that there are a total of N cameras in the multi-camera system, the control device respectively sends a hardware time code acquisition request to each of the multi-camera systems, Each camera receives the request, reads its own hardware time code through its own time code register and returns it to the control device. The control device receives the hardware time code sent by each camera in the multi-camera system and counts as Initial hardware time code T 1 , T 2 , T 3 ... T N . At this time, initial hardware time codes T 1 , T 2 , T 3 ... T N corresponding to each camera in the multi-camera system are obtained.
步骤S102，根据所述多个初始硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值。Step S102, determining a synchronization reference value according to the plurality of initial hardware time codes, and determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera.
在获取多个相机对应的初始硬件时间码之后，便可根据所述多个初始硬件时间码确定同步基准值。在确定同步基准值时，可以选择多个初始硬件时间码的任意一个值作为同步基准值，也可以选择所述多个初始硬件时间码中的最大值或最小值作为同步基准值，当然也可以计算所述多个初始硬件时间码的平均值，并将该平均值作为同步基准值。然后根据该同步基准值和获取到的多个初始硬件时间码，确定每个相机的帧补偿值。帧扩展寄存器根据同步基准值与相机的初始硬件时间码的偏差，计算每个相机的对应的帧补偿值。After acquiring the initial hardware time code corresponding to the plurality of cameras, the synchronization reference value may be determined according to the plurality of initial hardware time codes. When determining the synchronization reference value, any one of the plurality of initial hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of initial hardware time codes may be selected as the synchronization reference value, of course, An average of the plurality of initial hardware time codes is calculated and used as a synchronization reference value. A frame compensation value for each camera is then determined based on the synchronization reference value and the plurality of initial hardware time codes obtained. The frame extension register calculates a corresponding frame compensation value for each camera based on the deviation of the synchronization reference value from the camera's initial hardware time code.
步骤S103，将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便于每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。Step S103, transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems The camera's exposure time is synchronized.
在本发明实施例中，控制装置计算每个相机对应的帧补偿值后将帧补偿值发送给多相机***中的对应相机。多相机***中每个相机接收到对应的帧补偿值之后，更新相机内部的帧扩展寄存器中存储的图像的帧长，以便多相机***中的每个相机在采集图像时能够对齐曝光时间。需要说明的是，若确定出的同步基准值和相机的初始硬件时间码没有偏差，则表示此相机在本次曝光同步过程中不需要进行帧补偿。In the embodiment of the present invention, the control device calculates the frame compensation value corresponding to each camera and transmits the frame compensation value to the corresponding camera in the multi-camera system. After each camera in the multi-camera system receives the corresponding frame compensation value, the frame length of the image stored in the frame extension register inside the camera is updated so that each camera in the multi-camera system can align the exposure time when acquiring the image. It should be noted that if the determined synchronization reference value does not deviate from the initial hardware time code of the camera, it means that the camera does not need to perform frame compensation during the current exposure synchronization.
需要说明的另一点是，在根据帧补偿值调整图像的帧长时，可以调整当前图像帧的帧长，这样多相机***中的相机在采集下帧图像时便可实现曝光同步，即是说，下帧图像就是待同步的图像。当然，相机也可以根据帧补偿值调整下一帧图像的帧长，这样多相机***中的相机在采集下下帧图像时便可实现曝光同步，即是说，下下帧图像就是待同步的图像，以此类推。Another point to be explained is that when the frame length of the image is adjusted according to the frame compensation value, the frame length of the current image frame can be adjusted, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is to say, The next frame image is the image to be synchronized. Of course, the camera can also adjust the frame length of the next frame image according to the frame compensation value, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is, the next frame image is to be synchronized. Image, and so on.
本发明实施例的多相机***同步曝光的方法，在接收到曝光同步的指令之后，获取每个相机的初始硬件时间码，并根据该多个初始硬件时间码确定同步基准值，以及根据每个相机的初始硬件时间码与同步基准值之间的偏差，确定图像的帧补偿值，并将该确定出来的帧补偿值发送至多相机***中对应的相机，以便多相机***中的每个相机能够根据接收到的帧补偿值对应调整图像的帧长，从而使得多相机***中的每个相机在采集图像时曝光时间同步。The method for synchronously exposing a multi-camera system according to an embodiment of the present invention acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and determines a synchronization reference value according to the plurality of initial hardware time codes, and according to each Deviation between the initial hardware time code of the camera and the synchronization reference value, determining a frame compensation value of the image, and transmitting the determined frame compensation value to a corresponding camera in the multi-camera system so that each camera in the multi-camera system can The frame length of the image is adjusted correspondingly according to the received frame compensation value, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image.
可以理解的是，造成多相机***中多个相机曝光时间不同步的原因除了晶振振荡周期不相等之外，还有其他的一些因素。例如，获取与多相机***中每个相机的初始时间码时的网络延时，获取每一个相机的初始时间码时的操作延时，以及多相机***中每个相机本身硬件累计延时值。由于获取的初始硬件时间码中包括上述这些延时值，因此为进一步提高多相机***的同步精准度，在获取初始硬件时间码之后，确定帧补偿值之前，还可以对获取的初始硬件时间码进行延时修正处理。可以理解的是，延时修正处理可以包括：网络延时修正处理、操作延时修正处理以及硬件累计修正处理中的至少一种。相应的，对初始硬件时间码做修正处理后，获得同步基准值时具体是采用修正后的硬件时间码，同时确定每个相机的帧补偿值时也采用修正后的硬件时间码。下面，将以延时修正处理包括网络延时修正处理、操作延时修正处理以及硬件累计修正处理为例进行详细说明。It can be understood that the reason for causing the multiple camera exposure times in the multi-camera system to be out of sync is that there are other factors besides the crystal oscillation period is not equal. For example, the network delay when acquiring the initial time code of each camera in the multi-camera system, the operation delay when acquiring the initial time code of each camera, and the hardware cumulative delay value of each camera itself in the multi-camera system. Since the obtained initial hardware time code includes the above delay values, in order to further improve the synchronization accuracy of the multi-camera system, after the initial hardware time code is acquired, before the frame compensation value is determined, the acquired initial hardware time code may also be acquired. Perform delay correction processing. It can be understood that the delay correction processing may include at least one of a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing. Correspondingly, after the initial hardware time code is corrected, the corrected hardware time code is obtained when the synchronization reference value is obtained, and the corrected hardware time code is also used when determining the frame compensation value of each camera. Hereinafter, the delay correction processing including the network delay correction processing, the operation delay correction processing, and the hardware cumulative correction processing will be described in detail as an example.
请参见图2，是本发明提供的多相机***同步曝光的控制方法的第二实施例的流程示意图，如图2所示该多相机***同步曝光的控制方法可以包括以下步骤：2 is a schematic flowchart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention. The control method for synchronous exposure of the multi-camera system shown in FIG. 2 may include the following steps:
步骤S201，在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码。Step S201, after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
步骤S202，对得到的多个初始硬件时间码依次进行硬件累计修正处理、网络延时处理以及操作延时处理。Step S202, performing hardware accumulation correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes in sequence.
下面，将分别描述硬件累计修正处理、网络延时处理以及操作延时处理的具体操作过程。In the following, specific operational procedures of hardware cumulative correction processing, network delay processing, and operation delay processing will be separately described.
第一，对多个初始硬件时间码依次进行硬件累计修正处理。First, hardware cumulative correction processing is sequentially performed on a plurality of initial hardware time codes.
在进行硬件累计修正处理时，由于相机传感器开始工作后，硬件计时器就开始计时了，在工作一定时间之后，晶振则会产生一定的累计误差，为消除此误差，因此可以在每帧图像的预设位置（比如第100个像素点的位置）设置触发CPU硬件中断，当中断触发后，硬件计时器会自动锁存当前时刻的时间码，每经过一次中断触发，会锁存一个时间码。When the hardware cumulative correction processing is performed, the hardware timer starts to count after the camera sensor starts to work. After a certain period of operation, the crystal oscillator generates a certain cumulative error. To eliminate this error, it can be used in each frame of the image. The preset position (such as the position of the 100th pixel) is set to trigger the CPU hardware interrupt. When the interrupt is triggered, the hardware timer will automatically latch the time code of the current time. Each time an interrupt is triggered, a time code is latched.
在步骤202中，对多个初始硬件时间码进行硬件累计修正处理时，操作流程可以按照如图3所示的流程进行。In step 202, when hardware initial correction processing is performed on a plurality of initial hardware time codes, the operation flow may be performed according to the flow shown in FIG.
如图3所示，是步骤202中进行硬件累计修正处理的流程示意图时，可以包括如下步骤：As shown in FIG. 3, when the flowchart of the hardware cumulative correction processing in step 202 is performed, the following steps may be included:
步骤301，对应获取每个相机最近一次中断触发时刻的中断时间码。Step 301: Corresponding to acquiring an interrupt time code of the last interrupt trigger time of each camera.
步骤302，将所述多个初始硬件时间码与获取的所述中断时间码做差值运算，对应得到每个相机的修正硬件时间码。Step 302: Perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtain a corrected hardware time code of each camera.
根据前文的描述可知，在产生一次中断时，硬件计时器可以自动锁存一个时间码。因此，每个相机在读取自身的初始硬件时间码之后，还读取最近一次中断触发时刻的硬件时间码，记为中断时间码。然后，将获取的初始硬件时间码与中断时间码做差，便可消除晶振产生的误差，得到经硬件累积延时处理后的修正硬件时间码。According to the foregoing description, the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, by taking the difference between the obtained initial hardware time code and the interrupt time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
需要说明的是，对于多相机***中每个相机的硬件累计修正处理操作，可以由控制装置完成，当然也可以由多相机***中每个相机完成。当由控制装置来完成多相机***中每个相机的硬件累计修正处理操作时，每个相机在读取自身的初始硬件时间码和自身最近一次中断时刻的中断时间码之后，还需要将其发送给控制装置。控制装置将多相机***中每个相机的初始硬件时间码和每个相机的中断时间码做差值运算，便可得到经硬件累计修正处理后的每个相机的修正硬件时间码。It should be noted that the hardware cumulative correction processing operation for each camera in the multi-camera system can be performed by the control device, or of course, by each camera in the multi-camera system. When the hardware cumulative correction processing operation of each camera in the multi-camera system is completed by the control device, each camera needs to send it after reading its own initial hardware time code and the interruption time code of its own last interruption time. Give the control device. The control device compares the initial hardware time code of each camera in the multi-camera system with the interrupt time code of each camera, and obtains the corrected hardware time code of each camera after the hardware cumulative correction processing.
举例来说，在对相机的初始硬件时间码进行修正时，若相机的初始硬件时间码为T i，获取到的相机的最近一次中断的中断时间码为T i0，那么经硬件累计修正处理操作后的相机的修正硬件时间码K i= T i- T i0 ，其中T i为相机i的初始硬件时间码，T i0 为相机i最近一次中断的中断时间码，i的取值范围为1到N 。 For example, when the initial hardware time code of the camera is corrected, if the initial hardware time code of the camera is T i , and the acquired interrupt time code of the last interrupt of the camera is T i0 , the hardware cumulative correction processing operation is performed. The corrected hardware time code K i = T i - T i0 of the following camera , where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the most recent interruption of camera i, i ranges from 1 to N.
第二，对初始硬件时间码进行网络延时修正处理。Second, network delay correction processing is performed on the initial hardware time code.
在获取相机的初始硬件时间码时，需要向每个相机发送一个硬件时间码的获取请求，多相机***中的相机在接收到控制装置发来的硬件时间码的获取请求之后，通过自身的时间码寄存器读取自身的硬件时间码并发送给控制装置。从控制装置发送请求到多相机***中的相机接收到请求所消耗的时间，便是本发明实施例中的网络延时。When acquiring the initial hardware time code of the camera, it is necessary to send a request for acquiring a hardware time code to each camera, and the camera in the multi-camera system passes its own time after receiving the acquisition request of the hardware time code sent by the control device. The code register reads its own hardware time code and sends it to the control device. The time delay from the time the control device sends a request to the time the camera in the multi-camera system receives the request is the network delay in the embodiment of the present invention.
具体地，在对初始硬件时间码进行网络延时修正处理时，具体可以按照图4所示的流程进行操作。Specifically, when the network delay correction processing is performed on the initial hardware time code, the operation may be specifically performed according to the flow shown in FIG. 4.
如图4所示，是对初始硬件时间码进行网络延时修正处理的实施例的流程示意图，具体方法可以包括：As shown in FIG. 4, it is a schematic flowchart of an embodiment of performing network delay correction processing on an initial hardware time code, and the specific method may include:
步骤401，获取本地与多相机***中每个相机之间的网络延时值。Step 401: Acquire a network delay value between each camera in the local and multi-camera system.
步骤402，根据所述多个初始硬件时间码和获取的本地与所述多相机***中每个相机之间的网络延时值计算获得每个相机的修正硬件时间码。Step 402: Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera in the multi-camera system.
步骤401具体操作时，具体操作方法例如可以是：When the specific operation of step 401, the specific operation method may be, for example:
在曝光同步任务启动后，可以通过IEEE 1588 精密时钟同步协议测量本地与多相机***中每个相机之间的网络延时，并以传感器时钟晶振周期为单位进行换算，由此可以得出此时本地与多相机***中每个相机之间的网络延时。After the exposure synchronization task is started, the network delay between each camera in the local and multi-camera systems can be measured by the IEEE 1588 Precision Clock Synchronization Protocol and converted in units of the sensor clock crystal period. Network latency between each camera in the local and multi-camera systems.
需要说明的一点是，在步骤402中，在将所述多个初始硬件时间码与获取的本地与每个相机之间的网络延时值对应做差值运算后，相机的修正硬件时间码M i= T i- Y i,其中T i为相机i的初始硬件时间码，Y i相机i的网络延时值，i的取值范围为1到N 。 It should be noted that, in step 402, after the difference between the plurality of initial hardware time codes and the acquired network delay value between the local and each camera, the corrected hardware time code of the camera M i = T i - Y i, where T i is the initial time code camera i, hardware, network latency value Y i of the camera i, i ranges from 1 to N.
同时，由于步骤202中已经对初始硬件延时码进行了硬件累计修正处理，因此本步骤中在进行差值运算时应该使用经硬件累计修正处理后的硬件时间码，即T i- T i0,其中i的取值范围为1到N。若前文没有进行硬件累计修正处理，那么本步骤中进行差值运算时，使用的便应该是初始的硬件时间码T i，其中i的取值范围为1到N。 At the same time, since the hardware cumulative correction processing has been performed on the initial hardware delay code in step 202, the hardware time code after the hardware cumulative correction processing, that is, T i - T i0 , should be used in the difference calculation in this step. Where i ranges from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
下面，将通过具体的例子进行详细说明。Hereinafter, a detailed description will be given by way of specific examples.
假设，获取到的本地与多相机***中每个相机之间的网络延时值分别为Y 1、Y 2、Y 3……Y N。 Assume that the network delay values between each of the acquired local and multi-camera systems are Y 1 , Y 2 , Y 3 ... Y N , respectively .
那么经硬件累计修正处理以及网络延时修正处理之后，得到的相机的修正硬件时间码M i=K i- Y i= T i- T i0- Y i，其中，其中i的取值范围为1到N。 Then, after the hardware cumulative correction processing and the network delay correction processing, the corrected hardware time code M i =K i - Y i = T i - T i0 - Y i of the obtained camera, wherein i has a value range of 1 To N.
第三，对初始硬件时间码进行操作延时修正处理。Third, the operation delay correction processing is performed on the initial hardware time code.
如前所述可知，在获取多相机***中每个相机的初始硬件时间码时，需要分别获取每一个相机的初始硬件时间码，在获取了某个相机的初始硬件时间码后，由于多相机***中存在多个相机，发出读取下一个相机的初始硬件时间码的指令时的时间已经发生了变化了。因此，在对初始硬件时间码进行修正时，还需要考虑操作延时。As can be seen from the foregoing, when acquiring the initial hardware time code of each camera in the multi-camera system, it is necessary to separately obtain the initial hardware time code of each camera, after acquiring the initial hardware time code of a certain camera, due to the multi-camera There are multiple cameras in the system, and the time to issue an instruction to read the initial hardware time code of the next camera has changed. Therefore, when correcting the initial hardware time code, you also need to consider the operation delay.
特别需要说明的是，若逐个读取多相机***中每个相机的时间码，则需要进行操作延时修正处理，若采用广播的方式读取多相机***中每个相机的时间码时，则可以不考虑操作延时。具体可根据实际情况选择。In particular, if the time code of each camera in the multi-camera system is read one by one, an operation delay correction process is required. If the time code of each camera in the multi-camera system is read by broadcasting, then The operation delay can be ignored. Specific can be selected according to the actual situation.
如图5所示，是对初始硬件时间码做操作延时修正处理的实施例的流程示意图，包括如下步骤：As shown in FIG. 5, it is a schematic flowchart of an embodiment of performing an operation delay correction process on an initial hardware time code, and includes the following steps:
步骤501，当获取每个相机的初始硬件时间码时，读取本地的瞬时硬件时间码。Step 501: When acquiring the initial hardware time code of each camera, reading the local instantaneous hardware time code.
步骤502，将每一次读取的本地的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取的本地的瞬时硬件时间码做差值运算，得到本地与每个相机之间的操作延时值。Step 502: Perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when the initial hardware time code of the first camera is obtained, to obtain a local and each camera Operation delay value.
步骤503，根据所述多个初始硬件时间码和得到的本地与每个相机之间的操作延时值计算获得每个相机的修正硬件时间码。Step 503: Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained operation delay value between the local and each camera.
步骤501中，在向每个相机发出初始硬件时间码的获取请求之前都先读取一遍控制装置本身的硬件时间码，记为瞬时硬件时间码。这也就要求控制装置本身也是含有时间码寄存器的，用于读写当前控制装置的硬件时间码。然后再向多相机***中的每个相机发出初始硬件时间码的获取请求，多相机***中的相机接收到所述请求后读取本相机的硬件时间码并返回本相机的硬件时间码给控制装置。这时需要将读取第一个相机的初始硬件时间码时读取的控制装置本身的瞬时硬件时间码作为一个参照；每次获取其它相机的初始硬件时间码时，控制装置读取的自身的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取到的自身的瞬时硬件时间码之间的差值就是操作延时值，因为控制装置在读取到自身的瞬时硬件时间码之后会立即获取第一个相机的初始硬件时间码，因此可以认为控制装置与第一个相机之间没有操作延时，而控制装置在获取其它相机的初始硬件时间码的时间点是在获取第一个相机的初始硬件时间码之后，因此与控制装置获取第一个相机的初始硬件时间码时读取到的控制装置本身的瞬时硬件时间码之间是存在操作延时的，因此需要计算这段时间的操作延时。In step 501, the hardware time code of the control device itself is read once before the initial hardware time code acquisition request is sent to each camera, and is recorded as the instantaneous hardware time code. This also requires that the control device itself also contains a time code register for reading and writing the hardware time code of the current control device. Then, an initial hardware time code acquisition request is sent to each camera in the multi-camera system, and the camera in the multi-camera system reads the hardware time code of the camera and returns the hardware time code of the camera to the control after receiving the request. Device. At this time, it is necessary to take the instantaneous hardware time code of the control device itself read when the initial hardware time code of the first camera is read as a reference; each time the initial hardware time code of the other camera is acquired, the control device reads its own The difference between the instantaneous hardware time code and its own instantaneous hardware time code read when the initial hardware time code of the first camera is acquired is the operation delay value because the control device is reading its own instantaneous hardware time code. Immediately afterwards, the initial hardware time code of the first camera is obtained, so it can be considered that there is no operation delay between the control device and the first camera, and the control device acquires the first time when acquiring the initial hardware time code of the other camera. There is an operational delay between the initial hardware time code of a camera and therefore the instantaneous hardware time code of the control device itself read by the control device when acquiring the initial hardware time code of the first camera, so this needs to be calculated The operation delay of the segment time.
即操作延时值C i=T 1i- T 11。其中，T 1i为获取第i相机初始硬件时间码之前读取的控制装置本地的瞬时硬件时间码，T 11为读取第1个相机的初始硬件时间码时读取的控制装置本地的瞬时硬件时间码。其中，其中i的取值范围为1到N。当i等于1时表示控制装置与第一个相机之间的操作延时值。其中，控制装置与第一个相机的操作延时值为0，也可以记为C 1=T 11- T 11。 That is, the operation delay value C i = T 1i - T 11 . Where T 1i is the instantaneous hardware time code local to the control device read before the initial hardware time code of the i-th camera is acquired, and T 11 is the instantaneous hardware local to the control device read when the initial hardware time code of the first camera is read. Time code. Where i has a value ranging from 1 to N. When i is equal to 1, it indicates the operation delay value between the control device and the first camera. Wherein, the operation delay value of the control device and the first camera is 0, and may also be recorded as C 1 =T 11 - T 11 .
那么在经操作延时修正处理之后，相机的修正硬件时间码N i= T i- C i,其中T i为相机i的初始硬件时间码，C i相机i的操作延时值，i的取值范围为1到N。经操作延时修正处理后的相机的修正硬件时间码N i= T i- C i，其中i的取值范围为1到N。 Then after a delay by the correction processing operation, the camera hardware time corrected code N i = T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N. The corrected hardware time code N i = T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
需要说明的一点是，由于步骤202中已经对初始硬件延时码进行了硬件累计修正处理以及网络延时修正处理，因此本步骤中在进行差值运算时应该使用经硬件累计修正处理、网络延时修正处理后的硬件时间码，即M i,其中i的取值范围为1到N。若前文没有进行硬件累计修正处理和网络延时修正处理，那么本步骤中进行差值运算时，使用的便应该是初始的硬件时间码T i，其中i的取值范围为1到N。 It should be noted that since the hardware cumulative correction processing and the network delay correction processing have been performed on the initial hardware delay code in step 202, the hardware cumulative correction processing and network extension should be used in the difference calculation in this step. The hardware time code after processing is corrected, that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
下面，将通过具体的例子进行详细说明。Hereinafter, a detailed description will be given by way of specific examples.
假设，获取到的与多相机***中每个相机之间的操作延时值分别为C 1、C 2、C 3……C N。 It is assumed that the operational delay values obtained between each of the cameras in the multi-camera system are C 1 , C 2 , C 3 ... C N , respectively .
那么经硬件累计修正处理、网络延时修正处理以及操作延时处理之后，得到的相机的修正硬件时间码N i=K i- Y i-C i= T i- T i0- Y i-( T 1i- T 11)，其中，其中i的取值范围为1到N。 Then, after the hardware cumulative correction processing, the network delay correction processing, and the operation delay processing, the obtained corrected hardware time code of the camera is N i =K i - Y i -C i = T i - T i0 - Y i -( T 1i - T 11 ), where i has a value ranging from 1 to N.
需要说明的是，以上实施例虽然是同时采用了网络延时修正处理、操作延时修正处理、硬件累计修正处理三项延时修正处理，但是在实际应用中，可以只选择其中的一项或者两项延时修正处理。在将初始硬件时间码做延时修正处理得到修正硬件时间码的过程中选择了哪项延时修正处理，相应的就将哪项延时修正处理获得的延时结果和初始硬件时间码一起计算获得修正硬件时间码，具体以上三项延时修正处理中只选择其中的一项延时修正处理的实施例和选择其中任两项延时修正处理的实施例在此不再举例说明，可参照上述实施例的计算过程获得。It should be noted that although the above embodiment adopts three delay correction processings of network delay correction processing, operation delay correction processing, and hardware cumulative correction processing, in practical applications, only one of them may be selected or Two delay correction processing. Which delay correction processing is selected in the process of correcting the hardware time code by performing the delay correction processing on the initial hardware time code, and correspondingly, which delay correction result obtained by the delay correction processing is calculated together with the initial hardware time code Obtaining the modified hardware time code, the embodiment of selecting only one of the delay correction processing in the above three delay correction processing and the embodiment for selecting any two of the delay correction processing are not illustrated herein, and can be referred to The calculation process of the above embodiment is obtained.
步骤203，根据所述多个修正硬件时间码确定所述同步基准值，并根据所述同步基准值和每个相机的修正硬件时间码，确定所述多相机***中每个相机对应的帧补偿值。Step 203: Determine the synchronization reference value according to the plurality of modified hardware time codes, and determine frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and the corrected hardware time code of each camera. value.
由于步骤202中，对初始硬件时间码进行了延时修正处理，那么在本步骤中，确定同步基准值时，则需要根据经修正处理后的多个修正时间码确定同步基准值。具体地，在确定同步基准值时，可以选择多个修正硬件时间码的任意一个值作为同步基准值，也可以选择所述多个修正硬件时间码中的最大值或最小值作为同步基准值，当然也可以计算所述多个修正硬件时间码的平均值，并将该平均值作为同步基准值。Since the initial hardware time code is subjected to the delay correction processing in step 202, in this step, when the synchronization reference value is determined, it is necessary to determine the synchronization reference value based on the corrected plurality of corrected time codes. Specifically, when determining the synchronization reference value, any one of the plurality of modified hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of modified hardware time codes may be selected as the synchronization reference value. It is of course also possible to calculate an average of the plurality of modified hardware time codes and use the average as a synchronization reference value.
若只对初始硬件时间码进行了硬件累计修正处理，那么则需要根据经硬件累计修正处理后的硬件时间码确定同步基准值，以此类推。由于本发明实施例中，对初始硬件时间码进行了硬件累计修正处理、网络延时修正处理以及操作延时修正处理，那么则应该根据经硬件累计修正处理、网络延时修正处理以及操作延时处理之后得到的每个相机的修正硬件时间码确定同步基准值。在选取同步基准值之后，所有的相机都与选取的同步基准值对应的相机对齐。If only the hardware cumulative correction processing is performed on the initial hardware time code, then the synchronization reference value needs to be determined according to the hardware time code after the hardware cumulative correction processing, and so on. In the embodiment of the present invention, the hardware cumulative correction processing, the network delay correction processing, and the operation delay correction processing are performed on the initial hardware time code, then the hardware cumulative correction processing, the network delay correction processing, and the operation delay should be performed according to the hardware. The corrected hardware time code of each camera obtained after processing determines the synchronization reference value. After selecting the sync reference value, all cameras are aligned with the camera corresponding to the selected sync reference value.
步骤S204，将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便于每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。Step S204, sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems The camera's exposure time is synchronized.
在本发明实施例中，由于曝光是根据时间码来确定的，若相机的修正硬件时间码大于同步基准值，则说明相机的硬件时间码相对与同步基准值走的快了，需要将相机当前帧图像或下一帧图像延长一定时间，以便于同步基准值对应的相机的时间码能赶上，从而在采集下帧或下下帧图像时曝光时间同步；若是相机的修正硬件时间码小于同步基准值，则说明相机的时间码走的慢了，需要赶上同步基准值对应的相机的时间码，因此需要将相机当前帧图像或下一帧图像缩短一定时间，以便于同步基准值对应的相机的时间码能赶上，从而在采集下帧或下下帧图像时曝光同步。In the embodiment of the present invention, since the exposure is determined according to the time code, if the corrected hardware time code of the camera is greater than the synchronization reference value, the hardware time code of the camera is relatively faster than the synchronization reference value, and the current camera needs to be The frame image or the next frame image is extended for a certain period of time, so that the time code of the camera corresponding to the synchronization reference value can be caught up, so that the exposure time is synchronized when the lower frame or the lower frame image is acquired; if the corrected hardware time code of the camera is smaller than the synchronization The reference value indicates that the time code of the camera is slow, and it is necessary to catch up with the time code of the camera corresponding to the synchronization reference value. Therefore, it is necessary to shorten the current frame image or the next frame image of the camera for a certain time so as to synchronize the reference value. The camera's time code can be caught up so that the exposure is synchronized when the next frame or the next frame is captured.
本发明实施例的多相机***同步曝光的方法，在接收到曝光同步的指令之后，获取每个相机的初始硬件时间码，并对该多个初始硬件时间码进行延时修正处理，得到多个修正硬件时间码，以及根据该多个修正硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值，以及将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机能够根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。The method for synchronously exposing a multi-camera system according to an embodiment of the present invention acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes to obtain a plurality of Correcting a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera And transmitting a frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value so that all of the multi-camera system The camera's exposure time is synchronized.
本发明的实施例可以将该方法应用于控制装置，用于控制多相机***中每个相机实现同步曝光，同时，该方法还可以应用于多相机***中的一个相机，也就是可以把该控制装置集成到多相机***中的一个相机当中。可以选取多相机***中一个相机作为主相机，其它命名为从相机，控制装置集成到主相机中，或者该方法直接应用于主相机中。获取的控制装置与每个相机的延时值用主相机与每个相机的延时值代替。主相机与自身的延时值可以采用数值0代替。具体将所述主相机作为控制装置实现多相机***中每个相机同步曝光可以通过以上内容获取，在此不一一举例。Embodiments of the present invention may apply the method to a control device for controlling simultaneous synchronization of each camera in a multi-camera system, and the method may also be applied to one camera in a multi-camera system, that is, the control may be The device is integrated into one of the cameras in the multi-camera system. One camera in the multi-camera system can be selected as the main camera, the other is named as the slave camera, the control device is integrated into the main camera, or the method is directly applied to the main camera. The acquired control device and the delay value of each camera are replaced by the delay values of the main camera and each camera. The delay value of the main camera and itself can be replaced by the value 0. Specifically, the main camera is used as a control device to realize synchronous exposure of each camera in the multi-camera system, which can be obtained by the above content, which is not exemplified herein.
请参见图6，是本发明提供的多相机***同步曝光的控制方法的第三实施例的流程示意图，如图6所示该多相机***同步曝光的控制方法可以包括以下步骤：6 is a schematic flowchart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention. The control method for synchronous exposure of the multi-camera system shown in FIG. 6 may include the following steps:
步骤S601，在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码。Step S601, after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
步骤S602，对得到的多个初始硬件时间码依次进行硬件累计修正处理、网络延时处理以及操作延时处理，得到多个修正硬件时间码。Step S602, sequentially performing hardware cumulative correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes to obtain a plurality of modified hardware time codes.
步骤S603，根据所述多个修正硬件时间码确定同步基准值。Step S603, determining a synchronization reference value according to the plurality of modified hardware time codes.
步骤S604，按照公式：B i=(N i-S)%FrameLength计算相机的帧补偿值。 In step S604, the frame compensation value of the camera is calculated according to the formula: B i = (N i - S) % FrameLength.
步骤S605，将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机。Step S605, sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system.
本实施例与图2所示的实施例的区别有：The difference between this embodiment and the embodiment shown in FIG. 2 is as follows:
在确定帧补偿值时，具体采用步骤S604中的公式。步骤S604中，B i表示相机i的帧补偿值，N i表示经延时修正处理后相机i的修正硬件时间码，S表示同步基准值，FrameLength表示所述多相机***中每个相机的一帧图像以振荡周期为单位的大小，%表示模运算。具体的帧补偿值是相机一帧图像以晶振的振荡周期为单位的大小，也就是转换成时间码的大小。取模运算是为了相机的帧补偿值不会超过一帧图像的帧长（转换成时间码的一帧图像的时间长度），只要对齐时钟的相位就可以。 When determining the frame compensation value, the formula in step S604 is specifically employed. In step S604, B i represents the frame compensation value of the camera i, N i represents the corrected hardware time code of the camera i after the delay correction processing, S represents the synchronization reference value, and FrameLength represents one of each camera in the multi-camera system. The size of the frame image in units of oscillation periods, and % indicates the modulo operation. The specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code. The modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
第二，在具体处理时，确定出的同步基准值可以是多个修正硬件时间码中的最小值，即从多个修正硬件时间码中选取一个最小值作为同步基准值。选取修正硬件时间码中的最小值得目的是为了简化计算，同时也为了获取更好的效果。这时，所有的相机都只需要拉长当前帧或下帧图像的时间长度就行。Second, in the specific processing, the determined synchronization reference value may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value. The minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
在计算出相机的帧补偿值之后，多相机***中的相机便可根据该帧补偿值相应调整当前帧或下一帧图像的长度，使得在相机在采集下帧或下下帧图像时可以实现曝光同步。After calculating the frame compensation value of the camera, the camera in the multi-camera system can adjust the length of the current frame or the next frame image according to the frame compensation value, so that the camera can realize when acquiring the lower frame or the lower frame image. Exposure sync.
应理解，在上述实施例中，各步骤的序号的大小并不意味着执行顺序的先后，各步骤的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。It should be understood that, in the above embodiments, the size of the serial number of each step does not mean the order of execution sequence, and the order of execution of each step should be determined by its function and internal logic, and should not constitute any implementation process of the embodiment of the present invention. limited.
上述图1至图6对多相机***同步曝光的控制方法进行了详细的描述，下面将结合附图，对多相机***同步曝光的控制装饰着、终端设备以及计算机可读存储介质进行详细描述。为避免赘述，上文中已经描述的术语在下文中可能不再做重复说明。The above-mentioned FIG. 1 to FIG. 6 describe in detail the control method of the synchronous exposure of the multi-camera system. The control decoration of the multi-camera system synchronous exposure, the terminal device and the computer-readable storage medium will be described in detail below with reference to the accompanying drawings. To avoid redundancy, the terms already described above may not be repeatedly described below.
请参见图7，图7是本发明提供的控制装置700的结构框图，为了便于说明，仅示出与本发明实施例相关的部分。该控制装置700可以是内置于终端设备（手机、电脑、计算机）内的软件单元、硬件单元或者软硬结合的单元，也可以作为独立的挂件集成到所述终端设备中；还可以是内置于主相机内的软件单元、硬件单元或者软硬结合的单元，也可以作为独立的挂件集成到所述主相机中。该同步曝光的装置700包括：Referring to FIG. 7, FIG. 7 is a structural block diagram of a control apparatus 700 provided by the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown. The control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant. The synchronous exposure apparatus 700 includes:
获取模块701，用于在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码。The obtaining module 701 is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes.
确定模块702，用于根据所述多个初始硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；a determining module 702, configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera Compensation value
发送模块703，用于将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。The sending module 703 is configured to send the frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system The exposure time of all cameras is synchronized.
如图8所示，是本发明提供的应用于多相机***同步曝光的控制装置700的结构框图，为了便于说明，仅示出与本发明实施例相关的部分。该控制装置700可以是内置于终端设备（手机、电脑、计算机）内的软件单元、硬件单元或者软硬结合的单元，也可以作为独立的挂件集成到所述终端设备中；还可以是内置于主相机内的软件单元、硬件单元或者软硬结合的单元，也可以作为独立的挂件集成到所述主相机中。As shown in FIG. 8, it is a structural block diagram of a control apparatus 700 for simultaneous exposure of a multi-camera system provided by the present invention, and for convenience of explanation, only parts related to the embodiment of the present invention are shown. The control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant.
该控制装置700包括：获取模块701、确定模块702、发送模块703。本发明实施例与图7所示的实施例的区别在于：还包括修正模块704。The control device 700 includes an acquisition module 701, a determination module 702, and a transmission module 703. The difference between the embodiment of the present invention and the embodiment shown in FIG. 7 is that the correction module 704 is further included.
修正模块704，用于对获取模块701获得的多个初始硬件时间码依次进行硬件累计修正处理、网络延时处理以及操作延时处理。The correction module 704 is configured to sequentially perform hardware cumulative correction processing, network delay processing, and operation delay processing on the plurality of initial hardware time codes obtained by the obtaining module 701.
下面，将分别描述修正模块704对多个初始硬件时间码进行硬件累计修正处理、网络延时处理以及操作延时处理的具体操作过程。In the following, a specific operation process of the hardware accumulation correction processing, the network delay processing, and the operation delay processing of the plurality of initial hardware time codes by the correction module 704 will be respectively described.
修正模块704具体可以包括：获取单元，以及修正单元。具体地，修正模块704对多个初始硬件时间码进行硬件累计修正处理时，具体操作方法可以是：The correction module 704 may specifically include: an obtaining unit, and a correcting unit. Specifically, when the correction module 704 performs hardware cumulative correction processing on a plurality of initial hardware time codes, the specific operation method may be:
获取单元对应获取每个相机最近一次中断触发时刻的中断时间码，修正单元将所述多个初始硬件时间码与获取的所述中断时间码做差值运算，对应得到每个相机的修正硬件时间码。The acquiring unit is configured to obtain an interrupt time code of the last interrupt trigger time of each camera, and the correcting unit performs a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtains a corrected hardware time of each camera. code.
根据前文的描述可知，在产生一次中断时，硬件计时器可以自动锁存一个时间码。因此，每个相机在读取自身的初始硬件时间码之后，还读取最近一次中断触发时刻的硬件时间码，记为中断时间码。然后，将读取的初始硬件时间码与中断时间码发送至获取单元，获取单元将从相机处获取的初始硬件时间码与中断时间码传送至修正单元处理。修正单元初始硬件时间码与中断时间码做差，便可消除晶振产生的误差，得到经硬件累积延时处理后的修正硬件时间码。According to the foregoing description, the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, the read initial hardware time code and the interrupt time code are sent to the acquiring unit, and the acquiring unit transmits the initial hardware time code and the interrupt time code acquired from the camera to the correcting unit process. By correcting the initial hardware time code of the correction unit and the interruption time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
举例来说，修正模块704在对相机的初始硬件时间码进行修正时，若相机的初始硬件时间码为T i，获取到的相机的最近一次中断的中断时间码为T i0，那么经硬件累计修正处理操作后相机的修正硬件时间码K i= T i- T i0，其中T i为相机i的初始硬件时间码，T i0 为相机i最近一次中断的中断时间码，i的取值范围为1到N。 For example, when the correction module 704 corrects the initial hardware time code of the camera, if the initial hardware time code of the camera is T i , and the acquired interrupt time code of the latest interrupt of the camera is T i0 , then the hardware is accumulated. Correct the corrected hardware time code K i = T i - T i0 of the camera after the processing operation, where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the last interrupt of camera i, and the value range of i is 1 to N.
具体地，修正模块704对多个初始硬件时间码进行网络延时修正处理时，具体操作方法可以是：Specifically, when the correction module 704 performs network delay correction processing on multiple initial hardware time codes, the specific operation method may be:
获取单元获取本地与多相机***中每个相机之间的网络延时值，修正单元将所述多个初始硬件时间码与获取单元获取的本地与每个相机之间的网络延时值对应做差值运算，对应得到每个相机的修正硬件时间码。The obtaining unit acquires a network delay value between each camera in the local and multi-camera system, and the correcting unit associates the plurality of initial hardware time codes with a network delay value between the local and each camera acquired by the acquiring unit The difference operation corresponds to the corrected hardware time code of each camera.
获取单元在获取网络延时值时，操作方法可以是：通过IEEE 1588 精密时钟同步协议测量与每个相机之间的网络延时，并以传感器时钟晶振周期为单位进行换算，由此可以得出此时控制装置本身与每个相机之间的网络延时。When the acquisition unit obtains the network delay value, the operation method may be: measuring the network delay between each camera and the camera through the IEEE 1588 precision clock synchronization protocol, and performing conversion in units of the sensor clock crystal period, thereby obtaining At this point, the network delay between the control device itself and each camera.
其中，修正单元在将所述多个初始硬件时间码与获取的控制装置本地与每个相机之间的网络延时值对应做差值运算时，在经网络延时修正处理之后，相机的修正硬件时间码M i= T i- Y i，其中T i为相机i的初始硬件时间码，Y i是相机i的网络延时值，i的取值范围为1到N。 Wherein, the correction unit performs the difference correction after the network delay correction processing is performed on the network delay value between the plurality of initial hardware time codes and the acquired control device and each camera. The hardware time code M i = T i - Y i , where T i is the initial hardware time code of camera i, Y i is the network delay value of camera i, and i ranges from 1 to N.
同时，由于已经对初始硬件延时码进行了硬件累计修正处理，因此此时在进行差值运算时应该使用经硬件累计修正处理后的硬件时间码，即T i- T i0,其中i的取值范围为1到N。若前文没有进行硬件累计修正处理，那么此时进行差值运算时，使用的便应该是初始的硬件时间码T i，其中i的取值范围为1到N。 At the same time, since the hardware cumulative correction processing has been performed on the initial hardware delay code, the hardware time code after the hardware cumulative correction processing, that is, T i - T i0 , where i is taken, should be used when performing the difference calculation. Values range from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
具体地，修正模块703对多个初始硬件时间码进行操作延时修正处理时，具体操作方法可以是：Specifically, when the correction module 703 performs an operation delay correction process on a plurality of initial hardware time codes, the specific operation method may be:
当获取每个相机的初始硬件时间码时，获取单元读取所述控制装置本地的瞬时硬件时间码，修正单元将每一次读取的控制装置本地的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取的控制装置本地的瞬时硬件时间码做差值运算，得到控制装置与每个相机之间的操作延时值，以及根据所述多个初始硬件时间码和获取的所述操作延时值计算获得每个相机的修正硬件时间码。When acquiring the initial hardware time code of each camera, the obtaining unit reads the instantaneous hardware time code local to the control device, and the correction unit will respectively read the instantaneous hardware time code local to the control device and acquire the first camera. The instantaneous hardware time code local to the control device read at the initial hardware time code is subjected to a difference operation to obtain an operation delay value between the control device and each camera, and according to the plurality of initial hardware time codes and acquired locations. The operation delay value is calculated to obtain the corrected hardware time code of each camera.
在向每个相机发出初始硬件时间码的获取请求之前，获取单元都先读取一遍控制装置本地的硬件时间码，计为瞬时硬件时间码。然后再向每个相机发出初始硬件时间码的获取请求，每个相机接收到所述请求后读取每个相机本身的硬件时间码并发送给控制装置。修正单元将读取单元读取的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取的控制装置本地的瞬时硬件时间码做差值运算，便可得到控制装置与每个相机之间的操作延时值。即操作延时值C i=T 1i- T 11。其中，T 1i为获取第i相机初始硬件时间码之前读取的控制装置本身的瞬时硬件时间码，T 11为读取第1个相机的硬件时间码时读取的控制装置本身的瞬时硬件时间码。其中，其中i的取值范围为1到N，当i等于1时表示控制装置与第一个相机之间的操作延时值。其中，控制装置与第一个相机的操作延时值为0，也可以记为C 1=T 11- T 11。 Before the initial hardware time code acquisition request is sent to each camera, the acquisition unit first reads the hardware time code local to the control device, which is counted as the instantaneous hardware time code. An acquisition request for the initial hardware time code is then sent to each camera, and each camera receives the request and reads the hardware time code of each camera itself and sends it to the control device. The correction unit compares the instantaneous hardware time code read by the reading unit with the instantaneous hardware time code of the control device read when the initial hardware time code of the first camera is acquired, thereby obtaining the control device and each camera The operation delay value between. That is, the operation delay value C i = T 1i - T 11 . Where T 1i is the instantaneous hardware time code of the control device itself read before acquiring the initial hardware time code of the i-th camera, and T 11 is the instantaneous hardware time of the control device itself read when reading the hardware time code of the first camera code. Where i is in the range of 1 to N, and when i is equal to 1, it represents the operational delay value between the control device and the first camera. Wherein, the operation delay value of the control device and the first camera is 0, and may also be recorded as C 1 =T 11 - T 11 .
那么在经操作延时修正处理之后，相机的修正硬件时间码N i= T i- C i,其中T i为相机i的初始硬件时间码，C i相机i的操作延时值，i的取值范围为1到N。经操作延时修正处理后的相机的修正硬件时间码N i= T i- C i，其中i的取值范围为1到N。 Then after a delay by the correction processing operation, the camera hardware time corrected code N i = T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N. The corrected hardware time code N i = T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
需要说明的一点是，由于已经对初始硬件延时码进行了硬件累计修正处理以及网络延时修正处理，因此此时在进行差值运算时应该使用经硬件累计修正处理、网络延时修正处理后的硬件时间码，即M i,其中i的取值范围为1到N。若前文没有进行硬件累计修正处理和网络延时修正处理，那么此时进行差值运算时，使用的便应该是初始的硬件时间码T i，其中i的取值范围为1到N。 It should be noted that since the hardware cumulative correction processing and the network delay correction processing have been performed on the initial hardware delay code, the hardware cumulative correction processing and the network delay correction processing should be used when performing the difference calculation. The hardware time code, that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
本发明实施例的多相机***同步曝光的控制装置，在接收到曝光同步的指令之后，通过获取每个相机的初始硬件时间码，并对该多个初始硬件时间码进行延时修正处理，得到多个修正硬件时间码，以及根据该多个修正硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；最后将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机能够根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。The control device for synchronous exposure of the multi-camera system of the embodiment of the present invention obtains the initial hardware time code of each camera after receiving the instruction of the exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes. Determining a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining, corresponding to each camera in the multi-camera system, according to the synchronization reference value and an initial hardware time code of each camera Frame compensation value; finally transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value such that the multi-camera system The exposure time of all cameras is synchronized.
进一步需要说明的是，本发明实施例的确定模块702具体包括：同步基准值单元以及帧补偿值确定单元。It should be further noted that the determining module 702 of the embodiment of the present invention specifically includes: a synchronization reference value unit and a frame compensation value determining unit.
帧补偿值确定单元在确定帧补偿值时，具体采用B i=(N i-S)%FrameLength计算相机的帧补偿值。其中，B i表示相机i的帧补偿值，N i表示经延时修正处理后相机i的修正硬件时间码，S表示同步基准值，FrameLength表示所述多相机***中每个相机的一帧图像以振荡周期为单位的大小，%表示模运算。具体的帧补偿值是相机一帧图像以晶振的振荡周期为单位的大小，也就是转换成时间码的大小。取模运算是为了相机的帧补偿值不会超过一帧图像的帧长（转换成时间码的一帧图像的时间长度），只要对齐时钟的相位就可以。 The frame compensation value determining unit calculates the frame compensation value of the camera by specifically using B i =(N i -S)%FrameLength when determining the frame compensation value. Wherein, B i represents the frame compensation value of the camera i, N i represents the corrected hardware time code of the camera i after the delay correction processing, S represents the synchronization reference value, and FrameLength represents a frame image of each camera in the multi-camera system. The size in units of oscillation periods, % represents the modulo operation. The specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code. The modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
具体处理时，同步基准值确定单元确定出的同步基准值可以是多个修正硬件时间码中的最小值，即从多个修正硬件时间码中选取一个最小值作为同步基准值。选取修正硬件时间码中的最小值得目的是为了简化计算，同时也为了获取更好的效果。这时，所有的相机都只需要拉长当前帧或下帧图像的时间长度就行。In the specific processing, the synchronization reference value determined by the synchronization reference value determining unit may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value. The minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上集成在一个单元或者模块中，上述集成的单元或模块既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述控制装置中模块的具体工作过程，可以参考前述方法实施例的对应过程，在此不再赘述。It will be clearly understood by those skilled in the art that for the convenience and brevity of the description, only the division of each functional module described above is exemplified. In practical applications, the above function assignment can be completed by different functional units and modules as needed. That is, the internal structure of the device is divided into different functional units or modules to perform all or part of the functions described above. Each functional module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit or module, and the integrated unit or module may adopt hardware. The form is implemented in the form of a software functional unit. In addition, the specific names of the respective functional modules are only for the purpose of distinguishing from each other, and are not intended to limit the scope of protection of the present application. For the specific working process of the module in the foregoing control device, refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
图9是本发明一实施例提供的终端设备的示意框图。该终端设备还可以是多相机***中的一个相机，例如主相机。如图9所示，该实施例的终端设备9包括：一个或多个处理器90、存储器91以及存储在所述存储器91中并可在所述处理器90上运行的计算机程序92。所述处理器90执行所述计算机程序92时实现上述各个多相机***同步曝光的控制方法实施例中的步骤，例如图1所示的步骤S101至S103。或者，所述处理器90执行所述计算机程序92时实现上述终端设备实施例中各模块/单元的功能，例如图7所示模块701至703的功能。FIG. 9 is a schematic block diagram of a terminal device according to an embodiment of the present invention. The terminal device can also be a camera in a multi-camera system, such as a main camera. As shown in FIG. 9, the terminal device 9 of this embodiment includes: one or more processors 90, a memory 91, and a computer program 92 stored in the memory 91 and operable on the processor 90. The steps in the embodiment of the control method for realizing the synchronous exposure of each of the multi-camera systems described above when the processor 90 executes the computer program 92, such as steps S101 to S103 shown in FIG. Alternatively, the processor 90 implements the functions of the modules/units in the terminal device embodiment when the computer program 92 is executed, such as the functions of the modules 701 to 703 shown in FIG.
示例性的，所述计算机程序92可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器91中，并由所述处理器90执行，以完成本发明。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序92在所述终端设备9中的执行过程。例如，所述计算机程序92可以被分割成获取模块、确定模块、发送模块。Illustratively, the computer program 92 can be partitioned into one or more modules/units that are stored in the memory 91 and executed by the processor 90 to complete this invention. The one or more modules/units may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program 92 in the terminal device 9. For example, the computer program 92 can be partitioned into an acquisition module, a determination module, and a transmission module.
所述获取模块，用于在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码；The acquiring module is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes;
所述确定模块，用于根据所述多个初始硬件时间码确定同步基准值，以及根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；The determining module is configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine, according to the synchronization reference value and an initial hardware time code of each camera, each camera in the multi-camera system Frame compensation value;
所述发送模块，用于将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。The sending module is configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value to make the multi-camera The exposure time of all cameras in the system is synchronized.
可选的，还可以包括：Optionally, it may also include:
修正模块，用于对所述多个初始硬件时间码做延时修正处理，得到多个修正硬件时间码；a correction module, configured to perform delay correction processing on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes;
所述延时修正处理包括以下至少一项:网络延时修正处理、操作延时修正处理以及硬件累计修正处理；The delay correction processing includes at least one of the following: a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing;
可选的，所述确定模块具体用于：Optionally, the determining module is specifically configured to:
根据所述多个修正硬件时间码确定所述同步基准值；Determining the synchronization reference value according to the plurality of modified hardware time codes;
根据所述同步基准值和所述多个修正硬件时间码，确定所述多相机***中每个相机对应的帧补偿值。And determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and the plurality of modified hardware time codes.
可选的，所述修正模块包括：Optionally, the modification module includes:
获取单元，用于获取本地与所述多相机***中每个相机之间的网络延时值；An obtaining unit, configured to acquire a network delay value between the local and each camera in the multi-camera system;
修正单元，用于根据所述多个初始硬件时间码和获取的本地与每个相机之间的网络延时值计算获得每个相机的修正硬件时间码。And a correction unit, configured to obtain a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera.
可选的，所述获取单元，还用于当获取每个相机的初始硬件时间码时，读取本地的瞬时硬件时间码；Optionally, the acquiring unit is further configured to: when acquiring an initial hardware time code of each camera, read a local instantaneous hardware time code;
所述修正单元，还用于将每一次读取的本地的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取的本地的瞬时硬件时间码做差值运算，得到本地与每个相机之间的操作延时值；The correcting unit is further configured to perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when acquiring the initial hardware time code of the first camera, to obtain local and per Operation delay value between cameras;
根据所述多个初始硬件时间码和得到的本地与每个相机之间的操作延时值计算获得每个相机的修正硬件时间码。A corrected hardware time code for each camera is obtained based on the plurality of initial hardware time codes and the resulting operational delay value between the local and each camera.
可选的，所述获取单元，还用于对应获取每个相机最近一次中断触发时刻的中断时间码；Optionally, the acquiring unit is further configured to: correspondingly acquire an interrupt time code of a last interrupt trigger time of each camera;
所述修正单元，还用于将所述多个初始硬件时间码与获取的所述中断时间码做差值运算，对应得到每个相机的修正硬件时间码。The modifying unit is further configured to perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and correspondingly obtain a corrected hardware time code of each camera.
可选的，所述确定模块具体用于：Optionally, the determining module is specifically configured to:
根据公式B i=(N i-S)%FrameLength计算相机的帧补偿值； Calculating the frame compensation value of the camera according to the formula B i =(N i -S)%FrameLength;
其中，B i表示相机i的帧补偿值，N i表示经延时修正处理后相机i的修正硬件时间码，S表示同步基准值，FrameLength表示所述多相机***中每个相机的一帧图像以振荡周期为单位的大小，%表示模运算。 Wherein, B i represents the frame compensation value of the camera i, N i represents the corrected hardware time code of the camera i after the delay correction processing, S represents the synchronization reference value, and FrameLength represents a frame image of each camera in the multi-camera system. The size in units of oscillation periods, % represents the modulo operation.
所述终端设备包括但不仅限于处理器90、存储器91。本领域技术人员可以理解，图9仅仅是终端设备9的示例，并不构成对终端设备9的限定，该终端设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述终端设备还可以包括输入设备、输出设备、网络接入设备、总线等。The terminal device includes, but is not limited to, a processor 90 and a memory 91. It will be understood by those skilled in the art that FIG. 9 is only an example of the terminal device 9, and does not constitute a limitation on the terminal device 9, which may include more or less components than those illustrated, or combine some components, or Different components, such as the terminal device, may also include input devices, output devices, network access devices, buses, and the like.
所述存储器91，用于存储软件程序、模块、单元以及终端设备中需要的数据信息，所述处理器90通过运行存储在所述存储器91的软件程序、模块以及单元，从而执行各种功能应用以及数据处理，提高复杂场景下多相机***中多个相机同步曝光的精度。该存储器91可以包括只读存储器和随机存取存储器，并向处理器 90 提供指令和数据。存储器 91的一部分还可以包括非易失性随机存取存储器。例如，存储器91还可以存储设备类型的信息。The memory 91 is configured to store software programs, modules, units, and data information required in the terminal device, and the processor 90 executes various functional applications by running software programs, modules, and units stored in the memory 91. And data processing to improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in complex scenes. The memory 91 can include read only memory and random access memory and provides instructions and data to the processor 90. A portion of the memory 91 may also include a non-volatile random access memory. For example, the memory 91 can also store information of the device type.
所述处理器90可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器 (Digital Signal Processor，DSP)、专用集成电路 (Application Specific Integrated Circuit，ASIC)、现成可编程门阵列 (Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。所述处理器是所述终端设备的控制中心，利用各种接口和线路连接整个终端设备的各个部分。The processor 90 can be a central processing unit (Central) Processing Unit (CPU), which can also be other general-purpose processors, digital signal processors (DSPs), and application specific integrated circuits (Application). Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like. The processor is a control center of the terminal device, and connects various parts of the entire terminal device by using various interfaces and lines.
具体实现中，本发明实施例中所描述的处理器90和存储器91可执行本发明实施例提供的多相机***同步曝光的控制方法的实施例中所描述的实现方式，也可执行多相机***同步曝光的控制装置的实施例中所描述的实现方式，在此不再赘述。In a specific implementation, the processor 90 and the memory 91 described in the embodiments of the present invention may implement the implementation manner described in the embodiment of the method for controlling the synchronous exposure of the multi-camera system provided by the embodiment of the present invention, and may also execute the multi-camera system. The implementation described in the embodiment of the synchronous exposure control device will not be described herein.
最后，为了更好的理解本发明实施例与多相机***的关系，通过图10说明本发明所述方法应用于的控制装置7与多相机***10的关系。如图所示，控制装置用于控制多相机***，多相机***包括多个相机101，控制装置7可以表示以上控制装置任一实施例所述的控制装置，还可以表示以上任一实施例所述的终端设备。Finally, in order to better understand the relationship between the embodiment of the present invention and the multi-camera system, the relationship between the control device 7 to which the method of the present invention is applied and the multi-camera system 10 will be described with reference to FIG. As shown, the control device is used to control a multi-camera system, and the multi-camera system includes a plurality of cameras 101. The control device 7 can represent the control device described in any of the above control devices, and can also represent any of the above embodiments. The terminal device described.
另外，本发明实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被一个或多个处理器执行时实现本发明实施例提供的应用于多相机***同步曝光的控制方法的步骤。In addition, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the application provided by the embodiment of the present invention is implemented. The steps of the control method for simultaneous exposure of a multi-camera system.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above embodiments, the descriptions of the various embodiments are different, and the parts that are not detailed or described in a certain embodiment can be referred to the related descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的模块及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art will appreciate that the modules and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
在本发明所提供的实施例中，应该理解到，所揭露的控制方法、控制装置以及终端设备，可以通过其它的方式实现。例如，以上所描述的实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。In the embodiments provided by the present invention, it should be understood that the disclosed control method, control device, and terminal device may be implemented in other manners. For example, the embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外，在本发明各个实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备（可以是个人计算机，服务器，或者网络设备等）或处理器（processor）执行本发明实施例各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage. The medium includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), and a random access memory (RAM, Random Access). A variety of media that can store program code, such as a memory, a disk, or an optical disk.
以上所述实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明实施例各实施例技术方案的精神和范围。The embodiments described above are only for explaining the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that The technical solutions described in the examples are modified, or the equivalents of the technical features are replaced by the equivalents of the technical solutions of the embodiments of the present invention.

Claims (10)

  1. 一种多相机***同步曝光的控制方法，其特征在于，所述控制方法包括：A control method for synchronous exposure of a multi-camera system, characterized in that the control method comprises:
    在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码；After receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes;
    根据所述多个初始硬件时间码确定同步基准值，并根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；Determining a synchronization reference value according to the plurality of initial hardware time codes, and determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera;
    将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。Transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value such that the exposure time of all the cameras in the multi-camera system Synchronize.
  2. 根据权利要求1所述的方法，其特征在于，在所述获取所述多相机***中每个相机的初始硬件时间码之后，所述控制方法还包括：The method according to claim 1, wherein after the obtaining the initial hardware time code of each camera in the multi-camera system, the control method further comprises:
    对所述多个初始硬件时间码做延时修正处理，得到多个修正硬件时间码；Performing delay correction processing on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes;
    所述延时修正处理包括以下至少一项:网络延时修正处理、操作延时修正处理以及硬件累计修正处理；The delay correction processing includes at least one of the following: a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing;
    所述根据所述多个初始硬件时间码确定同步基准值的步骤，具体包括：The step of determining a synchronization reference value according to the multiple initial hardware time codes includes:
    根据所述多个修正硬件时间码确定所述同步基准值；Determining the synchronization reference value according to the plurality of modified hardware time codes;
    所述根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值，具体包括：Determining, according to the synchronization reference value and an initial hardware time code of each camera, a frame compensation value corresponding to each camera in the multi-camera system, specifically:
    根据所述同步基准值和所述多个修正硬件时间码，确定所述多相机***中每个相机对应的帧补偿值。And determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and the plurality of modified hardware time codes.
  3. 根据权利要求2所述的方法，其特征在于，所述延时修正处理包括：The method of claim 2 wherein said delay correction process comprises:
    网络延时修正处理；Network delay correction processing;
    所述对所述多个初始硬件时间码做延时修正处理，得到多个修正硬件时间码的步骤，具体包括：The step of performing a delay correction process on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes, specifically including:
    获取本地与多相机***中每个相机之间的网络延时值；Get network latency values between each camera in the local and multi-camera systems;
    根据所述多个初始硬件时间码和获取的本地与每个相机之间的网络延时值计算获得每个相机的修正硬件时间码。A corrected hardware time code for each camera is obtained based on the plurality of initial hardware time codes and the acquired network delay value between the local and each camera.
  4. 根据权利要求2所述的方法，其特征在于，所述延时修正处理包括：操作延时修正处理；The method according to claim 2, wherein said delay correction processing comprises: operation delay correction processing;
    所述对所述多个初始硬件时间码做延时修正处理，得到多个修正硬件时间码的步骤，具体包括：The step of performing a delay correction process on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes, specifically including:
    当获取每个相机的初始硬件时间码时，读取本地的瞬时硬件时间码；Reading the local instantaneous hardware time code when acquiring the initial hardware time code of each camera;
    将每一次读取的本地的瞬时硬件时间码与获取第一个相机的初始硬件时间码时读取的本地的瞬时硬件时间码做差值运算，得到本地与每个相机之间的操作延时值；The difference between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when the initial camera time code of the first camera is obtained is obtained, and the operation delay between the local and each camera is obtained. value;
    根据所述多个初始硬件时间码和得到的本地与每个相机之间的操作延时值计算获得每个相机的修正硬件时间码。A corrected hardware time code for each camera is obtained based on the plurality of initial hardware time codes and the resulting operational delay value between the local and each camera.
  5. 根据权利要求2所述的方法，其特征在于，所述延时修正处理包括：The method of claim 2 wherein said delay correction process comprises:
    硬件累计修正处理；Hardware cumulative correction processing;
    所述对所述多个初始硬件时间码做延时修正处理，得到多个修正硬件时间码的步骤，具体包括：The step of performing a delay correction process on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes, specifically including:
    对应获取每个相机最近一次中断触发时刻的中断时间码；Corresponding to the interrupt time code for obtaining the last interrupt trigger time of each camera;
    将所述多个初始硬件时间码与获取的所述中断时间码做差值运算，对应得到每个相机的修正硬件时间码。Performing a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and correspondingly obtaining a corrected hardware time code of each camera.
  6. 根据权利要求2所述的方法，其特征在于，所述根据所述同步基准值和每个相机的修正硬件时间码，确定所述多相机***中每个相机对应的帧补偿值的步骤，具体为：The method according to claim 2, wherein said step of determining a frame compensation value corresponding to each camera in said multi-camera system based on said synchronization reference value and a corrected hardware time code of each camera, for:
    根据公式B i=(N i-S)%FrameLength计算相机的帧补偿值； Calculating the frame compensation value of the camera according to the formula B i =(N i -S)%FrameLength;
    其中，B i表示相机i的帧补偿值，N i表示经延时修正处理后相机i的修正硬件时间码，S表示同步基准值，FrameLength表示所述多相机***中每个相机的一帧图像以振荡周期为单位的大小，%表示模运算。 Wherein, B i represents the frame compensation value of the camera i, N i represents the corrected hardware time code of the camera i after the delay correction processing, S represents the synchronization reference value, and FrameLength represents a frame image of each camera in the multi-camera system. The size in units of oscillation periods, % represents the modulo operation.
  7. 根据权利要求1至6任一项所述的方法，其特征在于，所述根据所述多个初始硬件时间码确定同步基准值，具体包括：The method according to any one of claims 1 to 6, wherein the determining the synchronization reference value according to the plurality of initial hardware time codes comprises:
    选择所述多个初始硬件时间码中的最大值或最小值作为同步基准值；或者，Selecting a maximum value or a minimum value of the plurality of initial hardware time codes as a synchronization reference value; or
    计算所述多个初始硬件时间码的平均值，并将所述平均值作为同步基准值。An average of the plurality of initial hardware time codes is calculated and used as a synchronization reference value.
  8. 一种多相机***同步曝光的控制装置，其特征在于，所述控制装置包括：A control device for synchronous exposure of a multi-camera system, characterized in that the control device comprises:
    获取模块，用于在接收到曝光同步的指令后，获取所述多相机***中每个相机的初始硬件时间码，得到多个初始硬件时间码；An acquiring module, configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes;
    确定模块，用于根据所述多个初始硬件时间码确定同步基准值，以及根据所述同步基准值和每个相机的初始硬件时间码，确定所述多相机***中每个相机对应的帧补偿值；a determining module, configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera value;
    发送模块，用于将所述每个相机对应的帧补偿值发送至多相机***中对应的每个相机，以便每个相机根据对应的所述帧补偿值调整图像的长度使得所述多相机***中所有相机的曝光时间同步。a sending module, configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system The exposure time of all cameras is synchronized.
  9. 一种终端设备，其特征在于，所述终端设备包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现权利要求1至7中任一项所述的方法的步骤。A terminal device, comprising: a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor implementing the right when executing the computer program The method of any one of claims 1 to 7.
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述方法的步骤。A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 7.
PCT/CN2018/090744 2017-06-12 2018-06-12 Control method and apparatus for synchronous exposure of multi-camera system, and terminal device WO2018228353A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710439586.5 2017-06-12
CN201710439586.5A CN107277385B (en) 2017-06-12 2017-06-12 Multi-camera system synchronous exposure control method and device and terminal equipment

Publications (1)

Publication Number Publication Date
WO2018228353A1 true WO2018228353A1 (en) 2018-12-20

Family

ID=60066084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/090744 WO2018228353A1 (en) 2017-06-12 2018-06-12 Control method and apparatus for synchronous exposure of multi-camera system, and terminal device

Country Status (2)

Country Link
CN (1) CN107277385B (en)
WO (1) WO2018228353A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107277385B (en) * 2017-06-12 2020-04-17 深圳市瑞立视多媒体科技有限公司 Multi-camera system synchronous exposure control method and device and terminal equipment
CN107911575B (en) * 2017-11-02 2018-10-26 中国矿业大学 A kind of high-precision low-power consumption polyphaser synchronous triggering system
CN110740251B (en) * 2018-07-20 2021-09-24 杭州海康机器人技术有限公司 Multi-camera synchronous stream taking method, device and system
CN111343401B (en) * 2018-12-18 2021-06-01 华为技术有限公司 Frame synchronization method and device
CN109743501A (en) * 2019-01-15 2019-05-10 上海曼恒数字技术股份有限公司 A kind of polyphaser synchronous trigger method, device, equipment and storage medium
CN109842737B (en) * 2019-02-01 2021-04-09 初速度（苏州）科技有限公司 Image exposure method and device and vehicle-mounted terminal
WO2020181494A1 (en) * 2019-03-12 2020-09-17 深圳市大疆创新科技有限公司 Parameter synchronization method, image capture apparatus, and movable platform
CN110186957B (en) * 2019-05-30 2021-08-31 哈尔滨工业大学 Synchronous information acquisition and processing system and method
CN110290287B (en) * 2019-06-27 2022-04-12 上海玄彩美科网络科技有限公司 Multi-camera frame synchronization method
CN113096128B (en) * 2021-06-08 2021-09-14 上海启迪睿视智能科技有限公司 Method for ensuring camera picture alignment based on software
CN114371722B (en) * 2021-12-03 2024-05-10 深圳供电局有限公司 Data acquisition method, device, unmanned aerial vehicle and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103108125A (en) * 2013-01-07 2013-05-15 华中科技大学 Picture-taking synchronous control equipment of multi-camera system and method thereof
JP2013098877A (en) * 2011-11-02 2013-05-20 Sharp Corp Stereo imaging device, synchronous imaging method and electronic information apparatus
CN103792841A (en) * 2014-01-23 2014-05-14 中国科学院长春光学精密机械与物理研究所 Image time hack information generating system of space camera
CN107277385A (en) * 2017-06-12 2017-10-20 深圳市瑞立视多媒体科技有限公司 Control method, device and terminal device that a kind of multicamera system synchronously exposes

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101959022B (en) * 2009-07-17 2012-11-28 深圳泰山在线科技有限公司 Synchronous circuit, and image pick-up device and synchronous exposure control method thereof
JP6241467B2 (en) * 2015-09-11 2017-12-06 カシオ計算機株式会社 Imaging apparatus, imaging control apparatus, imaging method, imaging control method, and program
CN105680969A (en) * 2015-12-31 2016-06-15 浙江中控技术股份有限公司 Clock synchronization method and device
CN105681632B (en) * 2015-12-31 2019-06-28 深圳市华途数字技术有限公司 The method of multi-lens camera and its frame synchronization
CN105680975B (en) * 2016-03-07 2018-06-15 浙江大学 A kind of method for synchronizing time of host-guest architecture multinode network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013098877A (en) * 2011-11-02 2013-05-20 Sharp Corp Stereo imaging device, synchronous imaging method and electronic information apparatus
CN103108125A (en) * 2013-01-07 2013-05-15 华中科技大学 Picture-taking synchronous control equipment of multi-camera system and method thereof
CN103792841A (en) * 2014-01-23 2014-05-14 中国科学院长春光学精密机械与物理研究所 Image time hack information generating system of space camera
CN107277385A (en) * 2017-06-12 2017-10-20 深圳市瑞立视多媒体科技有限公司 Control method, device and terminal device that a kind of multicamera system synchronously exposes

Also Published As

Publication number Publication date
CN107277385A (en) 2017-10-20
CN107277385B (en) 2020-04-17

Similar Documents

Publication Publication Date Title
WO2018228353A1 (en) Control method and apparatus for synchronous exposure of multi-camera system, and terminal device
WO2018228352A1 (en) Synchronous exposure method and apparatus and terminal device
CN109104259B (en) Multi-sensor time synchronization system and method
CN107439000B (en) Synchronous exposure method and device and terminal equipment
KR101389789B1 (en) Image pickup apparatus, image pickup system, image pickup method and computer readable non-transitory recording medium
JP5817664B2 (en) Imaging apparatus, imaging method, and program
US20160088209A1 (en) Synchronous photographing system that controls synchronous photographing by pluraltiy of image capture apparatus
JP6075357B2 (en) Imaging apparatus, imaging control method, and program
Ansari et al. Wireless software synchronization of multiple distributed cameras
JP2019087791A (en) Information processing apparatus, information processing method, and program
CN109391799B (en) Monitoring video synchronization method and device and video acquisition equipment
US20160088210A1 (en) Photographing control apparatus that controls synchronous photographing by plurality of image capture apparatus
WO2020125643A1 (en) Frame synchronization method and apparatus
WO2024060763A1 (en) Wireless smart wearable device and image acquisition method thereof
US20230412918A1 (en) System for wireless synchronized capturing, and smart device
CN107455006B (en) Synchronous exposure method and device and terminal equipment
JP5802727B2 (en) Synchronous camera
WO2019183759A1 (en) Image fusion method, image capturing apparatus, and mobile platform system
JP2022096504A (en) Communication device, method for controlling communication device, and program
JP2005253056A (en) Imaging apparatus
CN117336419A (en) Shooting method, shooting device, electronic equipment and readable storage medium
TWM634644U (en) frame alignment system
WO2019010626A1 (en) Capturing method and terminal
JP2023078537A (en) Communication device
KR20210117635A (en) Synchronization device for camera and synchronization method for camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18818182

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18818182

Country of ref document: EP

Kind code of ref document: A1