WO2016059785A1 - Inspection vehicle control device, control method, and computer program - Google Patents
Inspection vehicle control device, control method, and computer program Download PDFInfo
- Publication number
- WO2016059785A1 WO2016059785A1 PCT/JP2015/005142 JP2015005142W WO2016059785A1 WO 2016059785 A1 WO2016059785 A1 WO 2016059785A1 JP 2015005142 W JP2015005142 W JP 2015005142W WO 2016059785 A1 WO2016059785 A1 WO 2016059785A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- hovering camera
- vehicle
- information
- image
- control terminal
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 139
- 238000007689 inspection Methods 0.000 title description 82
- 238000004590 computer program Methods 0.000 title description 8
- 238000003384 imaging method Methods 0.000 claims abstract description 180
- 239000002131 composite material Substances 0.000 claims abstract description 58
- 238000012545 processing Methods 0.000 claims description 20
- 238000013459 approach Methods 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 description 86
- 238000010586 diagram Methods 0.000 description 78
- 230000010365 information processing Effects 0.000 description 49
- 238000004891 communication Methods 0.000 description 34
- 230000006870 function Effects 0.000 description 33
- 238000005259 measurement Methods 0.000 description 19
- 230000001133 acceleration Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 7
- 238000010276 construction Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 7
- 230000005856 abnormality Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 239000004567 concrete Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000003550 marker Substances 0.000 description 3
- 239000011150 reinforced concrete Substances 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000001151 other effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 235000008733 Citrus aurantifolia Nutrition 0.000 description 1
- 235000011941 Tilia x europaea Nutrition 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005339 levitation Methods 0.000 description 1
- 239000004571 lime Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000011513 prestressed concrete Substances 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0094—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
- G01C11/025—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures by scanning the object
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
- G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/20—Control system inputs
- G05D1/22—Command input arrangements
- G05D1/221—Remote-control arrangements
- G05D1/222—Remote-control arrangements operated by humans
- G05D1/224—Output arrangements on the remote controller, e.g. displays, haptics or speakers
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/60—Intended control result
- G05D1/656—Interaction with payloads or external entities
- G05D1/689—Pointing payloads towards fixed or moving targets
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0047—Navigation or guidance aids for a single aircraft
- G08G5/0069—Navigation or guidance aids for a single aircraft specially adapted for an unmanned aircraft
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0073—Surveillance aids
- G08G5/0086—Surveillance aids for monitoring terrain
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/04—Anti-collision systems
- G08G5/045—Navigation or guidance aids, e.g. determination of anti-collision manoeuvers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/10—Rotorcrafts
- B64U10/13—Flying platforms
- B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/25—UAVs specially adapted for particular uses or applications for manufacturing or servicing
- B64U2101/26—UAVs specially adapted for particular uses or applications for manufacturing or servicing for manufacturing, inspections or repairs
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
- B64U2201/104—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] using satellite radio beacon positioning systems, e.g. GPS
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/20—Remote controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U30/00—Means for producing lift; Empennages; Arrangements thereof
- B64U30/20—Rotors; Rotor supports
Definitions
- the present disclosure relates to a control device, a control method, and a computer program.
- a technology relating to a method for capturing photographs using a camera installed in a radio-controllable flying body has been disclosed (for example, refer to Patent Literature 1).
- Using the camera installed in such a flying body it is possible to capture photographs from the sky or a position in which a tripod is difficult to set.
- Capturing using a camera installed in a flying body brings various advantages in that costs can be suppressed, and safe capturing, capturing at a low altitude or in a narrow place, capturing in proximity to a target, and the like are possible in comparison to when a real aircraft or helicopter is used.
- a dedicated controller is typically necessary.
- the user can simply designate a movement instruction to the mobile object using an image captured by the mobile object, even the user who is unaccustomed to an operation of a mobile object is considered to be able to move the mobile object to a desired position easily.
- the present disclosure proposes a control system, a control method, and a computer program, which are novel and improved and capable of giving an instruction to move the mobile object intuitively using an image obtained by capturing the mobile object.
- a vehicle control system comprising at least one imaging device attached to a vehicle and configured to capture a plurality of images; and a control circuit configured to generate a composite image from the plurality of images, and to display the composite image on a display unit, wherein the vehicle is operated according to a user operation on a portion of the display unit on which the composite image is being displayed.
- a vehicle control method comprising capturing, via at least one imaging device attached to a vehicle, a plurality of images; generating a composite image from the plurality of images, and displaying the composite image on a display unit; and operating the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
- computer system comprising: at least one processing unit; and a memory, the memory including a non-transitory computer-readable medium storing instructions that, when executed by the at least one processing unit, cause the computer system to cause at least one imaging device attached to a vehicle to capture a plurality of images, generate a composite image from the plurality of images, display the composite image on a display unit, and operate the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
- a control system, a control method, and a computer program which are novel and improved and capable of giving an instruction to move the mobile object intuitively using an image captured by the mobile object are provided.
- FIG. 1 is an explanatory diagram for describing an overview of an embodiment of the present disclosure.
- FIG. 2 is an explanatory diagram illustrating an exemplary system configuration of an inspection system 10 according to an embodiment of the present disclosure.
- FIG. 3 is an explanatory diagram illustrating an exemplary function configuration of a hovering camera 100 according to an embodiment of the present disclosure.
- FIG. 4 is an explanatory diagram illustrating an exemplary function configuration of a control terminal 200 according to an embodiment of the present disclosure.
- FIG. 5 is a flowchart illustrating an exemplary operation of an inspection system 10 according to an embodiment of the present disclosure.
- FIG. 6 is an explanatory diagram illustrating an exemplary screen displayed on a display unit 210 of a control terminal 200.
- FIG. 1 is an explanatory diagram for describing an overview of an embodiment of the present disclosure.
- FIG. 2 is an explanatory diagram illustrating an exemplary system configuration of an inspection system 10 according to an embodiment of the present disclosure.
- FIG. 3 is an explanatory diagram illustrating an
- FIG. 7 is an explanatory diagram illustrating an exemplary screen displayed on a display unit 210 of a control terminal 200.
- FIG. 8 is an explanatory diagram illustrating an exemplary screen displayed on a display unit 210 of a control terminal 200.
- FIG. 9 is an explanatory diagram conceptually illustrating an example of capturing of a bottom surface of a bridge 1 by a hovering camera 100.
- FIG. 10 is an explanatory diagram conceptually illustrating an operation of a hovering camera 100 in an inspection system 10 according to an embodiment of the present disclosure.
- FIG. 11 is an explanatory diagram conceptually illustrating an operation of a hovering camera 100 in an inspection system 10 according to an embodiment of the present disclosure.
- FIG. 12 is an explanatory diagram illustrating an exemplary screen displayed on a display unit 210 of a control terminal 200.
- FIG. 13 is an explanatory diagram illustrating an overview when a bottom surface of a bridge girder 3 is inspected.
- FIG. 14 is an explanatory diagram illustrating an example of an image 20 obtained by stitching still images captured by a hovering camera 100.
- FIG. 15 is an explanatory diagram illustrating an exemplary function configuration of an information processing device 300 according to an embodiment of the present disclosure.
- FIG. 16 is a flowchart illustrating an exemplary operation of an information processing device 300 according to an embodiment of the present disclosure.
- FIG. 17 is a flowchart illustrating an exemplary operation of a control terminal 200 according to an embodiment of the present disclosure.
- FIG. 18 is an explanatory diagram illustrating an example in which a hovering camera 100 is caused to capture a ground direction.
- FIG. 19 is an explanatory diagram illustrating an example in which a user is caused to designate a flight path of a hovering camera 100 using a combined image.
- FIG. 20 is an explanatory diagram illustrating an example in which a hovering camera 100 is caused to capture an upward direction (a back surface of a bridge).
- FIG. 21 is an explanatory diagram illustrating an example in which a user is caused to designate a flight path of a hovering camera 100 using a combined image.
- FIG. 22 is an explanatory diagram illustrating an example in which a control terminal 200 generates and displays a combined image.
- FIG. 23 is an explanatory diagram for describing a flight path generation process of a control terminal 200 based on a user's input on a combined image.
- FIG. 24 is an explanatory diagram illustrating an exemplary combined image.
- FIG. 25 is an explanatory diagram illustrating an exemplary combined image.
- FIG. 26 is a flowchart illustrating an exemplary operation of a control terminal 200 according to an embodiment of the present disclosure.
- Embodiment of present disclosure (1.1. Overview) In detailed description of an embodiment of the present disclosure, an overview of an embodiment of the present disclosure will be first described.
- Checking a state of a structure by humans is indispensable in operation and maintenance of a structure such as a road, a bridge, a tunnel, or a building.
- a worker approaches a structure, and visually checks whether or not damage such as corrosion or a crack or looseness of a coupling member such as a bolt has occurred in a structure or performs a hammering test to check the presence or absence of such abnormalities.
- a bridge For operation and maintenance of a bridge, particularly, a concrete bridge, for example, it is necessary to set up a scaffold at a back side portion of a bridge pier or a bridge girder for a worker who performs a visual inspection and a hammering test of a bridge girder or a bridge pier, or it is necessary to close some lanes or all lanes in order to secure safety of a worker or place a work vehicle. For this reason, a cost necessary for an inspection, a cost necessary for a placement of a road guide person due to road closing, and a traffic jam of a detour occurring by road closing can be problematic.
- the disclosers of the present application have reviewed a technique capable of implementing an inspection of a structure at a low cost with high safety without influencing traffic in view of such circumstances. Further, the disclosers of the present application have ended up with a proposal of a technique capable of implementing an inspection at a low cost with high safety without influencing traffic using a flying body equipped with an imaging device (in the following description, the flying body equipped with the imaging device is also referred to as a "hovering camera") which will be described below.
- FIG. 1 is an explanatory diagram for describing an overview of an embodiment of the present disclosure.
- FIG. 1 schematically illustrates, for example, a bridge 1 constructed of concrete.
- a bridge 1 constructed of concrete When the bridge 1 constructed of concrete is inspected, in a related art, it is necessary to set up a scaffold at a back side portion of a bridge pier 2 or a bridge girder 3 in order for a worker to visually inspect whether or not damage such as a crack or corrosion has occurred, or it is necessary to close some lanes or all lanes in order to secure safety of a worker or place a work vehicle.
- a hovering camera 100 is used when the bridge 1 is inspected.
- the hovering camera 100 is a flying body equipped with an imaging device which is configured to perform an automatic flight according to flight information (including a flight path and information of an imaging position of a still image in the present embodiment) which is set in advance.
- flight information including a flight path and information of an imaging position of a still image in the present embodiment
- Examples of the information of the imaging position of the still image include a position at which an imaging process is executed, an imaging direction, and a traveling time to a position at which a next imaging process is executed.
- the hovering camera 100 is operated to perform an automatic flight to capture the back side of the bridge girder 3.
- the hovering camera 100 By causing the hovering camera 100 to capture the back side of the bridge girder 3, it is unnecessary to set up a scaffold at the back side portion of the bridge pier 2 or the bridge girder 3 for an inspection of the bridge girder 3, the frequency of lane closing is reduced or it is unnecessary to perform lane closing.
- the hovering camera 100 is operated to perform an automatic flight to capture the side of the bridge girder 3.
- the hovering camera 100 In order to cause the hovering camera 100 to perform an automatic flight to capture the back side of the bridge girder 3, it is necessary to set a flight path of the hovering camera 100 and set information of an imaging position of a still image at the position of the back side of the bridge girder 3. In embodiment of the present disclosure, it is a purpose to make it possible to perform an efficient inspection of the bridge 1 by efficiently creating flight information to be set for the hovering camera 100 using information related to a typical condition of the bridge 1.
- FIG. 2 is an explanatory diagram illustrating an exemplary system configuration of an inspection system 10 according to an embodiment of the present disclosure.
- the inspection system 10 according to the embodiment of the present disclosure illustrated in FIG. 2 is a system that is configured to efficiently inspect a structure, for example, the bridge 1.
- An exemplary system configuration of the inspection system 10 according to the embodiment of the present disclosure will be described below with reference to FIG. 2.
- the inspection system 10 includes the hovering camera 100, a control terminal 200, an information processing device 300, a wireless relay node 400, a position estimation node 500, a base station 600, a charging station 700, and a server device 800.
- the hovering camera 100 is an exemplary imaging device of the present disclosure and serves as the flying body equipped with the imaging device described above.
- the hovering camera 100 is a flying body configured to be able to perform an automatic flight based on a designated flight path and capture a still image at a designated imaging position through the imaging device.
- the hovering camera 100 can fly, for example, through four rotors and fly while moving upward, downward, or forward by controlling the rotation of each rotor.
- the number of rotors is not limited to the relevant example.
- a flight path from a flight start position to a flight end position and the imaging position set for the hovering camera 100 are set as position information of a global positioning system (GPS), for example.
- GPS global positioning system
- a GPS receiver that receives radio waves from GPS satellites and calculates a current position may be incorporated into the hovering camera 100.
- the flight path set for the hovering camera 100 may be set using all of a latitude, a longitude, and an altitude as GPS position information or may be set using only a latitude and a longitude as the GPS position information, and, for example, a relative height from the base station 600 which will be described below may be set as an altitude.
- the control terminal 200 is an exemplary control device of the present disclosure and serves as a terminal that executes control related to a flight of the hovering camera 100.
- the control terminal 200 As the control related to the flight of the hovering camera 100, for example, the control terminal 200 generates flight information to be transmitted to the hovering camera 100, gives a takeoff instruction to the hovering camera 100, gives a return instruction to the base station 600 which will be described below, or flies the hovering camera 100 when the hovering camera 100 does not fly automatically due to a certain reason.
- a generation process of the flight information of the hovering camera 100 by the control terminal 200 will be described in detail below but will be described briefly here.
- the control terminal 200 reads the information related to the typical condition of the bridge 1 to be inspected, for example, a typical condition diagram of the bridge 1 to be inspected, and causes the read information to be displayed on a screen.
- Points on the typical condition diagram of the bridge 1 are associated with points on map data including more detailed GPS information.
- the associating is preferably performed by at least two sets of points.
- the typical condition diagram of the bridge 1 is associated with points on the map data including detailed GPS information in advance, and thus the flight path of the hovering camera 100 is defined as GPS values.
- the control terminal 200 generates the flight path of the hovering camera 100 based on the typical condition diagram of the bridge 1.
- the flight path of the hovering camera 100 is displayed on the typical condition diagram in a superimposed manner so that it is easily understood by the user (structure inspection worker).
- the control terminal 200 may consider a structure or dimension of the bridge 1 or a portion of the bridge 1 to be captured by the hovering camera 100 when generating the flight information of the hovering camera 100.
- the control terminal 200 may generate the flight information for causing the hovering camera 100 to capture a portion, in detail, considered likely to be damaged when generating the flight information of the hovering camera 100.
- the flight path set to the hovering camera 100 may be set using all of a latitude, a longitude, and an altitude as the GPS position information, but a case in which no altitude data is included in the typical condition diagram of the bridge 1 is considered.
- the flight path set to the hovering camera 100 is set using only a latitude and a longitude as the GPS position information, and, for example, a relative height from the base station 600 may be set as an altitude.
- the control terminal 200 When the flight information is set for the hovering camera 100, the control terminal 200 preferably generates the flight information so that a distance from an imaging target surface becomes constant when the hovering camera 100 captures the bridge 1. Since the flight information is generated so that the distance from the imaging target surface becomes constant when the hovering camera 100 captures the bridge 1, the control terminal 200 can cause the hovering camera 100 to generate images having the same scale.
- the control terminal 200 is a portable device such as a laptop computer or a tablet terminal, and performs wireless transmission and reception of information to/from the hovering camera 100.
- the control terminal 200 may perform wireless communication with the hovering camera 100 directly with the hovering camera 100, but since there are cases in which the hovering camera 100 flies beyond a communication range of the control terminal 200 in an inspection of a structure, particularly, the bridge 1, the control terminal 200 may perform wireless communication with the hovering camera 100 through the wireless relay node 400 installed at the time of inspection.
- the control terminal 200 acquires an image captured by the imaging device while the hovering camera 100 is flying, and displays the acquired image as necessary.
- the control terminal 200 may acquire a moving image captured by the imaging device in a streaming manner while the hovering camera 100 is flying and display the acquired moving image. Since the moving image captured by the imaging device is acquired in the streaming manner while the hovering camera 100 is flying and displayed, the control terminal 200 can present a position at which the hovering camera 100 is flying to the user.
- the information processing device 300 is a device that processes a variety of information and may be, for example, a device having a function of processing information such as a personal computer (PC), a game machine, or the like.
- the information processing device 300 is a device having a function of displaying, particularly, an image captured by the hovering camera 100 and enables the user to check the state of the bridge 1.
- the information processing device 300 has a function of calculating an absolute position of damage of the bridge girder 3 from the image captured by the hovering camera 100 and generating damage data which will be described below.
- the information processing device 300 may have a function of transmitting the generated damage data to the server device 800.
- the control terminal 200 may have the function of calculating an absolute position of damage of the bridge girder 3 from the image captured by the hovering camera 100 and generating damage data which will be described below.
- the information processing device 300 acquires the image captured by the hovering camera 100, for example, from the control terminal 200.
- the acquiring of the image captured by the hovering camera 100 by the information processing device 300 is not limited to a specific time, and, for example, the information processing device 300 may acquire the image captured by the hovering camera 100 from the control terminal 200 at a time at which one flight of the hovering camera 100 ends.
- the wireless relay node 400 is a device that relays wireless communication between the hovering camera 100 and the control terminal 200. As described above, the hovering camera 100 may fly beyond the communication range of the control terminal 200 at the time of inspection of a structure, particularly, the bridge 1. Thus, wireless communication between the hovering camera 100 and the control terminal 200 can be performed through the wireless relay node 400 installed at the time of inspection of a structure.
- the number of wireless relay nodes 400 is not limited to 1, and a plurality of wireless relay nodes 400 may be installed depending on an inspection range of the bridge 1. Thus, wireless communication between the hovering camera 100 and the control terminal 200 may be performed through a plurality of wireless relay nodes 400.
- the hovering camera 100 can switch a communication destination between the control terminal 200 and the wireless relay node 400 according to a situation of the radio waves.
- the wireless relay node 400 may be installed at an appropriate position on a bridge face (preferably, on a sidewalk) at the time of inspection of the bridge 1.
- the wireless relay node 400 may be installed so as to be suspended from a parapet of the bridge girder 3. Further, before the inspection of the bridge 1, it is desirable to check whether or not the wireless relay node 400 operates normally, for example, using the control terminal 200 by a certain method.
- the position estimation node 500 is a device that causes the hovering camera 100 to estimate a current position.
- the flight path of the hovering camera 100 is set, for example, using the GPS position information.
- the hovering camera 100 can detect the current position with a high degree of accuracy.
- the hovering camera 100 flies under the bridge girder 3 and so the radio waves from the GPS satellites are blocked by the bridge girder 3 or a multipath occurs, for example, due to reflection of the radio waves by the bridge 1, the hovering camera 100 is unlikely to detect the current position with a high degree of accuracy.
- the position estimation node 500 is installed under the bridge girder 3 in order to enable the hovering camera 100 to acquire the current position accurately.
- an augmented reality (AR) marker or a GPS signal transmitter may be used as the position estimation node 500.
- the hovering camera 100 In order to enable the hovering camera 100 to recognize the current position, for example, position estimation nodes 500 are suspended from both ends of the bridge 1, and the hovering camera 100 is caused to capture the position estimation node 500. Further, the hovering camera 100 that has captured the position estimation node 500 is caused to fly between the designated position estimation nodes 500.
- the hovering camera 100 can detect the position between the position estimation nodes 500, for example, based on an integration value of a sensor (for example, an inertial measurement unit (IMU) sensor) installed in the hovering camera 100 and a distance to the position estimation node 500 of the movement destination calculated from the captured image.
- IMU inertial measurement unit
- position estimation nodes 500 are installed at opposing corners or four corners of the bridge 1.
- the hovering camera 100 receives the GPS signal transmitted from the position estimation node 500 and thus can acquire the current position accurately even under the bridge girder 3.
- the base station 600 is a device installed for takeoff and landing of the hovering camera 100.
- the base station 600 includes a GPS receiver, and receives the radio waves from the GPS satellites and calculates the current position.
- the current position calculated by the base station 600 is transmitted to the control terminal 200. Since the current position calculated by the base station 600 is transmitted to the control terminal 200, the control terminal 200 can cause the position of the base station 600 to be displayed on the typical condition diagram of the bridge 1.
- the base station 600 may have a function of checking an operation of the hovering camera 100.
- Examples of the operation check of the hovering camera 100 performed by the base station 600 include a communication function check, an imaging function check, a flight function check, and calibration of various types of sensors.
- the calibration method of the sensors of the hovering camera 100 is not limited to the method of using the base station 600.
- a method of fixing the hovering camera 100 in a dedicated calibration and correcting the sensors by rotating the hovering camera 100 in a pitch direction or a roll direction may be used as the calibration method of the sensors of the hovering camera 100.
- the charging station 700 electrically charges a secondary battery installed in the hovering camera 100.
- the hovering camera 100 uses a battery as a power source, and expends electrical power accumulated in the battery during the flight or the capturing.
- the charging station 700 can restore electric power expended by the hovering camera 100 by charging the battery.
- the charging station 700 may charge the hovering camera 100 by connecting a cable or the like to the hovering camera 100 and supplying electric power to the hovering camera 100 or may charge the hovering camera 100 by supplying electric power to the hovering camera 100 by a non-contact power transmission scheme.
- the server device 800 is a device that stores various types of data.
- the server device 800 may store damage data generated by the information processing device 300.
- the inspection system 10 has the configuration illustrated in FIG. 2 and can cause the hovering camera 100 to capture the bridge 1 and acquire the image of the bridge 1. Since the hovering camera 100 is caused to capture the bridge 1, in the inspection system 10 according to the embodiment of the present disclosure, it is unnecessary to set up a scaffold at a bridge pier or a bridge girder, the frequency in which some lanes or all lanes are closed in order to secure safety of a worker is reduced, and it is unnecessary to close lanes, and thus the inspection of the bridge 1 can be efficiently performed at a low cost.
- FIG. 3 is an explanatory diagram illustrating an exemplary function configuration of the hovering camera 100 according to an embodiment of the present disclosure.
- An exemplary function configuration of the hovering camera 100 according to an embodiment of the present disclosure will be described below with reference to FIG. 3.
- the hovering camera 100 is configured to include an imaging device 101, rotors 104a to 104d, motors 108a to 108d, a control unit 110, a communication unit 120, a sensor unit 130, a position information acquisition unit 132, a storage unit 140, and a battery 150.
- the control unit 110 controls an operation of the hovering camera 100.
- the control unit 110 can control an adjustment of the rotational speed of the rotors 104a to 104d by an adjustment of the rotational speed of the motors 108a to 108d, the imaging process by the imaging device 101, the transmission and reception processes of information to/from other devices (for example, the control terminal 200) through the communication unit 120, and storage and reading of information in and from the storage unit 140.
- control unit 110 controls a flight in which the rotational speed of the motors 108a to 108d is adjusted and execution of the imaging process of the still image by the imaging device 101 based on the flight information transmitted from the control terminal 200.
- the control unit 110 controls the motors 108a to 108d or the imaging device 101 based on the flight information transmitted from the control terminal 200 and thus can provide an image to the control terminal 200 based on a request of the control terminal 200.
- the imaging device 101 is configured with a lens, an image sensor such as a CCD image sensor or a CMOS image sensor, a flash, and the like.
- the imaging device 101 installed in the hovering camera 100 captures a still image or a moving image according to control from the control terminal 200.
- the image captured by the imaging device 101 is transmitted from the communication unit 120 to the control terminal 200.
- the imaging device 101 performs the imaging process based on the information of the imaging position of the still image included in the flight information transmitted from the control terminal 200.
- the image obtained by the imaging process of the imaging device 101 is stored in the storage unit 140 or transmitted from the communication unit 120 to the control terminal 200.
- the imaging device 101 can change the imaging direction, for example, to an arbitrary direction by the control from the control unit 110. For example, when the horizontal direction of the hovering camera is assumed to be 0°, the capturing can be performed in an imaging direction indicated by a range of ⁇ 90° vertically. As the imaging device 101 changes the imaging direction, the hovering camera 100 can capture an image in a certain direction and provides a captured image to the control terminal 200. Then, the control unit 110 associates position information (which may include position information obtained by position measurement using the GPS or position measurement using the position estimation node 500.
- position information which may include position information obtained by position measurement using the GPS or position measurement using the position estimation node 500.
- the position measurement using the position estimation node 500 will be described below) of the hovering camera 100 when the imaging device 101 captures a still image, fuselage information (for example, a yaw angle, a pitch angle, acceleration, and an angular velocity) at the time of capturing, and information of the imaging direction as metadata of the still image.
- fuselage information for example, a yaw angle, a pitch angle, acceleration, and an angular velocity
- information of the imaging direction as metadata of the still image.
- the metadata may be added to an additional information region (for example, a specific region of an Exif format) of still image data, or the metadata may be recorded in an image file, a separate file, or the like as separate data.
- the rotors 104a to 104d cause the hovering camera 100 to fly by generating a lift force from rotation thereof. Rotation of the rotors 104a to 104d is caused by rotation of the motors 108a to 108d.
- the motors 108a to 108d cause the rotors 104a to 104d to rotate. The rotation of the motors 108a to 108d can be controlled by the control unit 110.
- the communication unit 120 performs transmission and reception processes of information to/from the control terminal 200 through wireless communication.
- the hovering camera 100 transmits images captured by the imaging device 101 from the communication unit 120 to the control terminal 200.
- the hovering camera 100 receives instructions relating to flight from the control terminal 200 using the communication unit 120.
- the sensor unit 130 is a group of devices that acquire a state of the hovering camera 100, and may include, for example, an acceleration sensor, a gyro sensor, an ultrasonic sensor, a pneumatic sensor, an optical flow sensor, a laser range finder, and the like.
- the sensor unit 130 can convert an acquired state of the hovering camera 100 into a predetermined signal, and provide the signal to the control unit 110 when necessary.
- the position information acquisition unit 132 acquires information of a current position of the hovering camera 100 using, for example, the GPS, a vision sensor, or the like.
- the position information acquisition unit 132 can provide the acquired information of the current position of the hovering camera 100 to the control unit 110 when necessary.
- the control unit 110 executes control of the flight of the hovering camera 100 based on the flight information received from the control terminal 200 using the information of the current position of the hovering camera 100 acquired by the position information acquisition unit 132.
- the sensor unit 130 detects an obstacle that may interfere with a flight at the time of the flight. As the sensor unit 130 detects an obstacle, the hovering camera 100 can provide information related to the detected obstacle to the control terminal 200.
- the storage unit 140 stores a variety of information. Examples of the information stored in the storage unit 140 include the flight information of the hovering camera 100 transmitted from the control terminal 200 and an image captured by the imaging device 101.
- the battery 150 accumulates electric power for operating the hovering camera 100.
- the battery 150 may be a primary battery in which only discharging is possible or may be a secondary battery in which charging is also possible, but when the battery 150 is the secondary battery, for example, the battery 150 can be supplied with electric power from the charging station 700 illustrated in FIG. 2.
- the hovering camera 100 may have the configuration illustrated in FIG. 3 and thus can perform an automatic flight based on the flight path included in the flight information transmitted from the control terminal 200 and execute the imaging process based on the information of the imaging position of the still image included in the flight information transmitted from the control terminal 200.
- the exemplary function configuration of the hovering camera 100 according to an embodiment of the present disclosure has been described above with reference to FIG. 3.
- an exemplary function configuration of the control terminal 200 according to an embodiment of the present disclosure will be described.
- FIG. 4 is an explanatory diagram illustrating an exemplary function configuration of the control terminal 200 according to an embodiment of the present disclosure. An exemplary function configuration of the control terminal 200 according to an embodiment of the present disclosure will be described below with reference to FIG. 4.
- control terminal 200 is configured to include a display unit 210, a communication unit 220, a control unit 230, and a storage unit 240.
- the display unit 210 includes a flat display device, for example, a liquid crystal display device, an organic EL display device, or the like.
- the display unit 210 can display, for example, images captured by the imaging device 101 or information for controlling operations of the hovering camera 100.
- the display unit 210 is provided with a touch panel, and thus a user can perform a direct operation with respect to the information displayed on the display unit 210 by touching the display unit 210 with his or her finger, or the like.
- the communication unit 220 transmits and receives information to/from the hovering camera 100 through wireless communication.
- the control terminal 200 receives images captured by the imaging device 101 from the hovering camera 100 using the communication unit 220.
- the control terminal 200 transmits instructions relating to the flight of the hovering camera 100 to the hovering camera 100 from the communication unit 220. Commands relating to the flight of the hovering camera 100 can be generated by the control unit 230.
- the control unit 230 controls an operation of the control terminal 200.
- the control unit 230 can control a process of displaying text, figures, images, or other information on the display unit 210 and the transmission and reception processes of information to/from other devices (for example, the hovering camera 100) through the communication unit 220.
- the control unit 230 is configured to include a flight information generating unit 232 and a display control unit 234.
- the flight information generating unit 232 generates the flight information to be transmitted to the hovering camera 100. At the time of generation of the flight information, for example, the flight information generating unit 232 uses information related to a structure of an inspection target stored in the storage unit 240 which will be described below. When the flight information is generated, the flight information generating unit 232 causes the generated flight information to be transmitted from the communication unit 220 before takeoff of the hovering camera 100.
- the flight information generation process by the flight information generating unit 232 will be described below, but an example of the flight information generation process by the flight information generating unit 232 will be briefly described.
- the flight information generating unit 232 reads the typical condition diagram of the bridge 1 to be inspected when generating the flight information of the hovering camera 100.
- the read typical condition diagram of the bridge 1 is displayed on the display unit 210 through the display control unit 234.
- points on the typical condition diagram of the bridge 1 are associated with points on the map data including detailed GPS information in advance.
- the associating is preferably performed by at least two sets of points.
- the typical condition diagram of the bridge 1 is associated with points on the map data including detailed GPS information in advance, and thus the flight path of the hovering camera 100 is defined using GPS values (a set of a latitude and a longitude).
- the flight information generating unit 232 uses information related to a structure such as a construction method, a width, and a span length of the bridge 1, an available flight period of time of the hovering camera 100, and information such as an inspection method of the bridge 1 when generating the flight path of the hovering camera 100.
- Concrete bridges are classified into reinforced concrete (RC) and prestressed concrete (PC) according to an reinforcement method and are classified into, for example, a RCT girder bridge, a PCT girder bridge, a PC hollow slab bridge, a RC box-girder bridge, a PC box-girder bridge, and the like.
- the flight information generating unit 232 can generate a flight path suitable for the construction method of the bridge 1. Then, the flight information generating unit 232 causes the flight path of the hovering camera 100 to be displayed on the typical condition diagram of the bridge 1 in a superimposed manner.
- the flight information generating unit 232 defines the flight path of the hovering camera 100 using GPS values (a set of a latitude and a longitude) as described above. As the flight information generating unit 232 defines the flight path of the hovering camera 100 using the GPS value, the hovering camera 100 can determine a position at which the imaging process is executed at the time of flight based on the GPS value.
- the display control unit 234 controls the display of text, figures, images, and other information on the display unit 210. Display of text, figures, symbols, images, and other information on the display unit 210 in drawings to be referred to in following descriptions is assumed to be controlled by the display control unit 234. For example, when the flight information generating unit 232 generates the flight information to be transmitted to the hovering camera 100, the display control unit 234 executes control such that the typical condition diagram of the structure (the bridge 1) of the inspection target and the generated flight information are displayed on the display unit 210.
- the storage unit 240 stores various types of information. Examples of the information stored in the storage unit 240 include information related to the structure of the inspection target. Examples of the information related to the structure of the inspection target include the typical condition diagram of the structure (the bridge 1) of the inspection target and the construction method of the structure of the inspection target. Further, when a location of the structure of the inspection target which is considered likely to be damaged is known in advance, the information related to the structure of the inspection target may include information of a portion that is considered likely to be damaged.
- control terminal 200 may receive the information related to the structure of the inspection target, for example, from the information processing device 300 at the time of inspection of the structure.
- the control terminal 200 has the configuration illustrated in FIG. 4 and can generate the flight information to be transmitted to the hovering camera 100 based on the information related to the structure (the bridge 1) of the inspection target and acquire the image captured based on the flight information by the hovering camera 100 that flies based on the flight information.
- control terminal 200 The exemplary function configuration of the control terminal 200 according to an embodiment of the present disclosure has been described above with reference to FIG. 4. Next, an exemplary operation of the inspection system 10 according to the embodiment of the present disclosure will be described.
- FIG. 5 is a flowchart illustrating an exemplary operation of the inspection system 10 according to the embodiment of the present disclosure.
- FIG. 5 illustrates an exemplary operation of the inspection system 10 according to the embodiment of the present disclosure when the bridge 1 is inspected by causing the hovering camera 100 to fly and causing the hovering camera 100 to capture the bridge 1. Further, when the bridge 1 is inspected using the hovering camera 100, the wireless relay node 400 or the position estimation node 500 is assumed to be installed at an appropriate position of the bridge 1 in advance. An exemplary operation of the inspection system 10 according to the embodiment of the present disclosure will be described below with reference to FIG. 5.
- the control terminal 200 that generates the flight information of the hovering camera 100 reads information related to the bridge 1 including the typical condition diagram of the bridge 1 (the inspection target), and causes the typical condition diagram of the bridge 1 to be displayed on the display unit 210 (step S101).
- the reading of the information related to the bridge 1 is executed, for example, by the flight information generating unit 232, and the displaying of the typical condition diagram of the bridge 1 on the display unit 210 is executed, for example, by the display control unit 234.
- the control terminal 200 in which the typical condition diagram of the bridge 1 is being displayed on the display unit 210 enables the user to designate a region of the bridge 1 to be inspected using the typical condition diagram of the bridge 1 being displayed on the display unit 210 (step S102).
- the process of enabling the user to designate in step S102 is executed, for example, by the flight information generating unit 232.
- the control terminal 200 when a part of the bridge 1 is set as the inspection target, the control terminal 200 enables the user to designate an inspection target region in the typical condition diagram of the bridge 1 being displayed on the display unit 210. Further, for example, when the entire bridge 1 is set as the inspection target, the control terminal 200 enables the user to designate all regions of the bridge 1 in the typical condition diagram of the bridge 1 being displayed on the display unit 210.
- FIG. 6 is an explanatory diagram illustrating an exemplary screen displayed on the display unit 210 of the control terminal 200.
- FIG. 6 illustrates an exemplary screen displayed on the display unit 210 when the user is requested to designate the region of the bridge 1 to be inspected in step S102.
- a screen displayed on the display unit 210 when the bridge girder is designated as the region of the bridge 1 to be inspected is assumed to be displayed.
- the control terminal 200 may include, for example, a touch panel as an input unit (not illustrated) and enable the user to designate the region of the bridge 1 by enabling the user to drag on the screen or enabling the user to select the span of the inspection target.
- a method of enabling the user to designate the region of the bridge 1 to be inspected is not limited to the relevant example.
- the displaying of the region designated by the user is not limited to the example illustrated in FIG. 6.
- FIG. 6 illustrates an example in which a mark B1 indicating the position of the base station 600 is displayed on the typical condition diagram of the bridge 1 in a superimposed manner.
- the base station 600 may include a GPS receiver, and receive the radio waves from the GPS satellites and calculate the current position.
- the control terminal 200 can cause the mark B1 indicating the position of the base station 600 to be displayed on the typical condition diagram of the bridge 1 in a superimposed manner based on the information of the current position calculated by the base station 600.
- the control terminal 200 When the region of the bridge 1 to be inspected is designated by the user, the control terminal 200 then generates the flight information of the hovering camera 100 in the inspection region designated by the user based on the information related to the bridge 1 (step S103).
- the flight information generation process in step S103 is executed, for example, by the flight information generating unit 232.
- the control terminal 200 uses information related to a structure such as a construction method, a width, and a span length of the bridge 1, an available flight period of time of the hovering camera 100, and information such as an inspection method of the bridge 1 when generating the flight information of the hovering camera 100 in step S103. For example, when a T girder is used in the construction method of the bridge 1, the control terminal 200 generates a flight path in which the hovering camera 100 repeats levitation and descending at the bottom side of the bridge 1 as the flight information. Further, the control terminal 200 may use information of an image target surface of the bridge 1 when generating the flight information of the hovering camera 100 in step S103.
- the control terminal 200 when the user selects capturing of the side of the bridge 1, the control terminal 200, the control terminal 200 generates a flight path along the side of the bridge 1 as the flight information, and when the user selects capturing of the bottom surface of the bridge 1, the control terminal 200 generates a flight path in which it travels back and forth under the bottom side of the bridge 1 as the flight information.
- a list of positions at which the imaging process is executed may be designated in the following format: ID: (relative coordinates of imaging point, imaging direction, speed at time of imaging, traveling time to next imaging point, and others)
- ID relative coordinates of imaging point, imaging direction, speed at time of imaging, traveling time to next imaging point, and others
- the relative coordinates of an imaging point are designated by three points of an X axis, a Y axis, and a Z axis.
- the X axis is set as a latitude direction
- the Y axis is set as a longitude direction
- the Z axis is set as a height direction.
- information used to control special capturing may be included as other information.
- Examples of the information used to control special capturing include information for capturing the same position in a plurality of imaging directions, information related to a parameter for bracket capturing (which indicates capturing by different exposures, different shutter speeds, different ISO sensitivities, and the like at the same position and in the same imaging direction), and information on a wavelength of infrared rays at the time of capturing.
- the flight information generated by the control terminal 200 can be configured with the following list of following various values: 0:(0,0,0,0,0,2,1.0) 1:(5,0,0,0,0,2,1.0) 2:(7,0,0,0,0,2,1.0) 3:(9,0,0,0,0,2,1.0)
- the imaging point included in the flight information generated by the control terminal 200 may be designated, for example, relative coordinates from a reference point by using absolute coordinates of the base station 600 or absolute coordinates of an arbitrary position such as a first imaging position as the reference point.
- the hovering camera 100 may convert the relative coordinates from the absolute coordinates of the reference point into the absolute coordinates and refer to the converted coordinates at the time of flight.
- the imaging point included in the flight information generated by the control terminal 200 may be designated by the absolute coordinates instead of the relative coordinates.
- a certain value may be stored in the information used to control special capturing included in the flight information generated by the control terminal 200.
- a value such as 1: capturing in a plurality of imaging directions), 2: bracket capturing (a change in a shutter speed), 3: bracket capturing (a change in ISO sensitivity), or the like may be stored in the information used to control special capturing.
- the control terminal 200 may cause the information used to control special capturing to be included in the flight information, for example, for a location of the bridge girder 3 which is considered likely to be damaged and stored in the storage unit 240.
- the control terminal 200 may generate the flight information for causing the hovering camera 100 to capture, for example, the back surface of the bridge girder 3 of the bridge 1 at equal intervals at the time of the flight information generation process of step S103.
- the control terminal 200 may generate the flight information so that the imaging positions of the still images are equal intervals at the time of the flight information generation process of step S103.
- the control terminal 200 may read the stored information and generate the flight information so that the portion is captured in detail by the hovering camera 100 when generating the flight information of the hovering camera 100 in step S103.
- the control terminal 200 may cause the information used to control special capturing to be included in the flight information.
- information of a portion considered likely to be damaged may not be stored in the storage unit 140 in advance, and in this case, information of a portion considered likely to be damaged may be input by the user at the time of inspection.
- the available flight period of time of the hovering camera 100 may be obtained based on the capacity of the battery 150, power consumption of the motors 108a to 108d for driving the rotors 104a to 104d, power consumption of the imaging device 101, the control unit 110, and the communication unit 120, or the like in advance.
- the control terminal 200 may divide the generated flight path into several paths.
- control terminal 200 may generate a plurality of flight paths and cause the plurality of flight paths to be displayed on the display unit 210 when generating the flight information of the hovering camera 100 in step S103.
- FIG. 7 is an explanatory diagram illustrating an exemplary screen displayed on the display unit 210 of the control terminal 200.
- FIG. 7 illustrates an example of a state in which a plurality of flight paths are generated, and then flight paths R1 and R2 are displayed on the display unit 210 when the flight information of the hovering camera 100 is generated in S103.
- the control terminal 200 causes a plurality of flight paths to be displayed on the display unit 210 and enables the user to select one flight path.
- the control terminal 200 generates the flight information based on the flight path selected by the user.
- the control terminal 200 When the flight information of the hovering camera 100 is generated in step S103, the control terminal 200 then transmits the generated flight information to the hovering camera 100, and transmits a takeoff instruction to the hovering camera 100 (step S104).
- the transmitting of the generated flight information and the transmitting of the takeoff instruction are performed, for example, by the flight information generating unit 232 through the communication unit 220.
- FIG. 8 is an explanatory diagram illustrating an exemplary screen displayed on the display unit 210 of the control terminal 200.
- FIG. 8 an exemplary screen displayed on the display unit 210 of the control terminal 200 when the takeoff instruction is transmitted to the hovering camera 100.
- the user can cause the takeoff instruction to be transmitted from the control terminal 200 to the hovering camera 100 by touching a takeoff instruction button 211 displayed on the display unit 210.
- the flight information generated in step S103 may be transmitted from the control terminal 200 to the hovering camera 100 before the takeoff instruction is transmitted, but the flight information generated in step S103 may be transmitted from the control terminal 200 to the hovering camera 100 after the takeoff instruction is transmitted from the control terminal 200 to the hovering camera 100.
- the hovering camera 100 that has received the flight information and the takeoff instruction from the control terminal 200 and then taken off from the base station 600 flies based on the flight information transmitted from the control terminal 200, performs the imaging process, and obtains a still image (step S105).
- the hovering camera 100 acquires position information when the imaging process of acquiring a still image is executed or fuselage information at the time of the imaging process, and associates the acquired information with the still image. For example, information such as a yaw angle, a pitch angle, acceleration, or an angular velocity may be included in the fuselage information at the time of the imaging process.
- the hovering camera 100 may transmit a moving image being captured by the imaging device 101 during flight to the control terminal 200 in a streaming manner. As the control terminal 200 acquires and displays the moving image being captured through the imaging device during flight by the hovering camera 100, the control terminal 200 can present a position at which the hovering camera 100 is flying to the user.
- the hovering camera 100 maintains constant a distance from the image target surface (for example, the side surface or the bottom surface of the bridge girder 3) at all the imaging points when executing the imaging process. As the distance from the image target surface is maintained constant at all the imaging points, the hovering camera 100 can obtain still images captured with the same size.
- the image target surface for example, the side surface or the bottom surface of the bridge girder 3
- the hovering camera 100 may change the imaging direction of the imaging device, use infrared rays having different wavelengths, or change a shutter speed for the portion and then capture a plurality of still images. Further, when a portion considered likely to be damaged is included in the flight path of the hovering camera 100, the hovering camera 100 may narrow an interval of positions at which the imaging process of the portion is performed so as to be smaller than that of other portions.
- FIG. 9 is an explanatory diagram conceptually illustrating an operation of the hovering camera 100 in the inspection system 10 according to the embodiment of the present disclosure.
- the hovering camera 100 flies under the bottom side of the bridge 1 based on the flight information, for example, the hovering camera 100 stops at a time t 1 and captures the bottom surface of the bridge 1, flies to and stops at a position at which capturing is to be performed at a time t 2 after the imaging, captures the bottom surface of the bridge 1 at a different position at the time t 2 and then repeats flying, stopping, and capturing up to a time t n .
- the hovering camera 100 repeats flying, stopping, and capturing, the images of the bottom surface of the bridge 1 are obtained.
- the hovering camera 100 When the hovering camera 100 flies based on the flight information, it is possible to detect the current position accurately when it is possible to receive the radio waves from the GPS satellites without interference. However, it is difficult for the hovering camera 100 to detect the current position accurately at a position at which it is difficult to receive the radio waves from the GPS satellites such as a position under the bridge 1. In this regard, in the present embodiment, the position estimation node 500 is used, and thus the hovering camera 100 detects the current position accurately at a position at which it is difficult to receive the radio waves from the GPS satellites.
- FIG. 10 is an explanatory diagram conceptually illustrating an operation of the hovering camera 100 in the inspection system 10 according to the embodiment of the present disclosure.
- the hovering camera 100 receives the radio waves from the GPS satellites 30 without interference, and moves back and forth in a GPS position measurement area 40 in which position measurement is performed and a sensor position measurement area 50 in which the current position is estimated, for example, using a vision sensor.
- the hovering camera 100 detects the current position using the radio waves received from the GPS satellites 30.
- the hovering camera 100 detects the position between the position estimation nodes 500, that is, the current position based on the integration value of the sensors (for example, the IMU sensor) installed in the hovering camera 100 and the distance to the position estimation node 500 of the movement destination calculated from the image captured by the imaging device 101 when the position estimation node 500 is the AR marker.
- the position estimation node 500 is the GPS signal transmitter
- the hovering camera 100 detects the current position using a signal transmitted from the position estimation node 500.
- the hovering camera 100 can detect the accurate current position even when the hovering camera 100 moves to the position at which the radio waves from the GPS satellites are hardly received.
- the hovering camera 100 automatically flies to the base station 600 in order to return to the base station 600 (step S106). Then, the control terminal 200 acquires the image captured by the hovering camera 100 that has returned to the base station 600 from the hovering camera 100 (step S107). The acquiring of the image captured by the hovering camera 100 may be performed after the hovering camera 100 returns to the base station 600 as described above, but the control terminal 200 may acquire a still image sequentially each time the hovering camera 100 executes the imaging process and acquires the still image.
- the inspection system 10 can generate the flight information to be transmitted to the hovering camera 100 based on the information related to the structure (the bridge 1) of the inspection target through the control terminal 200, capture an image based on the flight information through the hovering camera 100 that flies based on the flight information, and acquire the image captured by the hovering camera 100 through the control terminal 200.
- the user is assumed to have found a portion that is desired to be captured in detail after viewing a moving image captured by the hovering camera 100 while the hovering camera 100 is flying.
- the user may operate the control terminal 200 to stop the automatic flight of the hovering camera 100 and cause an instruction to switch to a manual operation to be transmitted from the control terminal 200.
- the hovering camera 100 performs an automatic flight based on the generated flight information and executes the imaging process.
- a case in which an obstacle not found on the typical condition diagram of the bridge 1 is present in the flight path is also considered.
- FIG. 11 is an explanatory diagram conceptually illustrating an operation of the hovering camera 100 in the inspection system 10 according to the embodiment of the present disclosure.
- FIG. 11 illustrates an example in which a tree 4 is under the bridge girder 3.
- the tree 4 is an obstacle that is not shown on the typical condition diagram of the bridge 1, and there are cases in which the presence of the tree is found at the time of flight of the hovering camera 100 for the first time.
- it may be checked whether or not there is an obstacle in the flight path included in the flight information by causing the hovering camera 100 to perform a test flight once based on the flight information generated by the control terminal 200.
- the control terminal 200 may receive a moving image being captured by the hovering camera 100 in a streaming manner, and the user may check whether or not there is an obstacle in the flight path included in the flight information while viewing the moving image.
- An obstacle may be detected through the sensor unit 130 of the hovering camera 100.
- a detailed position of an obstacle can be detected when a stereo camera is installed as the imaging device 101 of the hovering camera 100, and a distance to an obstacle is detected by capturing performed by the stereo camera, or a direction of an obstacle is specified according to a direction of the hovering camera 100.
- the hovering camera 100 when the hovering camera 100 is caused to perform a test flight, when there is an obstacle in the flight path, the hovering camera 100 may stop an automatic flight, move in a hovering state, and may be on standby for an operation from the user or may return to the base station 600 automatically.
- the control terminal 200 may register a location of the obstacle in the typical condition diagram of the bridge 1.
- the location of the obstacle may be manually input by the user, and when the hovering camera 100 detects an obstacle through the sensor unit 130, the detected location of the obstacle may be acquired from the hovering camera 100, and then the location of the obstacle may be registered in the typical condition diagram of the bridge 1.
- FIG. 12 is an explanatory diagram illustrating an exemplary screen displayed on the display unit 210 of the control terminal 200.
- FIG. 12 is an exemplary screen displayed on the display unit 210 when it is found that there is an obstacle in the flight path through the test flight of the hovering camera 100.
- the control terminal 200 causes a mark O1 indicating the location of the obstacle to be displayed on the typical condition diagram of the bridge 1 in a superimposed manner.
- the control terminal 200 regenerates flight information including a flight path avoiding the location of the obstacle, and transmits the generated flight information to the hovering camera 100.
- the hovering camera 100 flies based on the flight information regenerated by the control terminal 200 and thus perform the flight and the imaging process while avoiding the obstacle (the tree 4).
- a method of causing the hovering camera 100 to fly and detecting the location of the obstacle is not limited to the relevant example.
- the hovering camera 100 may be caused to fly along an outer circumference of the flight path generated by the control terminal 200 through a simple path while capturing a moving image through the imaging device 101, and it may be checked whether or not there is an obstacle under the bridge girder 3.
- a location which is not easily accessible such as the bottom surface of the bridge girder 3 can be detected by causing the hovering camera 100 to fly and capture the bridge 1.
- the still image captured by the hovering camera 100 is associated with, for example, the position information (which may include position information obtained by position measurement using the GPS or position measurement using the position estimation node 500) of the hovering camera 100 that has captured the still image, the fuselage information (for example, a yaw angle, a pitch angle, acceleration, and an angular velocity) at the time of capturing, and information of the imaging direction.
- the hovering camera 100 performs the capturing at all the imaging points while maintaining the distance from the image target surface constant, a relative position of a location at which damage is occurring in an image is detected.
- the still image captured by the hovering camera 100 includes a damaged portion of the bridge girder 3, it is possible to detect an absolute location of the damaged portion.
- position information of a damaged portion is obtained by setting a center of a still image as a point of origin, calculating a relative value of a damaged portion, and calculating the relative value of the position information of the hovering camera 100 when the image is captured.
- the following data may be recorded as position information of a damaged portion.
- the information of the imaging position of the still image is recorded as the position of the damaged portion (a relative value (offset) is not recorded).
- the relative value (offset) corresponding to the information of the imaging position of the still image and damaged portion is recorded as the position of the damaged portion.
- the absolute value used as a reference for example, as will be described below, the imaging position of the still images of four corners considered to be highly accurate in position information, or coordinates of the position estimation node 500
- the relative value (offset) are recorded as the position of the damaged portion.
- the calculated absolute value for example, a latitude, a longitude, and an altitude is recorded as the position of the damaged portion.
- a technique of obtaining a physical size of an imaging range using a numerical value such as a focal distance of a lens, a size of an image sensor, a distance to an imaging target, or the like is known.
- the physical size of the imaging range of the hovering camera 100 may be estimated using distance information from the hovering camera 100 to the imaging target (for example, the back surface or the side surface of the bridge girder 3) or angle of view information of the imaging device 101.
- Physical position information of a damaged portion is determined by setting a central position (a position at which the hovering camera 100 performs capturing) of a captured image as a point of origin, estimating a physical relative position from the point of origin to the damaged portion, and adding position coordinates of the point of origin of the captured image to the relative position. Further, when the distance information and the angle of view information may be acquired through the sensors installed in the hovering camera 100 at the time of capturing, information recorded in association with the image may be used, and a value set for the hovering camera 100 or the imaging device 101 may be used.
- the position information of the damaged portion may be calculated using the fuselage information (for example, a yaw angle, a pitch angle, acceleration, and an angular velocity) at the time of capturing and the information of the imaging direction rather than the imaging position information, the distance information, and the angle of view information.
- the fuselage information for example, a yaw angle, a pitch angle, acceleration, and an angular velocity
- the detecting of the damaged portion based on the still image captured by the hovering camera 100 may be visually performed by the user but may be automatically performed through imaging processing, for example, by the information processing device 300.
- an image processing technique such as a pattern matching may be used.
- a data configuration of damage data is defined, for example, in the following format: (image ID, damage ID, position information of damaged portion, coordinates of damaged portion on image, damage type ID, damage degree)
- the damage type ID refers to an ID allocated to a type of damage such as a crack, peeling, a water leak, or free lime. Further, a maximum width of data, a length of a damaged portion in an image, or the like may be recorded in the damage degree field.
- the inspection system 10 according to the present embodiment can generate damage data according to the above format from the still image captured by the hovering camera 100 through a manual input of the user or an automatic process by the information processing device 300. Further, the damage data generated by the inspection system 10 according to the present embodiment may be used for a process of placing an order to a construction contractor who repairs the damage occurring in the bridge 1.
- the hovering camera 100 captures a number of still images during a single inspection flight. Thus, checking the still images captured by the hovering camera 100 during an inspection flight one by one increases a burden on the user.
- one image is obtained by stitching the still images captured by the hovering camera 100.
- the still images captured by the hovering camera 100 are stitched, for example, an appearance of the bottom surface of the bridge girder 3 corresponding to one span is obtained as one image.
- the user can check whether there is damage to the bottom surface of the bridge girder 3.
- the still image stitching process may be performed by the control terminal 200 or may be performed by the information processing device 300.
- FIG. 13 is an explanatory diagram illustrating an overview when the bottom surface of the bridge girder 3 is inspected based on the still images captured by the hovering camera 100.
- One image 20 obtained by capturing the bottom surface of the bridge girder 3 is obtained by capturing a certain portion of the bottom surface of the bridge girder 3 (for example, a portion of the bridge girder 3 corresponding to one span length) and stitching the still images captured by the hovering camera 100.
- a reference numeral 21 indicates an image captured in a single imaging process of the hovering camera 100.
- position information that is relatively highly accurate position information at the time of capturing in the stitched image can be selected as a reference point.
- the position information of the hovering camera 100 of the still images of four corners serving as the basis of the stitched image at the time of capturing may be used as the reference point.
- the still images of four corners serving as the basis of the stitched image have the smallest distortion
- the GPS position measurement area has a small error in position information
- position measurement status information information indicating a state in which 2D position measurement is being performed, a state in which 3D position measurement is being performed, an position measurement disable state or data such as the number of reception satellites
- GPS position measurement data may be used as the accuracy of position information.
- FIG. 14 is an explanatory diagram illustrating an example of the image 20 obtained by stitching the still images captured by the hovering camera 100.
- Each of centers G1 to G4 of the still images C1 to C4 of four corners serving as the basis of the image 20 corresponds to the position of the hovering camera 100 when each still image is captured.
- the absolute position of the damaged portion in the image 20 is calculated using the position information of the hovering camera 100 corresponding to the still images C1 to C4 of the four corners.
- a data configuration of the damage data is defined, for example, in the following format.
- an image ID is deleted from the damage data.
- the image ID of the stitched image may be generated and included in the damage data.
- the inspection system 10 according to the present embodiment can generate the damage data according to the above format from the stitched image through the manual input of the user or an automatic process by the information processing device 300.
- FIG. 15 is an explanatory diagram illustrating an exemplary function configuration of the information processing device 300 according to an embodiment of the present disclosure.
- FIG. 15 illustrates an exemplary function configuration of the information processing device 300 according to an embodiment of the present disclosure which has a function of obtaining the absolute position of damage of the bridge girder 3 from the still image captured by the hovering camera 100 and generating the damage data.
- An exemplary function configuration of the information processing device 300 according to an embodiment of the present disclosure will be described below with reference to FIG. 15.
- the information processing device 300 includes a display unit 310, a communication unit 320, a control unit 330, and a storage unit 340.
- the display unit 310 is configured with a flat panel display device such as a liquid crystal display (LCD) device or an organic EL display device.
- the display unit 310 may display an image captured by the imaging device 101 of the hovering camera 100, information related to damage of the bridge 1 obtained by the image captured by the imaging device 101, and the like.
- the communication unit 320 performs transmission and reception of information to/from the control terminal 200 through wireless communication.
- the information processing device 300 receives the image captured by the hovering camera 100 from the control terminal 200 through the communication unit 320 together with information of an absolute imaging position of the image.
- the control unit 330 controls an operation of the information processing device 300.
- the control unit 330 can control a process of displaying text, figures, images, or other information on the display unit 210 and the transmission and reception processes of information to/from other devices (for example, the control terminal 200) through the communication unit 320.
- the control unit 330 includes an imaging position information acquisition unit 332, a damage position calculating unit 334, an image combining unit 336, and a damage data generating unit 338.
- the imaging position information acquisition unit 332 acquires information of the imaging position at the time of capturing which is acquired by the hovering camera 100 when the hovering camera 100 captures the bridge 1.
- the damage position calculating unit 334 detects the damaged portion of the bridge 1 from the image captured by the hovering camera 100, for example, using an image processing technique such as pattern matching, and calculates the absolute position of the damaged portion using the information of the imaging position acquired by the imaging position information acquisition unit 332.
- the image combining unit 336 performs the image process of stitching the still images captured by the hovering camera 100 and generating one image.
- the image combining unit 336 may use the information of the imaging positions of the still images at the time of capturing when stitching the still images captured by the hovering camera 100.
- the damage position calculating unit 334 may use the information of the imaging positions of the captured images (for example, each of the four corners) of the corners among the captured images serving as the basis of the image stitched by the image combining unit 336. As described above, since the still images of the four corners among the captured images serving as the basis of the stitched image are considered to be smallest in distortion, the damage position calculating unit 334 can obtain the more accurate damage position using the information of the imaging positions of the still images of the four corners among the captured images serving as the basis of the stitched image.
- the damage data generating unit 338 generates the damage data using the absolute position of the damaged portion of the bridge 1 calculated by the damage position calculating unit 334.
- the damage data generating unit 338 may generate the damage data in units of still images or may generate damage data on the one image stitched by the image combining unit 336.
- the storage unit 340 stores various types of information.
- the information stored in the storage unit 340 may include, for example, the still images captured by the imaging device 101 of the hovering camera 100, the information of the absolute imaging position of the hovering camera 100 when the still images are captured, and information of the damage data generated by the damage data generating unit 338.
- the information processing device 300 has the configuration illustrated in FIG. 15 and can generate the damage data from the still image captured by the hovering camera 100, and thus, the information processing device 300 according to an embodiment of the present disclosure can efficiently generate the inspection result of the bridge 1 serving as the structure of the inspection target.
- the damage data may be generated by the control terminal 200 rather than the information processing device 300.
- the control terminal 200 may have the configuration of the control unit 330 of the information processing device 300 illustrated in FIG. 15.
- the inspection result of the bridge 1 serving as the structure of the inspection target may be accumulated in a public or private database and used.
- the damage data may be generated by the control terminal 200 rather than the information processing device 300.
- the control terminal 200 may have the configuration of the control unit 330 of the information processing device 300 illustrated in FIG. 15.
- the exemplary function configuration of the information processing device 300 according to an embodiment of the present disclosure has been described above with reference to FIG. 15. Next, an exemplary operation of the information processing device 300 according to an embodiment of the present disclosure will be described.
- FIG. 16 is a flowchart illustrating an exemplary operation of the information processing device 300 according to an embodiment of the present disclosure.
- FIG. 16 illustrates an exemplary operation of the information processing device 300 according to an embodiment of the present disclosure when the absolute position of damage of the bridge girder 3 is acquired from the still image captured by the hovering camera 100, and the damage data is generated.
- An exemplary operation of the information processing device 300 according to an embodiment of the present disclosure will be described below with reference to FIG. 16.
- the information processing device 300 acquires the image that is associated with the information of the imaging position and captured by the hovering camera 100 flying over the periphery of the bridge 1 (step S301).
- the information processing device 300 detects the damaged portion from the image, for example, using an image processing technique such as pattern matching (step S302).
- the damaged portion detection process of step S302 may be executed, for example, by the damage position calculating unit 334.
- the information processing device 300 calculates the absolute position of the damaged portion (step S303).
- the calculation process of step S303 may be executed, for example, by the damage position calculating unit 334.
- the information processing device 300 performs the calculating of the absolute position of the damaged portion in step S303 based on the information of the imaging position of the still image captured by the hovering camera 100.
- the information processing device 300 may estimate the physical size of the imaging range of the hovering camera 100 based on the distance information from the hovering camera 100 to the imaging target (for example, the back surface or the side surface of the bridge girder 3) or the angle of view information of the imaging device 101.
- the information processing device 300 can determine the physical position information of the damaged portion by estimating the physical relative position from the center of the captured image to the damaged portion and adding position coordinates of the captured image serving as the point of origin to the relative position.
- the information processing device 300 generates the damage data including the absolute position of the damaged portion (step S304).
- the damage data generation process of step S304 may be executed, for example, by the damage data generating unit 338.
- the information processing device 300 can generate the damage data from the still image captured by the hovering camera 100 by performing the operation illustrated in FIG. 15, and thus, the information processing device 300 according to an embodiment of the present disclosure can efficiently generate the inspection result of the bridge 1 serving as the structure of the inspection target. Further, as described above, the damage data may be generated by the control terminal 200 rather than the information processing device 300. Thus, the operation illustrated in FIG. 16 may be executed by the control terminal 200.
- FIG. 17 is a flowchart illustrating an exemplary operation of the control terminal 200 according to an embodiment of the present disclosure.
- FIG. 17 illustrates an example of the flight information generation process by the control terminal 200 using the damage data generated by the information processing device 300.
- An exemplary operation of the control terminal 200 according to an embodiment of the present disclosure will be described below with reference to FIG. 17.
- the control terminal 200 acquires the damage position of the designated portion from the damage data generated by the information processing device 300 (step S312).
- the method of designating a damaged portion is not limited, for example, the still image may be displayed, and the user may designate the damaged portion by touching the touch panel of the control terminal 200 with his/her finger.
- step S313 When the damage position of the portion designated by the user is acquired from the damage data, the control terminal 200 then generates the flight information of causing the hovering camera 100 to fly over the damage position acquired from the damage data and to capture the damage position (step S313).
- the process of step S313 may be executed, for example, by the flight information generating unit 232. Since the flight information generated in step S313 by the control terminal 200 is used to check the damage position in detail, flight information that instructs the hovering camera 100 to reduce an interval of imaging positions to be smaller than that in the flight information generated by the control terminal 200 described above with reference to FIG. 5 or to execute special capturing at each imaging position may be used.
- the control terminal 200 transmits the generated flight information to the hovering camera 100, and the hovering camera 100 executes a flight and the imaging process based on the flight information as described in steps S104 and S105 of FIG. 5. Then as described in steps S106 and S107 of FIG. 5, when the imaging process at the last imaging point is completed, the hovering camera 100 flies to the base station 600 automatically in order to return to the base station 600, and the control terminal 200 acquires the images captured by the hovering camera 100 from the hovering camera 100.
- the control terminal 200 can generate the flight information for causing the hovering camera 100 to capture the damaged portion of the bridge girder 3 in detail using the damage data generated by the information processing device 300 by executing the operation illustrated in FIG. 17.
- hovering camera provides on a flying body
- this disclosure is not so limited and may apply to any vehicle having at least one camera.
- a hovering vehicle such as a drone equipped with at least one imaging device and that may move along a two- or three-dimensional flight path.
- the following may also apply similarly to a land-based vehicle such as an automobile equipped with at least one imaging device, and that may move along a two-dimensional (ground-based) drive path.
- a hovering camera is illustrative only and not limiting.
- the control terminal described below may generate flight (drive) information for a non-linear or polyline flight (drive) path to be followed by the vehicle, instead of the linear path described below.
- the control terminal 200 can perform a process (a real-time stitching process) of combining the moving images captured by the hovering camera 100 in units of certain frames or combining the still images captured at certain intervals in real time.
- the real time mentioned herein is a process of sequentially updating the display of a composite image (especially a combined image) over time and includes a process in which there is a time difference between a time of capturing and a time of an image combination process or image display due to a processing delay or the like.
- the control terminal 200 can cause the user to designate the target position of the hovering camera 100 using the combined image. Then, the control terminal 200 transmits the flight information for flying to the target position designated by the user to the hovering camera 100.
- the control terminal 200 can generate the flight path of the hovering camera 100 for the user more intuitively by combining the images, causing the combined image to be displayed on the display unit 210, and generating the flight information for flying to the target position designated on the display screen of the display unit 210 including the combined image. Further, by generating the combined image from the moving image or the still images captured by the hovering camera 100 and causing the combined image to be displayed on the display unit 210, when it is difficult to obtain an aerial photograph, for example, even for a place such as the back side of the bridge, the control terminal 200 can cause the flight path to be designated using fine image information or a current situation that is hardly obtained through a map or an aerial photograph and does not have to access an external map information database or the like. Further, as the control terminal 200 generates the combined image from the moving image or the still images captured by the hovering camera 100, it is possible to designate the flight path of the hovering camera 100 indoors.
- FIG. 18 is an explanatory diagram illustrating an example in which the hovering camera 100 is caused to fly and image in a ground direction.
- FIG. 19 is an explanatory diagram illustrating an example in which the images captured by the hovering camera 100 are combined, and the user is caused to designate the flight path of the hovering camera 100 using the combined image.
- a current position mark 252 indicating the current position of the hovering camera 100 is displayed on a combined image 251 obtained as a result of capturing from the sky in a superimposed manner.
- the control terminal 200 uses, for example, feature point matching or the like when generating the combined image 251 as will be described below.
- the control terminal 200 when the user designates a target position 253 of the hovering camera 100 on the combined image 251 as illustrated in FIG. 19, the control terminal 200 generates the flight information for causing the hovering camera 100 to fly along a flight path 254 from the current position mark 252 to the target position 253, and transmits the flight information to the hovering camera 100.
- the hovering camera 100 performs a flight based on the flight information transmitted from the control terminal 200.
- FIG. 20 is an explanatory diagram illustrating an example in which the hovering camera 100 is caused to fly and capture in an upward direction (the back surface of the bridge).
- FIG. 21 is an explanatory diagram illustrating an example in which the images captured by the hovering camera 100 are combined, and the user is caused to designate the flight path of the hovering camera 100 using the combined image.
- a current position mark 262 indicating the current position of the hovering camera 100 is displayed on a combined image 261 obtained as a result of capturing the back surface of the bridge in a superimposed manner.
- the control terminal 200 uses, for example, feature point matching when generating the combined image 261 as will be described below.
- the control terminal 200 when the user designates a target position 263 of the hovering camera 100 on the combined image 261 as illustrated in FIG. 21, the control terminal 200 generates the flight information for causing the hovering camera 100 to fly along a flight path 264 from the current position mark 262 to the target position 263, and transmits the flight information to the hovering camera 100.
- the hovering camera 100 performs a flight based on the flight information transmitted from the control terminal 200.
- FIG. 22 is an explanatory diagram illustrating an example in which the combined image is generated by the control terminal 200 and displayed on the display unit 210.
- the following process may be executed, for example, the display control unit 234 of the control terminal 200.
- the display control unit 234 may function as an example of the image processing unit of the present disclosure.
- FIG. 22 illustrates an example in which the control terminal 200 generates a combined image, for example, using an image group 271 captured at times t-5 to t-1 and an image 272 captured at a time t.
- the control terminal 200 performs matching between a feature point of the image 272 captured at the time t and a feature point of the image captured at the time t-1 among the image group 271.
- the control terminal 200 performs matching between the feature point of the image 272 captured at the time t and a feature point of the combined image previously combined using the image group 271.
- the control terminal 200 obtains a rotation ⁇ translation parameter (or an affine transform parameter) that is smallest in a position error of a feature point. Then, the control terminal 200 combines (performs ⁇ blending on) the combined images generated by the stitching process to date based on the new image 272 through a transform using the rotation ⁇ translation parameter ( or the affine transform parameter).
- the control terminal 200 can generate, for example, a new combined image in which the image captured at the time t is located at the center of the combined image through this combination. Then, the control terminal 200 displays the combined image so that the center of the combined image, that is, a center 273 of the image captured at the time t is the position of the hovering camera 100.
- FIG. 23 is an explanatory diagram for describing the flight path generation process of the control terminal 200 based on the user's input on the combined image.
- a target position 276 on a combined image 275 generated by the control terminal 200 a direction from the current position of the hovering camera 100 to the target position is acquired. Since the combined image is generated based on the image captured by the hovering camera 100, the direction of the target position indicates a current direction from the hovering camera 100.
- the hovering camera 100 is in a mode in which acquisition of position information by the GPS is possible, autonomous flight is performed based on the GPS position information (an autonomous flight mode).
- the abnormality of the GPS refers to a state in which it is hard for the hovering camera 100 to stably receive the radio waves from the GPS satellites, for example, a state in which a GPS signal level is lower than a threshold value, the GPS position information drastically changes, or the position information acquired from the GPS satellites is different from a value of an acceleration sensor.
- the control unit 110 of the hovering camera 100 recognizes a real space through an image recognition process such as simultaneous localization and mapping (SLAM). Subsequently, the control unit 110 associates the real space with coordinates on the combined image, and thus it is possible to calculate a direction and a distance of a target position viewed from a current position.
- SLAM simultaneous localization and mapping
- movement amounts may be estimated by the control unit 110 using an acceleration sensor and a gyro sensor and combined in the image recognition process.
- Switching between the autonomous flight mode and the user operation flight mode is controlled by control unit 110 of the hovering camera 100 or control unit of the control terminal 200.
- the abnormality of the GPS is also detected by control unit 110 of the hovering camera 100 or control unit of the control terminal 200.
- the control terminal 200 can cause the user to change the reduction scale of the combined image appropriately. Particularly, when the user inputs a target position on the combined image, the control terminal 200 can set a more detailed target position by causing the user to change the reduction scale of the combined image. Further, it is possible to designate an arbitrary portion on the display screen of the display unit 210 including the combined image as the target position 276. In other words, in addition to a portion in the combined image, a portion (for example, a display area other than the combined image 275 in FIG. 25) that is not captured by the hovering camera 100 may be designated by the user.
- the imaging device is assumed to be installed in the hovering camera 100 downward so that an upward direction of a captured image is a front direction of the hovering camera 100, and a left direction is a left direction of the hovering camera 100.
- the target position 276 is in a left rear direction when viewed from the current position of the hovering camera 100 (that is, the center 273).
- the control terminal 200 determines a flight path 277 in which the hovering camera 100 flies in the left rear direction when viewed from the current position of the hovering camera 100, generates flight information for performing a horizontal flight along the flight path 277, and transmits the generated flight information to the hovering camera 100.
- the determining of the flight path 277 and the generating of the flight information may be executed, for example, by the flight information generating unit 232.
- the hovering camera 100 performs a horizontal flight in the left rear direction based on the flight information transmitted from the control terminal 200.
- the flight information may be only a direction to the target position 276 or may include a direction and a distance to the target position 276.
- the control unit 110 may determines whether or not the hovering camera 100 has arrived at the destination using the image recognition process. When the determining is performed using the image recognition process, arrival at the destination can be determined even in an environment in which it is difficult to receive the GPS information.
- a notification indicating that the hovering camera 100 has arrived near the destination can be given to the user by the control terminal 200. For example, when the remaining distance from the current position to the target position is below 5 pixels on the combined image, a notification indicating that the hovering camera 100 has arrived near the destination may be given to the user by the control terminal 200.
- FIG. 24 is an explanatory diagram illustrating an exemplary combined image updated using a new captured image obtained by movement of the target position 276 of the hovering camera 100.
- the position of the center 273 is substantially the same, and the target position 276 is approaching the center 273. In other words, it is understood from FIG. 24 that the target position 276 has moved from its position illustrated in FIG. 23.
- the control terminal 200 generates the flight information by performing feedback control such that the target position 276 approaches the center 273 by a certain distance or less through new capturing by the hovering camera 100 and updating of the combined image, and transmits the generated flight information to the hovering camera 100.
- the control terminal 200 may first generate the flight information for rotating the hovering camera 100 toward the target position and may generate the flight information for moving toward the target position after rotating the hovering camera 100. Even when the hovering camera 100 is caused to be rotated, the control terminal 200 performs the feature point matching as described above, and thus it is possible to rotate the combined image with the rotation of the hovering camera 100.
- a movement direction in an image differs according to a method of attaching the imaging device to the hovering camera 100.
- the control terminal 200 may control movement of the hovering camera 100 in the vertical direction in addition to movement of the hovering camera 100 in the horizontal direction.
- the control terminal 200 displays a slider for movement in the vertical direction on the screen, and the user may perform an operation of causing the hovering camera 100 to land by operating the slider.
- FIG. 25 is an explanatory diagram illustrating an exemplary combined image when the imaging device 101 is installed in the hovering camera 100 in the transverse direction.
- FIG. 25 illustrates a combined image 281 generated from the images captured by the hovering camera 100.
- a portion surrounded by a reference numeral 282 indicates an image that is captured at the current position by the hovering camera 100.
- control terminal 200 An exemplary operation of the control terminal 200 according to an embodiment of the present disclosure will be described based on the above description.
- FIG. 26 is a flowchart illustrating an exemplary operation of the control terminal 200 according to an embodiment of the present disclosure.
- FIG. 26 illustrates an exemplary operation of the control terminal 200 when the combined image is generated from the images captured by the hovering camera 100, the flight information is generated based on the combined image, and the flight information is transmitted to the hovering camera 100.
- the control terminal 200 first performs an initialization process through the display control unit 234 (step S401).
- the initialization process is a process of using the captured image initially transmitted from the hovering camera 100 as an input image and using the input image as a first combined image.
- control terminal 200 uses the image captured by the hovering camera 100 as the input image, and extracts the feature points of the input image and the combined image through the display control unit 234 (step S402).
- the control terminal 200 then performs matching of the extracted feature points, and calculates a movement amount or a rotation amount between the feature points through the display control unit 234 (step S403).
- control terminal 200 converts the combined image according to the movement amount or the rotation amount between the feature points obtained in step S403 through the display control unit 234 (step S404). At the time of the conversion, when the target position has been already designated, the control terminal 200 performs the conversion according to even the target position through the display control unit 234.
- control terminal 200 combines a previous combined image with the input image to generate a new combined image, and performs the process of displaying the combined image through the display control unit 234 (step S405).
- the control terminal 200 determines whether or not there is the user's input on the display unit 210 on which the combined image is being displayed through the display control unit 234 (step S406).
- the user's input on the display unit 210 on which the combined image is being displayed is the user's input on the touch panel installed in the display unit 210.
- the control terminal 200 detects a coordinate position of the image input by the user, for example, through the display control unit 234.
- the control terminal 200 performs a process of registering the user's touch position as the target position through the display control unit 234 (step S407).
- the control terminal 200 skips the process of step S407.
- control terminal 200 determines whether or not the target position has been registered through the display control unit 234 (step S408).
- the control terminal 200 obtains the direction of the target position from the current position through the display control unit 234 (step S409).
- the control terminal 200 When the direction of the target position from the current position is obtained, the control terminal 200 then performs conversion from the direction on the image to the movement direction of the fuselage of the hovering camera 100 based on the imaging direction information (according to the attachment) of the imaging device 101 (step S410). Then, the control terminal 200 generates a command for performing movement in the converted movement direction as the flight information through the flight information generating unit 232, and transmits the generated flight information to the hovering camera 100 (step S411).
- step S408 when the target position is determined to have not been registered in step S408 (No in step S408), the control terminal 200 skips the process of steps S409 to S411 and returns to the feature point extraction process of step S402.
- the control terminal 200 performs the above-described process and thus can combine the images captured by the hovering camera 100 as necessary, generate the combined image, generate the flight information for flying to the position designated on the combined image, and transmit the flight information to the hovering camera 100.
- control terminal 200 generates the flight information for controlling the flight of the hovering camera 100 according to the touch process on the combined image, but the present disclosure is not limited to the relevant example.
- control terminal 200 may generate the flight information for controlling the flight of the hovering camera 100 according to the reduction process by the pinch-in operation on the combined image, the enlargement process by the pinch-out operation on the combined image, or the rotation operation on the combined image.
- the control terminal 200 may generate the flight information for causing the hovering camera 100 to move far away from the imaging target. Further, when the pinch-out operation (the operation of enlarging the combined image) on the combined image is performed, the control terminal 200 may generate the flight information for causing the hovering camera 100 to approach the imaging target. Specifically, control is performed such that the flight information is generated based on the information indicating the imaging direction of the imaging device 101 installed in the hovering camera 100.
- the control terminal 200 may generate the flight information so that rotation is performed in a state in which the imaging device 101 of the hovering camera 100 faces the imaging target.
- the technology of the present embodiment can be applied to control of, for example, all mobile objects such as a robot equipped with an imaging device.
- the combination process of the images captured by the hovering camera 100 is performed by the control terminal 200, and the flight instruction is transmitted from the control terminal 200 to the hovering camera 100.
- the image captured by the hovering camera 100 is transmitted from the hovering camera 100 to the control terminal 200 each time capturing is performed, and the flight information generated by the control terminal 200, that is, the information for enabling the hovering camera 100 to directly interpret the movement direction is transmitted from the control terminal 200 to the hovering camera 100 each time the flight information is generated and updated.
- the hovering camera 100 may perform the extracting of the parameter by imaging processing of the image captured by the hovering camera 100 and the generating of the command for controlling the flight of the hovering camera 100, and the control terminal 200 may perform the combination process on the images captured by the hovering camera 100 and may receive only the input of the target position from the user.
- the image captured by the hovering camera 100 is transmitted from the hovering camera 100 to the control terminal 200 each time capturing is performed, and the information of the target position designated by the user is transmitted from the control terminal 200 to the hovering camera 100 each time the user designates the target position.
- the hovering camera 100 performs the extracting of the parameter by imaging processing of the image captured by the hovering camera 100 and the generating of the command for controlling the flight of the hovering camera 100, the feedback control is completed only inside the hovering camera 100. Thus, even when communication traffic necessary for exchange of information between the control terminal 200 and the hovering camera 100 is reduced or communication between the control terminal 200 and the hovering camera 100 is disconnected, it is possible to cause the hovering camera 100 to fly safely.
- the combined image is generated so that the image most recently captured by the hovering camera 100 is positioned at the center of the screen, but the present disclosure is not limited to the relevant example.
- the image captured by the hovering camera 100 may be combined with the combined image in a state in which the display position of the combined image is fixed.
- the target position is fixed, and the current position of the hovering camera 100 is moved as the combined image is updated.
- the current position and the target position may be updated by scrolling the entire combined image so that the most recently captured image fits within the screen.
- the imaging device 101 may not be attached to the hovering camera 100 so that the imaging direction is fixed, and, for example, the imaging device 101 may be attached to the hovering camera 100 so that the imaging direction is changed by a motor or the like.
- the control unit 110 may detect the imaging direction of the imaging device 101, and the combined image generation process and the process of designating the movement direction of the hovering camera 100 may be performed according to the imaging direction detected by the control unit 110.
- the hovering camera 100 that performs an automatic flight based on set flight information and captures a structure of an inspection target and the inspection system 10 that is capable of checking a damage state of a structure based on a still image captured by the hovering camera 100 are provided.
- the inspection system 10 uses information related to a structure of an inspection target when generating flight information to be transmitted to the hovering camera 100. Using the information related to the structure of the inspection target, the control terminal 200 can generate flight information for causing the hovering camera 100 to fly and efficiently inspecting a structure of an inspection target.
- control terminal 200 that is capable of generating a combined image from images captured by the hovering camera 100 and generating flight information for moving the hovering camera 100 according to an input on the combined image.
- the control terminal 200 according to an embodiment of the present disclosure can enable the user to intuitively operate the hovering camera 100 by generating a combined image from images captured by the hovering camera 100 and generating flight information for moving the hovering camera 100 according to an input on the combined image.
- the control terminal 200 according to an embodiment of the present disclosure can enable the user to operate the hovering camera 100 easily without forcing the user to perform a complicated operation.
- the hovering camera 100 may capture a moving image of the bridge 1 while flying, and the information processing device 300 may generate damage data using the moving image captured by the hovering camera 100.
- the hovering camera 100 acquires position information periodically when a moving image imaging process is performed and associates an imaging time of a moving image with an acquisition time of position information, and thus the information processing device 300 can generate damage data using a moving image.
- each step of a process executed by each device of the present specification may be performed in an order different from the order described as a flowchart, or may be performed in parallel.
- a computer program for causing hardware such as a CPU, a ROM, and a RAM installed in each device to exhibit the equivalent functions to those of each of the devices described above can also be created.
- a storage medium in which such a computer program is stored can also be provided.
- each of the functional blocks shown in the functional block diagram to be hardware or a hardware circuit, a series of processes can also be realized using hardware or a hardware circuit.
- some or all functional blocks illustrated in the functional block diagrams used in the above description may be implemented by a server device connected via a network such as the Internet.
- each of components of functional blocks illustrated in the functional block diagrams used in the above description may be implemented by a single device or may be implemented by a system in which a plurality of devices collaborate with each other.
- Examples of the system in which a plurality of devices collaborate with each other include a combination of a plurality of server devices and a combination of a server device and a terminal device.
- the system can be applied to an automobile. For example, a driver can touch a preferable parking space on the composite image. Then, the automobile can be automatically moved to the preferable parking space according to the touch process.
- the present disclosure may also take the following configurations.
- a vehicle control system comprising: at least one imaging device attached to a vehicle and configured to capture a plurality of images; and a control circuit configured to generate a composite image from the plurality of images, and to display the composite image on a display unit, wherein the vehicle is operated according to a user operation on a portion of the display unit on which the composite image is being displayed.
- control circuit is configured to generate a target position information based on an input position of the user operation on the display unit relative to a vehicle position displayed on the display unit, the input position representing a target position, and the target position information includes a direction from the vehicle position to the target position in a real coordinate system.
- control circuit configured to display the composite image such that the vehicle position on the composite image is located in a center of the display unit.
- control circuit is configured to change display of the composite image such that the target position on the display unit approaches a center of the display unit as the vehicle approaches the target position in the real coordinate system.
- control circuitry is configured to switch a flight mode between an autonomous flight mode and a user operation flight mode, and wherein the hovering machine is operated according to the user operation in the user operation flight mode.
- control circuitry is configured to switch from the autonomous flight mode to the user operation flight mode in accordance with an abnormality of a position information acquisition circuitry.
- a vehicle control method comprising: capturing, via at least one imaging device attached to a vehicle, a plurality of images; generating a composite image from the plurality of images, and displaying the composite image on a display unit; and operating the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
- a computer system comprising: at least one processing unit; and a memory, the memory including a non-transitory computer-readable medium storing instructions that, when executed by the at least one processing unit, cause the computer system to: cause at least one imaging device attached to a vehicle to capture a plurality of images, generate a composite image from the plurality of images, display the composite image on a display unit, and operate the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
- a control device including: an image processing unit configured to generate a combined image from images captured by a mobile object equipped with an imaging device; and a movement information generating unit configured to generate movement information for moving the mobile object according to an operation on the combined image generated by the image processing unit.
- a control method including: generating a combined image from images captured by a mobile object equipped with an imaging device; and generating movement information for moving the mobile object according to an operation on the generated combined image.
- a computer program causing a computer to execute: generating a combined image from images captured by a mobile object equipped with an imaging device; and generating movement information for moving the mobile object according to an operation on the generated combined image.
- inspection system 100 hovering camera 101 imaging device 104a to 104d rotor 108a to 108d motor 110 control unit 120 communication unit 130 sensor unit 132 position information acquisition unit 140 storage unit 150 battery 200 control terminal 300 information processing device 400 wireless relay node 500 position estimation node 600 base station 700 charging station
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Studio Devices (AREA)
- Closed-Circuit Television Systems (AREA)
- Traffic Control Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Vehicle Cleaning, Maintenance, Repair, Refitting, And Outriggers (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
Description
1. Embodiment of present disclosure
1.1. Overview
1.2. Exemplary system configuration
1.3. Exemplary function configuration
1.4. Exemplary operation
1.5. Exemplary damage data generation
1.5.1. Exemplary function configuration
1.5.2. Exemplary operation
1.6. Exemplary flight instruction using combined image
2. Conclusion
(1.1. Overview)
In detailed description of an embodiment of the present disclosure, an overview of an embodiment of the present disclosure will be first described.
FIG. 2 is an explanatory diagram illustrating an exemplary system configuration of an
An exemplary function configuration of the hovering
FIG. 5 is a flowchart illustrating an exemplary operation of the
ID: (relative coordinates of imaging point, imaging direction, speed at time of imaging, traveling time to next imaging point, and others)
The relative coordinates of an imaging point are designated by three points of an X axis, a Y axis, and a Z axis. The X axis is set as a latitude direction, the Y axis is set as a longitude direction, and the Z axis is set as a height direction. Further, for example, information used to control special capturing may be included as other information. Examples of the information used to control special capturing include information for capturing the same position in a plurality of imaging directions, information related to a parameter for bracket capturing (which indicates capturing by different exposures, different shutter speeds, different ISO sensitivities, and the like at the same position and in the same imaging direction), and information on a wavelength of infrared rays at the time of capturing. According to this format, the flight information generated by the
0:(0,0,0,0,0,2,1.0)
1:(5,0,0,0,0,2,1.0)
2:(7,0,0,0,0,2,1.0)
3:(9,0,0,0,0,2,1.0)
The imaging point included in the flight information generated by the
For example, a location which is not easily accessible such as the bottom surface of the
(1) The information of the imaging position of the still image is recorded as the position of the damaged portion (a relative value (offset) is not recorded).
(2) The relative value (offset) corresponding to the information of the imaging position of the still image and damaged portion is recorded as the position of the damaged portion.
(3) The absolute value used as a reference (for example, as will be described below, the imaging position of the still images of four corners considered to be highly accurate in position information, or coordinates of the position estimation node 500) and the relative value (offset) are recorded as the position of the damaged portion.
(4) The calculated absolute value (for example, a latitude, a longitude, and an altitude) is recorded as the position of the damaged portion.
(image ID, damage ID, position information of damaged portion, coordinates of damaged portion on image, damage type ID, damage degree)
The damage type ID refers to an ID allocated to a type of damage such as a crack, peeling, a water leak, or free lime. Further, a maximum width of data, a length of a damaged portion in an image, or the like may be recorded in the damage degree field. The
(damage ID, position information of damaged portion, coordinates of damaged portion on image, damage type ID, damage degree)
Further, the image ID of the stitched image may be generated and included in the damage data. The
FIG. 15 is an explanatory diagram illustrating an exemplary function configuration of the
FIG. 16 is a flowchart illustrating an exemplary operation of the
While the following description is provided using the example of a hovering camera provided on a flying body, this disclosure is not so limited and may apply to any vehicle having at least one camera. For example, the following may apply to a hovering vehicle such as a drone equipped with at least one imaging device and that may move along a two- or three-dimensional flight path. However, the following may also apply similarly to a land-based vehicle such as an automobile equipped with at least one imaging device, and that may move along a two-dimensional (ground-based) drive path. Thus, the following description of a "hovering camera" is illustrative only and not limiting. Additionally, the control terminal described below may generate flight (drive) information for a non-linear or polyline flight (drive) path to be followed by the vehicle, instead of the linear path described below.
As described above, according to an embodiment of the present disclosure, the hovering
the target position information includes a direction from the vehicle position to the target position in a real coordinate system.
100 hovering camera
101 imaging device
104a to 104d rotor
108a to 108d motor
110 control unit
120 communication unit
130 sensor unit
132 position information acquisition unit
140 storage unit
150 battery
200 control terminal
300 information processing device
400 wireless relay node
500 position estimation node
600 base station
700 charging station
Claims (25)
- A vehicle control system, comprising:
at least one imaging device attached to a vehicle and configured to capture a plurality of images; and
a control circuit configured to generate a composite image from the plurality of images, and to display the composite image on a display unit,
wherein the vehicle is operated according to a user operation on a portion of the display unit on which the composite image is being displayed.
- The vehicle control system according to claim 1, wherein the user operation is an operation on the composite image.
- The vehicle control system according to claim 1, wherein
the display unit includes a first display area for displaying the composite image,
the user operation is an operation on a second display area of the display unit different from the first display area.
- The vehicle control system according to claim 1, wherein
the control circuit is configured to generate a target position information based on an input position of the user operation on the display unit relative to a vehicle position displayed on the display unit, the input position representing a target position, and
the target position information includes a direction from the vehicle position to the target position in a real coordinate system.
- The vehicle control system according to claim 4, wherein the target position information includes a distance from the vehicle position to the target position in the real coordinate system.
- The vehicle control system according to claim 4, wherein the control circuit is configured to transmit the target position information to the vehicle.
- The vehicle control system according to claim 4, wherein the control circuit is configured to display the composite image such that the vehicle position on the composite image is located in a center of the display unit.
- The vehicle control system according to claim 4, wherein the control circuit is configured to change display of the composite image such that the target position on the display unit approaches a center of the display unit as the vehicle approaches the target position in the real coordinate system.
- The vehicle control system according to claim 4, wherein
the user operation is a pinch-in operation, and
the vehicle is configured to approach the target position in the real coordinate system in response to the pinch-in operation.
- The vehicle control system according to claim 1, wherein the vehicle is a hovering machine.
- The vehicle control system according to claim 1, wherein the vehicle is an automobile.
- The vehicle control system according to claim 1, wherein the composite image is a stitching image.
- A vehicle control method, comprising:
capturing, via at least one imaging device attached to a vehicle, a plurality of images;
generating a composite image from the plurality of images, and displaying the composite image on a display unit; and
operating the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
- The vehicle control method according to claim 13, wherein the user operation is an operation on the composite image.
- The vehicle control method according to claim 13, wherein
displaying the composite image on the display unit includes displaying the composite image on a first display area of the display unit, and
the user operation is an operation on a second display area of the display unit different from the first display area.
- The vehicle control method according to claim 13, further comprising:
generating a target position based on an input position of the user operation on the display unit relative to a vehicle position displayed on the display unit, the input position representing a target position,
wherein the target position information includes a direction from the vehicle position to the target position in a real coordinate system.
- The vehicle control method according to claim 16, wherein the target position information includes a distance from the vehicle position to the target position in the real coordinate system.
- The vehicle control method according to claim 16, further comprising:
transmitting the target position information to the vehicle.
- The vehicle control method according to claim 16, further comprising:
displaying the composite image such that the vehicle position on the composite image is located in the center of the display unit.
- The vehicle control method according to claim 16, further comprising:
changing display of the composite image such that the target position on the display unit approaches a center of the display unit as the vehicle approaches the target position in the real coordinate system.
- The vehicle control method according to claim 16, further comprising:
causing the vehicle to approach the target position in the real coordinate system in response to the user operation, wherein the user operation is a pinch-in operation.
- The vehicle control method according to claim 13, wherein the vehicle is a hovering machine.
- The vehicle control method according to claim 13, wherein the vehicle is an automobile.
- The vehicle control method according to claim 13, wherein the composite image is a stitching image.
- A computer system, comprising:
at least one processing unit; and
a memory, the memory including a non-transitory computer-readable medium storing instructions that, when executed by the at least one processing unit, cause the computer system to:
cause at least one imaging device attached to a vehicle to capture a plurality of images,
generate a composite image from the plurality of images,
display the composite image on a display unit, and
operate the vehicle according to a user operation on a portion of the display unit on which the composite image is being displayed.
Priority Applications (12)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2015331965A AU2015331965A1 (en) | 2014-10-17 | 2015-10-09 | Inspection vehicle control device, control method, and computer program |
CA2959471A CA2959471A1 (en) | 2014-10-17 | 2015-10-09 | Control device, control method, and computer program |
EP15787034.6A EP3206768B1 (en) | 2014-10-17 | 2015-10-09 | Inspection vehicle control device, control method, and computer program |
CN201811272015.8A CN109373975B (en) | 2014-10-17 | 2015-10-09 | Control device and control method for detecting vehicle |
CN201911346231.7A CN111009156B (en) | 2014-10-17 | 2015-10-09 | Control apparatus, control method, and computer program for detecting vehicle |
US15/505,541 US10198003B2 (en) | 2014-10-17 | 2015-10-09 | Control device, control method, and computer program |
CN201580054839.6A CN106796112B (en) | 2014-10-17 | 2015-10-09 | Control apparatus, control method, and computer program for detecting vehicle |
EP23190486.3A EP4253911A3 (en) | 2014-10-17 | 2015-10-09 | Inspection vehicle control device, control method, and computer program |
AU2018222910A AU2018222910B2 (en) | 2014-10-17 | 2018-08-28 | Inspection vehicle control device, control method, and computer program |
US16/232,699 US11520334B2 (en) | 2014-10-17 | 2018-12-26 | Control device, control method, and computer program |
US18/056,730 US11927960B2 (en) | 2014-10-17 | 2022-11-18 | Control device, control method, and computer program |
US18/443,346 US20240184290A1 (en) | 2014-10-17 | 2024-02-16 | Control device, control method, and computer program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-212953 | 2014-10-17 | ||
JP2014212953A JP6387782B2 (en) | 2014-10-17 | 2014-10-17 | Control device, control method, and computer program |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/505,541 A-371-Of-International US10198003B2 (en) | 2014-10-17 | 2015-10-09 | Control device, control method, and computer program |
US16/232,699 Continuation US11520334B2 (en) | 2014-10-17 | 2018-12-26 | Control device, control method, and computer program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016059785A1 true WO2016059785A1 (en) | 2016-04-21 |
Family
ID=54360508
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/005142 WO2016059785A1 (en) | 2014-10-17 | 2015-10-09 | Inspection vehicle control device, control method, and computer program |
Country Status (7)
Country | Link |
---|---|
US (4) | US10198003B2 (en) |
EP (2) | EP4253911A3 (en) |
JP (1) | JP6387782B2 (en) |
CN (3) | CN106796112B (en) |
AU (2) | AU2015331965A1 (en) |
CA (1) | CA2959471A1 (en) |
WO (1) | WO2016059785A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017166002A1 (en) * | 2016-03-28 | 2017-10-05 | 深圳市大疆创新科技有限公司 | Hovering control method and system for unmanned aerial vehicle, and unmanned aerial vehicle |
WO2018166645A1 (en) * | 2017-03-14 | 2018-09-20 | Bitmanagement Software GmbH | Method for determining a path along an object, system and method for automatically inspecting an object |
EP3460729A1 (en) * | 2017-09-26 | 2019-03-27 | Ricoh Company, Ltd. | Information processing apparatus, system of assessing structural object, method of assessing structural object system of assessing structural object, and carrier means |
CN110178099A (en) * | 2017-05-26 | 2019-08-27 | 广州极飞科技有限公司 | Unmanned plane course determines method and unmanned plane |
TWI682877B (en) * | 2018-06-04 | 2020-01-21 | 宏碁股份有限公司 | Movable device control method, server device and movable device control system |
Families Citing this family (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10618168B2 (en) * | 2016-05-13 | 2020-04-14 | General Electric Company | Robot system path planning for asset health management |
JP6387782B2 (en) * | 2014-10-17 | 2018-09-12 | ソニー株式会社 | Control device, control method, and computer program |
JP6454793B2 (en) | 2015-09-16 | 2019-01-16 | 富士フイルム株式会社 | Soundness determination device, soundness determination method, and soundness determination program |
JP6685742B2 (en) * | 2016-02-02 | 2020-04-22 | キヤノン株式会社 | Operating device, moving device, and control system thereof |
JP6626970B2 (en) * | 2016-06-14 | 2019-12-25 | 富士フイルム株式会社 | Server device, image processing system, and image processing method |
JP6156605B1 (en) * | 2016-07-20 | 2017-07-05 | 中国電力株式会社 | Unmanned air vehicle, charging system, and control method of charging system |
KR101850617B1 (en) * | 2016-09-26 | 2018-04-20 | 부산대학교 산학협력단 | System and method for monitoring deflection of cable bridge |
WO2018070354A1 (en) * | 2016-10-14 | 2018-04-19 | 富士フイルム株式会社 | Image-capturing plan generation device, image-capturing plan generation method, and program |
JP6716015B2 (en) * | 2017-03-16 | 2020-07-01 | 富士フイルム株式会社 | Imaging control device, imaging system, and imaging control method |
JP2018160196A (en) * | 2017-03-23 | 2018-10-11 | キヤノンマーケティングジャパン株式会社 | Unmanned aerial vehicle control system, control method of unmanned aerial vehicle control system, and program |
JP6811483B2 (en) * | 2017-03-30 | 2021-01-13 | 株式会社スカイマティクス | Systems and methods to support work with drones |
WO2018198313A1 (en) * | 2017-04-28 | 2018-11-01 | 株式会社オプティム | Unmanned aerial vehicle action plan creation system, method and program |
JP2019045898A (en) * | 2017-08-29 | 2019-03-22 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | Flying body control method, flying body, program and recording medium |
CN109708636B (en) * | 2017-10-26 | 2021-05-14 | 广州极飞科技股份有限公司 | Navigation chart configuration method, obstacle avoidance method and device, terminal and unmanned aerial vehicle |
US10872534B2 (en) * | 2017-11-01 | 2020-12-22 | Kespry, Inc. | Aerial vehicle inspection path planning |
JP7084571B2 (en) * | 2017-11-29 | 2022-06-15 | エンジェルスウィング インコーポレイテッド | A method and device for matching consumers and suppliers and providing drone data |
JP7153442B2 (en) * | 2017-12-19 | 2022-10-14 | 株式会社ダイヘン | moving body |
CN112106342B (en) * | 2018-02-26 | 2023-06-23 | 株式会社OPTiM | Computer system, unmanned aerial vehicle control method, and recording medium |
WO2019163119A1 (en) * | 2018-02-26 | 2019-08-29 | 株式会社オプティム | Computer system, drone control method, and program |
CN108647563A (en) * | 2018-03-27 | 2018-10-12 | 阿里巴巴集团控股有限公司 | A kind of method, apparatus and equipment of car damage identification |
JP6989849B2 (en) * | 2018-04-19 | 2022-01-12 | 株式会社Geoソリューションズ | Inspection system for the structure to be inspected |
JP6719738B2 (en) * | 2018-06-19 | 2020-07-08 | 電源開発株式会社 | Autonomous mobile shooting control system and autonomous mobile body |
KR102225112B1 (en) * | 2018-08-30 | 2021-03-09 | 한국공항공사 | Apparatus and method for inspecting navigation aid using flight vehicle |
JP7195117B2 (en) * | 2018-11-14 | 2022-12-23 | エヌ・ティ・ティ・コムウェア株式会社 | Information display system, information display method and information display program |
JP7472796B2 (en) | 2018-12-13 | 2024-04-23 | ソニーグループ株式会社 | Control device, control method, and control program |
CN109739234B (en) * | 2019-01-02 | 2022-05-17 | 中电海康集团有限公司 | Vehicle real-time image tracking method based on GPS track data |
WO2020145004A1 (en) * | 2019-01-10 | 2020-07-16 | 日本電気株式会社 | Photography guide device |
JP7412983B2 (en) * | 2019-02-04 | 2024-01-15 | キヤノン株式会社 | Information processing device, information processing method, and program |
WO2020179491A1 (en) * | 2019-03-06 | 2020-09-10 | ソニー株式会社 | Action control device, action control method, and program |
CN109982049B (en) * | 2019-04-04 | 2020-10-13 | 郑州和光电子科技有限公司 | Panorama fusion display control system |
JP2020201850A (en) * | 2019-06-12 | 2020-12-17 | 九州旅客鉄道株式会社 | Control method of flying body |
US20220297821A1 (en) * | 2019-08-26 | 2022-09-22 | Sony Group Corporation | Control device, control method, unmanned aircraft, information processing device, information processing method, and program |
JP7263983B2 (en) * | 2019-08-30 | 2023-04-25 | 富士通株式会社 | Photography omission detection device and photography omission detection method |
JP6684507B1 (en) * | 2019-10-09 | 2020-04-22 | 株式会社エイテック | Bridge inspection method and bridge inspection system |
CN112639735A (en) * | 2019-10-30 | 2021-04-09 | 深圳市大疆创新科技有限公司 | Distribution of calculated quantities |
JP7352908B2 (en) * | 2019-12-11 | 2023-09-29 | 株式会社Acsl | System, method, program, and storage medium storing the program for correcting the estimated position of an unmanned aircraft |
CN111174765B (en) * | 2020-02-24 | 2021-08-13 | 北京航天飞行控制中心 | Planet vehicle target detection control method and device based on visual guidance |
JPWO2021181553A1 (en) * | 2020-03-11 | 2021-09-16 | ||
JP6994087B2 (en) * | 2020-07-16 | 2022-01-14 | Kddi株式会社 | Flight equipment, flight control methods and programs |
JP2023117420A (en) * | 2020-07-16 | 2023-08-24 | ソニーグループ株式会社 | Information processing method, program, and system |
JP7047180B2 (en) * | 2020-07-16 | 2022-04-04 | Kddi株式会社 | Management device |
JP2021047205A (en) * | 2020-12-10 | 2021-03-25 | Kddi株式会社 | Determination device, determination method, and program |
US20220292633A1 (en) * | 2021-03-15 | 2022-09-15 | International Business Machines Corporation | Image stitching for high-resolution scans |
WO2023047859A1 (en) * | 2021-09-22 | 2023-03-30 | 富士フイルム株式会社 | Information processing device, method and program, and image data structure |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030216834A1 (en) * | 2000-05-01 | 2003-11-20 | Allard James R. | Method and system for remote control of mobile robot |
WO2005033629A2 (en) * | 2003-09-19 | 2005-04-14 | University Of Miami | Multi-camera inspection of underwater structures |
JP2006027448A (en) | 2004-07-16 | 2006-02-02 | Chugoku Electric Power Co Inc:The | Aerial photographing method and device using unmanned flying body |
US20080215204A1 (en) * | 2006-12-06 | 2008-09-04 | Mercury Computer Systems, Inc. | Methods, apparatus and systems for enhanced synthetic vision and multi-sensor data fusion to improve operational capabilities of unmanned aerial vehicles |
US20090015674A1 (en) * | 2006-04-28 | 2009-01-15 | Kevin Alley | Optical imaging system for unmanned aerial vehicle |
US20120232718A1 (en) * | 2011-03-08 | 2012-09-13 | Parrot | Method of piloting a multiple rotor rotary-wing drone to follow a curvilinear turn |
Family Cites Families (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE2923634C2 (en) * | 1979-06-11 | 1985-01-17 | Siemens AG, 1000 Berlin und 8000 München | Control system for individual traffic |
JPS57186111A (en) * | 1981-05-13 | 1982-11-16 | Nissan Motor Co Ltd | Map display device for vehicle |
JPS61245015A (en) * | 1985-04-23 | 1986-10-31 | Mitsubishi Electric Corp | Direction display device |
JP2891795B2 (en) * | 1991-04-12 | 1999-05-17 | パイオニア株式会社 | In-vehicle navigation device |
JPH0729099A (en) * | 1993-07-13 | 1995-01-31 | Kunimichi Takada | Device for displaying delivery map |
JP2001292444A (en) * | 2000-04-10 | 2001-10-19 | Mitsubishi Electric Corp | Image pickup system |
WO2001084260A2 (en) * | 2000-05-01 | 2001-11-08 | Irobot Corporation | Method and system for remote control of mobile robot |
JP2003195941A (en) | 2001-12-26 | 2003-07-11 | Matsushita Electric Ind Co Ltd | Method for providing service for moving mobile object |
JP2005269413A (en) | 2004-03-19 | 2005-09-29 | Chugoku Electric Power Co Inc:The | Aerial photography system using unmanned flying object |
JP2006113858A (en) | 2004-10-15 | 2006-04-27 | Mitsubishi Heavy Ind Ltd | Method and system for supporting remote operation for mobile object |
JP4502793B2 (en) * | 2004-12-14 | 2010-07-14 | 三菱電機株式会社 | Shooting video processing system |
JP2006308311A (en) | 2005-04-26 | 2006-11-09 | Xanavi Informatics Corp | Navigation apparatus |
JP4222355B2 (en) * | 2005-09-29 | 2009-02-12 | トヨタ自動車株式会社 | PARKING ASSISTANCE DEVICE AND POWER TRANSFER METHOD BETWEEN VEHICLE AND GROUND EQUIPMENT |
JP4306670B2 (en) * | 2005-10-31 | 2009-08-05 | トヨタ自動車株式会社 | Parking assistance device |
JP5140887B2 (en) * | 2006-11-07 | 2013-02-13 | 株式会社小松製作所 | Unmanned vehicle guided traveling control method and control device |
US20090281727A1 (en) * | 2008-05-06 | 2009-11-12 | Matsushita Electric Industrial Co., Ltd. | Car navigation system |
US20100286859A1 (en) | 2008-11-18 | 2010-11-11 | Honeywell International Inc. | Methods for generating a flight plan for an unmanned aerial vehicle based on a predicted camera path |
BRPI0920950A2 (en) * | 2008-11-20 | 2015-12-29 | Bae Systems Plc | aircraft and method of controlling the direction the flight direction of an aircraft |
EP2387526B1 (en) * | 2009-01-17 | 2019-08-14 | Boomerang Systems, Inc. | Omnidirectional drive and steering unit |
JP5402057B2 (en) | 2009-02-16 | 2014-01-29 | トヨタ自動車株式会社 | Mobile robot control system, route search method, route search program |
JP5326678B2 (en) | 2009-03-09 | 2013-10-30 | トヨタ自動車株式会社 | Navigation device |
JP5326794B2 (en) | 2009-05-15 | 2013-10-30 | トヨタ自動車株式会社 | Remote operation system and remote operation method |
JP2011065202A (en) | 2009-09-15 | 2011-03-31 | Hitachi Ltd | Autonomous mobile device |
CN201575814U (en) * | 2009-12-17 | 2010-09-08 | 北京华旗随身数码股份有限公司 | Visual vehicle-mounted navigation device |
DE202010000176U1 (en) * | 2010-02-12 | 2011-06-30 | AL-KO Kober AG, 89359 | remote control |
JP2011174799A (en) | 2010-02-24 | 2011-09-08 | Mitsubishi Electric Corp | Photographing route calculation device |
US8743199B2 (en) * | 2010-03-09 | 2014-06-03 | Physical Optics Corporation | Omnidirectional imaging optics with 360°-seamless telescopic resolution |
FR2957266B1 (en) * | 2010-03-11 | 2012-04-20 | Parrot | METHOD AND APPARATUS FOR REMOTE CONTROL OF A DRONE, IN PARTICULAR A ROTATING SAIL DRONE. |
JP5548002B2 (en) * | 2010-03-25 | 2014-07-16 | 富士通テン株式会社 | Image generation apparatus, image display system, and image generation method |
JP5610870B2 (en) | 2010-06-21 | 2014-10-22 | 三菱重工業株式会社 | Unmanned traveling vehicle guidance device and unmanned traveling vehicle guidance method |
EP2423873B1 (en) * | 2010-08-25 | 2013-12-11 | Lakeside Labs GmbH | Apparatus and Method for Generating an Overview Image of a Plurality of Images Using a Reference Plane |
CN101963806B (en) * | 2010-10-15 | 2012-06-27 | 农业部南京农业机械化研究所 | Unmanned helicopter pesticide applying operation automatic control system and method based on GPS (Global Positioning System) navigation |
JP5618840B2 (en) | 2011-01-04 | 2014-11-05 | 株式会社トプコン | Aircraft flight control system |
JP2012179682A (en) | 2011-03-01 | 2012-09-20 | Nec Corp | Mobile robot system, mobile robot control device, and moving control method and moving control program to be used for the control device |
DE102011017564B4 (en) * | 2011-04-26 | 2017-02-16 | Airbus Defence and Space GmbH | Method and system for inspecting a surface for material defects |
JP5828088B2 (en) | 2011-11-04 | 2015-12-02 | パナソニックIpマネジメント株式会社 | Remote control system |
CN102566580A (en) * | 2011-12-27 | 2012-07-11 | 中国直升机设计研究所 | Unmanned helicopter flight track planning method |
FR2988618B1 (en) | 2012-03-30 | 2014-05-09 | Parrot | ALTITUDE ESTIMER FOR MULTI-ROTOR ROTOR SAIL DRONE |
CN102982515A (en) * | 2012-10-23 | 2013-03-20 | 中国电子科技集团公司第二十七研究所 | Method of unmanned plane image real-time splicing |
US9581438B2 (en) | 2012-12-31 | 2017-02-28 | General Electric Company | Systems and methods for control of a non-destructive testing system |
US9215448B2 (en) * | 2013-01-31 | 2015-12-15 | Here Global B.V. | Stereo panoramic images |
US9429425B2 (en) * | 2013-03-05 | 2016-08-30 | Here Global B.V. | Aerial image collection |
US9075415B2 (en) * | 2013-03-11 | 2015-07-07 | Airphrame, Inc. | Unmanned aerial vehicle and methods for controlling same |
US9355368B2 (en) * | 2013-03-14 | 2016-05-31 | Toyota Motor Engineering & Manufacturing North America, Inc. | Computer-based method and system for providing active and automatic personal assistance using a robotic device/platform |
DE102013110580B4 (en) * | 2013-09-24 | 2024-05-23 | Faro Technologies, Inc. | Method for optically scanning and measuring a scene and laser scanner designed to carry out the method |
CN103604427B (en) * | 2013-12-10 | 2016-10-12 | 中国航天空气动力技术研究院 | UAS and method to ground moving target dynamic location |
CN103942273B (en) * | 2014-03-27 | 2017-03-15 | 北京空间机电研究所 | A kind of aerial quick response dynamic monitoring system and its dynamic monitoring method |
US9986154B2 (en) * | 2014-05-21 | 2018-05-29 | Here Global B.V. | Developing a panoramic image |
EP3786742A1 (en) * | 2014-05-30 | 2021-03-03 | SZ DJI Technology Co., Ltd. | Systems and methods for uav docking |
CN105518555B (en) | 2014-07-30 | 2017-11-03 | 深圳市大疆创新科技有限公司 | Target tracking system and method |
JP6387782B2 (en) * | 2014-10-17 | 2018-09-12 | ソニー株式会社 | Control device, control method, and computer program |
US20170221241A1 (en) * | 2016-01-28 | 2017-08-03 | 8681384 Canada Inc. | System, method and apparatus for generating building maps |
US9823658B1 (en) * | 2016-11-04 | 2017-11-21 | Loveland Innovations, LLC | Systems and methods for adaptive property analysis via autonomous vehicles |
US20180184073A1 (en) * | 2016-12-23 | 2018-06-28 | Red Hen Systems Llc | Systems and Methods For Recording Stereo Pairs From Independent Camera Platforms |
-
2014
- 2014-10-17 JP JP2014212953A patent/JP6387782B2/en active Active
-
2015
- 2015-10-09 US US15/505,541 patent/US10198003B2/en active Active
- 2015-10-09 CN CN201580054839.6A patent/CN106796112B/en active Active
- 2015-10-09 EP EP23190486.3A patent/EP4253911A3/en active Pending
- 2015-10-09 CN CN201811272015.8A patent/CN109373975B/en active Active
- 2015-10-09 WO PCT/JP2015/005142 patent/WO2016059785A1/en active Application Filing
- 2015-10-09 EP EP15787034.6A patent/EP3206768B1/en active Active
- 2015-10-09 AU AU2015331965A patent/AU2015331965A1/en not_active Abandoned
- 2015-10-09 CA CA2959471A patent/CA2959471A1/en active Pending
- 2015-10-09 CN CN201911346231.7A patent/CN111009156B/en active Active
-
2018
- 2018-08-28 AU AU2018222910A patent/AU2018222910B2/en active Active
- 2018-12-26 US US16/232,699 patent/US11520334B2/en active Active
-
2022
- 2022-11-18 US US18/056,730 patent/US11927960B2/en active Active
-
2024
- 2024-02-16 US US18/443,346 patent/US20240184290A1/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030216834A1 (en) * | 2000-05-01 | 2003-11-20 | Allard James R. | Method and system for remote control of mobile robot |
WO2005033629A2 (en) * | 2003-09-19 | 2005-04-14 | University Of Miami | Multi-camera inspection of underwater structures |
JP2006027448A (en) | 2004-07-16 | 2006-02-02 | Chugoku Electric Power Co Inc:The | Aerial photographing method and device using unmanned flying body |
US20090015674A1 (en) * | 2006-04-28 | 2009-01-15 | Kevin Alley | Optical imaging system for unmanned aerial vehicle |
US20080215204A1 (en) * | 2006-12-06 | 2008-09-04 | Mercury Computer Systems, Inc. | Methods, apparatus and systems for enhanced synthetic vision and multi-sensor data fusion to improve operational capabilities of unmanned aerial vehicles |
US20120232718A1 (en) * | 2011-03-08 | 2012-09-13 | Parrot | Method of piloting a multiple rotor rotary-wing drone to follow a curvilinear turn |
Non-Patent Citations (2)
Title |
---|
"IRIS Operation Manual - Developer Edition", 16 October 2013 (2013-10-16), XP055241311, Retrieved from the Internet <URL:https://3dr.com/wp-content/uploads/2013/10/operation-manual-compressed.pdf> [retrieved on 20160114] * |
DANIEL WISCHOUNIG-STRUCL: "RESOURCE AWARE AND INCREMENTAL MOSAICS OF WIDE AREAS FROM SMALL-SCALE UAVS", 1 August 2014 (2014-08-01), XP055241543, Retrieved from the Internet <URL:http://uav.lakeside-labs.com/fileadmin/user_upload/Dissertations/Wischounig-Dissertaion_resized.pdf> [retrieved on 20160114] * |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11073389B2 (en) | 2016-03-28 | 2021-07-27 | SZ DJI Technology Co., Ltd. | Hover control |
WO2017166002A1 (en) * | 2016-03-28 | 2017-10-05 | 深圳市大疆创新科技有限公司 | Hovering control method and system for unmanned aerial vehicle, and unmanned aerial vehicle |
US10754354B2 (en) | 2016-03-28 | 2020-08-25 | SZ DJI Technology Co., Ltd. | Hover control |
WO2018166645A1 (en) * | 2017-03-14 | 2018-09-20 | Bitmanagement Software GmbH | Method for determining a path along an object, system and method for automatically inspecting an object |
CN110637264B (en) * | 2017-03-14 | 2024-01-26 | 超级七两合有限责任公司 | Method for determining a path along an object, system and method for automatically inspecting an object |
US11880118B2 (en) | 2017-03-14 | 2024-01-23 | Top Seven Gmbh & Co. Kg | Method of determining a path along an object, system and method for automatically inspecting an object |
CN110637264A (en) * | 2017-03-14 | 2019-12-31 | 比特管理软件有限责任公司 | Method for determining a path along an object, system and method for automatically inspecting an object |
US11435650B2 (en) | 2017-03-14 | 2022-09-06 | Top Seven Gmbh & Co. Kg | Method of determining a path along an object, system and method for automatically inspecting an object |
EP3876065A1 (en) | 2017-03-14 | 2021-09-08 | TOP seven GmbH & Co. KG | Method for determining a path along an object, system and method for automatic inspecting an object |
EP3596570B1 (en) | 2017-03-14 | 2021-07-21 | TOP seven GmbH & Co. KG | Method for determining a path along an object, system and method for automatic inspecting an object |
CN110178099B (en) * | 2017-05-26 | 2022-05-10 | 广州极飞科技股份有限公司 | Unmanned aerial vehicle course determining method and unmanned aerial vehicle |
CN110178099A (en) * | 2017-05-26 | 2019-08-27 | 广州极飞科技有限公司 | Unmanned plane course determines method and unmanned plane |
US11037352B2 (en) | 2017-09-26 | 2021-06-15 | Ricoh Company, Ltd. | Information processing apparatus, system of assessing structural object, method of assessing structural object and storage medium |
US10565765B2 (en) | 2017-09-26 | 2020-02-18 | Ricoh Company, Ltd. | Information processing apparatus, system of assessing structural object, method of assessing structural object and storage medium |
EP3460729A1 (en) * | 2017-09-26 | 2019-03-27 | Ricoh Company, Ltd. | Information processing apparatus, system of assessing structural object, method of assessing structural object system of assessing structural object, and carrier means |
TWI682877B (en) * | 2018-06-04 | 2020-01-21 | 宏碁股份有限公司 | Movable device control method, server device and movable device control system |
Also Published As
Publication number | Publication date |
---|---|
AU2018222910A1 (en) | 2018-09-13 |
EP3206768A1 (en) | 2017-08-23 |
CN111009156B (en) | 2022-06-07 |
CN106796112B (en) | 2020-01-21 |
US11520334B2 (en) | 2022-12-06 |
US20190146503A1 (en) | 2019-05-16 |
US20170308088A1 (en) | 2017-10-26 |
US20230106526A1 (en) | 2023-04-06 |
US11927960B2 (en) | 2024-03-12 |
JP2016082441A (en) | 2016-05-16 |
CN109373975B (en) | 2022-08-30 |
EP3206768B1 (en) | 2023-11-29 |
AU2018222910B2 (en) | 2019-08-08 |
US20240184290A1 (en) | 2024-06-06 |
EP4253911A2 (en) | 2023-10-04 |
CN106796112A (en) | 2017-05-31 |
US10198003B2 (en) | 2019-02-05 |
AU2015331965A1 (en) | 2017-02-09 |
CN109373975A (en) | 2019-02-22 |
EP4253911A3 (en) | 2023-12-13 |
CN111009156A (en) | 2020-04-14 |
JP6387782B2 (en) | 2018-09-12 |
CA2959471A1 (en) | 2016-04-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11927960B2 (en) | Control device, control method, and computer program | |
US11657534B2 (en) | Information processing device, information processing method, and computer program | |
US11237560B2 (en) | Control device, imaging device, control method, imaging method, and computer program | |
JP2024020389A (en) | Control method, control device and program | |
JP2018174002A (en) | Mobile body |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15787034 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2015787034 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2015787034 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2015331965 Country of ref document: AU Date of ref document: 20151009 Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15505541 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2959471 Country of ref document: CA |
|
NENP | Non-entry into the national phase |
Ref country code: DE |