US20210092278A1 - Image processing system, method, apparatus and device of processing image data - Google Patents
Image processing system, method, apparatus and device of processing image data Download PDFInfo
- Publication number
- US20210092278A1 US20210092278A1 US17/114,331 US202017114331A US2021092278A1 US 20210092278 A1 US20210092278 A1 US 20210092278A1 US 202017114331 A US202017114331 A US 202017114331A US 2021092278 A1 US2021092278 A1 US 2021092278A1
- Authority
- US
- United States
- Prior art keywords
- image
- data
- instruction
- control
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000005540 biological transmission Effects 0.000 claims abstract description 28
- 238000004891 communication Methods 0.000 claims description 40
- 230000008569 process Effects 0.000 claims description 22
- 230000004044 response Effects 0.000 claims description 12
- 230000003993 interaction Effects 0.000 description 100
- 230000000694 effects Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008878 coupling Effects 0.000 description 6
- 238000010168 coupling process Methods 0.000 description 6
- 238000005859 coupling reaction Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- H04N5/23206—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C39/00—Aircraft not otherwise provided for
- B64C39/02—Aircraft not otherwise provided for characterised by special use
- B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D47/00—Equipment not otherwise provided for
- B64D47/08—Arrangements of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- H04N5/23216—
-
- H04N5/232933—
-
- H04N5/23296—
-
- B64C2201/024—
-
- B64C2201/027—
-
- B64C2201/108—
-
- B64C2201/127—
-
- B64C2201/146—
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/10—Rotorcrafts
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/10—Rotorcrafts
- B64U10/13—Flying platforms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/20—Remote controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U30/00—Means for producing lift; Empennages; Arrangements thereof
- B64U30/20—Rotors; Rotor supports
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
- G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
Definitions
- the present disclosure relates to image processing, and more particularly to an aircraft-based image processing system, and a method, an apparatus and a device of processing image data.
- Aerial photography is becoming more and more popular in recent years. Users can take photographs of the ground, landscape, scenery and people using various flight platforms including helicopters, hot air balloons and multi-rotor unmanned aerial vehicles capable of being remotely controlled. Aerial photography can be used in military, transportation, disaster relief applications in addition to photographic art.
- images can be transmitted to a user end via a wireless link (e.g., Wi-Fi), and the received images can be displayed at the user end in substantially real time such that the user can observe the images.
- a wireless link e.g., Wi-Fi
- the user end can only passively receive the aerial image data and no user control on the aerial image data is available.
- the disclosure provides an aircraft-based image processing system, and a method, an apparatus and a device of processing image data. With the technical solutions of the disclosure, the user's control on aerial data can be effected.
- a first aspect of the disclosure provides an aircraft-based image processing system, said image processing system comprising an aircraft, an image capturing device onboard the aircraft and a control device, the aircraft being provided with data processing device.
- the image capturing device can be configured to capture images in a flight of the aircraft.
- the data processing device can be configured to receive an image control instruction from the control device, communicate with the image capturing device, perform an image processing operation based upon the image control instruction and provide image data as a result of the image processing operation to the control device.
- the control device can be configured to generate the image control instruction, provide the image control instruction to the data processing device and receive data from the data processing device.
- the data processing device can be further configured to provide obtained human-computer interaction interface UI data to the control device.
- the control device can be further configured to display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- the data processing device can be further configured to obtain human-computer interaction interface UI data of the image capturing device and provide the obtained human-computer interaction interface UI data of the image capturing device to the control device.
- the data processing device can be further configured to provide obtained human-computer interaction interface UI data to the control device.
- the control device can be further configured to add an operating event to the received human-computer interaction interface UI, and display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data and the added operating event.
- the image control instruction generated by the control device can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- the data processing device can be configured to communicate with the image capturing device, obtain an image identified by the image control instruction and perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- the data processing device can be configured to communicate with the image capturing device, process the received image control instruction and provide the processed image control instruction to the image capturing device.
- the image capturing device can be further configured to perform an image processing operation based upon the processed image control instruction and provide image data as a result of the image processing operation to the data processing device.
- the image capturing device can be configured to perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the processed image control instruction.
- the data processing device can be further configured to, upon receiving the image control instruction from the control device, suspend an image transmission if the data processing device is currently transmitting images to the control device.
- control device can be further configured to, upon receiving an image processing operation from a user, temporarily save a currently received image and suspend an image transmission or a processing on a transmitted image if the control device is currently transmitting images to the data processing device.
- a second aspect of the disclosure further provides a method of processing image data, said method comprising an image capturing end receiving an image control instruction from a control end; the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction; and the image capturing end providing image data as a result of the image processing operation to the control end.
- the method can further comprise the image capturing end providing obtained human-computer interaction interface UI data to the control end, such that the control end can display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- the image capturing end can capture image data using an external camera, the method further comprising the image capturing end obtaining human-computer interaction interface UI data of the external camera and providing the obtained human-computer interaction interface UI data of the external camera to the control end.
- the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- the image processing operation performed by the image capturing end can comprise an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- the method can further comprise, upon receiving the image control instruction from the control end, the image capturing end suspending an image transmission if the image capturing end is currently transmitting images to the control end.
- a third aspect of the disclosure further provides an image capturing apparatus, said image capturing apparatus comprising a receiving circuit configured to receive an image control instruction from a control end; a processing circuit configured to, in response to the image control instruction, perform an image processing operation based upon the image control instruction; and a sending circuit configured to provide image data as a result of the image processing operation to the control end.
- the apparatus can further comprise an obtaining circuit configured to obtain human-computer interaction interface UI data.
- the sending circuit can further configured to provide a stored human-computer interaction interface UI data to the control end.
- the apparatus can capture image data using an external camera, the apparatus further comprising an obtaining circuit configured to obtain human-computer interaction interface UI data of the external camera.
- the sending circuit can further configured to provide the obtained human-computer interaction interface UI data of the external camera to the control end.
- the processing circuit can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- the processing circuit can be further configured to, upon the receiving circuit receives the image control instruction from the control end, suspend an image transmission if the processing circuit is currently transmitting images to the control end.
- a fourth aspect of the disclosure further provides an image capturing device, the image capturing device being connected with a control device, the image capturing device comprising a communication circuit and a processor.
- the communication circuit can be configured to communicate with the processor and the control device.
- the processor can be configured to receive an image control instruction from the control device via the communication circuit, perform, in response to the image control instruction, an image processing operation based upon the image control instruction, and provide image data as a result of the image processing operation to the control device via the communication circuit.
- the processor can be further configured to provide obtained human-computer interaction interface UI data to the control device via the communication circuit, such that the control device can display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- the device can capture image data using an external camera.
- the processor can be further configured to obtain human-computer interaction interface UI data of the external camera and provide the obtained human-computer interaction interface UI data of the external camera to the control device via the communication circuit.
- the processor can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- the processor can be further configured to, upon the communication circuit receives the image control instruction from the control device, suspend an image transmission if the processor is currently transmitting images to the control device.
- a fifth aspect of the disclosure further provides a method of processing image data, said method comprising a control end generating an image control instruction based upon a user operation, and providing the generated image control instruction to an image capturing end which is controlled by the control end; and the control end receiving processed image data, which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data.
- the method can further comprise the control end displaying the last received image data before displaying the processed image data.
- the method before the control end generating an image control instruction based upon a user operation, can further comprise the control end receiving human-computer interaction interface UI data from the image capturing end, parsing the human-computer interaction interface UI data and displaying the parsed human-computer interaction interface UI to receive the user operation on the UI and generate the image control instruction.
- control end parsing the human-computer interaction interface UI data can comprise the control end parsing the human-computer interaction interface UI data and adding an operating event.
- the added operating event can comprise a touch event.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing end.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- the image control instruction generated by the control end can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- a sixth aspect of the disclosure further provides a control apparatus, said control apparatus comprising a processing circuit configured to generate an image control instruction based upon a user operation and provide the generated image control instruction to an image capturing end which is controlled by the processing circuit; a receiving circuit configured to receive processed image data which is processed and returned by the image capturing end; and a display configured to display an image corresponding to the processed image data.
- the display can be further configured to display the last received image data before displaying the processed image data.
- the receiving circuit can be further configured to receive human-computer interaction interface UI data from the image capturing end.
- the processing circuit can be further configured to parse the human-computer interaction interface UI data.
- the display can be further configured to display the parsed human-computer interaction interface UI to receive the user operation on the UI for generating the image control instruction.
- the processing circuit can be configured to parse the human-computer interaction interface UI data and add an operating event.
- the operating event added by the processing circuit can comprise a touch event.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing end.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- the image control instruction generated by the processing circuit can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- a seventh aspect of the disclosure further provides a control device, the control device being connected with an image capturing device, and the control device comprising a communication circuit and a processor.
- the communication circuit can be configured to be connected with the image capturing device and the processor.
- the processor can be configured to generate an image control instruction based upon a user operation, provide the generated image control instruction via the communication circuit to an image capturing device which is controlled by the processor, receive via the communication circuit processed image data which is processed and returned by the image capturing device, and display an image corresponding to the processed image data.
- the processor can be further configured to display the last received image data before displaying the processed image data.
- the communication circuit can be further configured to receive human-computer interaction interface UI data from the image capturing device.
- the processor can be further configured to parse the human-computer interaction interface UI data and display a parsed human-computer interaction interface UI to receive the user operation on the UI and generate the image control instruction.
- the processor can be configured to parse the human-computer interaction interface UI data and add an operating event.
- the operating event added by the processing circuit can comprise a touch event.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing device.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing device.
- the image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images
- the user's manipulation on aerial data can be effected.
- FIG. 1 shows a structure of an aircraft-based image processing system in accordance with embodiments of the disclosure.
- FIG. 2 shows a user interface in accordance with embodiments of the disclosure.
- FIG. 3 shows a structure of an image capturing device in accordance with embodiments of the disclosure.
- FIG. 4 shows a flowchart of a method of processing image data in accordance with embodiments of the disclosure.
- FIG. 5 shows a flowchart of an alternative method of processing image data in accordance with embodiments of the disclosure.
- FIG. 6 shows a structure of an image capturing apparatus in accordance with embodiments of the disclosure.
- FIG. 7 shows a structure of a control device in accordance with embodiments of the disclosure.
- FIG. 8 shows a flowchart of a further method of processing image data in accordance with embodiments of the disclosure.
- FIG. 9 shows a flowchart of a still further method of processing image data in accordance with embodiments of the disclosure.
- FIG. 10 shows a structure of a control apparatus in accordance with embodiments of the disclosure.
- a user can communicate with an aircraft using a remote controller to control an image capturing device (e.g., a camera) for aerial photography, such that a control of aerial images can be effected.
- an image capturing device e.g., a camera
- the user can alternatively directly communicate with an image capturing device at an aircraft end to effect a control of aerial images.
- FIG. 1 shows a structure of an aircraft-based image processing system in accordance with embodiments of the disclosure.
- the aircraft-based image processing system can comprise an aircraft 101 , an image capturing device 102 onboard the aircraft 101 and a control device 103 .
- data processing device 104 can be provided in the aircraft 101 .
- the aircraft 101 can be an unmanned aerial vehicle (UAV) capable of being remotely controlled.
- UAV unmanned aerial vehicle
- the UAV can be a helicopter or a multi-rotor aircraft.
- the aircraft 101 can comprise a propulsion system and a flight control system.
- the image capturing device 102 can be a camera capable of capturing images, recording videos at various resolutions and transmitting the captured image data to the aircraft 101 or directly transmitting the captured image data to the control device 103 .
- the image capturing device 102 can be configured to save the captured image data to a memory (e.g., an SD card), such that the aircraft 101 or the control device 103 can read the captured image data.
- a memory e.g., an SD card
- the data processing device 104 can be an independent device provided in the aircraft 101 .
- the data processing device 104 can be a processor provided in the aircraft 101 and being in data connection with other circuits (e.g., a communication circuit).
- the data processing device 104 can be configured to receive an image control instruction from the control device 103 , communicate with the image capturing device 102 to perform an image processing operation based upon the image control instruction, and transmit image data obtained from the image processing operation to the control device 103 .
- the control device 103 can be a remote controller of a ground end, such as a smartphone, a tablet computer, a portable computer, an intelligent wearable device having a communication capability.
- the control device 103 can communicate with the aircraft 101 and/or the image capturing device 102 .
- the control device 103 can be configured to generate the image control instruction, send the image control instruction to the data processing device 104 , and receive data from the data processing device 104 .
- a transmission scheme, an encoding/decoding format and a control logic can be selected between the control device 103 and the data processing device 104 of the aircraft 101 .
- a transmission scheme, an encoding/decoding format and a control logic can be selected between the control device 103 and the image capturing device 102 .
- a transmission scheme, an encoding/decoding format and a control logic can be selected between the data processing device 104 of the aircraft 101 and the image capturing device 102 .
- Data encoding/decoding, data transmission and data control can thus be effected.
- the control device 103 can be provided with a mechanical button or an application-based touchscreen button to effect a human-computer interaction, such that the image control instruction can be generated.
- the user can effect various functions including an image playback, an image movement, an image scaling, an image paging up/down and an image deleting to images captured by the image capturing device 102 .
- the data processing device 104 can be configured to provide obtained human-computer interaction interface UI data to the control device 103 .
- the control device 103 can be configured to display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- a UI can be configured by the data processing device 104 .
- Various types of remote control devices after being connected to the data processing device 104 , can effect a control based upon a graphical interface and operating events in the UI without any application installed.
- the UI can be configured to display to a user in real time an image transmitted from the image processing device 104 via an image transmission link.
- the UI data can comprise data of an image display interface and data of operating events corresponding to the functions.
- the data processing device 104 can be configured to obtain human-computer interaction interface UI data of the image capturing device 102 and provide the obtained human-computer interaction interface UI data of the image capturing device 102 to the control device 103 .
- the control device 103 can be configured to display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data of the image capturing device 102 .
- the data processing device 104 can be configured to obtain data of an interface and operating events of a carried image capturing device 102 based upon a type or a model of the image capturing device 102 , and provide the data to the control device 103 .
- the control device 103 can be configured to generate an interaction interface based upon the data to receive operating instructions from the user.
- the data processing device 104 can be configured to provide the obtained human-computer interaction interface UI data to the control device 103 .
- the control device 103 can be configured to add one or more operating events to the received human-computer interaction interface UI, display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data and the added operating events.
- the data processing device 104 can be configured to transmit the UI data such as a graphical interface, while the control device 103 can be configured to add logical events corresponding to various functions including an image playback, an image movement, an image zooming, an image paging up/down and an image deleting.
- dedicated applications APPs can be installed on the control device 103 to effect various functions including an image playback, an image movement, an image zooming, an image paging up/down and an image deleting.
- FIG. 2 shows a UI (user interface) presented on the control device 103 to the user.
- the UI can comprise a display area 201 for displaying an image and a button area 202 .
- the display area 201 can at least display an aerial image received by the control device 103 and an image processed and transmitted by the data processing device 104 (e.g., an enlarged image, a flipped image).
- the button area 202 can comprise paging up/down buttons (for example a page down button 2021 and a page up button 2025 ), zooming buttons (for example, a zooming out button 2022 and a zooming in button 2024 ) and a delete button 2023 . Buttons in the button area 202 can be provided to meet actual requirements.
- the button area 202 can comprise an image translating button or a video playback button.
- user's operation performed within the button area 202 can be directed to an image currently displayed in the display area 201 .
- the page up 2025 can be tapped to view a previous image of the currently displayed image.
- the data processing device 104 can convert obtained UI data based upon a type of the control device 103 or a type of operating system of the control device 103 , such that the control device 103 can correctly process (e.g., parse and display) the UI data.
- the user can tap an icon or a button on the UI that is provided with the operating events as discussed hereinabove to trigger a corresponding function.
- the control device 103 can be configured to, based upon the user's tapping operation or pressing operation, generate an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images, and transmit the generated instruction to the data processing device 104 onboard the aircraft 101 via data transmission link.
- the image playback instruction, image movement instruction, zooming instruction, paging up/down instruction and/or deleting instruction can also comprise an identification of a target image (e.g., an identification of an image that is displayed when the user taps a button) and other relevant content.
- the data processing device 104 can be configured to, upon receiving the image playback instruction, the image movement instruction, the zooming instruction, the paging up/down instruction and/or the deleting instruction on one or more images, perform an image processing operation comprising communicating with the image capturing device 102 , obtaining an image identified by the image control instruction, and performing a playback, a movement, a zooming, a paging up/down and/or a deleting process to the obtained image based upon the image control instruction.
- Images captured by the image capturing device 102 and to be transmitted to the control device 103 can be stored in a memory unit of the image capturing device (e.g., an SD card).
- the data processing device 104 can communicate with the image capturing device 102 , locate a target image based upon information such as an image identification contained in the image control instruction, and obtain the target image to perform an image processing based upon the image control instruction.
- the image processing operations performed by the data processing device 104 can comprise communicating with the image capturing device 102 , processing the received image control instruction and providing the processed image control instruction to the image capturing device 102 ,
- the image capturing device 102 can be configured to perform an image processing operation based upon the processed image control instruction and provide image data as a result of the image processing operation to the data processing device 104 .
- the data processing device 104 can be configured to encode/decode a received image control instruction, extract an instruction therefrom, convert the extracted instruction to an instruction that is identifiable to the image capturing device 102 and provide the converted instruction to the image capturing device 102 , such that the image capturing device 102 can perform an image processing operation based upon the image control instruction.
- the image capturing device 102 can perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting based upon the image control instruction which is processed by the data processing device 104 , and provide processed image data to the data processing device 104 .
- the data processing device 104 can only perform a signal decoding, while other image processing operations can be performed by the image capturing device 102 .
- the data processing device 104 can be configured to, upon receiving an image control instruction from the control device 103 , suspend an image transmission if the data processing device 104 is currently transmitting images to the control device 103 . For instance, the data processing device 104 can stop transmitting a new aerial image if an image control instruction is received.
- control device 103 can be configured to, upon receiving an image processing operation from the user, temporarily save a currently received image and suspend an image transmission or a processing on a transmitted image if the control device 103 is currently transmitting images to the data processing device 104 .
- control device 103 can be configured to, upon detecting user's image processing, not receive a new aerial image or not display a new aerial image even if the new aerial image is received.
- the image control instruction can be transmitted between the control device 103 and the aircraft 101 via an existing control link.
- Data can be transmitted between the data processing device 104 and the image capturing device 102 of the aircraft 101 using a wired or a wireless connection.
- Data can be transmitted between the aircraft 101 and the control device 103 via a wireless image transmission link.
- the user can effectively manipulate the aerial images to satisfy user's control on aerial images.
- FIG. 3 shows a structure of an image capturing device in accordance with embodiments of the disclosure.
- the image capturing device described with reference to FIG. 3 can substantially identical to the image capturing device disclosed hereinabove.
- the image capturing device can be connected with a control device that is configured to control the image capturing device.
- the image capturing device can be a video camera comprising a lens and an optical sensor.
- the image capturing device can comprise a communication circuit 301 and a processor 302 .
- the communication circuit 301 can be configured to communicate with the processor 302 and the control device.
- the processor 302 can be configured to receive an image control instruction from the control device, perform an image processing operation based upon the image control instruction in response to the image control instruction, and provide image data as a result of the image processing operation to the control device via the communication circuit 301 .
- the communication circuit 301 can communicate with a communication circuit of the control device, such that data can be transmitted between the processor 302 and the control device in a wired or a wireless manner.
- the processor 302 can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting based upon the image control instruction.
- control device can be configured to effect an interaction with the image capturing device using an installed APP.
- the processor 302 can be configured to send a human-computer interaction interface UI, which is provided on the image capturing device, to the control device.
- the processor 302 can be configured to send the obtained human-computer interaction interface UI data to the control device via the communication circuit 301 , such that the control device can display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- control device can be configured to capture image data using an external camera.
- the processor 302 can be configured to obtain human-computer interaction interface UI data of the external camera, and send the obtained human-computer interaction interface UI data of the external camera to the control device via the communication circuit 301 .
- the processor 302 can be configured to send built-in UI data or UI data of an external camera to the control device. In some instances, the processor 302 can convert the UI data based upon a type of the control device or a type of an operating system of the control device, such that the control device can correctly process (e.g., parse and display) the UI data.
- the processor 302 can be configured to, upon the communication circuit 301 receiving an image control instruction from the control device, suspend an image transmission if the processor 302 is currently transmitting images to the control device.
- the processor 302 can suspend an transmission of aerial images, such that a processed image can be transmitted on an original image transmission link and the control device can receive the processed image.
- processor 302 of the image capturing device can be implemented with reference to relevant description of the data processing device as described hereinabove.
- FIG. 4 shows a flowchart of a method of processing image data in accordance with embodiments of the disclosure.
- the method can be performed at an image capturing end.
- the method can be performed by a camera that captures images in an aerial photography.
- the method can be performed by an aircraft in an aerial photography.
- the method in accordance with embodiments of the disclosure can comprise steps S 401 to S 403 .
- step S 401 a process of an image capturing end receiving an image control instruction from a control end can be performed.
- step S 402 a process of the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction can be performed.
- the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- step S 403 a process of the image capturing end providing image data as a result of the image processing operation to the control end can be performed.
- FIG. 5 shows a flowchart of another method of processing image data in accordance with embodiments of the disclosure.
- the method can be performed at an image capturing end.
- the method can be performed by a camera that captures images in an aerial photography.
- the method can be performed by an aircraft in an aerial photography.
- the method in accordance with embodiments of the disclosure can comprise steps S 501 to S 504 .
- step S 501 a process of an image capturing end providing obtained human-computer interaction interface UI data to a control end can be performed, such that the control end can display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- the human-computer interaction interface UI can be a UI which is preset at the control end based upon a connected camera.
- the human-computer interaction interface UI can be a human-computer interaction interface UI of a connected camera which is obtained by the image capturing end from the camera.
- the image capturing end can capture image data using an external camera.
- the step S 501 can comprise a process in which the image capturing end obtaining human-computer interaction interface UI data of the external camera, and providing the obtained human-computer interaction interface UI data of the external camera to the control end.
- the control end can be configured to, upon receiving the human-computer interaction interface UI data, perform a necessary data processing and display the UI to the user. One or more operating events can be added if needed.
- An example of the displayed interface is shown in FIG. 2 .
- the image capturing end can obtain and provide the UI data after establishing a connection with the control end.
- the image capturing end can obtain and provide the UI data after establishing a connection with the control end based upon a manual request from the user.
- step S 502 a process of the image capturing end receiving an image control instruction from the control end can be performed.
- the user can tap an icon button on an interface of the control end as shown in FIG. 2 , such that the control end (e.g., a smartphone, a tablet computer and the like) can generate and provide a corresponding image control instruction.
- the image control instruction can be sent from the control end through a control link which has been established with the image capturing end.
- step S 503 a process of the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction can be performed.
- the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- the image processing operation performed by the image capturing end can comprise an image playback, an image movement, an image zooming, an image paging up/down and/or an image deleting on one or more images.
- step S 504 a process of the image capturing end providing image data as a result of the image processing operation to the control end can be performed.
- the method in accordance with embodiments of the disclosure can further comprise a step S 505 in which the image capturing end, upon receiving the image control instruction from the control end, suspending an image transmission if the image capturing end is currently transmitting images to the control end.
- FIG. 6 shows a structure of an image capturing apparatus in accordance with embodiments of the disclosure.
- the image capturing apparatus can be provided to an image capturing device such as a camera.
- the image capturing apparatus can be provided to an aircraft which effects an aerial photography.
- the image capturing apparatus can comprise a receiving circuit 601 , a processing circuit 602 and a sending circuit 603 .
- the receiving circuit 601 can be configured to receive an image control instruction from a control end.
- the processing circuit 602 can be configured to, in response to the image control instruction, performing an image processing operation based upon the image control instruction.
- the sending circuit 603 can be configured to provide image data as a result of the image processing operation to the control end.
- the image capturing apparatus can further comprise an obtaining circuit 604 which is configured to obtain human-computer interaction interface UI data.
- the sending circuit 603 can further be configured to provide a stored human-computer interaction interface UI data to the control end.
- the image capturing apparatus can capture image data using an external camera.
- the image capturing apparatus can further comprise the obtaining circuit 604 configured to obtain human-computer interaction interface UI data of the external camera.
- the sending circuit 603 can further be configured to provide the obtained human-computer interaction interface UI data of the external camera to the control end.
- the processing circuit 602 can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and/or an image deleting on one or more images based upon the image control instruction.
- the processing circuit 602 can further be configured to, upon the receiving circuit 601 receiving the image control instruction from the control end, suspend an image transmission if the processing circuit 602 is currently transmitting images to the control end.
- circuits of the apparatus in accordance with the embodiments can be implemented with reference to relevant functions and steps described hereinabove.
- the user can effectively manipulate the aerial images to satisfy user's control on aerial images.
- FIG. 7 shows a structure of a control device in accordance with embodiments of the disclosure.
- the control device can be a dedicated remote control, a smartphone, a tablet computer, an intelligent wearable device or other devices having a communication capability.
- the control device can comprise a communication circuit 701 and a processor 702 .
- the communication circuit 701 can be configured to be in data connection with the image capturing device and the processor 702 .
- the processor 702 can be configured to generate an image control instruction based upon a user operation, send the generated image control instruction to the image capturing device which is controlled by the processor 702 using the communication circuit 701 , receive processed image data from the image capturing device via the communication circuit 701 , and display an image corresponding to the processed image data.
- the image capturing device can be a camera or an aircraft effecting an aerial photography.
- the communication circuit 701 can communicate data with the communication circuit 701 of the image capturing device via a wireless transmission.
- the processor 702 can be configured to detect a user operation and generate an image control instruction based upon a human-computer interaction interface UI.
- the human-computer interaction interface UI can be a UI of an APP installed and configured by the processor 702 .
- the human-computer interaction interface UI can be a UI interface downloaded from an image capturing device.
- the communication circuit 701 can be configured to receive the human-computer interaction interface UI data from the image capturing end.
- the processor 702 can be configured to parse the human-computer interaction interface UI data and display the parsed human-computer interaction interface UI, such that a user operation on the UI can be received and an image control instruction can be generated.
- the UI can comprise an interface graphic and relevant operating events.
- the UI can only comprise an interface graphics, while the operating events can be added by the processor 702 .
- the processor 702 can be configured to parse the human-computer interaction interface UI data and add operating events.
- the processor 702 can be configured to display the last received image data before displaying the processed image data.
- the added operating events can comprise a touch event.
- the human-computer interaction interface UI data can be an data associated with a human-computer interaction interface UI which is configured by the image capturing device.
- the human-computer interaction interface UI data can be an data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing device.
- the image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- FIG. 8 shows a flowchart of a further method of processing image data in accordance with embodiments of the disclosure.
- the method can be performed by a control end such as a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device.
- the control end can be configured to control an operation of an image capturing end.
- the image capturing end can be a camera.
- the image capturing end can be an aircraft effecting an aerial photography.
- the method can comprise steps S 801 and S 802 .
- step S 801 a process of a control end generating an image control instruction based upon a user operation and providing the generated image control instruction to an image capturing end which is controlled by the control end can be performed.
- step S 802 a process of the control end receiving processed image data, which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data can be performed.
- the control end can be configured to detect a user operation using an installed APP or a UI obtained from the image capturing end, and generate the image control instruction.
- the control end can transmit the image control instruction via a transmission link established with the image capturing end.
- control end Upon receiving the image data, the control end can display a corresponding image using the installed APP or the obtained UI.
- FIG. 9 shows a flowchart of a still further method of processing image data in accordance with embodiments of the disclosure.
- the method can be performed by a control end such as a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device.
- the method can comprise steps S 901 to S 905 .
- step S 901 a process can be performed in which the control end receiving human-computer interaction interface UI data from the image capturing end, parsing the human-computer interaction interface UI data, and displaying the parsed human-computer interaction interface UI to receive a user operation on the UI and generate an image control instruction.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured by the image capturing end.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- the process of the control end parsing the human-computer interaction interface UI data can comprise the control end parsing the human-computer interaction interface UI data and adding operating events.
- the added operating events can comprise a touch event.
- control end can request the UI data after establishing a connection with the image capturing end.
- control end can obtain the UI data of the image capturing end based upon a user's manual request after establishing a connection with the image capturing end.
- step S 902 a process of the control end generating an image control instruction based upon a user operation can be performed.
- the image control instruction generated by the control end can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- step S 903 a process of the control end displaying the last received image data can be performed.
- step S 904 a process of providing the generated image control instruction to an image capturing end which is controlled by the control end can be performed.
- step S 905 a process of the control end receiving processed image data which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data can be performed.
- FIG. 10 shows a structure of a control apparatus in accordance with embodiments of the disclosure.
- the control apparatus can be provided in a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device.
- the control apparatus can comprise a processing circuit 1001 , a receiving circuit 1002 and a display 1003 .
- the processing circuit 1001 can be configured to generate an image control instruction based upon a user operation and provide the generated image control instruction to an image capturing end which is controlled by the processing circuit.
- the receiving circuit 1002 can be configured to receive image data which is processed and returned by the image capturing end.
- the display 1003 can be configured to display an image corresponding to the processed image data.
- the display 1003 can further be configured to display the last received image data before displaying the processed image data.
- the receiving circuit 1002 can further be configured to receive human-computer interaction interface UI data from the image capturing end.
- the processing circuit 1001 can further be configured to parse the human-computer interaction interface UI data.
- the display 1003 can further be configured to display the parsed human-computer interaction interface UI, such that a user operation on the UI can be received and the image control instruction can be generated.
- the processing circuit 1001 can be configured to parse the human-computer interaction interface UI data and add operating events.
- the operating events added by the processing circuit 1001 can comprise a touch event.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured by the image capturing end.
- the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- the image control instruction generated by the processing circuit 1001 can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- the user can effectively manipulate the aerial images to satisfy user's control on aerial data.
- a division of circuits or units is merely a division based upon a logical function.
- Various division can be possible in actual implementation.
- multiple units or components can be combined or integrated on another system.
- some features can be ignored or not be performed.
- a mutual coupling, a direct coupling or a communication connection as shown or discussed can be an indirect coupling or a communication connection via an interface, a means or an unit.
- the coupling can be an electrical coupling or a mechanical coupling.
- the units illustrated as separate parts may or may not be physically separated.
- the parts shown as units may or may not be physical units.
- the parts can be provided at the same location or distributed over a plurality of network units. All or part of the circuits can be selected to implement the embodiments of the disclosure according to actual requirements.
- Various functional units in the embodiments of the disclosure may be integrated in one processing unit.
- the functional units can be separate and physical units. Two or more units may be integrated in one unit.
- the integrated units may be implemented as hardware or software functional units.
- the integrated units if implemented as software functional units and sold or used as independent products, may be stored in a computer-readable storage medium.
- essentially the technical solution of the disclosure, or a part making contribution over the prior art, or all or part of the technical solution may be embodied as a software product.
- the computer software product is stored in a storage medium and includes several instructions for causing a computer processor to execute all or part of steps of the method according to the various embodiments of the present disclosure.
- the above mentioned storage medium includes: various media capable of storing program code, such as a U disk, a removable hard disk, ROM (read-only memory), RAM (random access memory), a diskette, an optical disk, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Aviation & Aerospace Engineering (AREA)
- Studio Devices (AREA)
- User Interface Of Digital Computer (AREA)
- Editing Of Facsimile Originals (AREA)
Abstract
A method of processing image data includes displaying, by a control device, images corresponding to captured image data in real time as the captured image data is being transmitted via an image transmission link from an image capturing device that is communicatively connected to the control device; upon detecting a user operation with respect to a first image that is being displayed: generating and sending an image control instruction to the image capturing device, the image control instruction instructing the image capturing device to obtain processed image data according to the user operation; before the processed image data is received from the image capturing device, displaying the first image and controlling to not display newly received captured image data; and upon receiving the processed image data from the image capturing device via the image transmission link, displaying a second image corresponding to the processed image data.
Description
- This is a continuation application of application Ser. No. 15/860,143, filed on Jan. 2, 2018, which is a continuation of International Application No. PCT/CN2015/083219, filed on Jul. 2, 2015, the entire contents of both of which are incorporated herein by reference.
- The present disclosure relates to image processing, and more particularly to an aircraft-based image processing system, and a method, an apparatus and a device of processing image data.
- Aerial photography is becoming more and more popular in recent years. Users can take photographs of the ground, landscape, scenery and people using various flight platforms including helicopters, hot air balloons and multi-rotor unmanned aerial vehicles capable of being remotely controlled. Aerial photography can be used in military, transportation, disaster relief applications in addition to photographic art.
- In prior art aerial photography, images can be transmitted to a user end via a wireless link (e.g., Wi-Fi), and the received images can be displayed at the user end in substantially real time such that the user can observe the images. However, the user end can only passively receive the aerial image data and no user control on the aerial image data is available.
- The disclosure provides an aircraft-based image processing system, and a method, an apparatus and a device of processing image data. With the technical solutions of the disclosure, the user's control on aerial data can be effected.
- A first aspect of the disclosure provides an aircraft-based image processing system, said image processing system comprising an aircraft, an image capturing device onboard the aircraft and a control device, the aircraft being provided with data processing device. The image capturing device can be configured to capture images in a flight of the aircraft. The data processing device can be configured to receive an image control instruction from the control device, communicate with the image capturing device, perform an image processing operation based upon the image control instruction and provide image data as a result of the image processing operation to the control device. The control device can be configured to generate the image control instruction, provide the image control instruction to the data processing device and receive data from the data processing device.
- In some embodiments, the data processing device can be further configured to provide obtained human-computer interaction interface UI data to the control device. The control device can be further configured to display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- In some embodiments, the data processing device can be further configured to obtain human-computer interaction interface UI data of the image capturing device and provide the obtained human-computer interaction interface UI data of the image capturing device to the control device.
- In some embodiments, the data processing device can be further configured to provide obtained human-computer interaction interface UI data to the control device. The control device can be further configured to add an operating event to the received human-computer interaction interface UI, and display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data and the added operating event.
- In some embodiments, the image control instruction generated by the control device can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images. The data processing device can be configured to communicate with the image capturing device, obtain an image identified by the image control instruction and perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- In some embodiments, the data processing device can be configured to communicate with the image capturing device, process the received image control instruction and provide the processed image control instruction to the image capturing device. The image capturing device can be further configured to perform an image processing operation based upon the processed image control instruction and provide image data as a result of the image processing operation to the data processing device.
- In some embodiments, the image capturing device can be configured to perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the processed image control instruction.
- In some embodiments, the data processing device can be further configured to, upon receiving the image control instruction from the control device, suspend an image transmission if the data processing device is currently transmitting images to the control device.
- In some embodiments, the control device can be further configured to, upon receiving an image processing operation from a user, temporarily save a currently received image and suspend an image transmission or a processing on a transmitted image if the control device is currently transmitting images to the data processing device.
- A second aspect of the disclosure further provides a method of processing image data, said method comprising an image capturing end receiving an image control instruction from a control end; the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction; and the image capturing end providing image data as a result of the image processing operation to the control end.
- In some embodiments, the method can further comprise the image capturing end providing obtained human-computer interaction interface UI data to the control end, such that the control end can display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- In some embodiments, the image capturing end can capture image data using an external camera, the method further comprising the image capturing end obtaining human-computer interaction interface UI data of the external camera and providing the obtained human-computer interaction interface UI data of the external camera to the control end.
- In some embodiments, the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images. The image processing operation performed by the image capturing end can comprise an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- In some embodiments, the method can further comprise, upon receiving the image control instruction from the control end, the image capturing end suspending an image transmission if the image capturing end is currently transmitting images to the control end.
- A third aspect of the disclosure further provides an image capturing apparatus, said image capturing apparatus comprising a receiving circuit configured to receive an image control instruction from a control end; a processing circuit configured to, in response to the image control instruction, perform an image processing operation based upon the image control instruction; and a sending circuit configured to provide image data as a result of the image processing operation to the control end.
- In some embodiments, the apparatus can further comprise an obtaining circuit configured to obtain human-computer interaction interface UI data. The sending circuit can further configured to provide a stored human-computer interaction interface UI data to the control end. In some embodiments, the apparatus can capture image data using an external camera, the apparatus further comprising an obtaining circuit configured to obtain human-computer interaction interface UI data of the external camera. The sending circuit can further configured to provide the obtained human-computer interaction interface UI data of the external camera to the control end.
- In some embodiments, the processing circuit can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- In some embodiments, the processing circuit can be further configured to, upon the receiving circuit receives the image control instruction from the control end, suspend an image transmission if the processing circuit is currently transmitting images to the control end.
- A fourth aspect of the disclosure further provides an image capturing device, the image capturing device being connected with a control device, the image capturing device comprising a communication circuit and a processor. The communication circuit can be configured to communicate with the processor and the control device. The processor can be configured to receive an image control instruction from the control device via the communication circuit, perform, in response to the image control instruction, an image processing operation based upon the image control instruction, and provide image data as a result of the image processing operation to the control device via the communication circuit.
- In some embodiments, the processor can be further configured to provide obtained human-computer interaction interface UI data to the control device via the communication circuit, such that the control device can display an image and receive the image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- In some embodiments, the device can capture image data using an external camera. The processor can be further configured to obtain human-computer interaction interface UI data of the external camera and provide the obtained human-computer interaction interface UI data of the external camera to the control device via the communication circuit.
- In some embodiments, the processor can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting on one or more images based upon the image control instruction.
- In some embodiments, the processor can be further configured to, upon the communication circuit receives the image control instruction from the control device, suspend an image transmission if the processor is currently transmitting images to the control device.
- A fifth aspect of the disclosure further provides a method of processing image data, said method comprising a control end generating an image control instruction based upon a user operation, and providing the generated image control instruction to an image capturing end which is controlled by the control end; and the control end receiving processed image data, which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data.
- In some embodiments, the method can further comprise the control end displaying the last received image data before displaying the processed image data.
- In some embodiments, the method, before the control end generating an image control instruction based upon a user operation, can further comprise the control end receiving human-computer interaction interface UI data from the image capturing end, parsing the human-computer interaction interface UI data and displaying the parsed human-computer interaction interface UI to receive the user operation on the UI and generate the image control instruction.
- In some embodiments, the control end parsing the human-computer interaction interface UI data can comprise the control end parsing the human-computer interaction interface UI data and adding an operating event.
- In some embodiments, the added operating event can comprise a touch event.
- In some embodiments, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing end. Alternatively, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- In some embodiments, the image control instruction generated by the control end can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- A sixth aspect of the disclosure further provides a control apparatus, said control apparatus comprising a processing circuit configured to generate an image control instruction based upon a user operation and provide the generated image control instruction to an image capturing end which is controlled by the processing circuit; a receiving circuit configured to receive processed image data which is processed and returned by the image capturing end; and a display configured to display an image corresponding to the processed image data.
- In some embodiments, the display can be further configured to display the last received image data before displaying the processed image data.
- In some embodiments, the receiving circuit can be further configured to receive human-computer interaction interface UI data from the image capturing end. The processing circuit can be further configured to parse the human-computer interaction interface UI data. The display can be further configured to display the parsed human-computer interaction interface UI to receive the user operation on the UI for generating the image control instruction.
- In some embodiments, the processing circuit can be configured to parse the human-computer interaction interface UI data and add an operating event.
- In some embodiments, the operating event added by the processing circuit can comprise a touch event.
- In some embodiments, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing end. Alternatively, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- In some embodiments, the image control instruction generated by the processing circuit can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images.
- A seventh aspect of the disclosure further provides a control device, the control device being connected with an image capturing device, and the control device comprising a communication circuit and a processor. The communication circuit can be configured to be connected with the image capturing device and the processor. The processor can be configured to generate an image control instruction based upon a user operation, provide the generated image control instruction via the communication circuit to an image capturing device which is controlled by the processor, receive via the communication circuit processed image data which is processed and returned by the image capturing device, and display an image corresponding to the processed image data.
- In some embodiments, the processor can be further configured to display the last received image data before displaying the processed image data.
- In some embodiments, the communication circuit can be further configured to receive human-computer interaction interface UI data from the image capturing device. The processor can be further configured to parse the human-computer interaction interface UI data and display a parsed human-computer interaction interface UI to receive the user operation on the UI and generate the image control instruction.
- In some embodiments, the processor can be configured to parse the human-computer interaction interface UI data and add an operating event.
- In some embodiments, the operating event added by the processing circuit can comprise a touch event.
- In some embodiments, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured in the image capturing device. Alternatively, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing device.
- In some embodiments, the image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images
- With the technical solutions of the disclosure, the user's manipulation on aerial data can be effected.
-
FIG. 1 shows a structure of an aircraft-based image processing system in accordance with embodiments of the disclosure. -
FIG. 2 shows a user interface in accordance with embodiments of the disclosure. -
FIG. 3 shows a structure of an image capturing device in accordance with embodiments of the disclosure. -
FIG. 4 shows a flowchart of a method of processing image data in accordance with embodiments of the disclosure. -
FIG. 5 shows a flowchart of an alternative method of processing image data in accordance with embodiments of the disclosure. -
FIG. 6 shows a structure of an image capturing apparatus in accordance with embodiments of the disclosure. -
FIG. 7 shows a structure of a control device in accordance with embodiments of the disclosure. -
FIG. 8 shows a flowchart of a further method of processing image data in accordance with embodiments of the disclosure. -
FIG. 9 shows a flowchart of a still further method of processing image data in accordance with embodiments of the disclosure. -
FIG. 10 shows a structure of a control apparatus in accordance with embodiments of the disclosure. - A better understanding of the disclosure will be obtained by reference to the following detailed description that sets forth illustrative embodiments with reference to the drawings. It will be apparent that, the embodiments described herein are merely provided by way of example. Those skilled in the art can conceive various embodiments in light of those embodiments disclosed herein without inventive efforts, and all these embodiments are within the scope of the disclosure.
- With embodiments of the disclosure, a user can communicate with an aircraft using a remote controller to control an image capturing device (e.g., a camera) for aerial photography, such that a control of aerial images can be effected. The user can alternatively directly communicate with an image capturing device at an aircraft end to effect a control of aerial images.
-
FIG. 1 shows a structure of an aircraft-based image processing system in accordance with embodiments of the disclosure. The aircraft-based image processing system can comprise anaircraft 101, animage capturing device 102 onboard theaircraft 101 and acontrol device 103. In some instances,data processing device 104 can be provided in theaircraft 101. - The
aircraft 101 can be an unmanned aerial vehicle (UAV) capable of being remotely controlled. The UAV can be a helicopter or a multi-rotor aircraft. Theaircraft 101 can comprise a propulsion system and a flight control system. - The
image capturing device 102 can be a camera capable of capturing images, recording videos at various resolutions and transmitting the captured image data to theaircraft 101 or directly transmitting the captured image data to thecontrol device 103. In some instances, theimage capturing device 102 can be configured to save the captured image data to a memory (e.g., an SD card), such that theaircraft 101 or thecontrol device 103 can read the captured image data. - The
data processing device 104 can be an independent device provided in theaircraft 101. Optionally, thedata processing device 104 can be a processor provided in theaircraft 101 and being in data connection with other circuits (e.g., a communication circuit). In some instances, thedata processing device 104 can be configured to receive an image control instruction from thecontrol device 103, communicate with theimage capturing device 102 to perform an image processing operation based upon the image control instruction, and transmit image data obtained from the image processing operation to thecontrol device 103. - The
control device 103 can be a remote controller of a ground end, such as a smartphone, a tablet computer, a portable computer, an intelligent wearable device having a communication capability. Thecontrol device 103 can communicate with theaircraft 101 and/or theimage capturing device 102. In some instances, thecontrol device 103 can be configured to generate the image control instruction, send the image control instruction to thedata processing device 104, and receive data from thedata processing device 104. - A transmission scheme, an encoding/decoding format and a control logic can be selected between the
control device 103 and thedata processing device 104 of theaircraft 101. A transmission scheme, an encoding/decoding format and a control logic can be selected between thecontrol device 103 and theimage capturing device 102. A transmission scheme, an encoding/decoding format and a control logic can be selected between thedata processing device 104 of theaircraft 101 and theimage capturing device 102. Data encoding/decoding, data transmission and data control can thus be effected. - The
control device 103 can be provided with a mechanical button or an application-based touchscreen button to effect a human-computer interaction, such that the image control instruction can be generated. In some instances, based upon these buttons, the user can effect various functions including an image playback, an image movement, an image scaling, an image paging up/down and an image deleting to images captured by theimage capturing device 102. - In some embodiments, the
data processing device 104 can be configured to provide obtained human-computer interaction interface UI data to thecontrol device 103. Thecontrol device 103 can be configured to display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data. - In other words, a UI can be configured by the
data processing device 104. Various types of remote control devices, after being connected to thedata processing device 104, can effect a control based upon a graphical interface and operating events in the UI without any application installed. The UI can be configured to display to a user in real time an image transmitted from theimage processing device 104 via an image transmission link. The UI data can comprise data of an image display interface and data of operating events corresponding to the functions. - Additionally or alternatively, the
data processing device 104 can be configured to obtain human-computer interaction interface UI data of theimage capturing device 102 and provide the obtained human-computer interaction interface UI data of theimage capturing device 102 to thecontrol device 103. Thecontrol device 103 can be configured to display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data of theimage capturing device 102. - The
data processing device 104 can be configured to obtain data of an interface and operating events of a carriedimage capturing device 102 based upon a type or a model of theimage capturing device 102, and provide the data to thecontrol device 103. Thecontrol device 103 can be configured to generate an interaction interface based upon the data to receive operating instructions from the user. - Additionally or alternatively, the
data processing device 104 can be configured to provide the obtained human-computer interaction interface UI data to thecontrol device 103. Thecontrol device 103 can be configured to add one or more operating events to the received human-computer interaction interface UI, display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data and the added operating events. For instance, thedata processing device 104 can be configured to transmit the UI data such as a graphical interface, while thecontrol device 103 can be configured to add logical events corresponding to various functions including an image playback, an image movement, an image zooming, an image paging up/down and an image deleting. - In some instances, dedicated applications APPs can be installed on the
control device 103 to effect various functions including an image playback, an image movement, an image zooming, an image paging up/down and an image deleting. -
FIG. 2 shows a UI (user interface) presented on thecontrol device 103 to the user. The UI can comprise adisplay area 201 for displaying an image and abutton area 202. Thedisplay area 201 can at least display an aerial image received by thecontrol device 103 and an image processed and transmitted by the data processing device 104 (e.g., an enlarged image, a flipped image). Thebutton area 202 can comprise paging up/down buttons (for example a page downbutton 2021 and a page up button 2025), zooming buttons (for example, a zooming outbutton 2022 and a zooming in button 2024) and adelete button 2023. Buttons in thebutton area 202 can be provided to meet actual requirements. For example, thebutton area 202 can comprise an image translating button or a video playback button. In some instances, user's operation performed within thebutton area 202 can be directed to an image currently displayed in thedisplay area 201. For example, the page up 2025 can be tapped to view a previous image of the currently displayed image. - It will be appreciated that, the
data processing device 104 can convert obtained UI data based upon a type of thecontrol device 103 or a type of operating system of thecontrol device 103, such that thecontrol device 103 can correctly process (e.g., parse and display) the UI data. - In some instances, the user can tap an icon or a button on the UI that is provided with the operating events as discussed hereinabove to trigger a corresponding function. The
control device 103 can be configured to, based upon the user's tapping operation or pressing operation, generate an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction and/or an image deleting instruction on one or more images, and transmit the generated instruction to thedata processing device 104 onboard theaircraft 101 via data transmission link. It will be appreciated that, in addition to an operation instruction, the image playback instruction, image movement instruction, zooming instruction, paging up/down instruction and/or deleting instruction can also comprise an identification of a target image (e.g., an identification of an image that is displayed when the user taps a button) and other relevant content. - In some embodiments, the
data processing device 104 can be configured to, upon receiving the image playback instruction, the image movement instruction, the zooming instruction, the paging up/down instruction and/or the deleting instruction on one or more images, perform an image processing operation comprising communicating with theimage capturing device 102, obtaining an image identified by the image control instruction, and performing a playback, a movement, a zooming, a paging up/down and/or a deleting process to the obtained image based upon the image control instruction. - Images captured by the
image capturing device 102 and to be transmitted to thecontrol device 103 can be stored in a memory unit of the image capturing device (e.g., an SD card). Thedata processing device 104 can communicate with theimage capturing device 102, locate a target image based upon information such as an image identification contained in the image control instruction, and obtain the target image to perform an image processing based upon the image control instruction. - Alternatively or additionally, the image processing operations performed by the
data processing device 104 can comprise communicating with theimage capturing device 102, processing the received image control instruction and providing the processed image control instruction to theimage capturing device 102, Theimage capturing device 102 can be configured to perform an image processing operation based upon the processed image control instruction and provide image data as a result of the image processing operation to thedata processing device 104. - For instance, the
data processing device 104 can be configured to encode/decode a received image control instruction, extract an instruction therefrom, convert the extracted instruction to an instruction that is identifiable to theimage capturing device 102 and provide the converted instruction to theimage capturing device 102, such that theimage capturing device 102 can perform an image processing operation based upon the image control instruction. In some instances, theimage capturing device 102 can perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting based upon the image control instruction which is processed by thedata processing device 104, and provide processed image data to thedata processing device 104. It will be appreciated that, thedata processing device 104 can only perform a signal decoding, while other image processing operations can be performed by theimage capturing device 102. - Alternatively or additionally, the
data processing device 104 can be configured to, upon receiving an image control instruction from thecontrol device 103, suspend an image transmission if thedata processing device 104 is currently transmitting images to thecontrol device 103. For instance, thedata processing device 104 can stop transmitting a new aerial image if an image control instruction is received. - Alternatively or additionally, the
control device 103 can be configured to, upon receiving an image processing operation from the user, temporarily save a currently received image and suspend an image transmission or a processing on a transmitted image if thecontrol device 103 is currently transmitting images to thedata processing device 104. For instance, thecontrol device 103 can be configured to, upon detecting user's image processing, not receive a new aerial image or not display a new aerial image even if the new aerial image is received. - It will be appreciated that, the image control instruction can be transmitted between the
control device 103 and theaircraft 101 via an existing control link. Data can be transmitted between thedata processing device 104 and theimage capturing device 102 of theaircraft 101 using a wired or a wireless connection. Data can be transmitted between theaircraft 101 and thecontrol device 103 via a wireless image transmission link. - With the embodiments of the disclosure, the user can effectively manipulate the aerial images to satisfy user's control on aerial images.
-
FIG. 3 shows a structure of an image capturing device in accordance with embodiments of the disclosure. The image capturing device described with reference toFIG. 3 can substantially identical to the image capturing device disclosed hereinabove. The image capturing device can be connected with a control device that is configured to control the image capturing device. In some instances, the image capturing device can be a video camera comprising a lens and an optical sensor. In some embodiments of the disclosure, the image capturing device can comprise acommunication circuit 301 and aprocessor 302. - The
communication circuit 301 can be configured to communicate with theprocessor 302 and the control device. - The
processor 302 can be configured to receive an image control instruction from the control device, perform an image processing operation based upon the image control instruction in response to the image control instruction, and provide image data as a result of the image processing operation to the control device via thecommunication circuit 301. - The
communication circuit 301 can communicate with a communication circuit of the control device, such that data can be transmitted between theprocessor 302 and the control device in a wired or a wireless manner. Upon performing an image processing operation based upon the image control instruction, theprocessor 302 can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and an image deleting based upon the image control instruction. - In some instances, the control device can be configured to effect an interaction with the image capturing device using an installed APP. Optionally, the
processor 302 can be configured to send a human-computer interaction interface UI, which is provided on the image capturing device, to the control device. For instance, theprocessor 302 can be configured to send the obtained human-computer interaction interface UI data to the control device via thecommunication circuit 301, such that the control device can display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data. - In some instances, the control device can be configured to capture image data using an external camera. The
processor 302 can be configured to obtain human-computer interaction interface UI data of the external camera, and send the obtained human-computer interaction interface UI data of the external camera to the control device via thecommunication circuit 301. - The
processor 302 can be configured to send built-in UI data or UI data of an external camera to the control device. In some instances, theprocessor 302 can convert the UI data based upon a type of the control device or a type of an operating system of the control device, such that the control device can correctly process (e.g., parse and display) the UI data. - Alternatively or additionally, the
processor 302 can be configured to, upon thecommunication circuit 301 receiving an image control instruction from the control device, suspend an image transmission if theprocessor 302 is currently transmitting images to the control device. - For instance, upon receiving an image control instruction, the
processor 302 can suspend an transmission of aerial images, such that a processed image can be transmitted on an original image transmission link and the control device can receive the processed image. - It will be appreciated that, the
processor 302 of the image capturing device can be implemented with reference to relevant description of the data processing device as described hereinabove. -
FIG. 4 shows a flowchart of a method of processing image data in accordance with embodiments of the disclosure. The method can be performed at an image capturing end. In some instances, the method can be performed by a camera that captures images in an aerial photography. Optionally, the method can be performed by an aircraft in an aerial photography. The method in accordance with embodiments of the disclosure can comprise steps S401 to S403. - In step S401, a process of an image capturing end receiving an image control instruction from a control end can be performed.
- In step S402, a process of the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction can be performed.
- In some instances, the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- In step S403, a process of the image capturing end providing image data as a result of the image processing operation to the control end can be performed.
- It will be appreciated that, steps of the method in accordance with the embodiments can be implemented with reference to a functional description of relevant devices described with reference to
FIG. 1 . -
FIG. 5 shows a flowchart of another method of processing image data in accordance with embodiments of the disclosure. The method can be performed at an image capturing end. In some instances, the method can be performed by a camera that captures images in an aerial photography. Optionally, the method can be performed by an aircraft in an aerial photography. The method in accordance with embodiments of the disclosure can comprise steps S501 to S504. - In step S501, a process of an image capturing end providing obtained human-computer interaction interface UI data to a control end can be performed, such that the control end can display an image and receive an image control instruction to be applied onto the image based upon the human-computer interaction interface UI data.
- In some instances, the human-computer interaction interface UI can be a UI which is preset at the control end based upon a connected camera. Optionally, the human-computer interaction interface UI can be a human-computer interaction interface UI of a connected camera which is obtained by the image capturing end from the camera.
- In some instances, the image capturing end can capture image data using an external camera. The step S501 can comprise a process in which the image capturing end obtaining human-computer interaction interface UI data of the external camera, and providing the obtained human-computer interaction interface UI data of the external camera to the control end.
- The control end can be configured to, upon receiving the human-computer interaction interface UI data, perform a necessary data processing and display the UI to the user. One or more operating events can be added if needed. An example of the displayed interface is shown in
FIG. 2 . - In some instances, the image capturing end can obtain and provide the UI data after establishing a connection with the control end. Optionally, the image capturing end can obtain and provide the UI data after establishing a connection with the control end based upon a manual request from the user.
- In step S502, a process of the image capturing end receiving an image control instruction from the control end can be performed.
- In some instances, the user can tap an icon button on an interface of the control end as shown in
FIG. 2 , such that the control end (e.g., a smartphone, a tablet computer and the like) can generate and provide a corresponding image control instruction. The image control instruction can be sent from the control end through a control link which has been established with the image capturing end. - In step S503, a process of the image capturing end, in response to the image control instruction, performing an image processing operation based upon the image control instruction can be performed.
- In some instances, the received image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- In some instances, the image processing operation performed by the image capturing end can comprise an image playback, an image movement, an image zooming, an image paging up/down and/or an image deleting on one or more images.
- In step S504, a process of the image capturing end providing image data as a result of the image processing operation to the control end can be performed.
- Optionally, the method in accordance with embodiments of the disclosure can further comprise a step S505 in which the image capturing end, upon receiving the image control instruction from the control end, suspending an image transmission if the image capturing end is currently transmitting images to the control end.
- It will be appreciated that, steps of the method in accordance with the embodiments can be implemented with reference to a functional description of relevant devices described with reference to
FIG. 1 . -
FIG. 6 shows a structure of an image capturing apparatus in accordance with embodiments of the disclosure. In some instances, the image capturing apparatus can be provided to an image capturing device such as a camera. Optionally, the image capturing apparatus can be provided to an aircraft which effects an aerial photography. In some embodiments, the image capturing apparatus can comprise a receivingcircuit 601, aprocessing circuit 602 and a sendingcircuit 603. - The receiving
circuit 601 can be configured to receive an image control instruction from a control end. Theprocessing circuit 602 can be configured to, in response to the image control instruction, performing an image processing operation based upon the image control instruction. The sendingcircuit 603 can be configured to provide image data as a result of the image processing operation to the control end. - Optionally, the image capturing apparatus can further comprise an obtaining
circuit 604 which is configured to obtain human-computer interaction interface UI data. The sendingcircuit 603 can further be configured to provide a stored human-computer interaction interface UI data to the control end. - In some instances, the image capturing apparatus can capture image data using an external camera. The image capturing apparatus can further comprise the obtaining
circuit 604 configured to obtain human-computer interaction interface UI data of the external camera. The sendingcircuit 603 can further be configured to provide the obtained human-computer interaction interface UI data of the external camera to the control end. - In some instances, the
processing circuit 602 can be configured to, in response to the image control instruction, perform an image playback, an image movement, an image zooming, an image paging up/down and/or an image deleting on one or more images based upon the image control instruction. - Optionally, the
processing circuit 602 can further be configured to, upon the receivingcircuit 601 receiving the image control instruction from the control end, suspend an image transmission if theprocessing circuit 602 is currently transmitting images to the control end. - It will be appreciated that, circuits of the apparatus in accordance with the embodiments can be implemented with reference to relevant functions and steps described hereinabove.
- With the embodiments of the disclosure, the user can effectively manipulate the aerial images to satisfy user's control on aerial images.
-
FIG. 7 shows a structure of a control device in accordance with embodiments of the disclosure. The control device can be a dedicated remote control, a smartphone, a tablet computer, an intelligent wearable device or other devices having a communication capability. In some embodiments, the control device can comprise acommunication circuit 701 and aprocessor 702. - The
communication circuit 701 can be configured to be in data connection with the image capturing device and theprocessor 702. - The
processor 702 can be configured to generate an image control instruction based upon a user operation, send the generated image control instruction to the image capturing device which is controlled by theprocessor 702 using thecommunication circuit 701, receive processed image data from the image capturing device via thecommunication circuit 701, and display an image corresponding to the processed image data. - The image capturing device can be a camera or an aircraft effecting an aerial photography. The
communication circuit 701 can communicate data with thecommunication circuit 701 of the image capturing device via a wireless transmission. - The
processor 702 can be configured to detect a user operation and generate an image control instruction based upon a human-computer interaction interface UI. In some instances, the human-computer interaction interface UI can be a UI of an APP installed and configured by theprocessor 702. Optionally, the human-computer interaction interface UI can be a UI interface downloaded from an image capturing device. - In some instances, the
communication circuit 701 can be configured to receive the human-computer interaction interface UI data from the image capturing end. Theprocessor 702 can be configured to parse the human-computer interaction interface UI data and display the parsed human-computer interaction interface UI, such that a user operation on the UI can be received and an image control instruction can be generated. - In some instances, the UI can comprise an interface graphic and relevant operating events. Optionally, the UI can only comprise an interface graphics, while the operating events can be added by the
processor 702. Theprocessor 702 can be configured to parse the human-computer interaction interface UI data and add operating events. - In some instances, the
processor 702 can be configured to display the last received image data before displaying the processed image data. - In some instances, the added operating events can comprise a touch event.
- In some instances, the human-computer interaction interface UI data can be an data associated with a human-computer interaction interface UI which is configured by the image capturing device. Optionally, the human-computer interaction interface UI data can be an data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing device.
- In some instances, the image control instruction can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- It will be appreciated that, the
communication circuit 701 and theprocessor 702 of the control device in accordance with the embodiments can be implemented with reference to those embodiments described hereinabove. -
FIG. 8 shows a flowchart of a further method of processing image data in accordance with embodiments of the disclosure. The method can be performed by a control end such as a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device. The control end can be configured to control an operation of an image capturing end. In some instances, the image capturing end can be a camera. Optionally, the image capturing end can be an aircraft effecting an aerial photography. In some embodiments, the method can comprise steps S801 and S802. - In step S801, a process of a control end generating an image control instruction based upon a user operation and providing the generated image control instruction to an image capturing end which is controlled by the control end can be performed.
- In step S802, a process of the control end receiving processed image data, which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data can be performed.
- The control end can be configured to detect a user operation using an installed APP or a UI obtained from the image capturing end, and generate the image control instruction. The control end can transmit the image control instruction via a transmission link established with the image capturing end.
- Upon receiving the image data, the control end can display a corresponding image using the installed APP or the obtained UI.
-
FIG. 9 shows a flowchart of a still further method of processing image data in accordance with embodiments of the disclosure. The method can be performed by a control end such as a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device. In some embodiments, the method can comprise steps S901 to S905. - In step S901, a process can be performed in which the control end receiving human-computer interaction interface UI data from the image capturing end, parsing the human-computer interaction interface UI data, and displaying the parsed human-computer interaction interface UI to receive a user operation on the UI and generate an image control instruction.
- In some instances, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured by the image capturing end. Optionally, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- The process of the control end parsing the human-computer interaction interface UI data can comprise the control end parsing the human-computer interaction interface UI data and adding operating events. In some instances, the added operating events can comprise a touch event.
- In some instances, the control end can request the UI data after establishing a connection with the image capturing end. Optionally, the control end can obtain the UI data of the image capturing end based upon a user's manual request after establishing a connection with the image capturing end.
- In step S902, a process of the control end generating an image control instruction based upon a user operation can be performed.
- In some instances, the image control instruction generated by the control end can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images.
- In step S903, a process of the control end displaying the last received image data can be performed.
- In step S904, a process of providing the generated image control instruction to an image capturing end which is controlled by the control end can be performed.
- In step S905, a process of the control end receiving processed image data which is processed and returned by the image capturing end, and displaying an image corresponding to the processed image data can be performed.
- It will be appreciated that, the steps of method in accordance with the embodiments can be implemented with reference to those embodiments described hereinabove.
-
FIG. 10 shows a structure of a control apparatus in accordance with embodiments of the disclosure. The control apparatus can be provided in a dedicated remote control, a smartphone, a tablet computer or an intelligent wearable device. In some embodiments, the control apparatus can comprise aprocessing circuit 1001, a receivingcircuit 1002 and adisplay 1003. - The
processing circuit 1001 can be configured to generate an image control instruction based upon a user operation and provide the generated image control instruction to an image capturing end which is controlled by the processing circuit. The receivingcircuit 1002 can be configured to receive image data which is processed and returned by the image capturing end. Thedisplay 1003 can be configured to display an image corresponding to the processed image data. - Optionally, the
display 1003 can further be configured to display the last received image data before displaying the processed image data. - Optionally, the receiving
circuit 1002 can further be configured to receive human-computer interaction interface UI data from the image capturing end. Theprocessing circuit 1001 can further be configured to parse the human-computer interaction interface UI data. Thedisplay 1003 can further be configured to display the parsed human-computer interaction interface UI, such that a user operation on the UI can be received and the image control instruction can be generated. - Optionally, the
processing circuit 1001 can be configured to parse the human-computer interaction interface UI data and add operating events. - In some instances, the operating events added by the
processing circuit 1001 can comprise a touch event. - In some instances, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI which is configured by the image capturing end. Optionally, the human-computer interaction interface UI data can be data associated with a human-computer interaction interface UI of an external camera which is connected with the image capturing end.
- In some instances, the image control instruction generated by the
processing circuit 1001 can comprise an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up/down instruction, and/or an image deleting instruction on one or more images. - It will be appreciated that, various circuit units of the control apparatus in accordance with the embodiments can be implemented with reference to those embodiments described hereinabove.
- With the embodiments of the disclosure, the user can effectively manipulate the aerial images to satisfy user's control on aerial data.
- It will be appreciated that, the relevant device and method disclosed in embodiments of the disclosure can be implemented in other manners. For example, the described device embodiments are merely illustrative. For example, a division of circuits or units is merely a division based upon a logical function. Various division can be possible in actual implementation. For example, multiple units or components can be combined or integrated on another system. For example, some features can be ignored or not be performed. For example, a mutual coupling, a direct coupling or a communication connection as shown or discussed can be an indirect coupling or a communication connection via an interface, a means or an unit. The coupling can be an electrical coupling or a mechanical coupling.
- The units illustrated as separate parts may or may not be physically separated. The parts shown as units may or may not be physical units. For example, the parts can be provided at the same location or distributed over a plurality of network units. All or part of the circuits can be selected to implement the embodiments of the disclosure according to actual requirements.
- Various functional units in the embodiments of the disclosure may be integrated in one processing unit. The functional units can be separate and physical units. Two or more units may be integrated in one unit. The integrated units may be implemented as hardware or software functional units.
- The integrated units, if implemented as software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. With such an understanding, essentially the technical solution of the disclosure, or a part making contribution over the prior art, or all or part of the technical solution may be embodied as a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer processor to execute all or part of steps of the method according to the various embodiments of the present disclosure. The above mentioned storage medium includes: various media capable of storing program code, such as a U disk, a removable hard disk, ROM (read-only memory), RAM (random access memory), a diskette, an optical disk, etc.
- The foregoing embodiments are intended to merely illustrate rather than limit the scope of the present disclosure. Numerous equivalent structures or equivalent flow variations made in light of the specification and the accompanying drawings of the present disclosure, whether directly or indirectly applied to other related technical art, are within the scope of the present disclosure.
Claims (20)
1. A method of processing image data comprising:
displaying, by a control device, images corresponding to captured image data in real time as the captured image data is being transmitted via an image transmission link from an image capturing device that is communicatively connected to the control device; and
upon detecting a user operation with respect to a first image that is being displayed:
generating and sending an image control instruction to the image capturing device, the image control instruction instructing the image capturing device to obtain processed image data according to the user operation;
before the processed image data is received from the image capturing device, displaying the first image and controlling to not display newly received captured image data; and
upon receiving the processed image data from the image capturing device via the image transmission link, displaying a second image corresponding to the processed image data.
2. The method of claim 1 , wherein the image control instruction includes at least one of an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up instruction, an image paging down instruction, or an image deleting instruction.
3. The method of claim 1 , wherein the image control instruction includes:
image identification information for instructing the image capturing device to locate image data corresponding to the image control instruction; and
information for instructing the image capturing device to process the located image data to obtain the processed image data.
4. The method of claim 1 , further comprising:
upon detecting the user operation, suspending receiving newly captured image data from the image capturing device.
5. The system of claim 4 , wherein:
resuming receiving newly captured image data transmitted from the image capturing device after the second image corresponding to the processed image data is displayed.
6. The method of claim 1 , further comprising:
receiving user interface (UI) data from the image capturing device;
parsing the UI data to obtain a parsed UI; and
displaying the parsed UI to receive the user operation on the parsed UI.
7. The method of claim 6 , wherein the UI data includes:
data associated with a UI configured in the image capturing device, or
data associated with a UI of an external camera connected with the image capturing device.
8. The method of claim 6 , wherein the parsed UI includes:
a display area for displaying at least one of the images corresponding to the captured image data or the second image corresponding to the processed image data.
9. The method of claim 8 , wherein the parsed UI further includes:
a plurality of buttons for receiving the user operation directed to the first image currently displayed in the display area, the plurality of buttons including at least one of a paging up button, a paging down button, a zooming in button, a zooming out button, a delete button, a playback button, or an image translating button.
10. A control device comprising:
a communication circuit configured to:
communicate with an image capturing device controlled by the control device, and
a processor coupled to the communication circuit and configured to:
display images corresponding to captured image data in real time as the captured image data is being transmitted via an image transmission link from the image capturing device; and
upon detecting a user operation with respect to a first image that is being displayed:
generate and send an image control instruction to the image capturing device, the image control instruction instructing the image capturing device to obtain processed image data according to the user operation;
before the processed image data is received from the image capturing device, display the first image and control to not display newly received captured image data; and
upon receiving the processed image data from the image capturing device via the image transmission link, display a second image corresponding to the processed image data.
11. The control device of claim 10 , wherein the image control instruction includes at least one of an image playback instruction, an image movement instruction, an image zooming instruction, an image paging up instruction, an image paging down instruction, or an image deleting instruction.
12. The control device of claim 10 , wherein the image control instruction includes:
image identification information for instructing the image capturing device to locate image data corresponding to the image control instruction; and
information for instructing the image capturing device to process the located image data to obtain the processed image data.
13. The control device of claim 10 , wherein the processor is further configured to upon detecting the user operation, suspend receiving newly captured image data from the image capturing device.
14. The control device of claim 13 , wherein the processor is further configured to resume receiving newly captured image data transmitted from the image capturing device after the second image corresponding to the processed image data is displayed.
15. The control device of claim 13 , wherein the processor is further configured to:
receive user interface (UI) data from the image capturing device;
parse the UI data to obtain a parsed UI; and
display the parsed UI to receive the user operation on the parsed UI.
16. The control device of claim 15 , wherein the UI data includes:
data associated with a UI configured in the image capturing device, or
data associated with a UI of an external camera connected with the image capturing device.
17. The control device of claim 15 , wherein the parsed UI includes:
a display area for displaying at least one of the images corresponding to the captured image data or the second image corresponding to the processed image data.
18. The control device of claim 17 , wherein the parsed UI further includes:
a plurality of buttons for receiving the user operation directed to the first image currently displayed in the display area, the plurality of buttons including at least one of a paging up button, a paging down button, a zooming in button, a zooming out button, a delete button, a playback button, or an image translating button.
19. An image capturing device comprising:
a communication circuit configured to communicate with a control device; and
a processor coupled to the communication circuit and configured to:
transmit captured image data to the control device;
receive an image control instruction from the control device via the communication circuit, the image control instruction being generated based on a user operation on the control device,
perform, in response to the image control instruction, an image processing operation based upon the image control instruction to obtain processed image data, and
provide the processed image data to the control device via the communication circuit.
20. The device of claim 19 , wherein the processor is further configured to, upon the communication circuit receives the image control instruction from the control device, suspend an image transmission to the control device.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/114,331 US20210092278A1 (en) | 2015-07-02 | 2020-12-07 | Image processing system, method, apparatus and device of processing image data |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2015/083219 WO2017000313A1 (en) | 2015-07-02 | 2015-07-02 | Image processing system, image data processing method and apparatus and relevant device |
US15/860,143 US10863072B2 (en) | 2015-07-02 | 2018-01-02 | Image processing system, method, apparatus and device of processing image data |
US17/114,331 US20210092278A1 (en) | 2015-07-02 | 2020-12-07 | Image processing system, method, apparatus and device of processing image data |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,143 Continuation US10863072B2 (en) | 2015-07-02 | 2018-01-02 | Image processing system, method, apparatus and device of processing image data |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210092278A1 true US20210092278A1 (en) | 2021-03-25 |
Family
ID=57040272
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,143 Active US10863072B2 (en) | 2015-07-02 | 2018-01-02 | Image processing system, method, apparatus and device of processing image data |
US17/114,331 Abandoned US20210092278A1 (en) | 2015-07-02 | 2020-12-07 | Image processing system, method, apparatus and device of processing image data |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/860,143 Active US10863072B2 (en) | 2015-07-02 | 2018-01-02 | Image processing system, method, apparatus and device of processing image data |
Country Status (3)
Country | Link |
---|---|
US (2) | US10863072B2 (en) |
CN (1) | CN105993163B (en) |
WO (1) | WO2017000313A1 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018086133A1 (en) * | 2016-11-14 | 2018-05-17 | SZ DJI Technology Co., Ltd. | Methods and systems for selective sensor fusion |
CN106791483B (en) * | 2016-12-20 | 2020-07-10 | 北京小米移动软件有限公司 | Image transmission method and device and electronic equipment |
JP6919222B2 (en) * | 2017-02-27 | 2021-08-18 | セイコーエプソン株式会社 | Display device and control method of display device |
CN110574346A (en) * | 2018-04-10 | 2019-12-13 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle and working data storage method thereof |
CN110022460A (en) * | 2019-01-07 | 2019-07-16 | 深圳市道通智能航空技术有限公司 | A kind of image pickup method of aircraft, aircraft, terminal device and flight system |
CN113874175A (en) * | 2020-03-09 | 2021-12-31 | 深圳市大疆创新科技有限公司 | Control system, method, electronic device, removable device, and computer-readable storage medium |
CN111598880A (en) * | 2020-05-19 | 2020-08-28 | 上海杏脉信息科技有限公司 | Fluorescence in situ hybridization sample pathological detection system |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11266383A (en) * | 1998-03-18 | 1999-09-28 | Minolta Co Ltd | Digital camera system |
US6809759B1 (en) * | 2000-06-19 | 2004-10-26 | Benq Corporation | Remote control unit with previewing device for an image-capturing device |
US7746391B2 (en) * | 2006-03-30 | 2010-06-29 | Jai Pulnix, Inc. | Resolution proportional digital zoom |
JP5212110B2 (en) * | 2006-11-28 | 2013-06-19 | 日本電気株式会社 | Moving image photographing apparatus with zoom function, image processing and display method and program |
US20090115848A1 (en) * | 2007-11-07 | 2009-05-07 | Aochengtongli S&T Development ( Beijing ) Co., Ltd | Multidirectional traffic image capturing method and electronic police system |
JP5393340B2 (en) * | 2009-08-20 | 2014-01-22 | オリンパス株式会社 | Imaging terminal, display terminal, display method, and imaging system |
CN101979961A (en) * | 2010-05-18 | 2011-02-23 | 中国地震局地球物理研究所 | Disaster condition acquisition system |
JP6324063B2 (en) * | 2013-12-26 | 2018-05-16 | キヤノン株式会社 | Image reproducing apparatus and control method thereof |
WO2015112824A1 (en) * | 2014-01-24 | 2015-07-30 | Voxx International Corporation | Device for wirelessly controlling a camera |
CN103809600B (en) * | 2014-03-04 | 2016-08-17 | 北京航空航天大学 | A kind of human-computer interactive control system of unmanned airship |
CN103941746B (en) * | 2014-03-29 | 2016-06-01 | 国家电网公司 | Image processing system and method is patrolled and examined without man-machine |
US9798322B2 (en) * | 2014-06-19 | 2017-10-24 | Skydio, Inc. | Virtual camera interface and other user interaction paradigms for a flying digital assistant |
JP2016048835A (en) * | 2014-08-27 | 2016-04-07 | オリンパス株式会社 | Imaging device, imaging device control method, imaging device control program, and imaging system |
JP6504768B2 (en) * | 2014-09-03 | 2019-04-24 | キヤノン株式会社 | Electronic device and control method thereof |
JP6415286B2 (en) * | 2014-12-08 | 2018-10-31 | キヤノン株式会社 | Imaging device, control method thereof, and program |
-
2015
- 2015-07-02 WO PCT/CN2015/083219 patent/WO2017000313A1/en active Application Filing
- 2015-07-02 CN CN201580002809.0A patent/CN105993163B/en active Active
-
2018
- 2018-01-02 US US15/860,143 patent/US10863072B2/en active Active
-
2020
- 2020-12-07 US US17/114,331 patent/US20210092278A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
US20180124302A1 (en) | 2018-05-03 |
US10863072B2 (en) | 2020-12-08 |
CN105993163B (en) | 2018-12-14 |
CN105993163A (en) | 2016-10-05 |
WO2017000313A1 (en) | 2017-01-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20210092278A1 (en) | Image processing system, method, apparatus and device of processing image data | |
US11566915B2 (en) | Method, device and system for processing a flight task | |
US11997382B2 (en) | Method for providing different indicator for image based on shooting mode and electronic device thereof | |
US11095847B2 (en) | Methods and systems of video processing | |
US10606367B2 (en) | Command relay device, system and method for providing remote assistance/remote control | |
US9420324B2 (en) | Content isolation and processing for inline video playback | |
CN108475221B (en) | Method and apparatus for providing a multitasking view | |
US20130132859A1 (en) | Method and electronic device for collaborative editing by plurality of mobile devices | |
US20190317490A1 (en) | Control method, device, and remote control for vr apparatus | |
JP2016099731A5 (en) | ||
CN107948565B (en) | Method and device for realizing wired screen transmission | |
CN109587536A (en) | A kind of long-distance remote-control method, equipment, server and system | |
EP3156895A1 (en) | Image processing apparatus and image processing system | |
CN103957376A (en) | Infrared thermal-image probe controlling system and method based on mobile intelligent terminal | |
CN108961424B (en) | Virtual information processing method, device and storage medium | |
JP6286836B2 (en) | Projection system, projection apparatus, projection method, and projection program | |
US9086747B2 (en) | Information processing device, display device, display system, method for controlling information processing device, and method for controlling display device | |
CN112099529B (en) | Virtual reality equipment control system and method | |
CN104936003A (en) | Screenshot method and screenshot device | |
CN112689151A (en) | Live broadcast method and device, computer equipment and storage medium | |
CN111178306A (en) | Display control method and electronic equipment | |
JP2019129466A (en) | Video display device | |
US9148472B2 (en) | Server, electronic device, server control method, and computer-readable medium | |
EP3518198A1 (en) | Video image transmission apparatus, information processing apparatus, system, information processing method, and program | |
KR101759563B1 (en) | Apparatus and method for requesting contents and apparatus and method for transferring contents |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SZ DJI TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, SHUNNIAN;ZHOU, YOU;SIGNING DATES FROM 20190910 TO 20190911;REEL/FRAME:054569/0113 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STCB | Information on status: application discontinuation |
Free format text: EXPRESSLY ABANDONED -- DURING EXAMINATION |