WO2021120455A1 - 图像处理方法、装置、设备和存储介质 - Google Patents

图像处理方法、装置、设备和存储介质 Download PDF

Info

Publication number
WO2021120455A1
WO2021120455A1 PCT/CN2020/084018 CN2020084018W WO2021120455A1 WO 2021120455 A1 WO2021120455 A1 WO 2021120455A1 CN 2020084018 W CN2020084018 W CN 2020084018W WO 2021120455 A1 WO2021120455 A1 WO 2021120455A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processed
area
acquisition device
image processing
Prior art date
Application number
PCT/CN2020/084018
Other languages
English (en)
French (fr)
Inventor
杨凯
张展鹏
靳婉婷
刘家铭
成慧
高鸣岐
Original Assignee
深圳市商汤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市商汤科技有限公司 filed Critical 深圳市商汤科技有限公司
Priority to KR1020217014432A priority Critical patent/KR20210081364A/ko
Priority to JP2021526557A priority patent/JP2022518324A/ja
Publication of WO2021120455A1 publication Critical patent/WO2021120455A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present disclosure relates to computer vision technology, and in particular to an image processing method, device, electronic equipment, and computer storage medium.
  • the embodiments of the present disclosure are expected to provide a technical solution for image processing.
  • An embodiment of the present disclosure provides an image processing method, the method including:
  • Input the image to be processed into a neural network, which is trained based on sample images in different background environments;
  • Image processing is performed on the image to be processed based on the neural network to obtain an image processing result; the image processing result includes a target detection result and/or a semantic segmentation result.
  • the image to be processed is acquired by an image acquisition device; the method further includes:
  • the method for judging whether there is an obstacle in front of the image acquisition device can determine whether there is an obstacle in front of the image acquisition device, and then take follow-up measures.
  • the first preset condition includes at least one of the following:
  • the distance between at least one target in the image to be processed and the image acquisition device is less than or equal to a minimum distance safety threshold
  • the pixel area value of at least one target in the image to be processed is greater than or equal to the maximum area safety threshold.
  • the judgment criterion of the obstacle can be made more in line with actual needs and more in line with user needs.
  • the method further includes:
  • the image processing result of the image to be processed includes the target detection result
  • the relationship between each target in the image to be processed and the image acquisition device is obtained. And/or, according to the target detection result, obtain the pixel area value of each target in the image to be processed; wherein, the homography matrix is used to represent the world coordinate system of each pixel and The position mapping relationship between pixel coordinate systems.
  • the target in the image to be processed and the distance between the target and the image acquisition device can be accurately identified on the basis of the target detection result, and each target in the image can be obtained.
  • the corresponding pixel area value in pixel coordinates can be obtained.
  • the homography matrix is determined according to the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the homography matrix can be accurately obtained by determining the homography matrix based on the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the image acquisition device is arranged on a mobile carrier, and the method further includes:
  • the obstacle avoidance response of the mobile carrier is determined.
  • the mobile carrier can perform the corresponding obstacle avoidance response when the mobile carrier encounters an obstacle.
  • the determining the obstacle avoidance response of the mobile carrier includes:
  • the obstacle avoidance response of the mobile carrier is determined.
  • the above method for determining the obstacle avoidance response of the mobile carrier takes into account the types of obstacles. Therefore, the mobile carrier can implement different obstacle avoidance strategies for different obstacles, which is smarter and can better meet the needs of practical applications.
  • the background environment includes at least one of the following: lighting conditions and texture background.
  • the image to be processed is acquired by an image acquisition device; the method further includes:
  • this embodiment can accurately determine whether the image acquisition device has reached the boundary between the workable area and the non-workable area by determining whether the second preset condition is satisfied.
  • the second preset condition includes at least one of the following:
  • the average pixel height value of the boundary is less than or equal to the boundary pixel height threshold
  • the area value of the workable area in the image to be processed is less than or equal to the workable area area threshold
  • the area ratio of the workable area in the image to be processed is less than or equal to the workable area area ratio threshold.
  • the method further includes:
  • the area type of each pixel of the image to be processed is determined according to the result of semantic segmentation, and the workable area is determined according to the determined area type of each pixel And unworkable area; according to the determined workable area and unworkable area, obtain the area value of the workable area in the to-be-processed image, and/or determine the average pixel height value of the boundary.
  • this embodiment can obtain the area division of the image to be processed, and more accurately determine the workable area and the non-workable area, and the boundary of the workable area and the non-workable area, so that the area value and the workable area can be obtained later.
  • the average pixel height value of the border can be obtained later.
  • the image acquisition device is arranged on a mobile carrier, and the method further includes:
  • the action response of the moving carrier is determined.
  • the action response of the mobile carrier can be determined in time, so as to prevent the mobile carrier from reaching the non-workable area.
  • the action response of the moving carrier includes at least one of the following: stopping, turning, and turning around.
  • the mobile carrier can perform actions such as stopping, turning, and turning around in a timely manner, which is beneficial to prevent the mobile carrier from moving to an inoperable area.
  • the image acquisition device is a monocular image acquisition device.
  • the neural network is obtained by training through the following steps:
  • the sample image is input into the neural network, and the following steps are performed based on the neural network: image processing is performed on the sample image to obtain an image processing result; the image processing result includes the target detection result and/or the semantic segmentation result; wherein, The sample images are used to represent images in different background environments;
  • a neural network that can obtain image processing results is obtained to meet the actual needs for image processing results.
  • the training process of the network is realized based on the sample images in different background environments. Therefore, the neural network completed through the training processes the images, and the image processing results obtained are not easily affected by the background environment, and the stability and reliability are high. .
  • the method further includes:
  • the neural network can perform real-time updates of the neural network according to the tasks of the mobile carrier, thereby being able to adapt to new scenarios and tasks.
  • the embodiment of the present disclosure also provides an image processing device, which includes:
  • the processing module is configured to input the image to be processed into a neural network, the neural network is trained based on sample images in different background environments; image processing is performed on the image to be processed based on the neural network to obtain an image processing result ;
  • the image processing results include target detection results and/or semantic segmentation results.
  • the image to be processed is collected by an image acquisition device; the processing module is further configured to determine whether the first preset condition is satisfied according to the image processing result of the image to be processed; In this case, it is determined that there is an obstacle in front of the image acquisition device.
  • the method for judging whether there is an obstacle in front of the image acquisition device can determine whether there is an obstacle in front of the image acquisition device, and then take follow-up measures.
  • the first preset condition includes at least one of the following:
  • the distance between at least one target in the image to be processed and the image acquisition device is less than or equal to a minimum distance safety threshold
  • the pixel area value of at least one target in the image to be processed is greater than or equal to the maximum area safety threshold.
  • the judgment criterion of the obstacle can be made more in line with actual needs and more in line with user needs.
  • the processing module is further configured to obtain the to-be-processed image according to a pre-acquired homography matrix and the target detection result when the image processing result of the image to be processed includes the target detection result.
  • the distance value between each target in the image and the image acquisition device; and/or, according to the target detection result, the pixel area value of each target in the image to be processed is obtained respectively; wherein, the homography matrix It is used to represent the position mapping relationship between the world coordinate system of each pixel and the pixel coordinate system.
  • the target in the image to be processed and the distance between the target and the image acquisition device can be accurately identified on the basis of the target detection result, and each target in the image can be obtained.
  • the corresponding pixel area value in pixel coordinates can be obtained.
  • the homography matrix is determined according to the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the homography matrix can be accurately obtained by determining the homography matrix through the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the image acquisition device is arranged on a mobile carrier, and the processing module is further configured to determine the obstacle avoidance response of the mobile carrier when there is an obstacle in front of the image acquisition device.
  • the mobile carrier can perform the corresponding obstacle avoidance response when the mobile carrier encounters an obstacle.
  • the processing module is further configured to determine the category of the obstacle according to the result of the image processing; and to determine the movement of the obstacle according to the category of the obstacle.
  • the obstacle avoidance response of the carrier is further configured to determine the category of the obstacle according to the result of the image processing; and to determine the movement of the obstacle according to the category of the obstacle.
  • the above method for determining the obstacle avoidance response of the mobile carrier takes into account the types of obstacles. Therefore, the mobile carrier can implement different obstacle avoidance strategies for different obstacles, which is smarter and can better meet the needs of practical applications.
  • the background environment includes at least one of the following: lighting conditions and texture background.
  • the image to be processed is collected by an image acquisition device; the processing module is further configured to determine whether the second preset condition is satisfied according to the image processing result of the image to be processed; Under conditions, it is determined that the image acquisition device reaches the boundary between the workable area and the non-workable area.
  • this embodiment can accurately determine whether the image acquisition device has reached the boundary between the workable area and the non-workable area by determining whether the second preset condition is satisfied.
  • the second preset condition includes at least one of the following:
  • the average pixel height value of the boundary is less than or equal to the boundary pixel height threshold
  • the area value of the workable area in the image to be processed is less than or equal to the workable area area threshold
  • the area ratio of the workable area in the image to be processed is less than or equal to the workable area area ratio threshold.
  • the processing module is further configured to determine the area category of each pixel of the image to be processed according to the semantic segmentation result when the image processing result of the image to be processed includes the semantic segmentation result, and according to the determined
  • the area category of each pixel determines the workable area and the non-operable area; according to the determined workable area and non-operable area, the area value of the workable area in the image to be processed is obtained, and/or the boundary is determined
  • this embodiment can obtain the area division of the image to be processed, and more accurately determine the workable area and the non-workable area, and the boundary of the workable area and the non-workable area, so that the area value and the workable area can be obtained later.
  • the average pixel height value of the border can be obtained later.
  • the image acquisition device is arranged on a mobile carrier, and the processing module is further configured to determine an action response of the mobile carrier when the image acquisition device reaches the boundary.
  • the action response of the mobile carrier can be determined in time, so as to prevent the mobile carrier from reaching the non-workable area.
  • the action response of the moving carrier includes at least one of the following: stopping, turning, and turning around.
  • the mobile carrier can perform actions such as stopping, turning, and turning around in a timely manner, which is beneficial to prevent the mobile carrier from moving to an inoperable area.
  • the image acquisition device is a monocular image acquisition device.
  • the neural network is obtained by training through the following steps:
  • the sample image is input into the neural network, and the following steps are performed based on the neural network: image processing is performed on the sample image to obtain an image processing result; the image processing result includes the target detection result and/or the semantic segmentation result; wherein, The sample images are used to represent images in different background environments;
  • a neural network that can obtain image processing results is obtained to meet the actual needs for image processing results.
  • the training process of the network is realized based on the sample images in different background environments. Therefore, the neural network completed through the training processes the images, and the image processing results obtained are not easily affected by the background environment, and the stability and reliability are high. .
  • the processing module is further configured to obtain an annotation of the image to be processed
  • the neural network can perform real-time updates of the neural network according to the tasks of the mobile carrier, thereby being able to adapt to new scenarios and tasks.
  • the embodiment of the present disclosure also provides an electronic device, including a processor and a memory configured to store a computer program that can run on the processor; wherein,
  • any one of the image processing methods described above is executed.
  • the embodiments of the present disclosure also provide a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned image processing methods is implemented.
  • the embodiments of the present disclosure also provide a computer program, which, when executed by a processor, implements any one of the above-mentioned image processing methods.
  • the image processing method in the embodiment of the present disclosure can input the image to be processed into a neural network, which is trained based on sample images in different background environments, and based on the neural network.
  • Image processing is performed on the image to be processed to obtain an image processing result; the image processing result includes a target detection result and/or a semantic segmentation result. Since the neural network used in the image processing method is trained based on sample images in different background environments, the image processing results of the image to be processed obtained by the image processing method are not easily affected by the background environment and are stable High sex and credibility.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure
  • Fig. 2 is a flowchart of a neural network training method according to an embodiment of the disclosure
  • FIG. 3 is a schematic diagram of the composition structure of an image processing device according to an embodiment of the disclosure.
  • FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
  • the terms "including”, “including” or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements not only includes the explicitly stated Elements, but also include other elements not explicitly listed, or elements inherent to the implementation of the method or device. Without more restrictions, the element defined by the sentence "including a" does not exclude the existence of other related elements (such as steps or steps in the method) in the method or device that includes the element.
  • the unit in the device for example, the unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.).
  • the neural network training and image processing methods provided by the embodiments of the present disclosure include a series of steps, but the neural network training and image processing methods provided by the embodiments of the present disclosure are not limited to the recorded steps.
  • the embodiments of the present disclosure The provided neural network training and image processing device includes a series of modules, but the device provided by the embodiments of the present disclosure is not limited to include the explicitly recorded modules, and may also include settings required to obtain relevant information or to perform processing based on information. Module.
  • the embodiments of the present disclosure can be applied to a computer system composed of hardware or hardware such as terminals and servers, and can be operated with many other general-purpose or special-purpose computing system environments or configurations, or can be implemented by a processor running computer executable code. Disclosure of embodiments.
  • the terminal can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a small computer system, etc.
  • the server can be a server computer System small computer system, large computer system and distributed cloud computing technology environment including any of the above systems, etc.
  • Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system.
  • program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment. In the distributed cloud computing environment, tasks are executed by remote processing equipment linked through a communication network.
  • program modules may be located on a storage medium of a local or remote computing system including a storage device.
  • an image processing method is proposed.
  • the embodiments of the present disclosure can be applied to any image processing scene, for example, it can be applied to image processing scenes such as outdoor work robots and agricultural robots.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure. As shown in FIG. 1, the process may include:
  • Step 101 Input the image to be processed into a neural network, which is obtained by training based on sample images in different background environments.
  • the image to be processed may be a picture or a video file.
  • the processing here may be to determine the file type of the image to be processed, according to the file type of the image to be processed.
  • the type of processing file determines the processing operation corresponding to the file type. For example, when the image to be processed is a video file, it is necessary to perform frame selection operations on the image to be processed first to obtain the selected image, and perform image preprocessing operations on the selected image; in the case where the image to be processed is a picture file Next, only perform image preprocessing operations on the image to be processed.
  • the image preprocessing operation may be to process the size of the image to be processed, for example, it may be to convert all the images to be processed into images of a fixed size.
  • the size of the input image to be processed may be a fixed size.
  • the size of the image to be processed The size may be non-fixed.
  • the specific size of the image to be processed is not limited, and the size of the image to be processed may be a predetermined fixed size.
  • the neural network here refers to a neural network obtained through training, and the training here is implemented based on sample images in different background environments.
  • the type of neural network is not limited.
  • the neural network can be a single-shot multi-box detector (SSD), you only look once (YOLO), RetinaNet, fast area Convolutional Neural Networks (Faster Region-Convolutional Neural Networks, Faster RCNN) or other neural networks that achieve target detection, can also be Fully Convolutional Networks (Fully Convolutional Networks), U-net, SegNet, DeconvNet, or others to achieve semantic segmentation Neural network.
  • the sample images in different background environments can be multiple images acquired in different shooting background environments, and the sample images can be multiple images of the same subject in different background environments, or different subjects in different environments.
  • the sample images in different background environments there is no restriction on the subject of the image, as long as the background environment of the sample image is different.
  • the embodiment of the present disclosure does not limit the format and source of the sample image.
  • the sample image may be a sample image obtained in advance.
  • the sample image may be obtained from a local storage area or the network, such as ,
  • the sample image can be obtained through the public data set, where the public data set can be VOC data set, COCO data set, etc.; the format of the sample image can be Joint Photographic Experts Group (JPEG) image, bitmap (Bitmap) , BMP), Portable Network Graphics (PNG) or other formats.
  • JPEG Joint Photographic Experts Group
  • Bitmap Bitmap
  • BMP Portable Network Graphics
  • the neural network is trained based on sample images in different background environments, and has the ability to obtain the image processing results of the input image. Input the image to be processed into the neural network, and the processed image can be obtained. Image processing result.
  • Step 102 Perform image processing on the image to be processed based on the neural network to obtain an image processing result; the image processing result includes a target detection result and/or a semantic segmentation result.
  • image processing is performed on the to-be-processed image to obtain the image processing result, which may be a target detection result of the sample image to obtain the target detection result of the image, and/or the semantic segmentation of the sample image to obtain the semantics of the image Segmentation result.
  • the target detection result may include a bounding box representing the location and size of the target in the image.
  • the bounding box may be a rectangular check box or a check box of other shapes.
  • the bounding box is a rectangular check box
  • the target detection result may include the pixel coordinate position of the point on the upper left corner of the rectangular detection frame and the length and width of the rectangular detection frame.
  • the target detection result may include the point and the upper left corner of the detection frame. Position information such as the pixel coordinate position of the point in the lower right corner.
  • the result of semantic segmentation can include the category of each pixel in the image, and different colors can be used to indicate different categories of pixels. For example, all pixels corresponding to roads in the image can be represented by blue. All pixels corresponding to the car can be represented by red, and all pixels corresponding to the lawn in the picture are represented by green. Furthermore, different color areas can be obtained and different objects can be distinguished.
  • steps 101 to 102 can be implemented by a processor in an electronic device, and the above-mentioned processor can be an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), Digital signal processing device (Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), FPGA, Central Processing Unit (CPU), controller, microcontroller, microprocessor At least one.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Central Processing Unit
  • CPU Central Processing Unit
  • controller microcontroller
  • microprocessor At least one.
  • the image to be processed is captured by an image capture device; the image capture device may be a device that can capture images such as a camera.
  • the image processing method described above further includes: judging according to the image processing result of the image to be processed Whether the first preset condition is met; if the first preset condition is met, it is determined that there is an obstacle in front of the image capture device; if the first preset condition is not met, it is determined that the image capture device is not in front of the There are obstacles.
  • the image to be processed may be an image acquired in real time by the image acquisition device; the image processing result of the image to be processed may refer to the target detection result of the image acquired in real time by the image acquisition device.
  • the identified obstacle can be an obstacle in contact with the ground, such as a golf ball, a road cone, or a suspended obstacle, such as a pedestrian lifted. foot.
  • the method for judging whether there is an obstacle in front of the image acquisition device can determine whether there is an obstacle in front of the image acquisition device, and then take follow-up measures.
  • the first preset condition includes at least one of the following:
  • the distance between at least one target in the image to be processed and the image acquisition device is less than or equal to the minimum distance safety threshold
  • the pixel area value of at least one target in the image to be processed is greater than or equal to the maximum area safety threshold.
  • the distance between at least one target in the image to be processed and the image acquisition device is less than or equal to the minimum distance safety threshold, which may be one or more bounding boxes in all bounding boxes in the image to be processed.
  • the distance to the image acquisition device is less than or equal to the minimum distance safety threshold distance.
  • the distance between the closest bounding box to the image acquisition device and the acquisition device is less than or equal to the distance safety threshold.
  • the distance between the bounding box and the image acquisition device may be the distance between the position in the world coordinate system corresponding to the position point such as the center of the bounding box or the boundary point and the image acquisition device.
  • the pixel area value of at least one target in the image to be processed is greater than or equal to the maximum area safety threshold, which may be that the pixel area of one or more bounding boxes in all bounding boxes in the image to be processed is greater than or equal to the maximum
  • the safety threshold for example, may be that the pixel area of the bounding box with the largest pixel area is greater than or equal to the maximum safety threshold.
  • the minimum distance safety threshold and the maximum area safety threshold are specifically determined according to task requirements and user needs.
  • the specific sizes of the minimum distance safety threshold and the maximum area safety threshold are not limited here.
  • the judgment criterion of the obstacle can be made more in line with actual needs and more in line with user needs.
  • the above-mentioned image processing method further includes: in a case where the image processing result of the image to be processed includes the target detection result, obtaining the image to be processed according to the pre-acquired homography matrix and the target detection result.
  • the homography matrix here can be used to represent the relationship between the coordinates of the pixels on the image to be processed and the coordinates of the pixels on the image to be processed mapped on the world coordinate system, that is, through the homography matrix Obtain the coordinates of the pixel with known coordinates in the image to be processed in the world coordinate system, and then obtain the distance between the target corresponding to the pixel with known coordinates and the image acquisition device.
  • the homography matrix is determined according to the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the homography matrix can be determined in the following manner: First, the image acquisition device collects images when the calibration board is placed in different poses, and the internal parameters of the image acquisition device can be calculated, and then, according to the acquired image acquisition The internal parameters of the device can obtain a homography matrix based on a calibration board (ie an image) placed in a fixed position.
  • the internal parameters of the image acquisition device here include at least the focal length and pixel size of the image acquisition device.
  • the homography matrix can be accurately obtained by determining the homography matrix through the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • Using the homography matrix as a solution to obtain the depth matrix of the world coordinate system corresponding to the image pixel position is easy to implement, has a small amount of calculation, and can help the target detection algorithm to quickly obtain the distance of the detected object, which is conducive to obstacle avoidance judgment.
  • the target detection result may also include bounding box confidence.
  • the bounding box confidence is used to indicate the credibility of the bounding box. The higher the confidence, the higher the credibility of the bounding box.
  • the target detection result includes a bounding box with a confidence higher than the first threshold, that is, the target identified by target detection is a target corresponding to the bounding box with a confidence higher than the first threshold.
  • the first threshold is not limited
  • the first threshold may be 50%, 60%, or 80%.
  • the distance value between each target in the image to be processed and the image acquisition device is obtained according to the pre-acquired homography matrix and the target detection result
  • the position information of the target in the world coordinate system can be determined according to the above-mentioned bounding box and the position mapping relationship between the world coordinate system and the pixel coordinate system, that is, the position information of the target and the image acquisition device can be determined
  • the distance value between is not limited here. Specifically, the distance value between the target and the image acquisition device can be detected by lidar or ultrasound.
  • the area of the pixel occupied by all the bounding boxes on the image captured by the image capture device in real time can be obtained separately value.
  • the method of obtaining the area value of the pixel occupied by the bounding box on the image is not limited here. Specifically, the area value of the pixel occupied by the bounding box on the image can be determined by the GPS positioning system combined with the homography matrix.
  • the image acquisition device is arranged on a mobile carrier, and the mobile carrier may be a smart mobile device such as a mobile robot or a smart lawn mower.
  • the method further includes: there is an obstacle in front of the image acquisition device In the case of determining the obstacle avoidance response of the mobile carrier.
  • the obstacle avoidance reaction of the mobile carrier may refer to the reaction when the mobile carrier is moving when there is an obstacle in front and cannot proceed in the original direction.
  • the obstacle avoidance reaction may be stopping forward and waiting for the obstacle.
  • the removal can also be a turn or a U-turn, etc.
  • the obstacle avoidance response is not specifically limited here.
  • the mobile carrier can perform the corresponding obstacle avoidance response when the mobile carrier encounters an obstacle.
  • the determining the obstacle avoidance response of the mobile carrier includes: determining the category of the obstacle according to the image processing result; determining the category of the mobile carrier according to the category of the obstacle Obstacle avoidance response.
  • the training content of the target category may be added in the training stage of the neural network, so that the neural network obtained by training can obtain the input image
  • the ability of the target category, that is, the image processing result also includes the category of each target in the input image, and furthermore, the category of the obstacle can be determined.
  • the types of obstacles may be movable objects such as golf balls, immovable objects such as road cones and sprinklers, and movable characters such as pedestrians.
  • the obstacle can be a golf ball. If the mobile carrier is equipped with a robotic arm, the obstacle avoidance response can recover the golf ball; the obstacle is also It can be a road cone or other static objects, and the obstacle avoidance response can also be to keep a safe distance from the static obstacle to bypass; the obstacle can also be a pedestrian, if the pedestrian is moving, the obstacle avoidance response can also be a moving carrier waiting for the person to leave Moving forward, when the moving carrier needs to bypass a stationary pedestrian, the obstacle avoidance response may be that the moving carrier needs to reduce the speed and maintain a larger safety distance to ensure the safety of the pedestrian.
  • the above method for determining the obstacle avoidance response of the mobile carrier takes into account the types of obstacles. Therefore, the mobile carrier can implement different obstacle avoidance strategies for different obstacles, which is smarter and can better meet the needs of practical applications.
  • the moving carrier may refer to the determination that the moving carrier continues to move in the original direction when the image processing result does not meet the first preset condition, or it may be correct
  • the mobile carrier that is on the move does not take any intervention measures.
  • the mobile carrier can complete the work or task on time when there are no obstacles in front of the image acquisition device.
  • the background environment may include at least one of the following: lighting conditions and texture background.
  • the lighting condition may refer to the intensity of lighting or other lighting information
  • the texture background may be a linear pattern, a non-linear pattern or other texture background used as a background.
  • the image to be processed is captured by an image capture device; the image capture device may be a device that can capture images, such as a camera, a camera, or the like.
  • the above image processing method further includes: according to the image of the image to be processed According to the processing result, it is determined whether the second preset condition is satisfied; if the second preset condition is satisfied, it is determined that the image acquisition device has reached the boundary between the workable area and the non-workable area; in the case that the second preset condition is not met Next, it is determined that the image acquisition device has not reached the boundary between the workable area and the non-workable area.
  • the image to be processed may be an image acquired in real time by an image acquisition device; the image processing result of the image to be processed may refer to a semantic segmentation result of an image acquired in real time by the image acquisition device.
  • this embodiment can accurately determine whether the image acquisition device has reached the boundary between the workable area and the non-workable area by determining whether the second preset condition is satisfied.
  • the second preset condition includes at least one of the following: the average pixel height value of the boundary is less than or equal to the boundary pixel height threshold; the area value of the workable area in the image to be processed is less than or equal to the workable area area threshold ; The area ratio of the workable area in the image to be processed is less than or equal to the workable area area ratio threshold.
  • the average pixel height value of the boundary may refer to the average value of the distance between the boundary formed by the workable area and the non-workable area and the lower edge of the image. It can be understood that the smaller the average value, the workable area The closer the boundary formed by the unworkable area to the lower edge of the image, that is, the closer the image capture device is to the boundary, at this time, it can be determined that the image capture device has reached the boundary between the workable area and the unworkable area, and if the image capture device moves along the original direction If you move a little before, you may leave the workable area and arrive at the non-workable area.
  • the area value of the workable area may refer to the area value of the area occupied by the workable area of the image in the pixel coordinate system.
  • the area value of the workable area in the image to be processed is less than or equal to the workable area area threshold, it can be considered that the range of the workable area of the image acquisition device is not large enough.
  • the area ratio of the workable area in the image to be processed can refer to the ratio of the area of the workable area to the area of the entire image in the image to be processed, or the ratio of the area of the workable area to the area of the non-workable area. It can also refer to the ratio of the area of the workable area in the image to be processed to the area of the preset total workable area, which is not specifically limited here.
  • the area ratio of the workable area in the image to be processed is less than or equal to the workable area area ratio threshold, it indicates that the area of the workable area is relatively small.
  • the boundary pixel height threshold, workable area threshold, and workable area area ratio are specifically determined according to task requirements and user needs.
  • the boundary pixel height threshold, workable area threshold, and workable area area are not determined here.
  • the specific size of the proportion threshold is limited.
  • the above-mentioned image processing method further includes: when the image processing result of the image to be processed includes a semantic segmentation result, determining the area category of each pixel of the image to be processed according to the semantic segmentation result, and The determined area category of each pixel determines the workable area and the non-operable area; according to the determined workable area and non-operable area, the area value of the workable area in the image to be processed is obtained, and/or determined The average pixel height value of the boundary.
  • the area category of each pixel may refer to whether the specific area to which each pixel belongs is a workable area or a non-workable area.
  • the workable area may refer to a space area such as grass that can be cut
  • the non-workable area may refer to Areas that cannot be mowed on concrete floors, roads, etc.
  • this embodiment can obtain the area division of the image to be processed, and more accurately determine the workable area and the non-workable area, and the boundary of the workable area and the non-workable area, so that the area value and the workable area can be obtained later.
  • the average pixel height value of the border can be obtained later.
  • the image acquisition device is arranged on a mobile carrier, and the above-mentioned image processing method further includes: determining an action response of the mobile carrier when the image acquisition device reaches the boundary.
  • the action response of the mobile carrier can be determined in time, so as to prevent the mobile carrier from reaching the non-workable area.
  • the method further includes: determining that the moving carrier continues to move in the original direction when the image acquisition device does not reach the boundary.
  • the action response of the moving carrier includes at least one of the following: stopping, turning, and turning around.
  • the mobile carrier can perform actions such as stopping, turning, and turning around in a timely manner, which is beneficial to prevent the mobile carrier from moving to an inoperable area.
  • the image acquisition device is a monocular image acquisition device.
  • a monocular image acquisition device refers to an image acquisition device with a single camera, and, for example, it may be a monocular camera.
  • FIG. 2 is a flowchart of a neural network training method according to an embodiment of the present disclosure. As shown in FIG. 2, the above-mentioned neural network is trained through the following steps:
  • Step 201 Input a sample image into a neural network, and perform image processing on the sample image based on the neural network to obtain an image processing result;
  • the image processing result includes a target detection result and/or a semantic segmentation result;
  • the sample images are used to represent images in different background environments;
  • Step 202 Adjust network parameter values of the neural network according to the image processing result of the sample image and the label of the sample image;
  • Step 203 Determine whether the image processing result obtained by the neural network adjusted based on the network parameter value meets the set condition, if not, re-execute step 201 to step 203; if yes, execute step 204.
  • Step 204 Use the neural network with the adjusted network parameter values as the neural network that has been trained.
  • the neural network here may be an untrained neural network, or a neural network that has undergone neural network training, but the training does not include the training content of the present disclosure.
  • the label of the sample image can be a label box and label information, where the label box is used to frame the target in the sample image, and the label box can also Identify the location of the target. For example, you can mark the target's location such as people, animals, etc. in the sample image through the label box.
  • the labeling information is used to label the target category. For example, you can mark whether the target is an object, a person or an animal; Labeling can also be labeling information used to label the category of pixels in the image. Since multiple pixels can be of the same category, it can be labeling information for multiple area categories. For example, it can be used to mark the mowing area and Labeling information for areas where mowing is not allowed.
  • the adjustment of the network parameter value of the neural network according to the image processing result of the sample image and the annotation of the sample image may be based on the difference between the image processing result of the sample image and the annotation of the sample image .
  • the specific method for determining the loss function value may be determined according to the type of neural network, which is not limited in the embodiment of the present disclosure.
  • the setting condition can be that the number of times to adjust the network parameters of the neural network is equal to the set number of iterations, or the loss function of the neural network can reach the convergence condition.
  • the setting condition can also be tested on a fixed test set to reach the setting. Set accuracy rate.
  • the set number of iterations means the maximum number of times the network parameters of the neural network are adjusted, and the number of iterations is set to an integer greater than 1;
  • the convergence condition can be that the value of the loss function of the adjusted neural network is less than the set loss, and the set loss can be Pre-set according to actual application requirements.
  • the setting accuracy rate may be a preset percentage value, specifically, the setting The percentage value of can be 50% and a value greater than 50%.
  • a neural network that can obtain image processing results is obtained to meet the actual needs for image processing results.
  • the training process of the network is realized based on the sample images in different background environments. Therefore, the neural network completed through the training processes the images, and the image processing results obtained are not easily affected by the background environment, and the stability and reliability are high. .
  • the image processing method further includes: acquiring the annotation of the image to be processed; according to the image processing result of the image to be processed and the annotation of the image to be processed, On the basis of incremental training.
  • incremental training refers to a process of adjusting parameters of the neural network by using newly added data on the basis of the aforementioned neural network.
  • the modified loss function can be obtained by adding the loss function of the neural network to the preset regularization term; using the neural network to add data Perform image processing to obtain the image processing result of the sample image; determine the loss of the neural network according to the modified loss function and the label of the newly added data; adjust the network parameters of the neural network according to the loss of the neural network; repeat the above determination of the neural network And the steps of adjusting the network parameters of the neural network until the neural network after the adjustment of the network parameters meets the training end condition, and the trained neural network is obtained.
  • the neural network can perform real-time updates of the neural network according to the tasks of the mobile carrier, thereby being able to adapt to new scenarios and tasks.
  • an embodiment of the present disclosure proposes an image processing device.
  • FIG. 3 is a schematic diagram of the composition structure of an image processing apparatus according to an embodiment of the disclosure.
  • the apparatus may include: a processing module 301, wherein:
  • the processing module 301 is configured to input the image to be processed into a neural network, the neural network is trained based on sample images in different background environments; image processing is performed on the image to be processed based on the neural network to obtain image processing Results; the image processing results include target detection results and/or semantic segmentation results.
  • the processing module 301 is further configured to determine whether a first preset condition is satisfied according to the image processing result of the image to be processed; if the first preset condition is satisfied, determine the image acquisition device There is an obstacle ahead.
  • the first preset condition includes at least one of the following:
  • the distance between at least one target in the image to be processed and the image acquisition device is less than or equal to a minimum distance safety threshold
  • the pixel area value corresponding to at least one target in the image to be processed in the pixel coordinates is greater than or equal to the maximum area safety threshold.
  • the processing module 301 is further configured to obtain the to-be-processed image according to a pre-acquired homography matrix and the target detection result when the image processing result of the to-be-processed image includes the target detection result.
  • the homography matrix is used to represent the positional mapping relationship between the world coordinate system and the pixel coordinate system.
  • the homography matrix is determined according to the internal parameters of the image acquisition device and the known position of the calibration board relative to the image acquisition device.
  • the image acquisition device is arranged on a mobile carrier, and the processing module 301 is further configured to determine the obstacle avoidance response of the mobile carrier when there is an obstacle in front of the image acquisition device.
  • the processing module 301 is further configured to determine the category of the obstacle according to the image processing result; and determine the obstacle avoidance response of the mobile carrier according to the category of the obstacle.
  • the processing module 301 is further configured to determine that the moving carrier continues to move in the original direction when there is no obstacle in front of the image acquisition device.
  • the background environment includes at least one of the following: lighting conditions and texture background.
  • the image to be processed is collected by an image acquisition device, and the processing module 301 is further configured to determine whether the second preset condition is satisfied according to the image processing result of the image to be processed; Under the condition of setting conditions, it is determined that the image acquisition device reaches the boundary between the workable area and the non-workable area.
  • the second preset condition includes at least one of the following:
  • the average pixel height value of the boundary is less than or equal to the boundary pixel height threshold
  • the area value of the workable area in the image to be processed is less than or equal to the workable area area threshold
  • the area ratio of the workable area in the image to be processed is less than or equal to the workable area area ratio threshold.
  • the processing module 301 is further configured to determine the area category of each pixel of the image to be processed according to the semantic segmentation result when the image processing result of the image to be processed includes the semantic segmentation result, and according to the result of the semantic segmentation.
  • the determined area category of each pixel determines the workable area and the non-operable area; according to the determined workable area and non-operable area, the area value of the workable area in the image to be processed is obtained, and/or the The average pixel height value of the border.
  • the image acquisition device is set on a mobile carrier
  • the processing module 301 is further configured to determine the action response of the mobile carrier when the image acquisition device reaches the boundary.
  • the processing module 301 is further configured to determine that the moving carrier continues to move in the original direction when the image acquisition device does not reach the boundary.
  • the action response of the moving carrier includes at least one of the following: stopping, turning, and turning around.
  • the image acquisition device is a monocular image acquisition device.
  • the neural network is obtained by training through the following steps: inputting a sample image into the neural network, and performing image processing on the sample image based on the neural network to obtain an image processing result;
  • the image processing result includes Target detection results and/or semantic segmentation results; wherein the sample images are used to represent images in different background environments;
  • the processing module 301 is further configured to obtain the annotation of the image to be processed; according to the image processing result of the image to be processed and the annotation of the image to be processed, the training is completed on the basis of the neural network Incremental training.
  • the processing module 301 may be implemented by a processor in an electronic device, and the above-mentioned processor may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.
  • the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or It is said that the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • the computer program instructions corresponding to a neural network training method or image processing method in this embodiment can be stored on storage media such as optical disks, hard disks, USB flash drives, etc.
  • storage media such as optical disks, hard disks, USB flash drives, etc.
  • the embodiment of the present disclosure also provides a computer program, which implements any one of the above-mentioned image processing methods when the computer program is executed by a processor.
  • FIG. 4 shows an electronic device 400 provided by an embodiment of the present disclosure, which may include: a memory 401 and a processor 402; wherein,
  • the memory 401 is configured to store computer programs and data
  • the processor 402 is configured to execute a computer program stored in the memory to implement any one of the image processing methods in the foregoing embodiments.
  • the aforementioned memory 401 may be a volatile memory (volatile memory), such as RAM; or a non-volatile memory (non-volatile memory), such as ROM, flash memory, or hard disk (Hard Disk). Drive, HDD) or Solid-State Drive (SSD); or a combination of the foregoing types of memories, and provide instructions and data to the processor 402.
  • volatile memory volatile memory
  • non-volatile memory non-volatile memory
  • ROM read-only memory
  • flash memory read-only memory
  • HDD hard disk
  • SSD Solid-State Drive
  • the aforementioned processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that for different augmented reality cloud platforms, the electronic devices used to implement the above-mentioned processor functions may also be other, and the embodiment of the present disclosure does not specifically limit it.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • brevity, here No longer refer to the description of the above method embodiments.
  • the technical solution of the present disclosure essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to make a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present disclosure.
  • a terminal which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、装置、电子设备和计算机存储介质，图像处理方法包括：将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的（101）；基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果（102）。

Description

图像处理方法、装置、设备和存储介质
相关申请的交叉引用
本申请要求在2019年12月20日提交中国专利局、申请号为201911328268.7、申请名称为“图像处理方法、装置、设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机视觉技术，尤其涉及一种图像处理方法、装置、电子设备和计算机存储介质。
背景技术
随着机器人技术的发展，越来越多的机器人开始应用于作业范围不固定、障碍种类多且速度块的户外场景，因此，开发一种可以在户外作业范围内作业的机器人自主避障***显得尤为重要。
发明内容
本公开实施例期望提供图像处理的技术方案。
本公开实施例提供了一种图像处理方法，所述方法包括：
将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的；
基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
可选地，所述待处理图像是由图像采集设备采集的；所述方法还包括：
根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；
在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物。
可以看出，该判断图像采集设备前方是否存在障碍物的方法，能够确定图像采集设备前方是否存在障碍物，进而采取后续措施。
可选地，所述第一预设条件包括以下至少一项：
所述待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
所述待处理图像中的至少一个目标的像素面积值大于或等于最大面积安全阈值。
可以看出，通过对上述第一预设条件的限定，可以使得障碍物的判断标准更加符合实际需求，更符合用户需求。
可选地，所述方法还包括：
在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到所述待处理图像中的各目标与图像采集设备之间的距离值；和/或，根据所述目标检测结果，分别获得所述待处理图像中的各目标的像素面积值；其中，所述单应性矩阵用于表示各像素点的世界坐标系和像素坐标系之间的位置映射关系。
可以看出，采用本公开实施例的技术方案，可以在目标检测结果的基础上，准确识别待处理图像中的目标以及目标与图像采集设备之间的距离，并获得所述图像中的各目标在像素坐标上对应的像素面积值。
可选地，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置确定的。
可以看出，通过图像采集设备的内部参数以及标定板相对于所述图像采集设备的已 知位置来确定单应性矩阵，可以准确获得单应性矩阵。
可选地，所述图像采集设备设置于移动载体上，所述方法还包括：
在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
可以看出，采用本公开实施例的技术方案，可以在移动载体遇到障碍物的情况下，使得移动载体执行对应的避障反应。
可选地，所述确定所述移动载体的避障反应，包括：
根据所述图像处理结果，确定所述障碍物的类别；
根据所述障碍物的类别，确定所述移动载体的避障反应。
可以看出，上述确定移动载体避障反应的方法，由于考虑了障碍物的类别，因此，移动载体可以对不同的障碍物执行不同的避障策略，更智能，更能满足实际应用需求。
可选地，所述背景环境包括以下至少一项：光照条件、纹理背景。
可以看出，采用本公开实施例的技术方案，可以对不同光照条件和/或不同纹理背景的多个样本图像进行训练，获得训练完成的神经网络，由于训练过程是基于不同光照条件和/或纹理背景下的样本图像实现的，因此，该训练完成的神经网络更适用于光照条件变化较大的户外场景。
可选地，所述待处理图像是由图像采集设备采集的；所述方法还包括：
根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；
在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界。
可以看出，本实施例可以通过判断是否满足第二预设条件，来准确判断图像采集设备是否抵达可工作区域与不可工作区域的边界。
可选地，所述第二预设条件包括以下至少一项：
所述边界的平均像素高度值小于或等于边界像素高度阈值；
所述待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；
所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
可以看出，通过对上述第二预设条件的判断，可以使得是否抵达可工作区域与不可工作区域的边界的判断标准更能符合实际应用需求。
可选地，所述方法还包括：
在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
可以看出，本实施例可以得到待处理图像中的区域划分情况，较为准确地确定可工作区域和不可工作区域以及可工作区域和不可工作区域的边界，便于后面获得可工作区域的面积值和边界的平均像素高度值。
可选地，所述图像采集设备设置于移动载体上，所述方法还包括：
在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
可以看出，本实施例在移动载体抵达可工作区域和不可工作区域边界的情况下，可以及时确定移动载体的动作反应，避免移动载体抵达不可工作区域。
可选地，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
可以看出，采用本公开实施例的技术方案，移动载体可以及时地执行停止、转弯、掉头等动作反应，有利于避免移动载体移动至不可工作区域。
可选地，所述图像采集设备是单目图像采集设备。
可以看出，由于该单目图像采集设备成本低、重量轻，因此，可以应用于多种应用 场景，拓展了本实施例的应用范围。
可选地，所述神经网络是通过以下步骤训练得到的：
将样本图像输入至神经网络中，基于所述神经网络执行以下步骤：对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
重复执行上述步骤，直至网络参数值调整后的神经网络满足设定条件，得到训练完成的神经网络。
可以看出，在本公开实施例中，基于对不同背景环境下的样本图像进行图像处理的训练，得到可以获得图像处理结果的神经网络，以满足对图像的图像处理结果的实际需求，由于神经网络的训练过程是基于不同背景环境下的样本图像实现的，因此，通过该训练完成的神经网络对图像进行处理，获得的图像处理结果不易受背景环境的影响，稳定性和可信度较高。
可选地，所述方法还包括：
获取所述待处理图像的标注；
根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在所述训练完成的神经网络的基础上进行增量训练。
可以看出，通过该增量训练，神经网络可以根据移动载体的任务进行神经网络的实时更新，从而，能够适应新的场景和作业任务。
本公开实施例还提供了一种图像处理装置，所述装置包括：
处理模块，配置为将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的；基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
可选地，待处理图像是由图像采集设备采集的；所述处理模块还配置为根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物。
可以看出，该判断图像采集设备前方是否存在障碍物的方法，能够确定图像采集设备前方是否存在障碍物，进而采取后续措施。
可选地，所述第一预设条件包括以下至少一项：
所述待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
所述待处理图像中的至少一个目标的像素面积值大于或等于最大面积安全阈值。
可以看出，通过对上述第一预设条件的限定，可以使得障碍物的判断标准更加符合实际需求，更符合用户需求。
可选地，所述处理模块还配置为在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到所述待处理图像中的各目标与图像采集设备之间的距离值；和/或，根据所述目标检测结果，分别获得所述待处理图像中的各目标的像素面积值；其中，所述单应性矩阵用于表示各像素点的世界坐标系和像素坐标系之间的位置映射关系。
可以看出，采用本公开实施例的技术方案，可以在目标检测结果的基础上，准确识别待处理图像中的目标以及目标与图像采集设备之间的距离，并获得所述图像中的各目标在像素坐标上对应的像素面积值。
可选地，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所 述图像采集设备的已知位置确定的。
可以看出，通过图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置来确定单应性矩阵，可以准确获得单应性矩阵。
可选地，所述图像采集设备设置于移动载体上，所述处理模块还配置为在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
可以看出，采用本公开实施例的技术方案，可以在移动载体遇到障碍物的情况下，使得移动载体执行对应的避障反应。
可选地，所述确定所述移动载体的避障反应，所述处理模块还配置为根据所述图像处理结果，确定所述障碍物的类别；根据所述障碍物的类别，确定所述移动载体的避障反应。
可以看出，上述确定移动载体避障反应的方法，由于考虑了障碍物的类别，因此，移动载体可以对不同的障碍物执行不同的避障策略，更智能，更能满足实际应用需求。
可选地，所述背景环境包括以下至少一项：光照条件、纹理背景。
可以看出，采用本公开实施例的技术方案，可以对不同光照条件和/或不同纹理背景的多个样本图像进行训练，获得训练完成的神经网络，由于训练过程是基于不同光照条件和/或纹理背景下的样本图像实现的，因此，该训练完成的神经网络更适用于光照条件变化较大的户外场景。
可选地，所述待处理图像是由图像采集设备采集的；所述处理模块还配置为根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界。
可以看出，本实施例可以通过判断是否满足第二预设条件，来准确判断图像采集设备是否抵达可工作区域与不可工作区域的边界。
可选地，所述第二预设条件包括以下至少一项：
所述边界的平均像素高度值小于或等于边界像素高度阈值；
所述待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；
所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
可以看出，通过对上述第二预设条件的判断，可以使得是否抵达可工作区域与不可工作区域的边界的判断标准更能符合实际应用需求。
可选地，所述处理模块还配置为在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
可以看出，本实施例可以得到待处理图像中的区域划分情况，较为准确地确定可工作区域和不可工作区域以及可工作区域和不可工作区域的边界，便于后面获得可工作区域的面积值和边界的平均像素高度值。
可选地，所述图像采集设备设置于移动载体上，所述处理模块还配置为在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
可以看出，本实施例在移动载体抵达可工作区域和不可工作区域边界的情况下，可以及时确定移动载体的动作反应，避免移动载体抵达不可工作区域。
可选地，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
可以看出，采用本公开实施例的技术方案，移动载体可以及时地执行停止、转弯、掉头等动作反应，有利于避免移动载体移动至不可工作区域。
可选地，所述图像采集设备是单目图像采集设备。
可以看出，由于该单目图像采集设备成本低、重量轻，因此，可以应用于多种应用场景，拓展了本实施例的应用范围。
可选地，所述神经网络是通过以下步骤训练得到的：
将样本图像输入至神经网络中，基于所述神经网络执行以下步骤：对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
重复执行上述步骤，直至网络参数值调整后的神经网络满足设定条件，得到训练完成的神经网络。
可以看出，在本公开实施例中，基于对不同背景环境下的样本图像进行图像处理的训练，得到可以获得图像处理结果的神经网络，以满足对图像的图像处理结果的实际需求，由于神经网络的训练过程是基于不同背景环境下的样本图像实现的，因此，通过该训练完成的神经网络对图像进行处理，获得的图像处理结果不易受背景环境的影响，稳定性和可信度较高。
可选地，所述处理模块还配置为获取所述待处理图像的标注；
根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在所述训练完成的神经网络的基础上进行增量训练。
可以看出，通过该增量训练，神经网络可以根据移动载体的任务进行神经网络的实时更新，从而，能够适应新的场景和作业任务。
本公开实施例还提供了一种电子设备，包括处理器和配置为存储能够在处理器上运行的计算机程序的存储器；其中，
所述处理器配置为运行所述计算机程序时，执行上述任意一种所述的图像处理方法。
本公开实施例还提供了一种计算机存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述任意一种所述的图像处理方法。
本公开实施例还提供了一种计算机程序，所述计算机程序被处理器执行时实现上述任意一种图像处理方法。
可以看出，本公开实施例中的图像处理方法，可以将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的，并基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。由于该图像处理方法中所使用的神经网络是基于不同背景环境下的样本图像训练得到的，因此，通过该图像处理方法而获得的待处理图像的图像处理结果，不易受背景环境的影响，稳定性和可信度较高。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1为本公开实施例的图像处理方法的流程图；
图2为本公开实施例的神经网络训练方法的流程图；
图3为本公开实施例的图像处理装置的组成结构示意图；
图4为本公开实施例的电子设备的结构示意图。
具体实施方式
以下结合附图及实施例，对本公开进行进一步详细说明。应当理解，此处所提供的实施例仅仅用以解释本公开，并不用于限定本公开。另外，以下所提供的实施例是用于实施本公开的部分实施例，而非提供实施本公开的全部实施例，在不冲突的情况下，本公开实施例记载的技术方案可以任意组合的方式实施。
需要说明的是，在本公开实施例中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的方法或者装置不仅包括所明确记载的要素，而且还包括没有明确列出的其他要素，或者是还包括为实施方法或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个......”限定的要素，并不排除在包括该要素的方法或者装置中还存在另外的相关要素(例如方法中的步骤或者装置中的单元，例如的单元可以是部分电路、部分处理器、部分程序或软件等等)。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
例如，本公开实施例提供的神经网络训练及图像处理方法包含了一系列的步骤，但是本公开实施例提供的神经网络训练及图像处理方法不限于所记载的步骤，同样地，本公开实施例提供的神经网络训练及图像处理装置包括了一系列模块，但是本公开实施例提供的装置不限于包括所明确记载的模块，还可以包括为获取相关信息、或基于信息进行处理时所需要设置的模块。
本公开实施例可以应用于终端和服务器等硬件或硬件组成的计算机***中，并可以与众多其它通用或专用计算***环境或配置一起操作，或者可通过处理器运行计算机可执行代码的方式实现本公开实施例。这里，终端可以是瘦客户机、厚客户机、手持或膝上设备、基于微处理器的***、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机***，等等，服务器可以是服务器计算机***小型计算机***﹑大型计算机***和包括上述任何***的分布式云计算技术环境，等等。
终端、服务器等电子设备可以在由计算机***执行的计算机***可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机***/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算***存储介质上。
本公开的一些实施例中，提出了一种图像处理方法，本公开实施例可以应用于任意的图像处理场景，例如，可以应用于户外作业机器人、农业机器人等图像处理场景。
图1为本公开实施例的一种图像处理方法的流程图，如图1所示，该流程可以包括：
步骤101：将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的。
在一种实施方式中，待处理图像可以是图片或视频文件，在将待处理图像输入至神经网络之前，需要对待处理图像进行处理，这里的处理可以是判断待处理图像的文件类型，根据待处理文件的类型确定文件类型所对应的处理操作。例如，在待处理图像为视频类型文件的情况下，需要先对待处理图像进行选帧操作，以获取选取的图片，并对选取的图像进行图像预处理操作；在待处理图像为图片文件的情况下，仅对待处理图像进行图像预处理操作。这里，图像预处理操作可以是对待处理图像的尺寸进行处理，例如， 可以是将待处理图像都转化为固定尺寸大小的图像。
作为一种实施方式，当神经网络用于对待处理图像进行目标检测时，输入的待处理图像的尺寸可以是固定大小的，当神经网络用于对待处理图像进行语义分割时，待处理图像的尺寸可以是非固定大小的，这里，不对待处理图像的尺寸的具体大小进行限制，待处理图像的尺寸可以是预先设定的固定尺寸。
本公开实施例中，这里的神经网络是指经过训练得到的神经网络，且这里的训练是基于不同背景环境下的样本图像实现的。具体地，不对神经网络的种类进行限定，示例性地，神经网络可以是单步多框检测器(Single Shot MultiBox Detector，SSD)、只看一次(You Only Look Once，YOLO)、RetinaNet、快速区域卷积神经网络(Faster Region-Convolutional Neural Networks，Faster RCNN)或其他实现目标检测的神经网络，也可以是全卷积神经网络(Fully Convolutional Networks)、U-net、SegNet、DeconvNet或其他实现语义分割的神经网络。
这里，不同背景环境下的样本图像可以是在不同的拍摄背景环境下所获取的多个图像，样本图像可以是同一拍摄对象处于不同背景环境下的多个图像，也可以是不同拍摄对象处于不同背景环境下的多个图像，这里不对图像的拍摄对象进行限制，只要样本图像的背景环境不同即可。同时，本公开实施例并不对样本图像的格式和来源进行限定，在一种实施方式中，样本图像可以是预先获取的样本图像，示例性地，可以从本地存储区域或网络获取样本图像，例如，可以通过公共数据集获取样本图像，这里的公共数据集可以是VOC数据集、COCO数据集等；样本图像的格式可以是联合图像专家小组(Joint Photographic Experts GROUP，JPEG)图像、位图(Bitmap，BMP)、便携式网络图形(Portable Network Graphics，PNG)或其他格式。
在一种实施方式中，神经网络是基于不同背景环境下的样本图像训练得到的，且具备获取输入图像的图像处理结果能力，将待处理图像输入至该神经网络，则可以获得所处理图像的图像处理结果。
步骤102：基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
作为一种实施方式，对所述待处理图像进行图像处理，得到图像处理结果，可以是对样本图像进行目标检测获得图像的目标检测结果，和/或，对样本图像进行语义分割获得图像的语义分割结果。示例性地，目标检测结果可以包括图像中表示目标位置和大小的边界框(Bounding box)，例如，边界框可以是矩形检测框或其它形状的检测框，在边界框是矩形检测框的情况下，目标检测结果可以包括矩形检测框的左上角的点的像素坐标位置以及矩形检测框的长宽，在目标检测框不是矩形检测的情况下，目标检测结果可以是包括检测框左上角的点和右下角的点的像素坐标位置等位置信息。语义分割结果可以包括图像中每个像素点的类别，可以通过不同的颜色来分别表示像素点的不同类别，例如，对于图像中的马路对应的所有像素点可以用蓝色来表示，对于图像中汽车对应的所有像素点可以用红色来表示，对于图片中草坪对应的所有像素点采用绿色来表示，进而，可以获得不同的颜色区域，可区别不同的对象。
在实际应用中，步骤101至步骤102可以利用电子设备中的处理器实现，上述处理器可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、FPGA、中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。可以看出，由于该图像处理方法中所使用的神经网络是基于不同背景环境下的样本图像训练得到的，因此，通过该图像处理方法而获得的待处理图像的图像处理结果，不易受 背景环境的影响，稳定性和可信度较高。
在一种实施方式中，待处理图像是由图像采集设备采集的；图像采集设备可以是摄像头等可以采集图像的设备，上述图像处理方法还包括：根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物；在不满足第一预设条件的情况下，确定所述图像采集设备前方不存在障碍物。
在一个示例中，待处理图像可以是由图像采集设备实时采集的图像；待处理图像的图像处理结果，可以是指图像采集设备实时采集的图像的目标检测结果。这里，在确定图像采集设备前方存在障碍物的情况下，所识别出的障碍物可以是与地面接触的障碍物，如高尔夫球、路锥，也可以是悬空的障碍物，如行人抬起的脚。
可以看出，该判断图像采集设备前方是否存在障碍物的方法，能够确定图像采集设备前方是否存在障碍物，进而采取后续措施。
在一种实施方式中，所述第一预设条件包括以下至少一项：
待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
待处理图像中的至少一个目标的像素面积值大于或等于最大面积安全阈值。
在一个示例中，待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值，可以是待处理图像中的所有边界框中的一个或多个边界框分别与图像采集设备之间的距离值小于或等于最小距离安全阈值距离，例如，可以是距离图像采集设备最近的一个边界框与采集设备之间的距离小于等于距离安全阈值。其中，边界框与图像采集设备之间的距离可以是边界框的中心或边界点等位置点对应的世界坐标系下的位置与图像采集设备之间的距离。
作为一种实施方式，待处理图像中的至少一个目标的像素面积值大于或等于最大面积安全阈值，可以是待处理图像中的所有边界框中的一个或多个边界框的像素面积大于等于最大安全阈值，例如，可以是像素面积最大的边界框的像素面积大于等于最大安全阈值。
同时，这里的最小距离安全阈值和最大面积安全阈值是根据任务需求情况和用户需求而具体确定的，这里不对最小距离安全阈值和最大面积安全阈值的具体大小进行限定。
可以看出，通过对上述第一预设条件的限定，可以使得障碍物的判断标准更加符合实际需求，更符合用户需求。
在一种实施方式中，上述图像处理方法还包括：在待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到待处理图像中的各目标与图像采集设备之间的距离值；和/或，根据目标检测结果，分别获得待处理图像中的各目标的像素面积值；其中，单应性矩阵用于表示世界坐标系和像素坐标系之间的位置映射关系。
这里的单应性矩阵可以用于表示待处理图像的上的像素点的坐标与待处理图像的上的像素点映射在世界坐标系上的坐标之间的关系，即，通过单应性矩阵可以获得待处理图像中已知坐标的像素点在对应在世界坐标系上的坐标，进而获得该已知坐标像素点对应的目标与图像采集设备之间的距离。
在一种实施方式中，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置确定的。
作为一种实施方式，可以通过下述方式确定单应性矩阵：首先，图像采集设备采集不同位姿放置标定板时的图像，可计算获得图像采集设备的内部参数，然后，根据获得 的图像采集设备的内部参数，基于固定位置放置的标定板(即一张图像)可获得单应性矩阵。这里的图像采集设备的内部参数至少包括图像采集设备的焦距和像素大小。
可以看出，通过图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置来确定单应性矩阵，可以准确获取单应性矩阵。使用单应性矩阵作为获取图像像素位置对应的世界坐标系深度矩阵的方案，容易实现、运算量小，且能够帮助目标检测算法快速获取检测物体的距离，有利于进行避障判断。
作为一种实施方式，目标检测结果还可以包括边界框置信度，边界框置信度用于表示该边界框的可信程度，置信度越高，该边界框的可信程度越高，示例性地，目标检测结果包括置信度高于第一阈值的边界框，也就是说，通过目标检测所识别的目标是置信度高于第一阈值的边界框对应的目标，这里，不对第一阈值进行限定，例如，第一阈值可以是50％、60％或80％。
对于在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及目标检测结果，分别得到待处理图像中的各目标与图像采集设备之间的距离值的实现方式，在一个示例中，可以根据上述边界框，以及世界坐标系和像素坐标系之间的位置映射关系，确定目标在世界坐标系上的位置信息，即，确定目标与图像采集设备之间的距离值。这里不对确定目标与图像采集设备之间的距离值的方式进行限定，具体地可以通过激光雷达或超声波等来检测目标与图像采集设备之间的距离值。
对于根据所述目标检测结果，分别获得待处理图像中的各目标的像素面积值的实现方式，示例性地，可以分别获得图像采集设备实时采集的图像上的所有边界框所占据的像素的面积值。这里不对获取图像上的边界框所占据的像素的面积值的方式进行限定，具体地，可以通过GPS定位***结合单应性矩阵来确定图像上的边界框所占据的像素的面积值。
可以看出，采用本公开实施例的技术方案，可以在目标检测结果的基础上，准确识别待处理图像中的所有目标以及目标与图像采集设备之间的距离，并获得所述图像中的各目标在像素坐标上对应的像素面积值。
在一种实施方式中，所述图像采集设备设置于移动载体上，移动载体可以是移动机器人或智能割草机等智能移动设备，所述方法还包括：在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
在一示例中，所述移动载体的避障反应可以是指移动载体在移动时前方存在障碍物不能按原方向继续前进时的反应，具体地，所述避障反应可以是停止前进等待障碍物移除，也可以是转弯或掉头等，这里不对避障反应进行具体地限制。
可以看出，采用本公开实施例的技术方案，可以在移动载体遇到障碍物的情况下，使得移动载体执行对应的避障反应。
在一种实施方式中，所述确定所述移动载体的避障反应，包括：根据所述图像处理结果，确定所述障碍物的类别；根据所述障碍物的类别，确定所述移动载体的避障反应。
作为一种实施方式，对于根据图像处理结果，确定障碍物的类别，示例性地，可以是在神经网络的训练阶段加入对目标类别的训练内容，使得训练得到的神经网络可以具有获得输入图像中目标的类别的能力，即，图像处理结果也包含了输入图像中各目标的类别，进而，可以确定障碍物的类别。
示例性地，障碍物的类别可以是可以移动的物体如高尔夫球，不可移动的物体如路锥、洒水器，以及可以移动的人物，如行人等。
对于根据所述障碍物的类别，确定移动载体的避障反应，在一示例中，障碍物可以是高尔夫球，如果移动载体装有机械臂，避障反应可以将高尔夫球进行回收；障碍物也可以是路锥或其它静态物，避障反应也可以是保持与静态障碍物的安全距离进行绕行； 障碍物也可以是行人，如果行人在运动，避障反应也可以是移动载体等待人离开再前进，当移动载体需要绕过静止的行人时，所述避障反应可以是移动载体需要降低速度并保持较大的安全距离以保证行人安全。
可以看出，上述确定移动载体避障反应的方法，由于考虑了障碍物的类别，因此，移动载体可以对不同的障碍物执行不同的避障策略，更智能，更能满足实际应用需求。
在一种实施方式中，在所述图像采集设备前方不存在障碍物的情况下，确定所述移动载体沿原方向继续移动。
这里，在所述图像采集设备前方不存在障碍物的情况下，可以是指在图像处理结果不满足第一预设条件的情况下，确定所述移动载体沿原方向继续移动，也可以是对正在移动中的移动载体不做任何干预措施。
可以看出，采用本公开实施例的技术方案，可以在图像采集设备前方不存在障碍物的情况下，移动载体能够按时完成工作或任务。
在一种实施方式中，所述背景环境可以包括以下至少一项：光照条件、纹理背景。
这里，光照条件可以是指光照的强度或其它光照信息，纹理背景可以是作为背景使用的线形花纹、非线性花纹或其它纹理背景。
可以看出，采用本公开实施例的技术方案，可以对不同光照条件和/或不同纹理背景的多个样本图像进行训练，获得训练完成的神经网络，由于训练过程是基于不同光照条件和/或纹理背景下的样本图像实现的，因此，该训练完成的神经网络更适用于光照条件变化较大和/或低纹理背景的户外场景。
在一种实施方式中，所述待处理图像是由图像采集设备采集的；图像采集设备可以是相机、摄像头等可以采集图像的设备，上述图像处理方法还包括：根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界；在不满足第二预设条件的情况下，确定所述图像采集设备未抵达可工作区域与不可工作区域的边界。
在一个示例中，所述待处理图像可以是由图像采集设备实时采集的图像；所述待处理图像的图像处理结果，可以是指图像采集设备实时采集的图像的语义分割结果。
可以看出，本实施例可以通过判断是否满足第二预设条件，来准确判断图像采集设备是否抵达可工作区域与不可工作区域的边界。
在一实施方式中，第二预设条件包括以下至少一项：边界的平均像素高度值小于或等于边界像素高度阈值；待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
在一个示例中，边界的平均像素高度值可以是指可工作区域和不可工作区域形成的边界与图像下边缘之间的距离的平均值，可以理解的是，该平均值越小，可工作区域和不可工作区域形成的边界越靠近图像下边缘，即，图像采集设备越靠近边界，此时，可以确定图像采集设备抵达可工作区域与不可工作区域的边界，且如果图像采集设备沿原方向向前稍作移动，即可能离开可工作区域，抵达不可工作区域。
对于可工作区域的面积值的实现方式，示例性地，可以是指图像的可工作区域在像素坐标系中所占的区域面积值。这里，当待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值时，可以认为图像采集设备可以工作的区域范围不够大。
对于待处理图像中的可工作区域的面积占比可以是指待处理图像中可工作区域的面积与整个图像的面积的比值，也可以是可工作区域的面积与不可工作区域的面积的比值，还可以是指待处理图像中可工作区域的面积与预设总可工作区域的面积的比值，这里不做具体限定。当待处理图像中的可工作区域的面积占比小于或等于可工作区域面积 占比阈值时，表明可工作区域面积比较小。
这里的边界像素高度阈值、可工作区域面积阈值和可工作区域的面积占比是根据任务需求情况和用户需求而具体确定的，这里不对边界像素高度阈值、可工作区域面积阈值和可工作区域面积占比阈值的具体大小进行限定。
可以看出，通过对上述第二预设条件的判断，可以使得是否抵达可工作区域与不可工作区域的边界的判断标准更能符合实际应用需求。
在一实施方式中，上述图像处理方法还包括：在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
作为一种实施方式，各像素点的区域类别可以是指各像素点所属的具体区域是可作业区域还是不可作业区域。对于可工作区域和不可工作区域的划分方式，示例性地，对于设置有图像采集设备的割草机器人来说，可工作区域可以是指草地等可割草的空间区域，不可工作区域可以是指水泥地、马路等不可割草的区域。
可以看出，本实施例可以得到待处理图像中的区域划分情况，较为准确地确定可工作区域和不可工作区域以及可工作区域和不可工作区域的边界，便于后面获得可工作区域的面积值和边界的平均像素高度值。
在一种实施方式中，图像采集设备设置于移动载体上，上述图像处理方法还包括：在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
可以看出，本实施例在移动载体抵达可工作区域和不可工作区域边界的情况下，可以及时确定移动载体的动作反应，避免移动载体抵达不可工作区域。
在一种实施方式中，所述方法还包括：在所述图像采集设备未抵达所述边界的情况下，确定所述移动载体沿原方向继续移动。
可以看出，本实施例在图像采集设备未抵达可工作区域和不可工作区域边界的情况下，可以保证移动载体在可工作区域中按照需求完成任务。
在一种实施方式中，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
可以看出，采用本公开实施例的技术方案，移动载体可以及时地执行停止、转弯、掉头等动作反应，有利于避免移动载体移动至不可工作区域。
在一种实施方式中，所述图像采集设备是单目图像采集设备。
单目图像采集设备是指具有单个摄像头的图像采集设备，示例性地，可以是单目相机。
可以看出，由于该单目图像采集设备成本低、重量轻，因此，可以应用于多种应用场景，拓展了本实施例的应用范围。
图2本公开实施例的一种神经网络训练方法的流程图，如图2所示，上述神经网络是通过以下步骤训练得到的：
步骤201：将样本图像输入至神经网络中，基于所述神经网络对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
步骤202：根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
步骤203：判断基于网络参数值调整后的神经网络得到的图像处理结果是否满足设定条件，如果否，则重新执行步骤201至步骤203；如果是，则执行步骤204。
步骤204：将网络参数值调整后的神经网络作为训练完成的神经网络。
作为一种实施方式，这里的神经网络可以是未经训练的神经网络，也可以是经过神经网络训练，但所述训练不包含本公开的训练内容的神经网络。
在实际应用中，需要预先获取样本图像的标注；在一种实施方式中，样本图像的标注可以是标注框以及标注信息，其中，标注框用于框选样本图像中的目标，标注框也可以标识目标的位置，例如，可以通过标注框标注样本图像中的人物、动物等目标以及目标的位置，标注信息用于标注目标的类别，例如，可以标注目标是物体、人还是动物；样本图像的标注也可以是用于标注图像中像素点的类别的标注信息，由于多个像素点可以是同一类别，因此，可以是多个区域类别的标注信息，例如，可以是标注出可割草区域和不可割草区域的标注信息。
对于根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值，示例性地，可以是根据样本图像的图像处理结果与样本图像的标注之间的差异，以减少该差异为目标来调整所述神经网络的网络参数值，其中，该差异可以通过损坏神经网络的损失函数值来描述。具体的损失函数值确定方法可以根据神经网络的类型确定，本公开实施例不做限定。
这里，设定条件可以是调整神经网络的网络参数的次数等于设定迭代次数，也可以是神经网络的损失函数达到收敛条件，当然，设定条件也可以是在固定的测试集上测试达到设定的准确率。这里，设定迭代次数表示调整神经网络的网络参数的次数的最大值，设定迭代次数为大于1的整数；收敛条件可以是调整神经网络的损失函数的值小于设定损失，设定损失可以根据实际应用需求预先设置。需要说明的是，上述仅仅是对设定条件进行了示例性说明，本公开实施例的设定条件并不局限于此；设定的准确率可以是预先设置的百分比值，具体地，设定的百分比值可以是50％及大于50％的值。
可以看出，在本公开实施例中，基于对不同背景环境下的样本图像进行图像处理的训练，得到可以获得图像处理结果的神经网络，以满足对图像的图像处理结果的实际需求，由于神经网络的训练过程是基于不同背景环境下的样本图像实现的，因此，通过该训练完成的神经网络对图像进行处理，获得的图像处理结果不易受背景环境的影响，稳定性和可信度较高。
在一种实施方式中，图像处理方法还包括：获取所述待处理图像的标注；根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在所述训练完成的神经网络的基础上进行增量训练。
作为一种实施方式，增量训练表示在上述神经网络的基础上，利用新增数据对所述神经网络进行参数调整的过程。本实施例不对增量训练的实施方式进行具体限定，在一个示例中，可以根据将神经网络的损失函数加上预设的正则化项，得到修改后的损失函数；利用神经网络对新增数据进行图像处理，得到样本图像的图像处理结果；根据修改后的损失函数、新增数据的标注，确定神经网络的损失；根据神经网络的损失，调整神经网络的网络参数；重复执行上述确定神经网络的损失、以及调整神经网络的网络参数的步骤，直至网络参数调整后的神经网络满足训练结束条件，得到训练完成的神经网络。
可以看出，通过该增量训练，神经网络可以根据移动载体的任务进行神经网络的实时更新，从而，能够适应新的场景和作业任务。
在前述实施例提出的图像处理方法的基础上，本公开实施例提出了一种图像处理装置。
图3为本公开实施例的图像处理装置的组成结构示意图，如图3所示，该装置可以包括：处理模块301其中，
处理模块301，配置为将待处理图像输入至神经网络，所述神经网络是基于不同背 景环境下的样本图像训练得到的；基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
可选地，所述处理模块301，还配置为根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物。
可选地，所述第一预设条件包括以下至少一项：
所述待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
所述待处理图像中的至少一个目标在像素坐标上对应的像素面积值大于或等于最大面积安全阈值。
可选地，所述处理模块301还配置为在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到所述待处理图像中的各目标与图像采集设备之间的距离值；和/或根据所述目标检测结果，分别获得所述待处理图像中的各目标在像素坐标上对应的像素面积值；其中，所述单应性矩阵用于表示世界坐标系和像素坐标系之间的位置映射关系。
可选地，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置确定的。
可选地，所述图像采集设备设置于移动载体上，所述处理模块301还配置为在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
可选地，所述处理模块301还配置为根据所述图像处理结果，确定所述障碍物的类别；根据所述障碍物的类别，确定所述移动载体的避障反应。
可选地，处理模块301还配置为在所述图像采集设备前方不存在障碍物的情况下，确定所述移动载体沿原方向继续移动。
可选地，所述背景环境包括以下至少一项：光照条件、纹理背景。
可选地，所述待处理图像是由图像采集设备采集的，所述处理模块301还配置为根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界。
可选地，所述第二预设条件包括以下至少一项：
所述边界的平均像素高度值小于或等于边界像素高度阈值；
所述待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；
所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
可选地，所述处理模块301还配置为在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
可选地，所述图像采集设备设置于移动载体上，所述处理模块301还配置为在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
可选地，所述处理模块301还配置为在所述图像采集设备未抵达所述边界的情况下，确定所述移动载体沿原方向继续移动。
可选地，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
可选地，所述图像采集设备是单目图像采集设备。
可选地，所述神经网络是通过以下步骤训练得到的：将样本图像输入至神经网络中，基于所述神经网络对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结 果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
重复执行上述步骤，直至网络参数值调整后的神经网络满足设定条件，得到训练完成的神经网络。
可选地，所述处理模块301还配置为获取所述待处理图像的标注；根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在所述训练完成的神经网络的基础上进行增量训练。
实际应用中，处理模块301可以利用电子设备中的处理器实现，上述处理器可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
另外，在本实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
具体来讲，本实施例中的一种神经网络训练方法或图像处理方法对应的计算机程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的与一种神经网络训练方法或图像处理方法对应的计算机程序指令被一电子设备读取或被执行时，实现前述实施例的任意一种图像处理方法或任意一种神经网络训练方法。
基于前述实施例相同的技术构思，本公开实施例还提供了一种计算机程序，该计算机程序被处理器执行时实现上述任意一种图像处理方法。
基于前述实施例相同的技术构思，参见图4，其示出了本公开实施例提供的一种电子设备400，可以包括：存储器401和处理器402；其中，
所述存储器401，配置为存储计算机程序和数据；
所述处理器402，配置为执行所述存储器中存储的计算机程序，以实现前述实施例的任意一种图像处理方法。
在实际应用中，上述存储器401可以是易失性存储器(volatile memory)，例如RAM；或者非易失性存储器(non-volatile memory)，例如ROM，快闪存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合，并向处理器402提供指令和数据。
上述处理器402可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。可以理解地，对于不同的增强现实云平台，用于实现上述处理器功能的电子器件还可以为其它，本公开实施例不作具体限定。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之 处可以互相参考，为了简洁，本文不再赘述
本申请所提供的各方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。
本申请所提供的各产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。
本申请所提供的各方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本公开各个实施例所述的方法。
上面结合附图对本公开的实施例进行了描述，但是本公开并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本公开的启示下，在不脱离本公开宗旨和权利要求所保护的范围情况下，还可做出很多形式，这些均属于本公开的保护之内。

Claims (35)

  1. 一种图像处理方法，其中，所述方法包括：
    将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的；
    基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
  2. 根据权利要求1所述的方法，其中，所述待处理图像是由图像采集设备采集的；
    所述方法还包括：
    根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；
    在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物。
  3. 根据权利要求2所述的方法，其中，所述第一预设条件包括以下至少一项：
    所述待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
    所述待处理图像中的至少一个目标的像素面积值大于或等于最大面积安全阈值。
  4. 根据权利要求3所述的方法，其中，所述方法还包括：
    在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到所述待处理图像中的各目标与图像采集设备之间的距离值；和/或，根据所述目标检测结果，分别获得所述待处理图像中的各目标的像素面积值；其中，所述单应性矩阵用于表示各像素点的世界坐标系和像素坐标系之间的位置映射关系。
  5. 根据权利要求4所述的方法，其中，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置确定的。
  6. 根据权利要求2-5任一项所述的方法，其中，所述图像采集设备设置于移动载体上，
    所述方法还包括：
    在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
  7. 根据权利要求6所述的方法，其中，所述确定所述移动载体的避障反应，包括：
    根据所述图像处理结果，确定所述障碍物的类别；根据所述障碍物的类别，确定所述移动载体的避障反应。
  8. 根据权利要求1-7任一项所述的方法，其中，所述背景环境包括以下至少一项：光照条件、纹理背景。
  9. 根据权利要求1-7任一项所述的方法，其中，所述待处理图像是由图像采集设备采集的；
    所述方法还包括：
    根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；
    在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界。
  10. 根据权利要求9所述的方法，其中，所述第二预设条件包括以下至少一项：
    所述边界的平均像素高度值小于或等于边界像素高度阈值；
    所述待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；
    所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
  11. 根据权利要求10所述的方法，其中，所述方法还包括：
    在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
  12. 根据权利要求9-11任一项所述的方法，其中，所述图像采集设备设置于移动载体上，
    所述方法还包括：
    在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
  13. 根据权利要求12所述的方法，其中，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
  14. 根据权利要求2-13任一项所述的方法，其中，所述图像采集设备是单目图像采集设备。
  15. 根据权利要求1-14任一项所述的方法，其中，所述神经网络是通过以下步骤训练得到的：
    将样本图像输入至神经网络中，基于所述神经网络执行以下步骤：对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
    根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
    重复执行上述步骤，直至网络参数值调整后的神经网络满足设定条件，得到训练完成的神经网络。
  16. 根据权利要求15所述的方法，其中，所述方法还包括：
    获取所述待处理图像的标注；
    根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在所述训练完成的神经网络的基础上进行增量训练。
  17. 一种图像处理装置，其中，所述装置包括：处理模块，其中，
    处理模块，配置为将待处理图像输入至神经网络，所述神经网络是基于不同背景环境下的样本图像训练得到的；基于所述神经网络对所述待处理图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果。
  18. 根据权利要求17所述的装置，其中，所述待处理图像是由图像采集设备采集的；所述处理模块，还配置为根据所述待处理图像的图像处理结果，判断是否满足第一预设条件；在满足第一预设条件的情况下，确定所述图像采集设备前方存在障碍物。
  19. 根据权利要求18所述的装置，其中，所述第一预设条件包括以下至少一项：
    所述待处理图像中的至少一个目标分别与图像采集设备之间的距离值小于或等于最小距离安全阈值；
    所述待处理图像中的至少一个目标在像素坐标上对应的像素面积值大于或等于最大面积安全阈值。
  20. 根据权利要求19所述的装置，其中，所述处理模块还配置为在所述待处理图像的图像处理结果包括目标检测结果的情况下，根据预先获取的单应性矩阵以及所述目标检测结果，分别得到所述待处理图像中的各目标与图像采集设备之间的距离 值；和/或根据所述目标检测结果，分别获得所述待处理图像中的各目标在像素坐标上对应的像素面积值；其中，所述单应性矩阵用于表示世界坐标系和像素坐标系之间的位置映射关系。
  21. 根据权利要求20所述的装置，其中，所述单应性矩阵是根据所述图像采集设备的内部参数以及标定板相对于所述图像采集设备的已知位置确定的。
  22. 根据权利要求18-21任一项所述的装置，其中，所述图像采集设备设置于移动载体上，所述处理模块还配置为在所述图像采集设备前方存在障碍物的情况下，确定所述移动载体的避障反应。
  23. 根据权利要求22所述的装置，其中，所述处理模块还配置为根据所述图像处理结果，确定所述障碍物的类别；根据所述障碍物的类别，确定所述移动载体的避障反应。
  24. 根据权利要求17-23任一项所述的装置，其中，所述背景环境包括以下至少一项：光照条件、纹理背景。
  25. 根据权利要求17-23任一项所述的装置，其中，所述待处理图像是由图像采集设备采集的，所述处理模块还配置为根据所述待处理图像的图像处理结果，判断是否满足第二预设条件；在满足第二预设条件的情况下，确定所述图像采集设备抵达可工作区域与不可工作区域的边界。
  26. 根据权利要求25所述的装置，其中，所述第二预设条件包括以下至少一项：
    所述边界的平均像素高度值小于或等于边界像素高度阈值；
    所述待处理图像中的可工作区域的面积值小于或等于可工作区域面积阈值；
    所述待处理图像中的可工作区域的面积占比小于或等于可工作区域面积占比阈值。
  27. 根据权利要求26所述的装置，其中，所述处理模块还配置为在所述待处理图像的图像处理结果包括语义分割结果的情况下，根据语义分割结果确定所述待处理图像的各像素点的区域类别，根据所确定的各像素点的区域类别确定可作业区域和不可作业区域；根据所确定的可作业区域和不可作业区域，获得所述待处理图像中的可工作区域的面积值，和/或确定所述边界的平均像素高度值。
  28. 根据权利要求25-27任一项所述的装置，其中，所述图像采集设备设置于移动载体上，所述处理模块还配置为在所述图像采集设备抵达所述边界的情况下，确定所述移动载体的动作反应。
  29. 根据权利要求28所述的装置，其中，所述移动载体的动作反应包括以下至少一项：停止、转弯、掉头。
  30. 根据权利要求18至29任一项所述的装置，其中，所述图像采集设备是单目图像采集设备。
  31. 根据权利要求17-30任一项所述的装置，其中，所述神经网络是通过以下步骤训练得到的：将样本图像输入至神经网络中，基于所述神经网络对所述样本图像进行图像处理，得到图像处理结果；所述图像处理结果包括目标检测结果和/或语义分割结果；其中，所述样本图像用于表示不同背景环境下的图像；
    根据所述样本图像的图像处理结果以及所述样本图像的标注，调整所述神经网络的网络参数值；
    重复执行上述步骤，直至网络参数值调整后的神经网络满足设定条件，得到训练完成的神经网络。
  32. 根据权利要求31所述的装置，其中，所述处理模块还配置为获取所述待处理图像的标注；根据所述待处理图像的图像处理结果以及所述待处理图像的标注，在 所述训练完成的神经网络的基础上进行增量训练。
  33. 一种电子设备，其中，包括处理器和配置为存储能够在处理器上运行的计算机程序的存储器；其中，
    所述处理器配置为运行所述计算机程序时，执行权利要求1-16任一项所述的图像处理方法。
  34. 一种计算机存储介质，其上存储有计算机程序，其中，该计算机程序被处理器执行时实现权利要求1-16任一项所述的图像处理方法。
  35. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1-16中的任一权利要求所述的方法。
PCT/CN2020/084018 2019-12-20 2020-04-09 图像处理方法、装置、设备和存储介质 WO2021120455A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217014432A KR20210081364A (ko) 2019-12-20 2020-04-09 이미지 처리 방법, 장치, 기기 및 저장 매체
JP2021526557A JP2022518324A (ja) 2019-12-20 2020-04-09 画像処理方法、装置、機器及び記憶媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911328268.7A CN111126251B (zh) 2019-12-20 2019-12-20 图像处理方法、装置、设备和存储介质
CN201911328268.7 2019-12-20

Publications (1)

Publication Number Publication Date
WO2021120455A1 true WO2021120455A1 (zh) 2021-06-24

Family

ID=70500812

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/084018 WO2021120455A1 (zh) 2019-12-20 2020-04-09 图像处理方法、装置、设备和存储介质

Country Status (5)

Country Link
JP (1) JP2022518324A (zh)
KR (1) KR20210081364A (zh)
CN (1) CN111126251B (zh)
TW (1) TW202125316A (zh)
WO (1) WO2021120455A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113345034A (zh) * 2021-07-14 2021-09-03 山东诺德能源科技有限公司 一种利用图像识别方法快速测量钢筋直径的方法
CN113611079A (zh) * 2021-07-30 2021-11-05 北京市商汤科技开发有限公司 提示方法及装置、电子设备和存储介质
CN113706561A (zh) * 2021-10-29 2021-11-26 华南理工大学 一种基于区域分离的图像语义分割方法
CN114339046A (zh) * 2021-12-30 2022-04-12 中元汇吉生物技术股份有限公司 基于自动旋转试管的图像采集方法、装置、设备及介质
CN116132645A (zh) * 2023-01-06 2023-05-16 清华大学 图像处理方法、装置、设备和介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102378B (zh) * 2020-08-06 2024-06-11 深圳市优必选科技股份有限公司 图像配准方法、装置、终端设备及计算机可读存储介质
CN112183476B (zh) * 2020-10-28 2022-12-23 深圳市商汤科技有限公司 一种障碍检测方法、装置、电子设备以及存储介质
CN112418219A (zh) * 2020-11-24 2021-02-26 广东工业大学 一种服装面料裁片的颜色和形状识别方法及相关装置
CN112465794A (zh) * 2020-12-10 2021-03-09 无锡卡尔曼导航技术有限公司 一种基于YOLOv4和嵌入式平台的高尔夫球检测方法
CN112651931B (zh) * 2020-12-15 2024-04-26 浙江大华技术股份有限公司 建筑物变形监测方法、装置和计算机设备
CN112561897A (zh) * 2020-12-22 2021-03-26 电子科技大学中山学院 一种基于U-Net的光子晶体光纤端面结构提取方法
CN112862776A (zh) * 2021-02-02 2021-05-28 中电鸿信信息科技有限公司 基于ar和多重语义分割的智能测量方法
CN113910225A (zh) * 2021-10-09 2022-01-11 邦鼓思电子科技（上海）有限公司 一种基于视觉边界检测的机器人控制***及方法
TWI823491B (zh) * 2022-07-22 2023-11-21 鴻海精密工業股份有限公司 深度估計模型的優化方法、裝置、電子設備及存儲介質

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108321722A (zh) * 2018-03-08 2018-07-24 贵州电网有限责任公司 自动避障的可纵向弯曲树障清理空中机器人和避障方法
US20180253866A1 (en) * 2017-03-03 2018-09-06 General Electric Company Image analysis neural network systems
CN109063575A (zh) * 2018-07-05 2018-12-21 中国计量大学 一种基于单目视觉的智能割草机自主有序割草方法
CN110228484A (zh) * 2019-06-17 2019-09-13 福州视驰科技有限公司 一种具有辅助驾驶功能的低时延智能远程驾驶***
CN110502019A (zh) * 2019-09-06 2019-11-26 北京云迹科技有限公司 一种室内机器人的避障方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007148835A (ja) * 2005-11-28 2007-06-14 Fujitsu Ten Ltd 物体判別装置、報知制御装置、物体判別方法および物体判別プログラム
JP2016181068A (ja) * 2015-03-24 2016-10-13 株式会社明電舎 学習サンプル撮影装置
CN108303747B (zh) * 2017-01-12 2023-03-07 清华大学 检查设备和检测***的方法
CN107301646B (zh) * 2017-06-27 2019-09-17 深圳市云洲创新科技有限公司 基于单目视觉的无人船智能避障方法和装置
CN107895378A (zh) * 2017-10-12 2018-04-10 西安天和防务技术股份有限公司 目标检测方法和装置、存储介质、电子设备
JP6621220B2 (ja) * 2018-02-09 2019-12-18 株式会社ビコー 情報処理装置、情報処理方法およびプログラム
CN110245710B (zh) * 2019-06-18 2022-11-29 腾讯科技（深圳）有限公司 语义分割模型的训练方法、语义分割方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180253866A1 (en) * 2017-03-03 2018-09-06 General Electric Company Image analysis neural network systems
CN108321722A (zh) * 2018-03-08 2018-07-24 贵州电网有限责任公司 自动避障的可纵向弯曲树障清理空中机器人和避障方法
CN109063575A (zh) * 2018-07-05 2018-12-21 中国计量大学 一种基于单目视觉的智能割草机自主有序割草方法
CN110228484A (zh) * 2019-06-17 2019-09-13 福州视驰科技有限公司 一种具有辅助驾驶功能的低时延智能远程驾驶***
CN110502019A (zh) * 2019-09-06 2019-11-26 北京云迹科技有限公司 一种室内机器人的避障方法及装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113345034A (zh) * 2021-07-14 2021-09-03 山东诺德能源科技有限公司 一种利用图像识别方法快速测量钢筋直径的方法
CN113611079A (zh) * 2021-07-30 2021-11-05 北京市商汤科技开发有限公司 提示方法及装置、电子设备和存储介质
CN113706561A (zh) * 2021-10-29 2021-11-26 华南理工大学 一种基于区域分离的图像语义分割方法
CN114339046A (zh) * 2021-12-30 2022-04-12 中元汇吉生物技术股份有限公司 基于自动旋转试管的图像采集方法、装置、设备及介质
CN114339046B (zh) * 2021-12-30 2023-10-03 中元汇吉生物技术股份有限公司 基于自动旋转试管的图像采集方法、装置、设备及介质
CN116132645A (zh) * 2023-01-06 2023-05-16 清华大学 图像处理方法、装置、设备和介质

Also Published As

Publication number Publication date
KR20210081364A (ko) 2021-07-01
CN111126251B (zh) 2022-08-02
CN111126251A (zh) 2020-05-08
JP2022518324A (ja) 2022-03-15
TW202125316A (zh) 2021-07-01

Similar Documents

Publication Publication Date Title
WO2021120455A1 (zh) 图像处理方法、装置、设备和存储介质
US11878433B2 (en) Method for detecting grasping position of robot in grasping object
US20210390329A1 (en) Image processing method, device, movable platform, unmanned aerial vehicle, and storage medium
TWI726278B (zh) 行車偵測方法、車輛及行車處理裝置
WO2022088611A1 (zh) 障碍检测方法、装置、电子设备、存储介质及计算机程序
CN106650701A (zh) 基于双目视觉的室内阴影环境下障碍物检测方法及装置
CN111612823A (zh) 一种基于视觉的机器人自主跟踪方法
WO2023173950A1 (zh) 障碍物检测方法、移动机器人及机器可读存储介质
US20210216767A1 (en) Method and computing system for object recognition or object registration based on image classification
CN112605993A (zh) 基于双目视觉引导的自动抓取档案机器人控制***及方法
JP7153264B2 (ja) 画像解析システム、画像解析方法及び画像解析プログラム
CN109543634B (zh) 定位过程中的数据处理方法、装置、电子设备和存储介质
CN112784639A (zh) 路口检测、神经网络训练及智能行驶方法、装置和设备
WO2022061632A1 (zh) 障碍物检测方法、装置、无人机和存储介质
CN113435367A (zh) 社交距离评估方法、装置及存储介质
CN116503566B (zh) 一种三维建模方法、装置、电子设备及存储介质
CN116912517B (zh) 相机视野边界的检测方法及装置
WO2023216555A1 (zh) 基于双目视觉的避障方法、装置、机器人及介质
WO2022052853A1 (zh) 目标跟踪方法、装置、设备及计算机可读存储介质
CN115880643A (zh) 一种基于目标检测算法的社交距离监测方法和装置
CN115902977A (zh) 基于视觉和gps的变电站机器人双重定位方法及***
Ferreira et al. Human detection and tracking using a Kinect camera for an autonomous service robot
Velazquez et al. Using Autonomous Drone Interactions towards MobilePersonal Spaces for Indoor Environments
CN114333345B (zh) 停车位被遮挡时的预警方法、设备、存储介质及程序产品
Tian Effective image enhancement and fast object detection for improved UAV applications

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 20217014432

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021526557

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20903948

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.10.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20903948

Country of ref document: EP

Kind code of ref document: A1