CN113520233B

CN113520233B - Control method and device of sweeping robot with multiple sweeping modes and sweeping robot

Info

Publication number: CN113520233B
Application number: CN202111089588.9A
Authority: CN
Inventors: 汪洋
Original assignee: Shenzhen Yuanding Intelligent Innovation Co ltd
Current assignee: Shenzhen Yuanding Intelligent Innovation Co ltd
Priority date: 2021-09-16
Filing date: 2021-09-16
Publication date: 2023-04-07
Anticipated expiration: 2041-09-16
Also published as: CN113520233A

Abstract

The embodiment of the invention discloses a control method and a control device of a sweeping robot with multiple sweeping modes and the sweeping robot, wherein the method comprises the following steps: acquiring a target image of an area to be cleaned through a camera device and carrying out image recognition on the acquired target image so as to determine the area characteristics of the area to be cleaned; determining a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes according to the area characteristics; determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the target cleaning mode, and performing route planning on all the non-cleaned areas to obtain a first planned route; and controlling the sweeping robot to move according to the first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to the charging seat for charging. By adopting the invention, the sweeping accuracy of the sweeping robot can be improved.

Description

Control method and device of sweeping robot with multiple sweeping modes and sweeping robot

Technical Field

The invention relates to the technical field of sweeping robots, in particular to a route control method and device of a sweeping robot with multiple sweeping modes and the sweeping robot.

Background

With the continuous development of the artificial intelligence technology, a series of artificial intelligence products based on the artificial intelligence technology gradually enter the lives of people, and a great deal of convenience is provided for the lives of people. For example, the intelligent sweeping robot can automatically identify the obstacle, avoid the obstacle to perform a sweeping task on an object to be swept, and reduce the sweeping task for a user. However, a general sweeping robot has only one sweeping mode, and the same mode is adopted for sweeping different areas, so that different sweeping modes cannot be precisely performed for different areas.

Disclosure of Invention

In view of the above, it is necessary to provide a method and an apparatus for controlling a route of a sweeping robot with multiple sweeping modes, and a sweeping robot.

In a first aspect of the present invention, there is provided a route control method for a multi-sweeping mode sweeping robot, the method comprising:

acquiring a target image of an area to be cleaned through a camera device arranged on the sweeping robot, and carrying out image recognition on the acquired target image so as to determine the area characteristics of the area to be cleaned;

determining a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes according to the area characteristics;

determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaned areas to obtain a first planned route;

and controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to a charging seat for charging.

Optionally, the step of obtaining a target image of an area to be cleaned and performing image recognition on the collected target image by using the camera device arranged on the sweeping robot to determine the regional characteristics of the area to be cleaned further includes:

converting the target image into a gray scale image to obtain a target image of the gray scale image;

calculating the gray level average value of pixel points contained in each line of the target image, calculating the difference value between each pixel point and the gray level average value of the line as a gray level difference value, and calculating the gray level standard deviation under each line based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a first difference image according to the gray average value of each row;

determining a second differential image according to the absolute value of the gray difference between the target image and the first differential image, and performing binarization processing on the second differential image to obtain a binarized second differential image;

calculating the gray level average value of pixel points contained in each row of the target image, calculating the difference value between each pixel point and the gray level average value of the row as a gray level difference value, and calculating the gray level standard deviation of each row based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a third difference image according to the gray average value of each row;

determining a fourth differential image according to the absolute value of the gray difference between the target image and the third differential image, and performing binarization processing on the fourth differential image to obtain a binarized fourth differential image;

fusing the second differential image and the fourth differential image to obtain a fifth differential image, wherein the fifth differential image is a binary image;

identifying a fifth difference image to determine a ROI area in the target image.

Optionally, the step of obtaining a target image of an area to be cleaned and performing image recognition on the collected target image by using a camera device arranged on the sweeping robot to determine the regional characteristics of the area to be cleaned further includes:

acquiring an image of an ROI (region of interest) in the target image as a detection image;

acquiring a plurality of preset feature templates corresponding to the cleaning modes, and extracting features of the detection image based on the feature templates to acquire the features and confidence coefficients of the detection image under each feature template;

and determining the features under one feature template as the region features according to the confidence.

Optionally, the step of determining a target cleaning mode corresponding to the area to be cleaned in a plurality of preset cleaning modes according to the area characteristics further includes:

acquiring a plurality of preset cleaning modes, and determining the sample characteristics of each cleaning mode;

calculating the similarity between the regional characteristics of the region to be cleaned and the characteristics of the sample according to a preset characteristic matching algorithm,

and determining a cleaning mode corresponding to the sample feature with the maximum similarity as the target cleaning mode.

Optionally, the step of determining historical power consumption data of other non-cleaned areas except for the area to be cleaned, and a target cleaning mode and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaned areas to obtain a first planned route further includes:

determining preset mode power consumption data corresponding to each cleaning mode;

determining a first planned path of an area to be cleaned, and determining first power consumption of the area to be cleaned according to the first planned path and mode power consumption data corresponding to a target cleaning mode;

judging whether the first power consumption is less than or equal to the residual power of the sweeping robot, and if so, executing the step of controlling the sweeping robot to move according to a first planned route; if not, the sweeping robot is controlled to move to the charging seat for charging.

according to the historical data, determining historical power consumption data of other non-cleaned areas except the area to be cleaned, and determining second power consumption corresponding to the other non-cleaned areas except the area to be cleaned;

determining whether the sum of the first power consumption and the second power consumption is less than or equal to the residual power of the sweeping robot;

if yes, the step of controlling the sweeping robot to move according to a first planned route is executed;

and if not, determining the first power consumption of the area to be cleaned according to the first planned path and the mode power consumption data corresponding to the target cleaning mode.

determining a thermal value of a region to be cleaned according to historical data, wherein the thermal value of the region is used for representing the movement speed and the power consumption value of the region in the historical data, and the movement speeds corresponding to different cleaning modes are different;

and determining whether the area to be cleaned needs to be cleaned repeatedly according to the thermal value, and under the condition that the area needs to be cleaned repeatedly, planning the route of the area to be cleaned to obtain a first planned route, wherein the area repetition rate of the first planned route is greater than a preset value, and any place in the area to be cleaned is cleaned repeatedly for 2 times or more in the first planned route by the cleaning robot.

Optionally, the method further includes:

dividing a map into a plurality of areas, and executing the steps of acquiring a target image of an area to be cleaned and performing image recognition on the acquired target image through a camera device arranged on the sweeping robot for each area so as to determine the area characteristics of the area to be cleaned, wherein the step of dividing the map into the plurality of areas further comprises the following steps of:

dividing a map into a plurality of grids;

determining a plurality of basic grid points in the grids, and outwards expanding in the grids divided into the map based on each basic grid point to obtain an expanded rectangular area corresponding to each basic grid point, wherein expansion is stopped when an obstacle is encountered or the expansion is stopped when the number of the grids corresponding to the obstacle area is greater than a preset value;

for a plurality of rectangular regions:

performing first region processing operation on the plurality of rectangular regions according to the repeated relationship and the inclusion relationship among the rectangular regions, wherein the first region processing operation comprises one or more of deduplication operation, deletion operation and combination operation;

traversing each rectangular region, and calculating the ratio of the overlapping area between the traversed first rectangular region and other second rectangular regions with overlapping regions to the size of the region between the first rectangular region and the second rectangular region as a first ratio/a second ratio;

if the first ratio and the second ratio are both larger than a preset ratio threshold, merging the first rectangular area and the second rectangular area;

and if the first ratio is smaller than a preset ratio threshold value and/or the second ratio is smaller than a preset ratio threshold value, performing segmentation processing on the first rectangular area and the second rectangular area, wherein under the condition that the first ratio is smaller than the second ratio, the segmentation processing is to delete the intersection area of the first rectangular area and the second rectangular area from the first rectangular area.

In a second aspect of the present invention, there is provided a route control device for a multi-sweeping mode sweeping robot, the device comprising:

the image feature extraction module is used for acquiring a target image of an area to be cleaned through a camera device arranged on the sweeping robot and carrying out image recognition on the acquired target image so as to determine the regional features of the area to be cleaned;

the cleaning mode determining module is used for determining a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes according to the regional characteristics;

the route planning module is used for determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the target cleaning mode, and planning routes of all the non-cleaned areas to obtain a first planned route;

and the control sweeping module is used for controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, the control sweeping robot is controlled to move to the charging seat to charge.

In a third aspect of the present invention, there is provided a sweeping robot, comprising a memory and a processor, wherein the memory has an executable code, and when the executable code runs on the processor, the route control method of the sweeping robot with multiple sweeping modes as described in the first aspect is implemented.

The embodiment of the invention has the following beneficial effects:

after the sweeping robot and the route control method and device for the sweeping robot adopting the sweeping mode are adopted, a target image of a region to be swept is acquired through a camera device arranged on the sweeping robot, and the acquired target image is subjected to image recognition so as to determine the regional characteristics of the region to be swept; determining a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes according to the area characteristics; determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaned areas to obtain a first planned route; and controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to a charging seat for charging. That is to say, feature extraction is performed according to an image of each area, then a target cleaning mode is determined in a plurality of cleaning modes based on the extracted features, then the cleaning robot is controlled to clean the area to be cleaned based on the determined target cleaning mode and the planned path, and after the cleaning is completed, the cleaning robot is controlled to return to the charging seat for charging. By adopting the invention, the specific cleaning mode of each area can be customized according to the characteristics of each area, thereby improving the cleaning efficiency.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Wherein:

fig. 1 is a schematic flow chart illustrating a route control method of a multi-sweeping robot according to an embodiment;

fig. 2 is a schematic diagram illustrating a route control device of a multi-sweeping mode sweeping robot according to an embodiment;

fig. 3 is a schematic structural diagram of a sweeping robot operating the route control method of the sweeping robot in the sweeping mode in one embodiment.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.

In the embodiment, a route control method of a sweeping robot with multiple sweeping modes is provided.

In this embodiment, in order to sweep the whole house through the sweeping robot, a map of the whole house needs to be constructed in advance, and then a path planning is performed based on the constructed map, so that the sweeping robot can move according to the planned path, and the sweeping of the whole house is completed in the moving process, thereby reducing the sweeping task for the user.

Wherein, the robot of sweeping the floor can support different modes of cleaning, can adopt different modes of cleaning to clean different regions to obtain different effects of cleaning.

Specifically, referring to fig. 1, fig. 1 shows a flow chart of a route control method of the sweeping robot in the above-mentioned sweeping mode, where the method includes steps S101 to S104 shown in fig. 1:

step S101: through the camera device arranged on the sweeping robot, a target image of an area to be swept is acquired, and image recognition is carried out on the acquired target image so as to determine the regional characteristics of the area to be swept.

In this embodiment, a map corresponding to the sweeping robot is constructed in advance, and the map is an area where the sweeping robot moves in the sweeping process; the map is constructed by collecting surrounding environment data through a sensor arranged on the sweeping robot when the sweeping robot starts sweeping, for example, the map constructed based on SLAM technology.

The constructed map of the work of the sweeping robot needs to be further planned for the movement route of the sweeping robot in the sweeping process, and all areas of the map can be covered under the indication of the route of the sweeping robot. And in the process of working of the sweeping robot, moving based on the planned route so as to finish sweeping the area corresponding to the map.

In the process of sweeping by the sweeping robot, a corresponding target sweeping mode is respectively determined for each area to be swept. Specifically, only the image (target image) of the area to be cleaned is collected through the camera device arranged on the cleaning robot, and then what kind of multi-cleaning mode needs to be adopted for cleaning is determined according to the image.

In specific implementation, when an area to be cleaned is cleaned, a camera device arranged on the cleaning robot is used for collecting a target image of the area to be cleaned, and then the collected target image is identified so as to obtain the area characteristics of the area to be cleaned.

The following describes in detail how the target image is identified to obtain the region feature.

In the first step, a region (referred to as ROI region herein) including a region feature in the target image needs to be obtained, the ROI region includes a feature of the region to be cleaned, and the region feature of the region to be cleaned can be determined according to the feature of the ROI region. For example, there is water stain in a partial region (ROI region) on the floor, a region where the water stain is located may be acquired as the ROI region by image recognition, and then specific feature extraction is performed on the ROI region.

Firstly, the target image is converted into a gray scale image to obtain the target image of the gray scale image, and the outline of the ROI area is acquired by identifying the gray scale target image. Here, the gray-scale image is processed instead of the original target image, and the amount of calculation can be reduced.

Calculating the gray level average value of pixel points contained in each line of the target image, calculating the difference value between each pixel point and the gray level average value of the line as a gray level difference value, and calculating the gray level standard deviation under each line based on the gray level difference value; and calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a first difference image according to the gray average value of each row. And determining a second differential image according to the absolute value of the gray difference between the target image and the first differential image, and performing binarization processing on the second differential image to obtain a binarized second differential image. Calculating the gray level average value of pixel points contained in each row of the target image, calculating the difference value between each pixel point and the gray level average value of the row as a gray level difference value, and calculating the gray level standard deviation of each row based on the gray level difference value; and calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a third difference image according to the gray average value of each row. And determining a fourth differential image according to the absolute value of the gray difference between the target image and the third differential image, and performing binarization processing on the fourth differential image to obtain a binarized fourth differential image. Then, carrying out fusion processing on the second differential image and the fourth differential image to obtain a fifth differential image, wherein the fifth differential image is a binary image; and finally, identifying the fifth differential image to determine the ROI area in the target image.

In this case, the difference image is acquired, and the salient processing can be performed on the part of the target image that is different from the other area, so that the ROI area in the target image can be determined by simple recognition through the difference calculation process. Generally, the floor is pure color or fixed patterns, and differences in templates which are distinguished from the floor in the target image can be simply extracted through row-column difference processing, so that the ROI area in the target image can be simply identified.

Then, further feature extraction needs to be performed on the ROI region in the target image to obtain the region features of the region to be cleaned.

Specifically, the image of the ROI region in the target image is taken as the detection image, and then only the detection image is subjected to feature recognition, so that the amount of calculation of the features of the feature extraction can be reduced.

In a specific embodiment, a plurality of preset feature templates are obtained, where a feature template is a preset feature template corresponding to a cleaning mode, and each feature template includes features that should be possessed by an image in the corresponding cleaning mode. And then extracting features in the detected image based on the feature templates, specifically, inputting the feature templates and the detected image into a preset feature extraction model to calculate the features contained in the detected image under each feature template and the confidence degrees with the features, wherein the confidence degrees represent the similarity between the feature templates and the detected image, that is, the confidence degrees of the detected image containing the corresponding feature templates. Then, one target feature template is selected from the feature templates according to the confidence degrees, and features under the feature template are used as region features extracted from the target image of the region to be cleaned.

In a specific embodiment, the detection image is input into a preset convolutional neural network model, a confidence corresponding to each feature is output, and then the feature with the confidence higher than a preset value is taken as the region feature, or the feature with the maximum confidence is taken as the region feature.

Step S102: and determining a target cleaning mode corresponding to the area to be cleaned in a plurality of preset cleaning modes according to the area characteristics.

The sweeping robot can support multiple multi-sweeping modes, such as a water stain sweeping mode, a dust sweeping mode, a hair sweeping mode, an oil stain sweeping mode and a carpet sweeping mode, and what multi-sweeping mode is adopted for each area to sweep is determined according to the area characteristics corresponding to the areas.

In this embodiment, an image of an area corresponding to each of the scanning modes is obtained in advance, and features of the image are extracted as sample features corresponding to each of the multiple scanning modes; then, in this step, according to a preset feature matching algorithm, calculating a similarity between the regional feature of the region to be cleaned and the sample feature, which is also equivalent to calculating a similarity between the two features, and then determining which cleaning mode the regional feature specifically corresponds to according to the similarity, in specific implementation, determining the cleaning mode corresponding to the sample feature with the maximum similarity as the target cleaning mode.

Step S103: determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, and planning routes of all the non-cleaned areas to obtain a first planned route.

After the target cleaning mode of the area to be cleaned is determined, the area to be cleaned can be cleaned according to the target cleaning mode. Here, path planning is also needed to be performed on the cleaning action of the area to be cleaned, so as to obtain the travel path (i.e. the first planned path) of the cleaning process.

In a specific embodiment, a path planning needs to be performed on an area to be cleaned to determine a first planned path.

In another specific embodiment, the path planning needs to consider not only the area to be cleaned, but also other areas not to be cleaned, so that the first planned path is not only locally optimal for the area to be cleaned, but also globally optimal for the path of all the areas not to be cleaned.

Specifically, a path planning is performed on an area to be cleaned and all other areas which are not cleaned, so as to obtain a first planned path, wherein the first planned path includes the path planning of the area to be cleaned and the path planning of other areas which are not cleaned.

In other embodiments, when planning the path, the power of the sweeping robot needs to be considered to determine whether the remaining power of the sweeping robot can complete the sweeping operation of the area to be swept and other areas not swept.

Specifically, mode power consumption data corresponding to each preset cleaning mode is determined, wherein the mode power consumption data can be unit power consumption corresponding to each multi-cleaning mode, and a first power consumption of an area to be cleaned is determined according to a first planned path and the mode power consumption data corresponding to a target cleaning mode; then judging whether the first power consumption is less than or equal to the residual power of the sweeping robot, if so, executing the step of controlling the sweeping robot to move according to a first planned route; if not, the sweeping robot is controlled to move to the charging seat for charging, and the sweeping work of the area to be swept is executed after the charging is finished. Therefore, the situation that the electric quantity is insufficient in the middle of sweeping of one area by the sweeping robot is avoided.

Furthermore, the power consumption of other non-cleaned areas needs to be considered. Specifically, according to the historical data, determining historical power consumption data of other non-cleaned areas except the area to be cleaned, and determining second power consumption corresponding to the other non-cleaned areas except the area to be cleaned; and the second power consumption is the power consumption required to be consumed for cleaning the corresponding area, which is evaluated according to the historical data. Determining whether the sum of the first power consumption and the second power consumption is less than or equal to the residual power of the sweeping robot; if so, indicating that the residual electric quantity of the sweeping robot can support the completion of the sweeping work of the residual non-sweeping area, and under the condition, directly executing the step of controlling the sweeping robot to move according to the first planned route; on the contrary, it is indicated that the remaining electric quantity of the sweeping robot cannot support the completion of the sweeping work of the remaining non-sweeping area, a further judgment is needed to determine whether the sweeping work of the area to be swept can be supported, and if the remaining electric quantity of the sweeping robot can support the sweeping work of the area to be swept, the sweeping robot is controlled to charge after the area to be swept is swept.

For the path planning of the area to be cleaned, the cleaning condition of the area in the historical data also needs to be considered, and for the part of the area to be cleaned which needs to be cleaned strongly, each place in the area to be cleaned can be cleaned more than a plurality of times when the path planning is carried out. Specifically, according to historical data, a thermal value of a region to be cleaned is determined, wherein the thermal value of the region is used for representing the movement speed and the power consumption value of the region in the historical data, and the movement speeds corresponding to different cleaning modes are different. That is, the movement speed during the cleaning process is different for different dirty areas and different types of areas, and the corresponding power consumption is also different. And calculating a thermal value corresponding to the area to be cleaned according to the movement speed and the power consumption in the historical data. And then determining whether the area to be cleaned needs to be cleaned repeatedly according to the thermal value, for example, determining that the area needs to be cleaned repeatedly when the thermal value is larger than a preset value, and determining the number of times that the area needs to be cleaned repeatedly according to the magnitude of the thermal value. Then, when the path planning is carried out on the area to be cleaned to obtain at least one sub-path, each point in each sub-path where planning is needed needs to be repeatedly cleaned by two or more than two points, and the area repetition rate corresponding to the area to be cleaned is larger than a preset value.

Step S104: and controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to a charging seat for charging.

That is, after the sweeping robot finishes sweeping, the sweeping robot needs to be controlled to return to the charging seat for charging, so as to wait for the next charging.

Further, in this embodiment, the area to be cleaned may be an area obtained by dividing the map into areas in the process of planning the path, and each area may be cleaned in sequence. The division of the region may be determined as follows.

The method comprises the steps of obtaining a map of the sweeping robot, wherein the map is constructed in advance according to data collected by the sweeping robot, and for example, the map is a whole-house map. Generally, the area to be cleaned includes one or more rooms. In this embodiment, before performing path planning, the map needs to be divided into a plurality of areas, where each area corresponds to one room or one functional area. Wherein each region is a rectangular shape or a plurality of rectangular connected regions.

In a specific implementation, the map is divided into grids, and the size of the grid may be determined according to the specific shape of the map and the historical data, for example, according to the size of the smallest rectangle included in the map, and for example, according to the size of the area where the obstacle is located in the historical data. Generally, the size of the grid is smaller than the size of the smallest rectangle in the map, and also smaller than the size of the obstacle, and only if the grid is small enough, the subsequent partitioning of the grid-based area can be accurate enough.

A number of base grid points are selected evenly in the grid, wherein the number of base grid points is larger than the number of the finally determined regions, and preferably much larger than the number of regions. The expansion is performed on the basis of the basic grid points, wherein the expansion is performed towards the outer layer of grid, and the expansion is performed one layer by one layer to obtain the expanded grid in the condition that the expanded grid can not be expanded any more. Here, the expansion includes not only the expansion toward the periphery based on the grid points but also the expansion in one direction (i.e., the expansion toward the outside of one side of the rectangle). The number of base grid points is sufficient to cover the entire area of the map.

If an obstacle is encountered in the expansion process, if the number of grids in the area where the obstacle is located is smaller than or equal to a preset threshold value, the expansion is continued, but if the number of grids in the area where the obstacle is located is larger than the preset threshold value, the expansion is stopped when the obstacle is encountered. After the mesh expansion, rectangular areas of multiple meshes can be obtained. It should be noted that, here, the plurality of rectangular areas obtained by expansion completely cover the area where the map is located, and if not, the base grid points need to be further increased, so that the plurality of rectangular areas obtained by expansion of the base grid points can completely cover the area where the map is located.

Here, because outward expansion of each basic grid point is performed by itself, there is an overlap between rectangular areas expanded by different basic grid points, and further processing is required for these rectangular areas to obtain the final divided areas.

If the two rectangular areas are repeated, deleting the repeated rectangular areas and only reserving one rectangular area;

if one rectangular area A is completely contained in the other rectangular area B, deleting the rectangular area A and reserving the rectangular area B;

and for other rectangular areas with overlapping areas, processing according to the following method:

and selecting one rectangular region, determining all rectangular regions with overlapped regions, traversing each overlapped rectangular region, determining the corresponding overlapped area, calculating a first ratio of the overlapped area to the area of the selected rectangular region and a second ratio between edits of the traversed rectangular region, and determining whether to combine the two rectangular regions according to whether the first ratio and the second ratio are both greater than or equal to a preset threshold value. If so, merging, and if not, deleting the intersection areas of the 2 rectangular areas from one of the rectangular areas, for example, determining the sizes of the first ratio and the second ratio, determining the rectangular area with the smaller ratio, and deleting the intersection areas from the rectangular area with the smaller ratio to obtain the rectangular area after deleting the intersection areas and the complete rectangular area with the larger ratio.

Then, the region after the processing is also regarded as a rectangular region, and the processing of the overlapping area between the rectangular regions is continued, and the processing is terminated when there is no overlapping area between the rectangular regions.

In the process, in the acquisition process of the areas, the deletion action when the intersection areas between the local areas are deleted from one area is used for determining that the time of the two areas is communicated according to the intersection relation between the areas, and then the communication relation between the areas is determined. That is, in this step, by the above-described processing of the regions, a plurality of regions are obtained, each corresponding to one room or one relatively independent region.

When the path planning is carried out, the path planning is carried out by regions, and then all the regions are connected. That is, in the present embodiment, the path is formed by respectively planning the path for each area and then connecting the areas. Then, when moving to each area, taking the area as the area to be cleaned to determine a target cleaning mode corresponding to the area to be cleaned, cleaning based on the target cleaning mode, moving to the next area after the cleaning of the area is completed, and further determining the target cleaning mode of the next area until the cleaning of all the areas is completed.

In another embodiment, there is provided a route control device of a sweeping robot with multiple sweeping modes, as shown in fig. 2, the device including:

the image feature extraction module 101 is configured to acquire a target image of an area to be cleaned through a camera device arranged on the sweeping robot and perform image recognition on the acquired target image to determine an area feature of the area to be cleaned;

a cleaning mode determining module 102, configured to determine, according to the area characteristics, a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes;

the route planning module 103 is used for determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaned areas to obtain a first planned route;

and the control sweeping module 104 is used for controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to the charging seat for charging.

Fig. 3 shows an internal structure diagram of a sweeping robot (computer device) for implementing route control of the sweeping robot of the multi-sweeping mode in one embodiment. The computer device may specifically be a terminal, and may also be a server. As shown in fig. 3, the computer device includes a processor, a memory, and a network interface connected by a system bus. The memory comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by the processor, causes the processor to carry out the above method. The internal memory may also have a computer program stored thereon, which, when executed by the processor, causes the processor to perform the method described above. Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

After the sweeping robot and the route control method and device for the sweeping robot adopting the sweeping mode are adopted, a target image of an area to be swept is acquired through a camera device arranged on the sweeping robot, and the acquired target image is subjected to image recognition so as to determine the area characteristics of the area to be swept; determining a target cleaning mode corresponding to an area to be cleaned in a plurality of preset cleaning modes according to the area characteristics; determining historical power consumption data of other non-cleaned areas except the area to be cleaned, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaned areas to obtain a first planned route; and controlling the sweeping robot to move according to a first planned route so as to sweep the area to be swept and other areas which are not swept, and after sweeping is finished, controlling the sweeping robot to move to a charging seat for charging. That is to say, feature extraction is performed according to an image of each area, then a target cleaning mode is determined in a plurality of cleaning modes based on the extracted features, then the cleaning robot is controlled to clean the area to be cleaned based on the determined target cleaning mode and the planned path, and after the cleaning is completed, the cleaning robot is controlled to return to the charging seat for charging. By adopting the invention, the specific cleaning mode of each area can be customized according to the characteristics of each area, thereby improving the cleaning efficiency.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims. Please enter the implementation details.

Claims

1. A method for route control of a multi-sweeping mode sweeping robot, the method comprising:

acquiring a target image of an area to be cleaned through a camera device arranged on the sweeping robot, and carrying out image recognition on the acquired target image so as to determine the regional characteristics of the area to be cleaned;

wherein, through setting up the camera device on the robot of sweeping the floor, acquire the target image in the region of waiting to sweep and carry out image recognition to the target image who gathers to confirm the step of the regional characteristic in the region of waiting to sweep, still include: converting the target image into a gray level image to obtain a target image of the gray level image; calculating the gray level average value of pixel points contained in each line of the target image, calculating the difference value between each pixel point and the gray level average value of the line as a gray level difference value, and calculating the gray level standard difference of each line based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a first difference image according to the gray average value of each row; determining a second differential image according to the absolute value of the gray difference between the target image and the first differential image, and performing binarization processing on the second differential image to obtain a binarized second differential image; calculating the gray level average value of pixel points contained in each row of the target image, calculating the difference value between each pixel point and the gray level average value of the row as a gray level difference value, and calculating the gray level standard deviation of each row based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a third difference image according to the gray average value of each row; determining a fourth differential image according to the absolute value of the gray difference between the target image and the third differential image, and performing binarization processing on the fourth differential image to obtain a binarized fourth differential image; fusing the second differential image and the fourth differential image to obtain a fifth differential image, wherein the fifth differential image is a binary image; identifying a fifth differential image to determine a ROI area in the target image; acquiring an image of an ROI area in the target image as a detection image; acquiring a plurality of preset feature templates corresponding to the cleaning modes, and extracting features of the detection image based on the feature templates to acquire the features and confidence coefficients of the detection image under each feature template; determining the characteristics under a characteristic template as the region characteristics according to the confidence coefficient;

acquiring a plurality of preset cleaning modes, and determining the sample characteristics of each cleaning mode; according to a preset feature matching algorithm, calculating the similarity between the regional features of the region to be cleaned and the sample features, and determining a cleaning mode corresponding to the sample features with the maximum similarity as a target cleaning mode corresponding to the region to be cleaned;

determining historical power consumption data of other non-cleaned areas except the area to be cleaned, and a target cleaning mode and mode power consumption data of the area to be cleaned; determining historical power consumption data of other non-cleaned areas except the area to be cleaned according to the historical data, and determining second power consumption corresponding to the other non-cleaned areas except the area to be cleaned;

according to the second power consumption, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, route planning is carried out on all areas which are not cleaned, and a first planned route is obtained; determining a thermal value of a region to be cleaned according to historical data, wherein the thermal value of the region is used for indicating the movement speed and the power consumption value of the region in the historical data, and the movement speeds corresponding to different cleaning modes are different; determining whether the area to be cleaned needs to be cleaned repeatedly and the number of times of cleaning needs to be repeated according to whether the thermal value is larger than a preset value, and under the condition that the area needs to be cleaned repeatedly, planning the route of the area to be cleaned to obtain a first planned route, wherein the area repetition rate of the first planned route is larger than the preset value, and any place in the area to be cleaned is cleaned repeatedly for 2 times or more in the first planned route by the cleaning robot and is equal to the number of times of cleaning needs determined according to the person leaving;

2. The method for route control of a multi-cleaning-mode sweeping robot according to claim 1, wherein the step of determining historical power consumption data of other non-cleaning areas except for the area to be cleaned, and the target cleaning mode and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaning areas to obtain the first planned route further comprises:

judging whether the first power consumption is less than or equal to the residual power of the sweeping robot, if so, executing the step of controlling the sweeping robot to move according to a first planned route; if not, the sweeping robot is controlled to move to the charging seat for charging.

3. The method for route control of a multi-cleaning-mode sweeping robot according to claim 2, wherein the step of determining historical power consumption data of other non-cleaning areas except for the area to be cleaned, and the target cleaning mode and mode power consumption data of the area to be cleaned, and performing route planning on all the non-cleaning areas to obtain the first planned route further comprises:

4. The route control method of a multi-sweeping mode sweeping robot of claim 1, further comprising:

dividing a map into a plurality of grids;

for a plurality of rectangular regions:

performing first region processing operation on the plurality of rectangular regions according to the repetition relationship and the inclusion relationship among the rectangular regions, wherein the first region processing operation comprises one or more of deduplication operation, deletion operation and combination operation;

5. A route control device of a sweeping robot with multiple sweeping modes is characterized by comprising:

the image feature extraction module is used for acquiring a target image of an area to be cleaned through a camera device arranged on the sweeping robot and carrying out image recognition on the acquired target image so as to determine the regional features of the area to be cleaned; converting the target image into a gray image to obtain a target image of the gray image; calculating the gray level average value of pixel points contained in each line of the target image, calculating the difference value between each pixel point and the gray level average value of the line as a gray level difference value, and calculating the gray level standard deviation under each line based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a first difference image according to the gray average value of each row; determining a second differential image according to the absolute value of the gray difference between the target image and the first differential image, and performing binarization processing on the second differential image to obtain a binarized second differential image; calculating the gray level average value of pixel points contained in each row of the target image, calculating the difference value between each pixel point and the gray level average value of the row as a gray level difference value, and calculating the gray level standard deviation of each row based on the gray level difference value; calculating the gray average value of each row of the pixel points with the gray values smaller than the limit error threshold determined according to the gray standard deviation, and generating a third difference image according to the gray average value of each row; determining a fourth differential image according to the absolute value of the gray difference between the target image and the third differential image, and performing binarization processing on the fourth differential image to obtain a binarized fourth differential image; fusing the second differential image and the fourth differential image to obtain a fifth differential image, wherein the fifth differential image is a binary image; identifying a fifth difference image to determine an ROI area in the target image; acquiring an image of an ROI (region of interest) in the target image as a detection image; acquiring a plurality of preset feature templates corresponding to the cleaning modes, and extracting features of the detection image based on the feature templates to acquire the features and confidence coefficients of the detection image under each feature template; determining the characteristic under a characteristic template as the area characteristic according to the confidence coefficient;

the cleaning mode determining module is used for acquiring a plurality of preset cleaning modes and determining the sample characteristics of each cleaning mode; according to a preset feature matching algorithm, calculating the similarity between the regional features of the region to be cleaned and the sample features, and determining a cleaning mode corresponding to the sample features with the maximum similarity as a target cleaning mode corresponding to the region to be cleaned;

the route planning module is used for determining historical power consumption data of other non-cleaned areas except the area to be cleaned, and a target cleaning mode and mode power consumption data of the area to be cleaned; determining historical power consumption data of other non-cleaned areas except the area to be cleaned according to the historical data, and determining second power consumption corresponding to the other non-cleaned areas except the area to be cleaned; according to the second power consumption, a target cleaning mode of the area to be cleaned and mode power consumption data of the area to be cleaned, route planning is carried out on all areas which are not cleaned, and a first planned route is obtained; the method comprises the steps that a thermal power value of a region to be cleaned is determined according to historical data, wherein the thermal power value of the region is used for representing the movement speed and the power consumption value of the region in the historical data, and the movement speeds corresponding to different cleaning modes are different; determining whether the area to be cleaned needs to be cleaned repeatedly and the number of times of cleaning needs to be repeated according to whether the thermal value is larger than a preset value, and under the condition that the area needs to be cleaned repeatedly, planning the route of the area to be cleaned to obtain a first planned route, wherein the area repetition rate of the first planned route is larger than the preset value, and any place in the area to be cleaned is cleaned repeatedly for 2 times or more in the first planned route by the cleaning robot and is equal to the number of times of cleaning needs determined according to the person leaving;

6. A sweeping robot comprising a memory and a processor, wherein the memory has executable code, and when the executable code runs on the processor, the sweeping robot performs the route control method of the sweeping robot with multiple sweeping modes according to any one of claims 1 to 4.