CN116088533A

CN116088533A - Information determination method, remote terminal, device, mower and storage medium

Info

Publication number: CN116088533A
Application number: CN202310193827.8A
Authority: CN
Inventors: 陈子冲; 吕卓; 俞天宁; 李春红
Original assignee: Weilan Continental Beijing Technology Co ltd
Current assignee: Weilan Continental Beijing Technology Co ltd
Priority date: 2022-03-24
Filing date: 2023-02-28
Publication date: 2023-05-09
Anticipated expiration: 2043-02-28
Also published as: CN116088533B

Abstract

The embodiment of the application provides an information determining method, a remote terminal, equipment, a mower and a storage medium, wherein the information determining method comprises the following steps: acquiring a to-be-processed image aiming at an area to be identified by an image acquisition device of equipment to be controlled, wherein the area to be identified at least comprises the area to be processed and an obstacle area; processing the image to be processed, and determining the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located; and controlling the equipment to be controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the part boundary so as to determine the boundary of the area to be processed, wherein the boundary of the area to be processed is used for distinguishing the area to be processed from the obstacle area. The information determination method provided by the application can solve the problems of complex operation and low efficiency of the existing method for determining the boundary of the area.

Description

Information determination method, remote terminal, device, mower and storage medium

The present application claims priority to the application of the invention having application number "202210303714.4", patent name "an information determining method, apparatus and computer readable storage medium", 24 th year 2022, and claims priority to the application of the invention having application number "202210689378.1", patent name "method, apparatus, remote terminal and storage medium for establishing virtual work boundary", 2022, 06 th month 17 th year, patent name "202210689378.1", the entire contents of which are incorporated herein by reference.

Technical Field

The present disclosure relates to positioning technologies in the field of communications, and in particular, to an information determining method, a remote terminal, a device, a mower, and a storage medium.

Background

With the continuous development of computer technology, robots are becoming more and more popular. Among them, a garden robot such as a robot mower needs to determine the boundary of a working area (including an outer boundary and an inner obstacle) before the robot can work.

Currently, conventional robotic lawnmowers are commonly used to determine passable areas by cables laid under the lawn, or non-visual self-positioning lawnmowers determine passable area boundaries by remote control routes.

However, the above method for determining the boundary of the region has the problems of complicated operation and low efficiency.

Disclosure of Invention

In view of the foregoing, embodiments of the present application provide an information determining method, a remote terminal, an apparatus, a mower, and a storage medium to at least partially solve the above-mentioned problems.

According to a first aspect of embodiments of the present application, there is provided an information determining method, including: acquiring a to-be-processed image aiming at an area to be identified by an image acquisition device of equipment to be controlled, wherein the area to be identified at least comprises the area to be processed and an obstacle area; processing the image to be processed, and determining the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located; and controlling the equipment to be controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the part boundary so as to determine the boundary of the area to be processed, wherein the boundary of the area to be processed is used for distinguishing the area to be processed from the obstacle area.

According to a second aspect of embodiments of the present application, there is provided an information determining method applied to a remote terminal for controlling movement of a device to be controlled, the information determining method including: receiving information of a part boundary sent by the equipment to be controlled, wherein the information of the part boundary is used for indicating the part boundary of a region to be processed where the equipment to be controlled is currently located, the information of the part boundary is obtained by processing an image to be processed by the equipment to be controlled, the image to be processed is obtained by image acquisition of an area to be identified by an image acquisition device of the equipment to be controlled, and the area to be identified at least comprises the area to be processed and an obstacle area; and transmitting control information to the equipment to be controlled based on the information of the part boundary so that the equipment to be controlled can be switched between an automatic movement mode and a remote control movement mode according to the control information, and determining the boundary of the area to be treated, wherein the boundary of the area to be treated is used for distinguishing the area to be treated from the obstacle area.

According to a third aspect of embodiments of the present application, there is provided a method of establishing a virtual working boundary, applied to a mowing device, the method comprising: if the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device automatically advances, and the position information of the mowing device in the automatic advancing process is recorded; if the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the mowing device advances according to the received first user instruction, and the position information of the mowing device in the advancing process under the first user instruction is recorded; and establishing the virtual working boundary according to the recorded position information.

In a fourth aspect of the embodiments of the present application, a method for establishing a virtual working boundary is provided, and the method is applied to a remote terminal, and includes: acquiring environment information in the advancing process of the mowing device, and identifying the boundary of the working range of the mowing device according to the environment information; when the working range boundary of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device is made to automatically advance, and the position information in the automatic advancing process of the mowing device is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information; and when the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, enabling the mowing device to advance according to the received first user instruction, and recording the position information of the mowing device in the advancing process of the mowing device under the first user instruction, so that the mowing device can establish a virtual working boundary according to the recorded position information in the advancing process of the mowing device under the first user instruction.

According to a fifth aspect of embodiments of the present application, there is provided an apparatus for establishing a virtual working boundary, including an autonomous module configured to, if a working range boundary of a mowing apparatus is identified according to environmental information during traveling of the mowing apparatus, cause the mowing apparatus to autonomously travel, and record position information during autonomous traveling of the mowing apparatus; the manual module is used for enabling the mowing device to travel according to the received first user instruction and recording the position information of the mowing device in the traveling process under the first user instruction if the working range boundary of the mowing device cannot be identified according to the environmental information in the traveling process of the mowing device; and the mapping module is used for establishing the virtual working boundary according to the recorded position information.

According to a sixth aspect of embodiments of the present application, there is provided an information determining apparatus, including: a processor, a memory, and a communication bus; the communication bus is used for realizing communication connection between the processor and the memory; the processor is configured to execute the information determining program in the memory to implement the steps of the information determining method according to the first aspect of the embodiment of the present application.

According to a seventh aspect of embodiments of the present application, there is provided a lawn mower comprising an information determining apparatus according to the sixth aspect of embodiments of the present application.

According to an eighth aspect of embodiments of the present application, there is provided a remote terminal comprising a memory for storing instructions and a controller for executing the instructions to implement the method according to the second or fourth aspect of embodiments of the present application.

According to a ninth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to any one of the first, second, third and fourth aspects.

According to the information determining method provided by the embodiment of the application, the image collector of the equipment to be controlled is used for collecting the image to be processed aiming at the area to be identified, the area to be identified at least comprises the area to be processed and the barrier area, the image to be processed is processed to determine the information of the part of the boundary of the area to be processed where the equipment to be controlled is currently located, then the equipment to be controlled is controlled to switch between the automatic motion mode and the remote control motion mode based on the information of the part of the boundary to determine the boundary of the area to be processed, the boundary of the area to be processed is used for distinguishing the area to be processed from the barrier area, and therefore the problem that the operation is complex and the efficiency is low in the technical scheme of determining the boundary of the area to be determined is solved by combining the automatic motion mode and the remote control motion mode based on the information of the part of the boundary.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly introduce the drawings that are required to be used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings may also be obtained according to these drawings for a person having ordinary skill in the art.

Fig. 1 is a flowchart of an information determining method provided in an embodiment of the present application;

FIG. 2 is a flow chart of another information determination method provided by an embodiment of the present application;

FIG. 3 is a flow chart of yet another information determination method provided by an embodiment of the present application;

FIG. 4 is a schematic view of a mowing apparatus according to an embodiment of the present disclosure;

fig. 5 is a flowchart of an information determining method applied to a mowing device according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of an autonomous mapping mode according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a travel path in autonomous mapping mode according to an embodiment of the present disclosure;

fig. 8 is a flowchart of another information determining method applied to a mowing device according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a remote terminal display interface according to an embodiment of the present application;

FIG. 10 is a schematic flow chart of another autonomous mapping mode according to an embodiment of the present disclosure;

FIG. 11 is a schematic diagram of a remote terminal display interface according to an embodiment of the present application;

FIG. 12 is a schematic diagram of another remote terminal display interface provided by an embodiment of the present application;

FIG. 13 is a flowchart illustrating a method for establishing virtual boundaries according to an embodiment of the present disclosure;

fig. 14 is a schematic view of a mowing device according to an embodiment of the present disclosure;

FIG. 15 is a method for establishing virtual boundaries for a remote terminal according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of an apparatus for establishing a virtual working boundary according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of an information determining apparatus according to an embodiment of the present application;

FIG. 18 is a schematic view of a mower according to an embodiment of the present application;

fig. 19 is a schematic structural diagram of a remote terminal according to an embodiment of the present application.

Detailed Description

In order to better understand the technical solutions in the embodiments of the present application, the following descriptions will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the embodiments of the present application shall fall within the scope of protection of the embodiments of the present application.

The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

The embodiment of the application provides an information determining method, which can be applied to an information determining device, and is shown with reference to fig. 1, and the method comprises the following steps:

S101, acquiring a to-be-processed image aiming at an area to be identified through an image acquisition device of equipment to be controlled.

The area to be identified at least comprises an area to be processed and an obstacle area.

In the embodiment of the present application, the device to be controlled may be a device that performs a certain process on the area to be processed; moreover, the equipment to be controlled can be intelligent equipment with a certain data processing function and an image acquisition function; in one possible implementation, the device to be controlled may refer to a mobile machine device (or mobile robot) that is mobile; preferably, the device to be controlled may comprise a garden robot, also referred to as a mower.

It should be noted that the image collector may refer to a camera on the mower; the image to be processed can be obtained by continuously photographing a region to be identified, which at least comprises a region to be processed and an obstacle region, by using a camera of the mower; that is, the image to be processed may include a plurality of images. Furthermore, the image to be processed may include an image of the boundary of the region to be processed. The area to be processed may refer to an area in which an object in the area needs to be processed; the obstacle region may refer to a region other than the region to be treated.

S102, processing the image to be processed, and determining the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located.

In the embodiment of the present application, the information determining device may refer to a device to be controlled; the information of the partial boundary may refer to the case of the partial boundary. That is, the device to be controlled may perform semantic segmentation and processing on the acquired plurality of images to be processed, and determine, based on the result of the semantic segmentation and the processing result, the situation of the partial boundary of the region where the device to be controlled is currently located in the region to be processed.

S103, controlling the equipment to be controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the part of the boundary so as to determine the boundary of the area to be processed.

Wherein the boundary of the area to be treated is used for distinguishing the area to be treated from the obstacle area.

In the embodiment of the application, the equipment to be controlled can control the equipment to be controlled to switch back and forth between the automatic motion mode and the remote control motion mode according to the situation of the part boundary represented by the information of the part boundary, so that the boundary of the area to be processed is determined; that is, the device to be controlled can determine the boundary of the region to be processed by combining the automatic motion mode and the remote control motion mode, compared with the boundary of the region to be processed determined in a single mode, the manual operation is greatly reduced, the working efficiency is improved, and the labor cost is reduced. In one possible implementation, where the device to be controlled is a lawn mower, the area to be treated may refer to the lawn to be mowed.

According to the information determining method provided by the embodiment of the invention, the image collector of the equipment to be controlled is used for collecting the image to be processed aiming at the area to be identified, the area to be identified at least comprises the area to be processed and the barrier area, the image to be processed is processed to determine the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located, then the equipment to be controlled is controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the partial boundary so as to determine the boundary of the area to be processed, the boundary of the area to be processed is used for distinguishing the area to be processed from the barrier area, and therefore the problems that the operation is complicated and the efficiency is low in a scheme of determining the boundary of the area in the related technology are solved by combining the automatic motion mode and the remote control motion mode based on the information of the partial boundary.

Based on the foregoing embodiments, embodiments of the present application provide an information determining method, referring to fig. 2, which includes the steps of:

s201, in the movement process of the equipment to be controlled, the information determining equipment collects a plurality of images to be processed aiming at the area to be identified through an image collector of the equipment to be controlled.

In the embodiment of the application, the operation object can remotely control or place the mower on any boundary of the lawn to be mowed; and in the process that the mower starts to work and the mower moves along the boundary, a plurality of images to be processed are obtained by photographing the mower in real time through a camera of the mower. The area to be identified may be a predetermined area formed centering on the mower. In one possible implementation, the area to be identified may be a square area centered on the mower. It should be noted that, because the mower is placed at the boundary of the lawn to be mowed, the area to be identified includes both the lawn and other areas other than the lawn, and the boundary of the lawn will be in the area to be identified.

It should be noted that, the camera may set the shooting angle to be the direction of the right front of the mower to the next point when shooting, that is, the shot image to be processed is certainly the image of the partial area in the area to be identified.

S202, the information determining device performs semantic segmentation and processing on the plurality of images to be processed, and determines part of boundary information based on a semantic segmentation result and a processing result.

In the embodiment of the application, the object included in each image to be processed can be determined by performing semantic segmentation on each image to be processed, and then the plurality of images to be processed are processed based on the result of the semantic segmentation, so that the situation of the partial boundary of the area to be processed where the mower is currently located is determined.

Wherein step 202 may be followed by optional step 203 or step 204.

And S203, controlling the equipment to be controlled to be switched to work in an automatic motion mode by the information determining equipment under the condition that the information of the part boundary represents that the part boundary meets the target boundary condition so as to determine the boundary of the area to be processed.

In the embodiment of the present application, the target boundary condition may be a preset condition for determining whether to switch the working mode of the mower, and is related to the boundary condition; in one possible implementation, the target boundary conditions may include a clear and simple boundary. That is, if the determined information of the obtained partial boundary indicates that the partial boundary is clear and simple, the mower can be considered to be suitable for the automatic movement mode at the moment, the working mode of the mower is switched to the automatic movement mode, and therefore the mower works in the automatic movement mode to determine the boundary of the lawn to be trimmed.

S204, under the condition that the information of the part boundary characterizes that the part boundary does not meet the target boundary condition, the information determining device controls the device to be controlled to be switched to work in a remote control movement mode so as to determine the boundary of the region to be processed.

In other embodiments of the present application, if the determined information of the obtained partial boundary indicates that the partial boundary is not clear or simple, the mower may be considered to be suitable for the remote control movement mode at this time, and then the working mode of the mower may be switched to the remote control movement mode, so that the mower works in the remote control movement mode to determine the boundary of the lawn to be trimmed. Wherein, the remote control movement mode refers to that the operation object remotely controls the mower to move along the edge to determine the boundary.

It should be noted that, in the remote control movement mode or the automatic movement mode, the mower may determine the boundary of the lawn to be trimmed based on the movement track; alternatively, in the automatic movement mode, the mower can also identify the area of the lawn to be trimmed and the obstacle area in the area to be identified based on the movement track or by adopting a semantic segmentation technology, so as to determine the boundary of the lawn to be trimmed.

Steps 205 to 206 may be performed after both steps 203 and 204.

S205, the information determining apparatus receives an operation instruction for operating an operation object of the apparatus to be controlled.

In the embodiment of the application, the operation object may refer to a user for controlling the movement of the mower when the mower is in the remote control movement mode. It should be noted that, the operation instruction may be an instruction for controlling the mower to switch to a remote control movement mode; the operating instructions may be received by the mower during the switching operation between the remote control movement mode and the automatic movement mode as determined by steps 201 to 204.

S206, the information determining device controls the device to be controlled to be switched to work in a remote control movement mode based on the operation instruction so as to determine the boundary of the region to be processed.

In the embodiment of the application, in the process that the mower is switched back and forth between the remote control movement mode and the automatic movement mode to determine the boundary of the lawn to be trimmed, a user can forcedly take over the mower to control the mower to work in the remote control movement mode. After the user forcibly takes over the mower, the boundary of the lawn to be trimmed, which is determined by the part in the automatic movement mode, can be deleted, and the mower is controlled to work in the remote control movement mode to redetermine the boundary of the deleted part so as to ensure the accuracy of the obtained boundary.

It should be noted that, in this embodiment, the descriptions of the same steps and the same content as those in other embodiments may refer to the descriptions in other embodiments, and are not repeated here.

According to the information determination method provided by the embodiment of the invention, the image to be processed of the to-be-controlled equipment, which is acquired by the image acquisition device of the to-be-controlled equipment and aims at the to-be-processed image of the to-be-identified area comprising the to-be-processed area and the obstacle area, is processed to obtain the information of the part of the boundary of the to-be-processed area where the to-be-controlled equipment is currently located, the boundary of the to-be-processed area is determined by combining the automatic movement mode and the remote control movement mode based on the information of the part of the boundary, rather than determining the boundary of the area by singly adopting a certain fixed mode, and the boundary of the area is determined without burying lines in the area, so that the problems of complex operation and lower efficiency of the scheme for determining the boundary of the area in the related technology are solved.

Based on the foregoing embodiments, embodiments of the present application provide an information determining method, referring to fig. 3, which includes the steps of:

s301, in the motion process of equipment to be controlled, the information determining equipment collects a plurality of images to be processed aiming at an area to be identified through an image collector of the equipment to be controlled.

S302, the information determining device performs semantic segmentation processing on each image to be processed, and performs matching processing on each image to be processed based on a semantic segmentation result and a target map to obtain an unmatched area.

Wherein the target map is a map of a target area in the area to be identified.

In the embodiment of the application, semantic segmentation processing can be performed on each image to be processed to determine different objects in the image to be processed, and the image to be processed after the semantic segmentation processing is mapped into a target map; and then, carrying out superposition processing on the mapped multiple images to be processed, and determining areas corresponding to different objects at the same position in the target map based on the superposition processing result (namely, obtaining the fuzzy areas of the images superposed at the same position in the target map), so as to obtain unmatched areas.

S303, the information determining device performs semantic segmentation on each image to be processed, and maps each semantically segmented image to the target map based on the grid of the target map.

In embodiments of the present application, the target map may be grid-lined; each image to be processed can be subjected to semantic segmentation processing to obtain an object in the image to be processed, and then the object to be processed after semantic segmentation can be mapped into the target map according to the corresponding relation between the pixel points of the image to be processed after semantic segmentation and the grids in the target map.

It should be noted that semantic segmentation may refer to visual semantic segmentation, and may be an interpretable class that segments an image to be processed into different semantics, each pixel will have a certain class (e.g., car, building, plant, road surface, etc.). The common semantic segmentation method is semantic segmentation based on deep learning.

S304, the information determining device carries out contour recognition on the mapped image aiming at each image to be processed to obtain a boundary to be processed.

In the embodiment of the present application, for each image to be processed, the information determining apparatus may perform contour recognition on the mapped image, and determine the boundary to be processed based on the contour recognition result.

It should be noted that, the execution sequence between S302 and S303 to 304 is not sequential; that is, S302 and S303-304 may be executed simultaneously, or S303-304 may be executed after S302 is executed, or S302 may be executed after S303-304 is executed. Also, the operation of determining the smoothness of the boundary to be processed may be performed after S304 (i.e., before S305); that is, the execution order of S302 and the execution order of S303 to 304 and the operation of determining the smoothness of the boundary to be processed may be indiscriminately.

S305, the information determining apparatus determines the information of the partial boundary based on the smoothness of the boundary to be processed, the unmatched region, and the target region.

In this embodiment of the present application, the smoothness of the boundary to be processed may be that the boundary to be processed is smoothed to obtain a smooth boundary, and a ratio of the total length of the smooth boundary to the total length of the boundary to be processed is calculated, and then determined based on a relationship between the obtained ratio and a target ratio; if the obtained ratio is not in the target ratio range, the boundary to be processed is considered to be not smooth enough, and at the moment, the information of the part of the boundary can be considered to be not simple to characterize the part of the boundary; if the obtained ratio is in the target ratio range, the information of the part boundary is considered to represent the part boundary to be smooth, and the boundary to be processed can be considered to be simple; it should be noted that the target ratio may be a ratio value predetermined based on the historical data information.

In other embodiments of the present application, all the unmatched areas are added and summed, and if the proportion of the sum value in the target area is greater than the target proportion, the information of the partial boundary is considered to represent the partial boundary as unclear; and if the proportion of the sum value to the target area is smaller than or equal to the target proportion, the information of the part boundary is considered to represent the part boundary clearly.

S306, under the condition that the information of the part boundary characterizes that the part boundary meets the target boundary condition, the information determining device controls the device to be controlled to be switched to work in an automatic motion mode so as to determine the boundary of the region to be processed.

S307, under the condition that the information of the part boundary characterizes that the part boundary does not meet the target boundary condition, the information determining device controls the device to be controlled to be switched to work in a remote control movement mode so as to determine the boundary of the region to be processed.

S308, the information determining apparatus receives an operation instruction for operating an operation object of the apparatus to be controlled.

And S309, the information determining device controls the device to be controlled to switch to work in a remote control movement mode based on the operation instruction so as to determine the boundary of the region to be processed.

In other embodiments of the present application, if there are multiple lawn areas to be trimmed, the user remotely controls the mower to traverse different lawn areas to be trimmed to generate corresponding motion tracks; then, a channel between the areas of the grass to be trimmed can be established according to the movement track.

The method for determining information provided by the application is specifically described below by taking the device to be controlled as a mowing device as an example.

For convenience of explanation and understanding, the structure and working scene of the mowing device are briefly described as follows before explaining the method. As shown in fig. 4, the mowing device 100 mainly includes a mowing knife set, a driving wheel, a controller, a communication port, a positioning unit (not shown in the figure), an environmental information acquisition unit 10, and the like. The driving wheel group can drive the mowing device to move. The mowing knife set is used for cutting grass near the mowing device. The controller is capable of controlling the mower to travel within a continuous boundary marker (e.g., boundary line) which is intended to limit the mower to a lawn or other suitable area, thereby avoiding damage to non-grass planting areas such as a fountain outside the suitable area or intrusion into the neighborhood's territory. The boundary marks are typically continuous loops around the area to be sheared. The controller is respectively connected with the driving wheel set, the mowing knife set, the communication port, the positioning unit and other electrical signals so as to control and position the mowing device. The communication port allows the mower to communicate with a remote terminal 200 (e.g., a remote terminal such as a cell phone, a remote control dedicated to the mower, etc.), etc. The term "remote terminal" as used herein refers to a terminal that is not integrally provided with the grass cutting device, but is provided outside the grass cutting device body, for example, a dedicated remote controller or a mobile phone of the grass cutting device with related application programs, iPad, etc., and a user can interact with the grass cutting device through the application programs on the remote terminal 200.

The communication port may be a near field communication port such as bluetooth or WiFi, which makes the communication delay between the mowing device and the remote terminal 200 lower, so that the efficiency of establishing the virtual working boundary can be further improved, and the time of the user is saved. Of course, a network port may be used as the communication port.

The positioning unit may be, for example, a Global Positioning System (GPS) receiver provided on the grass cutting device 100, an Ultra Wide Band (UWB) positioning tag, or a position sensor capable of providing position information of the grass cutting device 100. Taking GPS as an example, the GPS receiver may estimate positional information of the grass cutting device 100 during traveling and may provide the positional information to a controller of the grass cutting device 100 (the controller of the grass cutting device 100 will be described in detail below). The differential GPS (Differential GPS), i.e. the global positioning system with the differential correction signal added outside the normal GPS, can be used to further improve the positioning accuracy. Taking ultra-wideband positioning navigation positioning as an example, when the mowing device travels, the positioning tag of the ultra-wideband positioning navigation on the mowing device 100 can transmit an ultra-wideband signal to a cloud server provided with a positioning algorithm, and the cloud server performs position calculation by using the positioning algorithm and displays the global tag position. It should be appreciated that techniques capable of acquiring positional information during travel of mower 100 may be used with embodiments of the present application.

The environmental information collection unit 10 is used for collecting environmental information of the mowing device. It should be understood that the term "environment" is used herein with respect to a grass cutting device, and is intended to mean information about objects or objects surrounding the grass cutting device, including information about the lawn area (curve 300 in fig. 1 illustrates the actual boundaries of the lawn area) and obstacles (such as the tree, road, cement, etc. shown in fig. 1). The environmental information collection unit 10 may be any sensor capable of acquiring environmental information of the mowing device 100, such as an image sensor (e.g., a camera), a laser radar, a sonar, a Radio Frequency Identification (RFID) device, etc. As shown in fig. 1, an image sensor provided on the mowing device 100 may acquire an image of an environment in front of the mowing device 100, and a dotted-line fan-shaped area in fig. 1 indicates a range of an environment image that the image sensor can capture at a current position of the mowing device. Because the mowing apparatus 100 is constantly moving, in one or more embodiments of the present application, the environmental information collection unit 10 can acquire images in real time at fixed time intervals (e.g., every 0.1 seconds). The working range boundaries of the grass cutting device 100 in the environment, such as boundary lines of grass and non-grass in the image, are identified by an image recognition algorithm.

In addition, the grass cutting device 100 may also acquire environmental information of the grass cutting device 100 from other information sources without the environmental information collection unit 10. For example, the grass cutting device 100 may obtain an aerial photograph of the environment of the grass cutting device 100 from a *** map, a hundred degree map, or a satellite through a communication port.

The implementation process of the information determining method provided by the application will be described in detail below when the device to be controlled is the mowing device.

As shown in fig. 5, which shows a flow chart of steps of an information determination method that can be applied to an apparatus to be controlled, wherein the apparatus to be controlled comprises a mowing device. The method comprises the following steps:

s501, acquiring a to-be-processed image aiming at an area to be identified through an environment information acquisition unit of the mowing device.

S502, processing the image to be processed, and determining the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located.

S503: if the boundary of the working range of the mowing device is determined according to the information of the part boundary, the mowing device automatically advances, and position information of the mowing device in the automatic advancing process is recorded;

s504: if the boundary of the working range of the mowing device can not be determined according to the information of the part boundary, the mowing device advances according to the received first user instruction, and the position information of the mowing device in the advancing process under the first user instruction is recorded;

S505: and establishing a virtual working boundary according to the recorded position information.

It should be noted that, the image to be processed of the area to be identified includes environmental information, that is, the environmental information is an image of an environment acquired by the environment acquisition unit during the traveling of the mowing device, and on this basis, information of a part of boundaries of the area to be processed where the mowing device is currently located can be determined, where the part of boundary information is a boundary between the area to be identified and the obstacle area, for example: a parting line between lawns and non-lawns. While the operating range boundary of the grass cutting device is a boundary that can satisfy the automatic movement mode (autonomous travel) of the grass cutting device, for example: as in the clear or simple boundaries in the above embodiments.

In the embodiment of the present application, the environmental information in the traveling of the mowing device may be acquired as the image to be processed of the area to be identified using the environmental information acquisition unit 10 on the mowing device shown in fig. 4. For example, when the mowing device 100 begins to travel over a mowing area to establish a virtual working boundary, the environmental information collection unit 10 may be activated to collect environmental information while the mowing device is traveling. The environmental information collection unit 10 shown in fig. 1 may be an image sensor that is self-contained with the mowing apparatus 100. The image sensor may be disposed in front of the mowing device 100, and the number thereof may be one or more. For example, an image sensor may be disposed on the head, left side, and right side of the mower. The image sensor continuously acquires image information of the surrounding environment of the mowing device 100 as the mowing device 100 moves (a dashed-line sector area in fig. 4 indicates the range of environments in which the image sensor can capture images at the current position of the mowing device, i.e., the field of view or the viewable area of the image sensor). Since the mowing apparatus 100 is constantly moving, the environment information acquisition unit 10 can acquire images of the environment at fixed time intervals (for example, every 0.1 seconds). It will be appreciated that the grass cutting device 100 may also receive environmental information from the grass cutting device 100 from *** maps, hundred degree maps, or satellites through its communication port. In addition, the environmental information collection unit 10 of various types may be provided in combination, for example, various types of sensors such as an image sensor, a collision sensor, and a depth sensor may be provided on the mowing device, so as to fuse the detection results of the various sensors, and further improve the detection accuracy of the environmental information.

According to the embodiment of the present application, whether the operation range boundary of the mowing device 100 is included in the environmental information can be identified from the information of the partial boundary included in the acquired environmental information. For example, the mowing device 100 (including the mowing device itself or a server (e.g., a cloud server) that establishes a communication connection with the mowing device) may be equipped with an identification algorithm of a working range boundary by which whether the working range boundary of the mowing device 100 is included in the environmental information is identified. Specifically, a neural network model may be used as an algorithm for identifying the boundary of the working range of the mowing device, and for example, various neural network models such as FCN (fully convolutional neural network), U Net network structure, or deep lab may be used. The initial neural network model may be trained by pre-collecting sample data to generate a neural network model. For example, images of multiple lawns can be collected, boundaries of grass and non-grass in the images can be manually marked, the marked image data is used for training an initial neural network model, and the trained neural network model can be used as an identification algorithm for identifying the boundary of the working range of the mowing device. In addition, the existing neural network model in a mowing device can be updated, so that the recognition accuracy is further improved. For example, with the increase of mowing devices used by us households, the collected lawn data can be continuously optimized to a neural network model, and then updated into the mowing devices, so as to improve the recognition accuracy. A confidence threshold for the recognition algorithm may be set. That is, if the calculation result of the trained neural network model on the image data is smaller than the set confidence threshold, it is determined that it is not identifiable. Otherwise, if the calculation result of the trained neural network model on the image data is larger than the set confidence threshold, the neural network model is determined to be identifiable.

In operation S503, if it is determined that the working range of the mowing device is boundary, the mowing device may travel autonomously, and enter an autonomous mode, i.e., an automatic movement mode, in which a virtual working boundary is established autonomously, without requiring a user to manually remotely control the mowing device. For example, the controller of the mowing device may store instructions, and when the environment information including the operating range boundary of the mowing device is identified according to the acquired environment information, that is, when the operating range boundary of the mowing device is determined through the information of the partial boundary, the controller executes the instructions to control the mowing device to autonomously travel. And, the mowing device can record position information in the autonomous traveling process. For example, a global positioning system or an ultra-bandwidth positioning navigation system may send geographic location coordinates of the mower during autonomous travel of the mower to a positioning unit of the mower. For example, stars 401 and 403 in fig. 4 represent two geographic locations traversed by a mower during autonomous travel. The GPS receiver of the mower 100 may receive position information (e.g., latitude and longitude coordinates of each position, or position coordinates in a local position coordinate system of the mowing area) of two geographic positions that the mower 100 walks through during autonomous travel of the mower represented by

stars

401 and 403, as transmitted by the GPS system. The mowing device 100 may record location information for both geographic locations in its memory. In one or more embodiments of the present application, the environmental information may be acquired at predetermined time intervals and identify whether the environmental information includes a boundary of the operating range of the grass cutting device 100. When the working range boundary of the mowing device included in the environment information can be identified, an automatic mode of autonomously establishing a virtual working boundary can be entered.

In operation S504, when the working range boundary of the mowing device cannot be determined according to the information of the partial boundary, that is, the working range boundary of the mowing device cannot be identified according to the environmental information during the traveling of the mowing device, the mowing device may enter a manual mode in which a virtual working boundary is manually established. Proceeding according to the received first user instruction. Specifically, when the identification algorithm on the mowing device or a server communicatively coupled to the mowing device cannot identify whether the acquired environmental information includes a boundary of the operating range of the mowing device, the user may intervene in, enter a first user command, such as a command to advance, retract, forward, or turn left 30 degrees through a display interface of the remote terminal 200. For convenience of user input of the first user command, the remote terminal 200 may display a virtual joystick in a remote control direction in the display interface, so that the user may click and drag the virtual joystick to control the traveling direction and distance of the mowing device. The mowing device advances according to the received first user instruction and records the position information in the advancing process under the first user instruction. For example,

ellipses

405 and 407 in fig. 4 represent two geographical locations traveled by the mowing device according to the received first user instruction, where the location unit may obtain the location coordinates of these geographical locations during the travel. For example, a GPS receiver of the mower 100 may receive the longitude and latitude coordinates of the two geographic locations shown in

ellipses

405 and 407 of the mower 100 transmitted by the GPS system. The mowing device may record location information for both geographic locations in its memory.

In operation S505, the mowing apparatus may establish a virtual work boundary according to the recorded position information. In one or more embodiments of the present application, a virtual working boundary of a lawn may be established after a mowing device travels around the lawn to be mowed, recording position information for a plurality of travel positions. For example, the mower may map the recorded position information into a local coordinate system of the mower to obtain the virtual working boundary.

The above embodiments of the present application select to enter an autonomous mapping mode or a manual mapping mode according to the recognition result of the environmental information. By introducing the autonomous mapping mode and combining the autonomous mapping mode and the manual mapping mode, the time of a user can be saved, the working efficiency of the user can be improved, and the user experience can be improved under the condition that the precision of the established virtual working boundary is not influenced.

Fig. 6 is a schematic flow chart of an autonomous mapping mode provided in an embodiment of the present application, and fig. 7 is a schematic diagram of a travel path in the autonomous mapping mode provided in an embodiment of the present application. According to one or more embodiments of the present application, operation S503 determines a working range boundary of the mowing device according to the information of the partial boundary, that is, identifies the working range boundary of the mowing device according to the environmental information during the traveling of the mowing device, and then the mowing device autonomously travels and records the position information during the autonomous traveling of the mowing device, which may include S503a: according to the identified working range boundary, obtaining a travel path of the mowing device and S503b: the mowing device travels autonomously along the travel path. For example, the mowing apparatus 100 includes an environmental information collection unit 10 that collects environmental image information, and when the presence of the working range boundary 21 in the image is recognized from the image information collected by the environmental information collection unit 10 at the position shown in fig. 7, the travel path of the mowing apparatus can be obtained from the recognized working range boundary 21. For example, the grass cutting device 100 may plan its autonomous travel path with reference to the identified working range boundary 21. The autonomous travel path of the grass cutting device 100 may be parallel to the working range boundary line 21 and within the working range boundary line 21, always being different from the working range boundary line 21 by a distance of half the width of the grass cutting device 100 (i.e., the distance from the center of the grass cutting device to the left or right side thereof), so that the grass cutting device may be prevented from being out of the working range boundary line 21 on its outer sides (i.e., left and right sides) while traveling.

The above-described embodiments of the present application plan an autonomous travel path of a grass cutting device with reference to a working range boundary of the grass cutting device identified from environmental information. The travel path planning method of the mowing device is simple, and meanwhile, the mowing device can be ensured not to walk out of a lawn or collide with obstacles when the mowing device travels autonomously.

According to one or more embodiments of the present application, when determining a working range boundary of the mowing device according to the information of the partial boundary, that is, identifying the working range boundary of the mowing device according to the environmental information during the traveling of the mowing device, and when receiving the first user command, making the mowing device travel according to the received first user command, and recording the position information of the mowing device during the traveling under the first user command. In this embodiment, even though the working range boundary of the grass cutting device can be identified from the environmental information, if the user inputs his instruction through the remote terminal, for example, the user inputs his desired target travel position by touching a blank area in the display screen, in which case the grass cutting device may no longer travel according to the identified working range boundary, but instead travel from the current position along a predefined path (which may be, for example, a straight line) to the target travel position set by the user. For example, when the user determines from the identified working range boundaries displayed on the remote terminal: the grass cutting device may fall along a path that is a distance of half the width of the grass cutting device from the boundary of the working range, e.g. the path passes through a steep slope, the user may guide the grass cutting device around this steep slope by entering a first user command.

In one or more embodiments of the present application, the first user instruction may include a target travel position set by a user, and the control program of the mowing device may be set to travel in a straight line from the current position of the mowing device to the target travel position. The embodiment can establish a more accurate virtual working boundary, can also receive the target position set by the user at any time, establishes the virtual working boundary which better accords with the hope of the user, and improves the user experience.

In one possible implementation, when the device to be controlled, for example: and the mowing device is used for controlling the equipment to be controlled to exit the automatic movement mode when at least one of the following conditions is met:

(i) The equipment to be controlled is disconnected with a remote terminal, wherein the remote terminal is used for controlling the movement of the remote terminal;

(ii) The equipment to be controlled moves to a preset target position.

When the equipment to be controlled is disconnected from the remote terminal, the equipment to be controlled is controlled to exit from the automatic motion mode, for example: when the device to be controlled is a mowing device, and when the mowing device cannot be connected to a remote terminal (such as a mobile phone and other remote control devices), the automatic movement mode is exited.

It will be appreciated that when the device to be controlled is manually turned off, it may also be understood that the device to be controlled is disconnected from the remote terminal and that the automatic movement mode is also exited as the device to be controlled is turned off.

And exiting the automatic movement mode when the device to be controlled moves to a preset target position, for example: the device to be controlled moves to the vicinity of the starting point, and at this time, the automatic movement mode is exited due to the movement to the starting point, so that the remote control movement mode can be switched, that is, the first instruction travel of the user is received.

In the embodiment of the present application, the device to be controlled, for example: and the mowing device and the like exit from the automatic movement mode after meeting specific conditions and are switched into the remote control movement mode, so that the problem that position information cannot be recorded due to disconnection of equipment to be controlled under the condition of the automatic movement mode and the equipment to be controlled can be switched into the remote control movement mode when reaching a preset target position is avoided, the equipment to be controlled can be automatically closed to avoid influence on a determined virtual working boundary caused by repeated recording of path information, and the problems of complex operation and lower efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the remote control movement mode, if the working range boundary of the equipment to be controlled is determined according to the information of the part boundary, first indication information is sent to the remote terminal, so that the remote terminal displays a first mode switching control part through a display interface according to the first indication information, if first control information sent by the remote terminal in response to the triggering of the first mode switching control part is received, the equipment to be controlled is switched to an automatic movement mode to automatically travel, position information in the autonomous travel process of the equipment to be controlled is recorded, and the boundary of the area to be processed is established according to the recorded position information.

When the device to be controlled is in the remote control motion mode, if the working range boundary of the device to be controlled can be determined according to the information (such as boundary between grass and non-grass) of the part boundary identified from the environment information, for example: the foregoing is a simple and/or smooth part boundary meeting the requirement of switching to the automatic motion mode in the embodiment, such as the working range boundary line 21 in fig. 7, the first indication information is sent to the remote terminal, where the first indication information may play a role of notifying the remote terminal that the device to be controlled identifies the working range boundary, so that the remote terminal may display the first mode switching control according to the first indication information through the display interface, and if the switching control is triggered at this time, the remote terminal sends the first control information to the device to be controlled, so as to switch the device to be controlled to the automatic motion mode.

It should be understood that after the remote terminal receives the first indication information, if the first mode switching control is not triggered, the remote terminal will not send the first control information to the device to be controlled, that is, the device to be controlled keeps the remote control movement mode.

When the device to be controlled is switched to the automatic motion mode, autonomous traveling can be performed according to the identified boundary of the working range, and position information in the traveling process can be recorded, specifically, recording the position information includes, but is not limited to, recording the boundary through an RTK (Real-time kinematic) carrier phase difference technology, and because the device is in the autonomous traveling process, the current position information can be recorded once at intervals, and finally, all the position recording points are connected, so that the boundary of the region to be processed can be established according to the recorded position information.

It should be noted that, the device to be controlled is initially in the remote control movement mode, and may receive the manual operation instruction to move, and when the corresponding working range boundary is identified, the above operation is performed, and when the device to be controlled is in the automatic movement mode, if the working range boundary of the device to be controlled is determined according to the information of the partial boundary, the autonomous traveling is performed in the automatic movement mode.

In the embodiment of the application, after the equipment to be controlled is in the remote control movement mode and the corresponding working range boundary is determined, the first indication information is sent to the remote terminal, and the first control information sent by the remote terminal based on the first indication information is received, so that the switching from the remote control movement mode to the automatic movement mode can be realized. The switching process requires the user to trigger the first mode switching control, so that the user has the right to select and keep the remote control movement mode to move when the user does not want to enter the automatic movement mode, the effect of prompting the user to enter the automatic movement mode can be achieved, the user can control the equipment to be controlled to enter the automatic movement mode to automatically advance in time, and the problems that the scheme for determining the boundary of the area in the related technology is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the automatic motion mode, if the working range boundary of the equipment to be controlled cannot be determined according to the information of the part boundary, second indication information is sent to the remote terminal, so that the remote terminal displays the first motion control through the display interface according to the second indication information, then receives second control information sent by the remote terminal in response to the triggering of the first motion control, proceeds according to the second control information, records position information of the equipment to be controlled in the proceeding process according to the second control information, and establishes the boundary of the area to be processed according to the recorded position information.

When the device to be controlled is in the automatic motion mode, if the working range boundary of the device to be controlled cannot be determined according to the information of the part boundary identified from the environment information, for example: the boundary in the information of the part boundary is a discontinuous boundary or a zigzag boundary, and the like, second indication information is sent to the remote terminal, and the second indication information can play a role in informing the remote terminal that the equipment to be controlled cannot identify the boundary of the working range, so that the remote terminal can display the first motion control according to the second indication information and send second control information to the equipment to be controlled based on the triggering of the first motion control.

When the device to be controlled receives the second control information, the device to be controlled is switched from the automatic motion mode to the remote control motion mode and proceeds according to the second control information, and specifically, the second control information includes a movement instruction generated by manually operating the first motion control, for example: the first motion control may be a virtual rocker, and the second control information may be a movement instruction generated by manually dragging the virtual rocker. Meanwhile, position information in the advancing process is recorded, specifically, the recording of the position information includes but is not limited to recording boundaries through an RTK carrier phase difference technology, the current position information is recorded once at regular time intervals, and finally all position recording points are connected, so that the boundary of the area to be processed is established according to the recorded position information.

It should be noted that, since the device to be controlled includes the automatic movement mode and/or the remote movement mode during the traveling process, the boundary of the complete area to be processed may be determined according to the position information recorded in the automatic movement mode and/or the position information recorded in the remote movement mode.

In the embodiment of the application, when the equipment to be controlled is in the automatic movement mode, if the corresponding working range boundary cannot be determined, the second indication information is sent to the remote terminal, and the second control information sent by the remote terminal based on the second indication information is received, so that the equipment to be controlled can be switched from the automatic movement mode to the remote movement mode, and can travel according to the operation instruction of the user under the condition that the condition of the automatic movement mode is not met, the condition that the equipment to be controlled cannot autonomously travel and stops in situ is avoided, the determined boundary of the area to be processed is more reliable, and the problems that the scheme for determining the boundary of the area in the related technology is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the remote control movement mode, if third control information sent by the remote terminal in response to the triggering of the second mode switching control is received, the equipment is switched to the automatic movement mode to automatically travel and record position information in the automatic travel process of the equipment to be controlled, wherein the second mode switching control is displayed through a display interface after the working range boundary of the equipment to be controlled is determined by the remote terminal, and the boundary of the area to be processed is established according to the recorded position information.

In the embodiment of the application, when the equipment to be controlled is in the remote control movement mode, if third control information sent by the remote terminal is received, the equipment is switched to the automatic movement mode, and the switching from the remote control movement mode to the automatic movement mode is realized. The third control information is that after the remote terminal identifies the boundary of the working range according to the information of the partial boundary, the second mode switching control is triggered to be sent, the process uses the remote terminal to identify the boundary of the working range, equipment to be controlled is not required to identify the boundary of the working range, therefore, the cost of a processor of the equipment to be controlled can be reduced, the second mode switching control is required to be triggered during switching, a user has the selected right, namely, the user can select to keep the remote control movement mode to move when the user does not want to enter the automatic movement mode, the user is prompted to enter the automatic movement mode, the user can control the equipment to be controlled to enter the automatic movement mode to automatically advance in time, and the problems that the scheme for determining the boundary of the area in the related technology is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the device to be controlled is in the automatic motion mode, if fourth control information sent by the remote terminal in response to the triggering of the second motion control is received, the device to be controlled proceeds according to the fourth control information, records position information in the proceeding process of the device to be controlled according to the fourth control information, and establishes a boundary of the region to be processed according to the recorded position information.

In the embodiment of the application, when the equipment to be controlled is in the automatic motion mode, if fourth control information sent by the remote terminal is received, the equipment is switched to the remote motion mode, and the switching from the automatic motion mode to the remote motion mode is realized. The fourth control information is that the remote terminal cannot identify the boundary of the working range according to the information of the partial boundary, and the second motion control is triggered to be sent, the process uses the remote terminal to identify the boundary of the working range, equipment to be controlled is not required to identify the boundary of the working range, therefore, the cost of a processor of the equipment to be controlled can be reduced, the equipment to be controlled can travel according to the operation instruction of a user under the condition that the condition of the automatic motion mode is not met, the situation that the equipment to be controlled cannot autonomously travel and stop in situ when the condition of the automatic motion mode is not met is avoided, the boundary of a determined area to be processed is more reliable, and the problems that the scheme for determining the boundary of the area in the related technology has complex operation and low efficiency are solved.

Another aspect of the present application provides an information determining method that may be used for a remote terminal, the method comprising: and receiving the information of the part of the boundary sent by the equipment to be controlled, and sending control information to the equipment to be controlled based on the information of the part of the boundary, so that the equipment to be controlled is switched between an automatic motion mode and a remote control motion mode according to the control information, and determining the boundary of the area to be treated, wherein the boundary of the area to be treated is used for distinguishing the area to be treated from the obstacle area.

In this embodiment of the present application, the information of the partial boundary is used to indicate a partial boundary of a to-be-processed area where the to-be-controlled device is currently located, the information of the partial boundary is obtained by processing an to-be-processed image by the to-be-controlled device, where the to-be-processed image is obtained by performing image acquisition on an to-be-identified area by an image acquirer of the to-be-controlled device, and the to-be-identified area at least includes the to-be-processed area and the obstacle area.

The remote terminal in the embodiment of the application comprises, but is not limited to, a mobile phone, a touch remote controller, a remote control device and the like.

In the embodiment of the application, the remote terminal receives the information of the part boundary sent by the equipment to be controlled, and sends the control signal to the equipment to be controlled based on the information of the part boundary, so that the equipment to be controlled is controlled to switch between an automatic movement mode and a remote control movement mode, the equipment to be controlled can automatically advance under the condition that the condition is met, the equipment to be controlled is not required to be manually controlled in the whole process, the labor intensity of manpower is reduced, and the equipment to be controlled is switched to the remote control movement mode under the condition that the condition is not met, so that the equipment to be controlled can receive a user instruction to advance under the condition that the equipment to be controlled faces a complex boundary, and the boundary of a region to be processed can be determined.

In the following, a method for determining information that may be used for a remote terminal will be specifically described by taking a device to be controlled as a mowing device, where an application program may be stored on the remote terminal 200 as shown in fig. 4, and the remote terminal 200 executes the application program to implement the method for establishing a virtual boundary. As shown in fig. 8, the method includes:

s801: acquiring environment information in the advancing process of the mowing device, and determining the boundary of the working range of the mowing device according to the information of the part boundary included in the environment information;

s802: when the working range boundary of the mowing device is determined according to the information of the part boundary, the mowing device is made to automatically travel, the position information of the mowing device in the automatic traveling process is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information, and the flow is ended;

s803: when the working range boundary of the mowing device cannot be determined according to the information of the part boundary, the mowing device is made to travel according to the received first user instruction, and the position information of the mowing device in the traveling process under the first user instruction is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information.

The environmental information in the travel of the mower may be collected by the mower's own environmental information collection unit 10 (e.g. image sensor, sonar, RFID, etc. any sensor capable of acquiring environmental information around the mower 100). The remote terminal 200 itself may also be used as a sensor such as a camera on a cell phone or a LiDAR (Light Identification Detection And Ranging light identification detection and ranging) sensor. Environmental information of the grass cutting device 100 may also be obtained from *** maps, hundred degree maps, or satellite. The remote terminal 200, such as a cell phone or a remote control of the mowing device, may acquire environmental information of the mowing device 100 through a communication interface (e.g., a bluetooth interface, etc.) thereof. According to one or more embodiments of the present application, environmental information of the grass cutting device 100 may be acquired at predetermined time intervals. For example, image information of the environment in front of the grass cutting device 100 may be acquired by an image acquisition unit on the grass cutting device 100. The image capturing unit may acquire image information of the environment in front of the mowing device 100 at predetermined time intervals (e.g., every 0.1 seconds), and the mowing device 100 may transmit the image information to the remote terminal 200 through a near field communication port such as WiFi. The remote terminal 200 may recognize whether the operating range boundary of the grass cutting device is included in the environment information according to the acquired environment information. The remote terminal 200 may be equipped with an identification algorithm by which it is identified whether the operating range boundaries of the grass cutting device are included in the environmental information. Specifically, the neural network model may be used as an identification algorithm for the boundary of the working range of the mowing device, and sample data may be collected in advance to train the initial neural network model to generate the neural network model. For example, images of multiple lawns can be collected, boundaries of grass and non-grass in the images can be manually marked, the marked image data are placed on a server to train an initial neural network model, and the trained neural network model can be used as an identification algorithm for identifying the boundary of the working range of the mowing device. When the remote terminal 200 can recognize the operation range boundary of the grass cutting device 100 from the environmental information through the recognition algorithm in operation S802, the remote terminal 200 may transmit an instruction to the grass cutting device 100 to make the grass cutting device 100 travel autonomously and record the position information during autonomous travel. For example, the grass cutting device 100 may be made to travel along an autonomously planned path and its position coordinates during autonomous travel recorded, so that the grass cutting device 100 can establish a virtual working boundary according to the recorded position information.

In operation S803, when the remote terminal 200 cannot identify the operation range boundary of the grass cutting device from the environmental information through the identification algorithm, the remote terminal 200 may transmit an instruction to the grass cutting device to cause the grass cutting device 100 to enter a manual mode in which a virtual operation boundary is manually established. Proceeding according to the received first user instruction. Specifically, when the remote terminal 200 cannot recognize whether the acquired environmental information includes the operating range boundary of the mowing device by using the recognition algorithm, the user may intervene, and send a first user instruction to the mowing device through the remote terminal 200, for example, the user may input an instruction to make the mowing device advance, retract, forward or backward through the display interface of the remote terminal 200. After receiving the first user command, the mowing device proceeds according to the received first user command and records the position information in the proceeding process under the first user command, so that the mowing device 100 can establish a virtual working boundary according to the recorded position information.

It will be appreciated that the virtual work boundary established in operation S802 and the virtual work boundary established in operation S803 of the grass cutting device 100 together form a complete virtual work boundary of the grass cutting device 100.

As shown in fig. 9, in performing the above operations, the display interface of the remote terminal 200 may display a variety of interaction information of the lawn mower and the user. In addition to displaying the identified working range boundaries 21, the display interface may also display historical trajectories 23 of the mower to facilitate a user's knowledge of the overall virtual working boundaries of the mower, as shown in fig. 7, and the user may manually manipulate the path of travel of the mower when the overall virtual working boundaries do not meet the user's expectations. The display interface may also include a virtual joystick 29 for remote control of direction and a virtual joystick 27 for remote control of throttle. The user may enter a command to advance, retract, forward or left the grass cutting device 100 by clicking or dragging the virtual joystick 29 in the remote control direction, or the like, i.e., to set the first user command. The user can also set the speed of the mower 100, etc. by clicking or dragging the virtual joystick 27 which remotely controls the throttle. The display interface may also include an environmental range 30 that the environmental information collection unit is currently capable of sensing, an exit button 31, a rollback erasure history track button 33, and an auto-recognize working range boundary activation switch 25. The display interface can enhance interaction between a user and the mowing device, and provides a user with a choice for enabling the mowing device to automatically enter a self-construction mode of the mowing device, so that time for the user to control the mowing device to manually set a travel path of the mowing device can be saved to the greatest extent, work efficiency of the user is improved, and user experience is improved.

As shown in fig. 10, according to one or more embodiments of the present method, when the remote terminal identifies the operating range boundary of the mowing device, the remote terminal may S804: the identified working range boundaries are displayed to the user. The remote terminal may also S805: the user is prompted to enter an automatic mode where virtual work boundaries are established autonomously.

As shown in fig. 7, the remote terminal may display the identified operating range boundary 21 through its display screen. The mowing device can plan its autonomous travel path with reference to the identified working range boundary 21. For example, the autonomous travel path of the mower may be parallel to the working range boundary line and within the working range boundary line 21, always a distance of half the mower width from the working range boundary. Displaying the working range boundary 21 by the remote terminal may make the user aware of the travel path of the mowing device. Thus, if the working range boundary line does not match the user's expectations, i.e. the user does not want the grass cutting device to travel according to the working range boundary 21, for example when the working range boundary 21 is further 1 meter from a fountain in a lawn, the user may manually set the target travel position of the user's own grass cutting device if he wishes to reserve grass within 1 meter of the fountain. The mowing device may travel from a current position, such as along a straight line, to a target travel position set by a user.

According to one or more embodiments of the present application, in response to a target travel position of the mowing device manually set by a user, a travel path from the mowing device to the target travel position may be displayed to the user on a display interface of the mobile terminal. This facilitates the user to see if the travel path meets the user's expectations and if not, the user can modify the travel path.

When the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the remote terminal can also prompt the user to enter an automatic mode of autonomously establishing the virtual working boundary through a display screen (for example, through a dialog box interacted with the user). By means of the explicit prompt, the user can more easily see or enter an automatic mode for autonomously establishing the virtual working boundary, and the user does not see the displayed working range boundary or can enter the automatic mode without knowing the displayed working range boundary on the remote terminal, and the mowing device can be manually controlled to travel.

Prompting the user to enter an automatic mode that autonomously establishes virtual work boundaries may also provide the user with an opportunity to select an automatic mode and a manual mode. Although the user may preferably make the mower travel autonomously when the remote terminal recognizes the boundary of the working range of the mower, and establish the virtual working boundary according to the position information during autonomous travel of the mower, the user may choose whether to enter the automatic mode according to his own intention or hope when the remote terminal prompts that the user can enter the automatic mode for autonomously establishing the virtual working boundary. The method gives more degrees of freedom to the user, enables the established virtual working boundary to more accord with the requirements or hopes of the user, and improves the user experience.

According to the embodiment of the application, when the working range boundary of the mowing device is determined according to the information of the part boundary, namely, the working range boundary of the mowing device is identified according to the environmental information in the advancing process of the mowing device, a user is reminded to enter an automatic mode for autonomously establishing the virtual working boundary. Similarly, when the working range boundary of the mowing device cannot be determined according to the information of the part boundary, namely, the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the user is reminded to enter a manual mode of manually establishing a virtual working boundary. I.e. to alert the user that an automatic mode or a manual mode can be entered before the grass cutting device is switched between the automatic mode and the manual mode. The remote terminal may include a user feedback unit that works with sound, light, electrical signals, etc., to give a prompt or reminder to the user. For example, a visual signal may be generated by a remote terminal display screen, an audible signal may be generated by a speaker on the remote terminal or mower, and a vibration alert may be generated by a tactile signal, such as a vibration unit. The embodiment can enhance interaction between the user and the mowing device and improve user experience. Meanwhile, when the boundary of the working range of the mowing device is identified, the user is clearly reminded, so that the user can utilize the autonomous mapping mode as much as possible, the time for the user to control the mowing device is saved, and the working efficiency of the user is improved.

In one possible implementation, when the remote terminal detects that the device to be controlled satisfies at least one of the following conditions, the automatic motion mode control interface is exited:

(i) The remote terminal is disconnected with the equipment to be controlled;

(ii) The equipment to be controlled moves to a preset target position.

When the remote terminal and the device to be controlled are disconnected, including but not limited to: and when any condition is met, the equipment to be controlled exits the automatic motion mode, and meanwhile, the remote terminal exits a control interface of the automatic motion mode, for example: and displaying a dialog box on the display interface to prompt the user that the equipment to be controlled has exited the automatic motion mode, wherein corresponding operation buttons can be displayed at the moment, so that the user can control the equipment to be controlled to travel through the operation buttons.

In the embodiment of the application, when the remote terminal detects the device to be controlled, for example: after the mowing device meets specific conditions, the remote terminal exits the automatic movement mode control interface and switches to the remote movement mode control interface, so that the situation that position information cannot be recorded due to disconnection of equipment to be controlled under the condition that the equipment to be controlled is in an automatic movement mode is avoided, and when the equipment to be controlled reaches a preset target position, the equipment to be controlled can be switched to the remote movement mode, so that the boundary can be closed by itself, or the equipment to be controlled can continue to advance according to a remote control instruction of a user, the influence on a determined virtual working boundary caused by repeated recording of path information is avoided, and the user can be reminded of the fact that the equipment to be controlled is disconnected or reaches the preset position by timely exiting the automatic movement control interface, and the problems of complex operation and low efficiency are solved.

In addition, the method for establishing the virtual boundary in the above embodiment is used on the remote terminal, and can utilize a larger computing power (such as a computing power of a mobile phone CPU) of the remote terminal. Thus, the mowing device can be provided with the processor with smaller calculation force, and the cost of the mowing device can be further reduced.

In one possible implementation manner, if first indication information sent by the device to be controlled is received, displaying a first mode switching control through a display interface, wherein the first indication information is sent by the device to be controlled when the following conditions are met: the equipment to be controlled is in a remote control movement mode, the boundary of the working range of the equipment to be controlled is determined according to the information of the part boundary, then first control information is sent to the equipment to be controlled in response to the triggering of a first mode switching control, so that the equipment to be controlled is switched from the remote control movement mode to an automatic movement mode to perform autonomous travelling according to the first control information, and the position information of the equipment to be controlled in the autonomous travelling process is recorded.

And after the remote terminal receives the first indication information sent by the equipment to be controlled, displaying a first mode switching control on the display interface, wherein the first mode switching control can be a dialog box popped up above the display interface, a prompt box popped up in a full screen mode of the display interface, or the like. When the user triggers the first mode switch control, for example: and the user clicks a confirmation button on the first mode switching control and the like to send first control information to the equipment to be controlled so as to enable the equipment to be controlled to be switched to an automatic motion mode. Optionally, when the first mode switching control is triggered, a motion control for remotely controlling the device to be controlled is hidden on a display interface of the remote terminal.

As shown in fig. 11 and fig. 12, fig. 11 is a diagram illustrating content displayed on a display interface after a remote terminal receives first indication information sent by a device to be controlled, where the content includes a first mode switching control 1101, a confirm button 1102 is included on the first mode switching control 1101, a motion control 1103, a device to be controlled 1104, a part of boundary information 1105, a motion track 1106 of the device to be controlled, and fig. 12 is content displayed on the remote terminal interface when entering an automatic motion mode, where the content includes a device to be controlled 1201, a motion track 1202 of the device to be controlled, an exit control 1203, and a part of boundary information 1204.

Optionally, if the device to be controlled enters the automatic movement mode for the first time, after the first mode switching control is triggered, notes and operation guidance of the automatic movement mode can be popped up on the display interface, so that a user can conveniently and quickly learn the control of the automatic movement mode.

In the embodiment of the application, the first indication information sent by the equipment to be controlled is received, the first mode switching control is displayed based on the first indication information, and the first control information is sent to the equipment to be controlled according to the triggering of the first mode switching control, so that the equipment to be controlled can be controlled is controlled to be switched from the remote control movement mode to the automatic movement mode.

In one possible implementation manner, if second indication information sent by the device to be controlled is received, displaying the first motion control through the display interface, wherein the second indication information is sent by the device to be controlled when the following conditions are met: the equipment to be controlled is in an automatic motion mode, the boundary of the working range of the equipment to be controlled cannot be determined according to the information of the partial boundary, then second control information is sent to the equipment to be controlled in response to the triggering of the first motion control, so that the equipment to be controlled advances according to the second control information, and position information of the equipment to be controlled in the advancing process according to the second control information is recorded.

When the remote terminal receives the second indication information sent by the equipment to be controlled, displaying a first motion control on a display interface, wherein the first motion control is a control capable of remotely controlling the equipment to be controlled, for example: virtual rockers, directional control buttons, and the like. When the user triggers the first motion control, for example: and the user drags the virtual rocker and the like to send second control information to the equipment to be controlled so as to enable the equipment to be controlled to be switched from the automatic movement mode to the remote control movement mode, and the equipment to be controlled moves according to remote control operation of the user included in the second control information.

For example, as shown in fig. 11, the interface shown in the figure is a display interface corresponding to the remote control movement mode, and the movement control 1103 in the figure is a first movement control displayed on the display interface by the remote terminal in the remote control movement mode.

In the embodiment of the application, the second indication information sent by the equipment to be controlled is received, the second control information is sent to the equipment to be controlled based on the second indication information and the first motion control displayed on the display interface, and based on the triggering condition of the first motion control, so that the equipment to be controlled can be controlled is controlled to be switched from the automatic motion mode to the remote control motion mode, the equipment to be controlled can advance according to the operation instruction of the user under the condition that the automatic motion mode condition is not met, the situation that the equipment to be controlled cannot advance independently and then stops in situ due to the condition that the automatic motion mode condition is not met is avoided, the determined boundary of the area to be processed is more reliable, and the problems that the scheme for determining the boundary of the area in the related art is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the remote control movement mode, if the working range boundary of the equipment to be controlled is determined according to the information of the part boundary, displaying a second mode switching control through a display interface, then responding to the triggering of the second mode switching control, sending third control information to the equipment to be controlled, so that the equipment to be controlled is switched to the automatic movement mode according to the third control information to perform autonomous movement, and recording the position information of the equipment to be controlled in the autonomous movement process.

The control process of switching the remote control movement mode to the automatic movement mode in the embodiment of the present application is similar to the control process of switching the remote control movement mode to the automatic movement mode in the above embodiment, and is not described herein.

In the embodiment of the application, after the remote terminal identifies the boundary of the working range of the equipment to be controlled, and when the second mode switching control is triggered, third control information is sent to the equipment to be controlled, so that the equipment to be controlled is controlled to be switched from the remote control movement mode to the automatic movement mode. The third control information is sent when the remote terminal recognizes the boundary of the working range according to the information of the part boundary and the second mode switching control is triggered, the remote terminal is used for recognizing the boundary of the working range, the equipment to be controlled does not need to recognize the boundary of the working range, therefore, the cost of a processor of the equipment to be controlled can be reduced, the second mode switching control is required to be triggered during switching, a user has the selected right, namely, the user can select to keep the remote control movement mode to move when the user does not want to enter the automatic movement mode, the user can enter the automatic movement mode through the second mode switching control, the user can control the equipment to be controlled to enter the automatic movement mode to automatically advance in time, and the problems that the scheme for determining the boundary of the area in the related technology is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the automatic motion mode, if the working range boundary of the equipment to be controlled cannot be determined according to the information of the part boundary, displaying a second motion control through a display interface, and then responding to the triggering of the second motion control, sending fourth control information to the equipment to be controlled, so that the equipment to be controlled advances according to the fourth control information, and recording position information of the equipment to be controlled in the advancing process according to the fourth control information.

The control process of switching the automatic movement mode to the remote control movement mode in the embodiment of the present application is similar to the control process of switching the automatic movement mode to the remote control movement mode in the above embodiment, and is not described herein.

In the embodiment of the application, when the remote terminal cannot identify the boundary of the working range of the equipment to be controlled, the second motion control is displayed, and fourth control information is sent according to the triggering condition of the second motion control, so that the equipment to be controlled is controlled to be switched from the automatic motion mode to the remote control motion mode. The fourth control information is that the remote terminal cannot identify the boundary of the working range according to the information of the partial boundary, and the second motion control is triggered to be sent, the process uses the remote terminal to identify the boundary of the working range, the boundary of the working range is not required to be identified through the equipment to be controlled, so that the cost of a processor of the equipment to be controlled can be reduced, the equipment to be controlled can be controlled to advance according to the operation instruction of a user under the condition that the condition of the automatic motion mode is not met, the situation that the equipment to be controlled cannot automatically advance to stop in situ when the condition of the automatic motion mode is not met is avoided, the determined boundary of the area to be processed is more reliable, and the problems that the scheme for determining the boundary of the area in the related art is complex in operation and low in efficiency are solved.

In one possible implementation manner, when the equipment to be controlled is in the automatic motion mode, the exit control is triggered, the third motion control is displayed through the display interface, wherein the exit control is displayed on the display interface, and fifth control information is sent to the equipment to be controlled in response to the third motion control being triggered, so that the equipment to be controlled advances according to the fifth control information, and position information of the equipment to be controlled in the advancing process according to the fifth control information is recorded.

As shown in fig. 12, when the device to be controlled is in the automatic motion mode, an exit control 1203 is displayed on the display interface, and if the exit control is triggered, the device to be controlled switches from the automatic motion mode to the remote control motion mode, and displays a third motion control on the display interface, and sends fifth control information to the device to be controlled based on the triggering of the third motion control, where the fifth control information includes a remote control command for remotely controlling the device to be controlled by a user, so that the device to be controlled can travel according to the fifth control information.

In the embodiment of the application, after the exit control displayed on the display interface is triggered, the third motion control is displayed, and fifth control information is sent according to the triggering of the third motion control, so that the equipment to be controlled is controlled to be switched from the automatic motion mode to the remote control motion mode. The automatic movement mode can be exited by triggering the exit control, so that the user can take over the equipment to be controlled at any time, namely, the user can exit the automatic movement mode at any time, so that the user has higher degree of freedom, the determined boundary of the area to be processed is more reliable, the requirements of the user are met, and the user experience is improved.

Another embodiment of the present application provides a method for establishing a virtual working boundary, which is applied to the mowing device 100 in the above embodiment.

As shown in fig. 13, a step flow diagram of a method of establishing a virtual boundary is shown, which may be applied to a mowing device. The method comprises the following steps:

s1301: if the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device automatically advances, and the position information of the mowing device in the automatic advancing process is recorded;

s1302: if the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the mowing device advances according to the received first user instruction, and the position information of the mowing device in the advancing process under the first user instruction is recorded;

s1303: and establishing a virtual working boundary according to the recorded position information.

In an embodiment of the present application, the environmental information on the mower illustrated in fig. 4 may be acquired by the environmental information acquisition unit 10 on the mower. For example, when the mowing device 100 begins to travel over a mowing area to establish a virtual working boundary, the environmental information collection unit 10 may be activated to collect environmental information while the mowing device is traveling. The environmental information collection unit 10 shown in fig. 1 may be an image sensor that is self-contained with the mowing apparatus 100. The image sensor may be disposed in front of the mowing device 100, and the number thereof may be one or more. For example, an image sensor may be disposed on the head, left side, and right side of the mower. The image sensor continuously acquires image information of the surrounding environment of the mowing device 100 as the mowing device 100 moves (a dashed-line sector area in fig. 4 indicates the range of environments in which the image sensor can capture images at the current position of the mowing device, i.e., the field of view or the viewable area of the image sensor). Since the mowing apparatus 100 is constantly moving, the environment information acquisition unit 10 can acquire images of the environment at fixed time intervals (for example, every 0.1 seconds). It will be appreciated that the grass cutting device 100 may also receive environmental information from the grass cutting device 100 from *** maps, hundred degree maps, or satellites through its communication port. In addition, the environmental information collection unit 10 of various types may be provided in combination, for example, various types of sensors such as an image sensor, a collision sensor, and a depth sensor may be provided on the mowing device, so as to fuse the detection results of the various sensors, and further improve the detection accuracy of the environmental information.

According to the embodiment of the present application, whether the operating range boundary of the mowing device 100 is included in the environmental information can be identified according to the acquired environmental information. For example, the mowing device 100 (including the mowing device itself or a server (e.g., a cloud server) that establishes a communication connection with the mowing device) may be equipped with an identification algorithm of a working range boundary by which whether the working range boundary of the mowing device 100 is included in the environmental information is identified. Specifically, a neural network model may be used as an algorithm for identifying the boundary of the working range of the mowing device, and for example, various neural network models such as FCN (fully convolutional neural network), U Net network structure, or deep lab may be used. The initial neural network model may be trained by pre-collecting sample data to generate a neural network model. For example, images of multiple lawns can be collected, boundaries of grass and non-grass in the images can be manually marked, the marked image data is used for training an initial neural network model, and the trained neural network model can be used as an identification algorithm for identifying the boundary of the working range of the mowing device. In addition, the existing neural network model in a mowing device can be updated, so that the recognition accuracy is further improved. For example, with the increase of mowing devices used by us households, the collected lawn data can be continuously optimized to a neural network model, and then updated into the mowing devices, so as to improve the recognition accuracy. A confidence threshold for the recognition algorithm may be set. That is, if the calculation result of the trained neural network model on the image data is smaller than the set confidence threshold, it is determined that it is not identifiable. Otherwise, if the calculation result of the trained neural network model on the image data is larger than the set confidence threshold, the neural network model is determined to be identifiable.

In operation S1301, if a working range boundary of the mowing apparatus is identified, for example, a boundary between grass and an obstacle exists in the environmental information, the mowing apparatus may autonomously travel, enter an autonomous mode in which a virtual working boundary is autonomously established, and a user does not need to manually remotely control the mowing apparatus. For example, instructions may be stored in a controller of the mower, and when the environmental information is identified from the acquired environmental information as including a boundary of a working range of the mower, execution of the instructions by the controller may control the mower to travel autonomously. And, the mowing device can record position information in the autonomous traveling process. For example, a global positioning system or an ultra-bandwidth positioning navigation system may send geographic location coordinates of the mower during autonomous travel of the mower to a positioning unit of the mower. For example, stars 401 and 403 in fig. 4 represent two geographic locations traversed by a mower during autonomous travel. The GPS receiver of the mower 100 may receive position information (e.g., latitude and longitude coordinates of each position, or position coordinates in a local position coordinate system of the mowing area) of two geographic positions that the mower 100 walks through during autonomous travel of the mower represented by

stars

In operation S1302, if the working range boundary of the mowing device cannot be identified according to the environmental information during the traveling of the mowing device, the mowing device may enter a manual mode in which a virtual working boundary is manually established. Proceeding according to the received first user instruction. Specifically, when the identification algorithm on the mowing device or a server communicatively coupled to the mowing device cannot identify whether the acquired environmental information includes a boundary of the operating range of the mowing device, the user may intervene in, enter a first user command, such as a command to advance, retract, forward, or turn left 30 degrees through a display interface of the remote terminal 200. For convenience of user input of the first user command, the remote terminal 200 may display a virtual joystick in a remote control direction in the display interface, so that the user may click and drag the virtual joystick to control the traveling direction and distance of the mowing device. The mowing device advances according to the received first user instruction and records the position information in the advancing process under the first user instruction. For example,

ellipses

In operation S1303, the mowing apparatus may establish a virtual work boundary according to the recorded position information. In one or more embodiments of the present application, a virtual working boundary of a lawn may be established after a mowing device travels around the lawn to be mowed, recording position information for a plurality of travel positions. For example, the mower may map the recorded position information into a local coordinate system of the mower to obtain the virtual working boundary.

As shown in fig. 6 and 7, according to one or more embodiments of the present application, if operation S1301 identifies a working range boundary of the mowing device according to environmental information during traveling of the mowing device, the mowing device autonomously travels and records position information during autonomous traveling of the mowing device, which may include S503a: according to the identified working range boundary, obtaining a travel path of the mowing device and S503b: the mowing device travels autonomously along the travel path. For example, the mowing apparatus 100 includes an environmental information collection unit 10 that collects environmental image information, and when the presence of the working range boundary 21 in the image is recognized from the image information collected by the environmental information collection unit 10 at the position shown in fig. 7, the travel path of the mowing apparatus can be obtained from the recognized working range boundary 21. For example, the grass cutting device 100 may plan its autonomous travel path with reference to the identified working range boundary 21. The autonomous travel path of the grass cutting device 100 may be parallel to the working range boundary line 21 and within the working range boundary line 21, always being different from the working range boundary line 21 by a distance of half the width of the grass cutting device 100 (i.e., the distance from the center of the grass cutting device to the left or right side thereof), so that the grass cutting device may be prevented from being out of the working range boundary line 21 on its outer sides (i.e., left and right sides) while traveling.

According to one or more embodiments of the present application, when a working range boundary of a grass cutting device is identified according to environmental information during traveling of the grass cutting device and a first user instruction is received, the grass cutting device is made to travel according to the received first user instruction, and position information of the grass cutting device during traveling under the first user instruction is recorded. In this embodiment, even though the working range boundary of the grass cutting device can be identified from the environmental information, if the user inputs his instruction through the remote terminal, for example, the user inputs his desired target travel position by touching a blank area in the display screen, in which case the grass cutting device may no longer travel according to the identified working range boundary, but instead travel from the current position along a predefined path (which may be, for example, a straight line) to the target travel position set by the user. For example, when the user determines from the identified working range boundaries displayed on the remote terminal: the grass cutting device may fall along a path that is a distance of half the width of the grass cutting device from the boundary of the working range, e.g. the path passes through a steep slope, the user may guide the grass cutting device around this steep slope by entering a first user command.

The method of establishing a virtual working boundary in the above embodiment is used on, i.e. performed by, a mowing apparatus, and as shown in fig. 14, the mowing apparatus 100 may include a memory 101 and a processor 103. The memory 101 stores instructions for implementing the method for establishing virtual working boundaries described above, which, when executed by the processor 103, may be implemented by the mowing device 100. Compared with the mode of identifying grass and non-grass boundaries by the remote terminal and establishing a virtual working boundary according to the recorded position information, the mode implemented by the mowing device 100 has low delay, does not need to send a large amount of environment information acquired by the environment information acquisition unit 10 to the remote terminal through a communication port, and does not need to receive the identification result of the remote terminal through a communication interface to establish the virtual working boundary. In this way, the mowing device 100 can recognize whether a boundary between grass and non-grass exists according to the environmental information acquired by the environmental information acquisition unit 10 in real time along with the movement of the mowing device, and can update the established virtual working boundary in real time according to the recognition result. Therefore, the efficiency and the precision of the graph construction can be further improved.

Another aspect of the present application provides a method of establishing virtual work boundaries that may be used with a remote terminal. An application program may be stored on the remote terminal 200 as shown in fig. 4, and the remote terminal 200 is caused to execute the application program to implement the method of establishing a virtual boundary. As shown in fig. 15, the method includes:

s1501: acquiring environment information in the advancing process of the mowing device, and identifying the boundary of the working range of the mowing device according to the environment information;

s1503: when the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device is made to automatically advance, and the position information in the automatic advancing process of the mowing device is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information;

s1505: when the boundary of the working range of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the mowing device is made to advance according to the received first user instruction, and the position information of the mowing device in the advancing process under the first user instruction is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information.

The environmental information in the travel of the mower may be collected by the mower's own environmental information collection unit 10 (e.g. image sensor, sonar, RFID, etc. any sensor capable of acquiring environmental information around the mower 100). The remote terminal 200 itself may also be used as a sensor such as a camera on a cell phone or a LiDAR (Light Identification Detection And Ranging light identification detection and ranging) sensor. Environmental information of the grass cutting device 100 may also be obtained from *** maps, hundred degree maps, or satellite. The remote terminal 200, such as a cell phone or a remote control of the mowing device, may acquire environmental information of the mowing device 100 through a communication interface (e.g., a bluetooth interface, etc.) thereof. According to one or more embodiments of the present application, environmental information of the grass cutting device 100 may be acquired at predetermined time intervals. For example, image information of the environment in front of the grass cutting device 100 may be acquired by an image acquisition unit on the grass cutting device 100. The image capturing unit may acquire image information of the environment in front of the mowing device 100 at predetermined time intervals (e.g., every 0.1 seconds), and the mowing device 100 may transmit the image information to the remote terminal 200 through a near field communication port such as WiFi. The remote terminal 200 may recognize whether the operating range boundary of the grass cutting device is included in the environment information according to the acquired environment information. The remote terminal 200 may be equipped with an identification algorithm by which it is identified whether the operating range boundaries of the grass cutting device are included in the environmental information. Specifically, the neural network model may be used as an identification algorithm for the boundary of the working range of the mowing device, and sample data may be collected in advance to train the initial neural network model to generate the neural network model. For example, images of multiple lawns can be collected, boundaries of grass and non-grass in the images can be manually marked, the marked image data are placed on a server to train an initial neural network model, and the trained neural network model can be used as an identification algorithm for identifying the boundary of the working range of the mowing device. When the remote terminal 200 can recognize the operation range boundary of the grass cutting device 100 from the environmental information through the recognition algorithm in operation S1503, the remote terminal 200 may transmit an instruction to the grass cutting device 100 to cause the grass cutting device 100 to autonomously travel and record position information during autonomous travel. For example, the grass cutting device 100 may be made to travel along an autonomously planned path and its position coordinates during autonomous travel recorded, so that the grass cutting device 100 can establish a virtual working boundary according to the recorded position information.

In operation S1505, when the remote terminal 200 cannot identify the operation range boundary of the grass cutting device from the environmental information through the identification algorithm, the remote terminal 200 may send a command to the grass cutting device to cause the grass cutting device 100 to enter a manual mode in which a virtual operation boundary is manually established. Proceeding according to the received first user instruction. Specifically, when the remote terminal 200 cannot recognize whether the acquired environmental information includes the operating range boundary of the mowing device by using the recognition algorithm, the user may intervene, and send a first user instruction to the mowing device through the remote terminal 200, for example, the user may input an instruction to make the mowing device advance, retract, forward or backward through the display interface of the remote terminal 200. After receiving the first user command, the mowing device proceeds according to the received first user command and records the position information in the proceeding process under the first user command, so that the mowing device 100 can establish a virtual working boundary according to the recorded position information.

It will be appreciated that the virtual work boundary established in operation S1503 and the virtual work boundary established in operation S1505 of the grass cutting device 100 together form a complete virtual work boundary of the grass cutting device 100.

As shown in fig. 9, in performing the above-described operations, the display interface of the remote terminal 200 may display a variety of interaction information of the lawn mower and the user. In addition to displaying the identified working range boundaries 21, as shown in fig. 9, the display interface may also display historical tracks 23 of the mower to facilitate a user's knowledge of the overall virtual working boundaries of the mower, and the user may manually manipulate the path of travel of the mower when the overall virtual working boundaries do not meet the user's expectations. The display interface may also include a virtual joystick 29 for remote control of direction and a virtual joystick 27 for remote control of throttle. The user may enter a command to advance, retract, forward or left the grass cutting device 100 by clicking or dragging the virtual joystick 29 in the remote control direction, or the like, i.e., to set the first user command. The user can also set the speed of the mower 100, etc. by clicking or dragging the virtual joystick 27 which remotely controls the throttle. The display interface may also include an environmental range 30 that the environmental information collection unit is currently capable of sensing, an exit button 31, a rollback erasure history track button 33, and an auto-recognize working range boundary activation switch 25. The display interface can enhance interaction between a user and the mowing device, and provides a user with a choice for enabling the mowing device to automatically enter a self-construction mode of the mowing device, so that time for the user to control the mowing device to manually set a travel path of the mowing device can be saved to the greatest extent, work efficiency of the user is improved, and user experience is improved.

According to the embodiment of the application, when the working range boundary of the mowing device is identified according to the environmental information in the advancing process of the mowing device, a user is reminded to enter an automatic mode for autonomously establishing the virtual working boundary. Similarly, when the working range boundary of the mowing device cannot be identified according to the environmental information during the traveling process of the mowing device, the user is reminded to enter a manual mode for manually establishing a virtual working boundary. I.e. to alert the user that an automatic mode or a manual mode can be entered before the grass cutting device is switched between the automatic mode and the manual mode. The remote terminal may include a user feedback unit that works with sound, light, electrical signals, etc., to give a prompt or reminder to the user. For example, a visual signal may be generated by a remote terminal display screen, an audible signal may be generated by a speaker on the remote terminal or mower, and a vibration alert may be generated by a tactile signal, such as a vibration unit. The embodiment can enhance interaction between the user and the mowing device and improve user experience. Meanwhile, when the boundary of the working range of the mowing device is identified, the user is clearly reminded, so that the user can utilize the autonomous mapping mode as much as possible, the time for the user to control the mowing device is saved, and the working efficiency of the user is improved.

Yet another aspect of the present application relates to an apparatus 300 for establishing virtual work boundaries. As shown in fig. 16, the apparatus 300 includes an autonomous module 301, a manual module 303, and a mapping module 305. The autonomous module 301, the manual module 303 and the mapping module 305 may be a functional module that can be implemented by a controller of the mowing device to execute instructions, or may be a functional module that can be implemented by a controller of the remote terminal to execute instructions. The specific functions of the individual modules are as follows. The autonomous module 301 is configured to, when identifying a working range boundary of the mowing device according to environmental information during traveling of the mowing device, cause the mowing device to autonomously travel, and record position information during autonomous traveling of the mowing device. The manual module 303 is configured to, when the working range boundary of the mowing device cannot be identified according to the environmental information during the traveling process of the mowing device, cause the mowing device to travel according to the received first user instruction, and record the position information of the mowing device during the traveling process under the first user instruction. The mapping module 305 is configured to establish the virtual working boundary according to the recorded location information.

Referring to fig. 17, a schematic structural diagram of an information determining apparatus according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the information determining apparatus.

As shown in fig. 17, the information determination device 1700 may include: a processor 1702, a communication interface (Communications Interface) 1704, a memory 1706, and a communication bus 1708.

Wherein:

processor 1702, communication interface 1704, and memory 1706 communicate with each other over a communication bus 1708.

A communication interface 1704 for communicating with other electronic devices or servers.

The processor 1702 is configured to execute the program 1710, and may specifically perform relevant steps in the above-described information determining method embodiment.

In particular, the program 1710 may include program code including computer operating instructions.

The processor 1702 may be a central processing unit CPU, or a graphics processor GPU (Graphics Processing Unit), or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors comprised by the smart device may be the same type of processor, such as one or more CPUs; one or more GPUs; but may also be different types of processors such as one or more CPUs and one or more GPUs and one or more ASICs.

Memory 1706 for storing programs 1710. Memory 1706 may comprise high-speed RAM memory or may also comprise non-volatile memory, such as at least one disk memory.

The program 1710 may be specifically configured to cause the processor 1702 to execute the information determining method in any of the foregoing embodiments.

The specific implementation of each step in the program 1710 may refer to corresponding steps and corresponding descriptions in the units in any one of the foregoing information determining method embodiments, which are not described herein. It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and modules described above may refer to corresponding procedure descriptions in the foregoing method embodiments, which are not repeated herein.

In the embodiment of the application, the image collector of the device to be controlled is used for collecting the image to be processed aiming at the region to be identified, the region to be identified at least comprises the region to be processed and the barrier region, the image to be processed is processed to determine the information of the partial boundary of the region to be processed where the device to be controlled is currently located, then the device to be controlled is controlled to switch between the automatic motion mode and the remote motion mode based on the information of the partial boundary so as to determine the boundary of the region to be processed, and the boundary of the region to be processed is used for distinguishing the region to be processed from the barrier region.

Still another aspect of the present application relates to a mower, as shown in fig. 18, the mowing apparatus 100 may include the information determining device 1700 described above, and instructions for implementing the information determining method described above are stored in a memory 1706 on the information determining device 1700, and when the processor 1701 executes the instructions, the mowing apparatus 100 may implement the method. Compared with the mode of identifying grass and non-grass boundaries by the remote terminal and establishing a virtual working boundary according to the recorded position information, the mode implemented by the mowing device 100 has low delay, does not need to send a large amount of environment information acquired by the environment information acquisition unit 10 to the remote terminal through a communication port, and does not need to receive the identification result of the remote terminal through a communication interface to establish the virtual working boundary. In this way, the mowing device 100 can recognize whether a boundary between grass and non-grass exists according to the environmental information acquired by the environmental information acquisition unit 10 in real time along with the movement of the mowing device, recognize whether a working range boundary is included according to the boundary between grass and non-grass, and can update the established virtual working boundary in real time according to the recognition result. Therefore, the efficiency and the precision of the graph construction can be further improved.

Yet another aspect of the present application relates to a remote terminal. As shown in fig. 19, the remote terminal 200 includes a memory 201 and a controller 203, the memory 201 is used for storing instructions, and the controller 203 is used for executing the instructions to implement operations corresponding to the above-mentioned method for establishing a virtual work boundary.

Yet another aspect of the present application relates to a computer storage medium. The computer storage medium stores a computer program which, when executed by the processor, implements operations corresponding to the above-described method of establishing virtual working boundaries.

Embodiments of the present application also provide a computer program product comprising computer instructions that instruct a computing device to perform operations corresponding to any one of the above-described method embodiments.

It should be noted that, according to implementation requirements, each component/step described in the embodiments of the present application may be split into more components/steps, and two or more components/steps or part of operations of the components/steps may be combined into new components/steps, so as to achieve the purposes of the embodiments of the present application.

The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored on such software processes on a recording medium using a general purpose computer, special purpose processor, or programmable or special purpose hardware such as an ASIC or FPGA. It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a memory component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the information determination methods described herein. Further, when the general-purpose computer accesses code for implementing the information determination method shown herein, execution of the code converts the general-purpose computer into a special-purpose computer for executing the information determination method shown herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

It should be noted that, although specific embodiments of the present application are described in detail with reference to the accompanying drawings, the scope of protection of the present application should not be construed as being limited. Various modifications and variations which may be made by those skilled in the art without the creative effort are within the scope of the claims.

Examples of embodiments of the present application are intended to concisely illustrate technical features of embodiments of the present application so that those skilled in the art may intuitively understand the technical features of embodiments of the present application, and are not meant to be undue limitations of embodiments of the present application.

Finally, the following description: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. An information determining method, the method comprising:

acquiring a to-be-processed image aiming at an area to be identified by an image acquisition device of equipment to be controlled, wherein the area to be identified at least comprises the area to be processed and an obstacle area;

processing the image to be processed, and determining the information of the partial boundary of the area to be processed where the equipment to be controlled is currently located;

and controlling the equipment to be controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the part boundary so as to determine the boundary of the area to be processed, wherein the boundary of the area to be processed is used for distinguishing the area to be processed from the obstacle area.

2. The method according to claim 1, wherein the capturing of the image to be processed for the area to be identified by the image capturing device of the device to be controlled comprises:

during the movement process of the equipment to be controlled, acquiring a plurality of images to be processed aiming at the area to be identified through the image acquisition device;

correspondingly, the processing the image to be processed, determining the information of the partial boundary of the area to be processed where the device to be controlled is currently located, includes:

And carrying out semantic segmentation and processing on the plurality of images to be processed, and determining the information of the part boundary based on a semantic segmentation result and a processing result.

3. The method according to claim 2, wherein the semantically segmenting and processing the plurality of images to be processed and determining the information of the partial boundary based on the semantically segmenting result and the processing result includes:

carrying out semantic segmentation on each image to be processed, and carrying out matching processing on each image to be processed based on a semantic segmentation result and a target map to obtain a non-matching region; wherein the target map is a map of a target area in the area to be identified;

carrying out semantic segmentation on each image to be processed, and mapping each semantically segmented image to be processed to the target map based on the grid of the target map;

for each image to be processed, carrying out contour recognition on the mapped image to obtain a boundary to be processed;

and determining the information of the part of boundary based on the smoothness of the boundary to be processed, the unmatched area and the target area.

4. The method according to claim 1, wherein the controlling the device to be controlled to switch between an automatic movement mode and a remote movement mode based on the information of the partial boundary to determine the boundary of the region to be processed comprises:

Under the condition that the information of the part boundary characterizes that the part boundary meets the target boundary condition, controlling the equipment to be controlled to be switched to work in an automatic motion mode so as to determine the boundary of the area to be processed;

and under the condition that the information of the part boundary characterizes that the part boundary does not meet the target boundary condition, controlling the equipment to be controlled to be switched to work in a remote control movement mode so as to determine the boundary of the area to be processed.

5. The method according to claim 1 or 4, characterized in that the method further comprises:

receiving an operation instruction for operating an operation object of the equipment to be controlled;

and controlling the equipment to be controlled to be switched to work in a remote control movement mode based on the operation instruction so as to determine the boundary of the area to be processed.

6. The method according to claim 1, characterized in that the device to be controlled comprises a mowing device;

the controlling the device to be controlled to switch between an automatic motion mode and a remote control motion mode based on the information of the partial boundary to determine the boundary of the region to be processed includes:

if the boundary of the working range of the mowing device is determined according to the information of the partial boundary, the mowing device is switched to an automatic movement mode to automatically travel, and position information of the mowing device in the automatic travel process is recorded;

If the boundary of the working range of the mowing device can not be determined according to the information of the partial boundary, the mowing device is switched to a remote control movement mode to travel according to the received first user instruction, and position information of the mowing device in the traveling process under the first user instruction is recorded;

and establishing a virtual working boundary according to the recorded position information, wherein the boundary of the area to be processed comprises the virtual working boundary.

7. The method of claim 6, wherein if the operating range boundary of the mowing device is determined based on the partial boundary information, the mowing device switches to an automatic motion mode to autonomously travel, comprising:

obtaining a travelling path of the mowing device according to the determined working range boundary;

the mowing device autonomously travels along the travel path.

8. The method of claim 6, wherein if the operating range boundary of the mowing device is determined according to the information of the partial boundary, the mowing device is switched to an automatic movement mode to autonomously travel, and position information during autonomous travel of the mowing device is recorded, comprising:

And if the working range boundary of the mowing device is determined according to the information of the part boundary and the first user instruction is received, the mowing device advances according to the received first user instruction and records the position information of the mowing device in the advancing process under the first user instruction.

9. The method of claim 6 or 8, wherein the first user instruction received comprises: a target travel position set by a user;

the mowing device proceeds according to the received first user instruction, comprising: the mowing device autonomously travels along a straight line to the target travel position.

10. The method according to any one of claims 1-4 and 6-8, further comprising:

and if the equipment to be controlled meets at least one of the following conditions, controlling the equipment to be controlled to exit the automatic motion mode:

(i) The equipment to be controlled is disconnected with a remote terminal, wherein the remote terminal is used for controlling the remote terminal to move;

(ii) And the equipment to be controlled moves to a preset target position.

11. The method according to claim 1, wherein the controlling the device to be controlled to switch between an automatic movement mode and a remote movement mode based on the information of the partial boundary to determine the boundary of the region to be processed comprises:

When the equipment to be controlled is in a remote control movement mode, if the working range boundary of the equipment to be controlled is determined according to the information of the part boundary, first indication information is sent to a remote terminal, so that the remote terminal displays a first mode switching control through a display interface according to the first indication information;

if first control information sent by the remote terminal in response to the triggering of the first mode switching control is received, switching to an automatic motion mode to automatically travel, and recording position information of the equipment to be controlled in the automatic travel process;

and establishing the boundary of the area to be processed according to the recorded position information.

12. The method according to claim 11, wherein the controlling the device to be controlled to switch between an automatic movement mode and a remote movement mode based on the information of the partial boundary to determine the boundary of the region to be processed comprises:

when the equipment to be controlled is in an automatic movement mode, if the boundary of the working range of the equipment to be controlled cannot be determined according to the information of the partial boundary, second indication information is sent to the remote terminal, so that the remote terminal displays a first movement control through a display interface according to the second indication information;

Receiving second control information sent by the remote terminal in response to the triggering of the first motion control, advancing according to the second control information, and recording position information of the equipment to be controlled in the advancing process according to the second control information;

13. The method according to claim 1, wherein the controlling the device to be controlled to switch between an automatic movement mode and a remote movement mode based on the information of the partial boundary to determine the boundary of the region to be processed comprises:

when the equipment to be controlled is in a remote control movement mode, if third control information sent by the remote terminal in response to triggering of a second mode switching control is received, switching to an automatic movement mode to automatically travel, and recording position information in the automatic travel process of the equipment to be controlled, wherein the second mode switching control is displayed through a display interface after the working range boundary of the equipment to be controlled is determined by the remote terminal;

14. The method of claim 13, wherein the controlling the device to be controlled to switch between an automatic motion mode and a remote motion mode based on the information of the partial boundary to determine the boundary of the area to be processed comprises:

when the equipment to be controlled is in an automatic motion mode, if fourth control information sent by the remote terminal in response to the triggering of the second motion control is received, advancing according to the fourth control information, and recording position information of the equipment to be controlled in the advancing process according to the fourth control information;

15. An information determining method applied to a remote terminal for controlling movement of a device to be controlled, the method comprising:

receiving information of a part boundary sent by the equipment to be controlled, wherein the information of the part boundary is used for indicating the part boundary of a region to be processed where the equipment to be controlled is currently located, the information of the part boundary is obtained by processing an image to be processed by the equipment to be controlled, the image to be processed is obtained by image acquisition of an area to be identified by an image acquisition device of the equipment to be controlled, and the area to be identified at least comprises the area to be processed and an obstacle area;

And transmitting control information to the equipment to be controlled based on the information of the part boundary so that the equipment to be controlled can be switched between an automatic movement mode and a remote control movement mode according to the control information, and determining the boundary of the area to be treated, wherein the boundary of the area to be treated is used for distinguishing the area to be treated from the obstacle area.

16. The method according to claim 15, wherein the device to be controlled comprises a mowing device;

the step of sending control information to the equipment to be controlled based on the information of the part boundary so that the equipment to be controlled can be switched between an automatic movement mode and a remote control movement mode according to the control information, and determining the boundary of the area to be processed comprises the following steps:

if the boundary of the working range of the mowing device is determined according to the information of the partial boundary, first control information is sent to control the mowing device to switch to an automatic movement mode so as to automatically travel, and position information of the mowing device in the automatic travel process is recorded;

if the boundary of the working range of the mowing device can not be determined according to the information of the partial boundary, sending second control information to control the mowing device to switch to a remote control movement mode so as to travel according to the received first user instruction, and recording the position information of the mowing device in the travelling process under the first user instruction;

And controlling the mowing device to establish a virtual working boundary according to the recorded position information, wherein the virtual working boundary comprises the boundary of the area to be treated.

17. The method according to claim 15 or 16, characterized in that the method further comprises:

if the remote terminal detects that the equipment to be controlled meets at least one of the following conditions, the automatic motion mode control interface is exited:

(i) The remote terminal is disconnected with the equipment to be controlled;

(ii) And the equipment to be controlled moves to a preset target position.

18. The method of claim 16, wherein if the operating range boundary of the mowing device is determined based on the partial boundary information, performing at least one of:

displaying the identified working range boundary to a user;

the user is prompted to enter an automatic movement mode where virtual work boundaries are established autonomously.

19. The method of claim 16, wherein the method further comprises:

when the boundary of the working range of the mowing device is determined according to the information of the part boundary, reminding a user to enter an automatic movement mode for independently establishing a virtual working boundary;

Or reminding a user to enter a remote control movement mode for manually establishing a virtual working boundary when the working range boundary of the mowing device cannot be determined according to the information of the part boundary.

20. The method of claim 15, wherein the method further comprises:

if the first indication information sent by the equipment to be controlled is received, a first mode switching control is displayed through a display interface, wherein the first indication information is sent by the equipment to be controlled when the following conditions are met: the equipment to be controlled is in a remote control movement mode, and the working range boundary of the equipment to be controlled is determined according to the information of the part boundary;

and responding to the first mode switching control to be triggered, sending first control information to the equipment to be controlled, enabling the equipment to be controlled to switch from a remote control movement mode to an automatic movement mode to conduct autonomous travel according to the first control information, and recording position information in the autonomous travel process of the equipment to be controlled.

21. The method of claim 20, wherein the method further comprises:

if the second indication information sent by the equipment to be controlled is received, displaying the first motion control through a display interface, wherein the second indication information is sent by the equipment to be controlled when the following conditions are met: the equipment to be controlled is in an automatic motion mode, and the working range boundary of the equipment to be controlled can not be determined according to the information of the part boundary;

And responding to the first motion control to be triggered, sending second control information to the equipment to be controlled, enabling the equipment to be controlled to travel according to the second control information, and recording position information of the equipment to be controlled in the traveling process according to the second control information.

22. The method of claim 15, wherein the sending control information to the device to be controlled based on the information of the partial boundary comprises:

when the equipment to be controlled is in the remote control movement mode, if the working range boundary of the equipment to be controlled is determined according to the information of the part boundary, a second mode switching control is displayed through a display interface;

and responding to the triggering of the second mode switching control, sending third control information to the equipment to be controlled, so that the equipment to be controlled is switched to an automatic movement mode according to the third control information to perform autonomous traveling, and recording position information in the autonomous traveling process of the equipment to be controlled.

23. The method of claim 22, wherein the sending control information to the device to be controlled based on the information of the partial boundary comprises:

When the equipment to be controlled is in the automatic motion mode, if the working range boundary of the equipment to be controlled can not be determined according to the information of the part boundary, displaying a second motion control through a display interface;

and responding to the triggering of the second motion control, and sending fourth control information to the equipment to be controlled so as to enable the equipment to be controlled to travel according to the fourth control information and record the position information of the equipment to be controlled in the traveling process according to the fourth control information.

24. The method of claim 15, wherein the method further comprises:

when the equipment to be controlled is in an automatic motion mode, responding to the trigger of an exit control, and displaying a third motion control through a display interface, wherein the exit control is displayed on the display interface;

and responding to the third motion control to be triggered, sending fifth control information to the equipment to be controlled, so that the equipment to be controlled advances according to the fifth control information, and recording position information of the equipment to be controlled in the advancing process according to the fifth control information.

25. A method for establishing a virtual working boundary, applied to a mowing device, comprising:

If the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device automatically advances, and the position information of the mowing device in the automatic advancing process is recorded;

if the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the mowing device advances according to the received first user instruction, and the position information of the mowing device in the advancing process under the first user instruction is recorded; and

and establishing the virtual working boundary according to the recorded position information.

26. The method of claim 25, wherein if the operating range boundary of the mower is identified based on environmental information during travel of the mower, the mower travels autonomously and records positional information during autonomous travel of the mower, comprising:

obtaining a travel path of the mowing device according to the identified working range boundary;

the mowing device autonomously travels along the travel path.

27. The method of claim 25, wherein if the operating range boundary of the mower is identified based on environmental information during travel of the mower, the mower travels autonomously and records positional information during autonomous travel of the mower, comprising:

If the working range boundary of the mowing device is identified according to the environmental information in the advancing process of the mowing device and the first user instruction is received, the mowing device advances according to the received first user instruction and records the position information of the mowing device in the advancing process under the first user instruction.

28. The method of claim 25 or 27, wherein the received first user instruction includes a user-set target travel position, and wherein the mowing device travels in accordance with the received first user instruction, including autonomous travel of the mowing device along a straight line to the target travel position.

29. The method of any one of claims 25-27, wherein the mowing apparatus comprises an image acquisition unit, the method further comprising:

obtaining image information of an environment in the advancing process of the mowing device;

and identifying whether the image of the environment comprises the working range boundary of the mowing device or not according to the image information of the environment through a preset neural network model.

30. A method of establishing a virtual work boundary for a remote terminal, the method comprising:

Acquiring environment information in the advancing process of the mowing device, and identifying the boundary of the working range of the mowing device according to the environment information;

when the working range boundary of the mowing device is identified according to the environmental information in the advancing process of the mowing device, the mowing device is made to automatically advance, and the position information in the automatic advancing process of the mowing device is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information; and

when the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device, the mowing device is made to advance according to the received first user instruction, and the position information of the mowing device in the advancing process of the first user instruction is recorded, so that the mowing device can establish a virtual working boundary according to the recorded position information in the advancing process of the first user instruction.

31. The method as recited in claim 30, further comprising:

acquiring image information of an environment when the mowing device advances; and

and identifying the boundary of the working range of the mowing device according to the image information of the environment through a preset neural network model.

32. The method of claim 30 or 31, wherein when the operating range boundary of the mowing device is identified, at least one of the following is performed:

displaying the identified working range boundary to a user;

the user is prompted to enter an automatic mode where virtual work boundaries are established autonomously.

33. The method according to claim 30 or 31, characterized in that the method further comprises:

when the boundary of the working range of the mowing device is identified according to the environmental information in the advancing process of the mowing device, reminding a user of entering an automatic mode for autonomously establishing a virtual working boundary; or (b)

And reminding a user of entering a manual mode of manually establishing a virtual working boundary when the working range boundary of the mowing device cannot be identified according to the environmental information in the advancing process of the mowing device.

34. An apparatus for establishing virtual work boundaries, comprising:

the autonomous module is used for enabling the mowing device to autonomously travel and recording position information in the autonomous travel process of the mowing device if the working range boundary of the mowing device is identified according to the environment information in the travel process of the mowing device;

The manual module is used for enabling the mowing device to travel according to the received first user instruction and recording the position information of the mowing device in the traveling process under the first user instruction if the working range boundary of the mowing device cannot be identified according to the environmental information in the traveling process of the mowing device; and

and the mapping module is used for establishing the virtual working boundary according to the recorded position information.

35. An information determining apparatus, characterized in that the apparatus comprises: a processor, a memory, and a communication bus;

the communication bus is used for realizing communication connection between the processor and the memory;

the processor is configured to execute an information determining program in a memory to implement the steps of the information determining method according to any one of claims 1-14.

36. A mower comprising the information determining apparatus according to claim 35.

37. A remote terminal comprising a memory for storing instructions and a controller for executing the instructions to implement the method of any one of claims 15-24 or claims 30-33.

38. A computer-readable storage medium storing one or more programs executable by one or more processors to implement the information determination method of any one of claims 1-24 or the steps of the method of establishing a virtual working boundary of any one of claims 25-33.