CN113762044A

CN113762044A - Road recognition method, road recognition device, computer equipment and storage medium

Info

Publication number: CN113762044A
Application number: CN202110488443.XA
Authority: CN
Inventors: 裴歌; 张甜; 唐梦云; 刘水生; 涂思嘉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-05-06
Filing date: 2021-05-06
Publication date: 2021-12-07

Abstract

The application relates to a road identification method, a road identification device, a computer device and a storage medium. The method comprises the following steps: performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image; determining pixel conversion values corresponding to all target pixel points based on the target road detection values, and arranging the pixel conversion values according to the pixel point arrangement sequence to obtain a conversion image corresponding to the target image; determining conversion value change information of a target pixel point in the conversion image relative to a reference pixel point in the conversion image, and determining an edge pixel corresponding to a road in the target image based on the conversion value change information; and identifying and obtaining a target road in the target image based on the edge pixels. The method can improve the accuracy of road identification. According to the road identification method, the target image can be subjected to road detection by using the artificial intelligence-based neural network model. The method is applied to the blind guiding scene, and the walking safety of the blind can be improved.

Description

Road recognition method, road recognition device, computer equipment and storage medium

Technical Field

The present application relates to the field of image recognition technologies, and in particular, to a road recognition method, apparatus, computer device, and storage medium.

Background

Along with the continuous construction of roads, the number of the roads is continuously increased, the relationship among the roads is more and more complex, and the real-time and accurate grasp of the distribution condition of the roads has important significance for vehicles or users moving on the roads. For example, for a vehicle traveling on a road, knowing the distribution of the road may guide the vehicle to travel correctly, and for a user traveling on a road, knowing the distribution of the road may guide the user to travel correctly.

At present, there are many methods for identifying roads, for example, an artificial intelligence-based neural network model can be used to detect an image and obtain road information in the image.

However, the existing road identification method has the condition that the road cannot be accurately identified, so that the accuracy of road identification is low.

Disclosure of Invention

In view of the above, it is necessary to provide a road identification method, apparatus, computer device and storage medium capable of improving the accuracy of road identification.

A method of road identification, the method comprising: acquiring a target image to be subjected to road identification; performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image; determining pixel conversion values corresponding to target pixel points in the target image based on the target road detection values, and arranging the pixel conversion values corresponding to the target pixel points according to a pixel point arrangement sequence to obtain a conversion image corresponding to the target image; determining transformation value change information of the target pixel points in the transformation image relative to reference pixel points in the transformation image, and determining edge pixels corresponding to roads in the target image based on the transformation value change information; and identifying and obtaining a target road in the target image based on the edge pixels.

A road identification device, the device comprising: the target image acquisition module is used for acquiring a target image to be subjected to road identification; a road detection value obtaining module, configured to perform road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image; a transformed image obtaining module, configured to determine pixel transformation values corresponding to target pixel points in the target image based on the target road detection value, and arrange the pixel transformation values corresponding to the target pixel points according to a pixel point arrangement sequence to obtain a transformed image corresponding to the target image; the edge pixel determining module is used for determining conversion value change information of the target pixel point in the conversion image relative to a reference pixel point in the conversion image and determining an edge pixel corresponding to a road in the target image based on the conversion value change information; and the target road obtaining module is used for identifying and obtaining the target road in the target image based on the edge pixels.

In some embodiments, the road detection value obtaining module includes: the target user type obtaining unit is used for obtaining a target user type corresponding to the target image; a first road detection model obtaining unit, configured to obtain a trained road detection model corresponding to the target user type, where the trained road detection model is obtained by training a target training image corresponding to the target user type; and the first road detection value obtaining unit is used for inputting the target image into the trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image.

In some embodiments, the road detection model derivation module that derives the trained road detection model comprises: the target user type determining unit is used for determining the target user type corresponding to the road detection model to be trained; a first target training image obtaining unit, configured to obtain candidate training images corresponding to multiple target user types from a candidate training image set, where the candidate training images are used as target training images; a pixel label value obtaining unit, configured to obtain a road position corresponding to a road in the target training image, and determine a pixel label value corresponding to each training pixel point in the target training image based on the road position; and the road detection model obtaining unit is used for training the road detection model to be trained based on the target training image and the pixel label value to obtain a trained road detection model.

In some embodiments, the target training image derivation module that derives the target training image comprises: the road behavior information acquisition unit is used for acquiring the road behavior information of the target user corresponding to the type of the target user; a mobile control information generating unit, configured to generate mobile control information corresponding to a virtual user based on the road behavior information; and the second target training image obtaining unit is used for controlling the virtual user to move along the road by using the movement control information, and obtaining an image acquired by the virtual user in the moving process as the target training image.

In some embodiments, the transformed image derivation module comprises: a road pixel point obtaining unit, configured to obtain, based on the target road detection value, a target pixel point that satisfies a road detection value screening condition from each target pixel point of the target image, and use the target pixel point as a road pixel point; and the pixel conversion value obtaining unit is used for taking the road pixel value corresponding to the road as the pixel conversion value corresponding to the road pixel point and taking the shielding pixel value as the pixel conversion value corresponding to the non-road pixel point in the target image.

In some embodiments, the edge pixel determination module comprises: a current pixel point obtaining unit, configured to obtain a current pixel point from each pixel point of the transformed image; a reference pixel point obtaining unit, configured to obtain an adjacent pixel point of the current pixel point from the transformed image, and determine a reference pixel point corresponding to the current pixel point based on the adjacent pixel point; and the conversion value change information obtaining unit is used for obtaining the conversion value difference between the current pixel point and the reference pixel point corresponding to the current pixel point and obtaining the conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference.

In some embodiments, the reference pixel point of the current pixel point includes an adjacent pixel point in a plurality of pixel arrangement directions, and the transformation value change information includes a transformation value change degree and a transformation value change direction angle; the conversion value change information obtaining unit is further configured to obtain a conversion value difference corresponding to each pixel arrangement direction of the current pixel point based on the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point of the current pixel point in each pixel arrangement direction; and determining the conversion value change degree and the conversion value change direction angle of the current pixel point relative to the reference pixel point by combining the conversion value difference corresponding to each pixel arrangement direction.

In some embodiments, the transformation value change information obtaining unit is further configured to perform statistical operation on a transformation value difference corresponding to each pixel arrangement direction of a current pixel point to obtain the transformation value change degree; and taking the conversion value difference corresponding to the current pixel point in each pixel arrangement direction as the side length of a direction side in the corresponding pixel arrangement direction, determining the angle of a connecting side connecting the direction sides based on the side length, and taking the angle as the conversion value change direction angle.

In some embodiments, the transform value change information includes a transform value change degree and a transform value change direction angle, the edge pixel determination module includes: a candidate pixel point obtaining unit, configured to obtain, from each pixel point of the transformed image, a pixel point satisfying a change degree screening condition as a candidate pixel point; a contrast transformation value change degree obtaining unit, configured to obtain a contrast transformation value change degree of the candidate pixel point at the transformation value change direction angle; a change degree difference obtaining unit for determining a change degree difference between the change degree of the candidate pixel point and the change degree of the contrast change value; and the edge pixel obtaining unit is used for taking the candidate pixel point as an edge pixel corresponding to the road in the target image when the change degree difference is larger than a change degree difference threshold value.

In some embodiments, the edge pixels are plural, and the target road obtaining module includes: a current edge line determining unit, configured to determine a current edge line corresponding to the edge pixel, determine a current edge line distance between the current edge line and a reference position, and determine a current edge line included angle between the current edge line and a preset reference direction line; a road edge line obtaining unit, configured to use a current edge line distance and a current edge line included angle as current edge line parameters to be adjusted, adjust the current edge line parameters of each current edge line until a parameter adjustment stop condition is met, and use an edge direction line corresponding to the current edge line parameter meeting the parameter adjustment condition as a road edge line corresponding to a road in the target image, where the parameter adjustment stop condition includes at least one of that a difference between each edge line distance is smaller than a distance difference threshold or that a difference between each edge line included angle is smaller than an included angle difference threshold; a target road determination unit for determining a target road in the target image based on the road edge line.

In some embodiments, the target image obtaining module is further configured to obtain a target image uploaded by a terminal, and use the target image as a target image to be subjected to road identification; the road detection value obtaining module includes: a second road detection model obtaining unit, configured to obtain a target user type corresponding to the terminal, and obtain a trained road detection model corresponding to the target user type, where the trained road detection model is obtained by training a target training image corresponding to the target user type; a second road detection value obtaining unit, configured to input the target image into the trained road detection model for road detection, so as to obtain a target road detection value corresponding to each target pixel point in the target image; the device further comprises: the user movement prompt information determining module is used for determining user movement prompt information based on the position corresponding to the target road and the position of the terminal; and the user movement prompt information sending module is used for sending the user movement prompt information to the terminal so that the terminal carries out movement prompt according to the user movement prompt information.

A computer device comprising a memory storing a computer program and a processor implementing the steps of the above-mentioned road identification method when executing the computer program.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned road identification method.

In some embodiments, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.

The road identification method, the device, the computer equipment and the storage medium acquire a target image to be subjected to road identification, perform road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image, determine a pixel conversion value corresponding to each target pixel point in the target image based on the target road detection value, arrange the pixel conversion values corresponding to the target pixel points according to the pixel point arrangement sequence to obtain a conversion image corresponding to the target image, determine conversion value change information of the target pixel points in the conversion image relative to reference pixel points in the conversion image, determine edge pixels corresponding to the road in the target image based on the conversion value change information, obtain the target road in the target image based on the edge pixel identification, and better embody the road information in the image due to the conversion based on the target road detection image value, therefore, the corresponding edge pixels in the target image can be accurately detected based on the conversion value change information of the target pixel points relative to the reference pixel points in the conversion image, and the accuracy of road identification is improved.

Drawings

FIG. 1 is a diagram of an environment in which a method for identifying roads may be used in some embodiments;

FIG. 2 is a schematic flow chart of a road identification method in some embodiments;

FIG. 3 is a schematic diagram of road detection using a road detection model in some embodiments;

FIG. 4 is a diagram illustrating pixel tag values in some embodiments;

FIG. 5 is a schematic diagram of road edge lines and road center lines in some embodiments;

FIG. 6 is a flow diagram of a road identification method in some embodiments;

FIG. 7A is a schematic diagram of a road edge line and a road centerline in some embodiments;

FIG. 7B is a schematic diagram of road edge lines and road center lines in some embodiments;

FIG. 8 is a block diagram of a road identifying apparatus in some embodiments;

FIG. 9 is a diagram of the internal structure of a computer device in some embodiments;

FIG. 10 is a diagram of the internal structure of a computer device in some embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Computer Vision technology (CV) Computer Vision is a science for researching how to make a machine "see", and further refers to that a camera and a Computer are used to replace human eyes to perform machine Vision such as identification, tracking and measurement on a target, and further image processing is performed, so that the Computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition and fingerprint recognition.

Key technologies for Speech Technology (Speech Technology) are automatic Speech recognition Technology (ASR) and Speech synthesis Technology (TTS), as well as voiceprint recognition Technology. The computer can listen, see, speak and feel, and the development direction of the future human-computer interaction is provided, wherein the voice becomes one of the best viewed human-computer interaction modes in the future.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.

The automatic driving technology generally comprises technologies such as high-precision maps, environment perception, behavior decision, path planning, motion control and the like, and the self-determined driving technology has wide application prospect,

with the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, and the like.

Cloud technology refers to a hosting technology for unifying serial resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.

The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied in the cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.

Cloud computing (cloud computing) refers to a delivery and use mode of an IT infrastructure, and refers to obtaining required resources in an on-demand and easily-extensible manner through a network; the generalized cloud computing refers to a delivery and use mode of a service, and refers to obtaining a required service in an on-demand and easily-extensible manner through a network. Such services may be IT and software, internet related, or other services. Cloud Computing is a product of development and fusion of traditional computers and Network Technologies, such as Grid Computing (Grid Computing), Distributed Computing (Distributed Computing), Parallel Computing (Parallel Computing), Utility Computing (Utility Computing), Network Storage (Network Storage Technologies), Virtualization (Virtualization), Load balancing (Load Balance), and the like.

With the development of diversification of internet, real-time data stream and connecting equipment and the promotion of demands of search service, social network, mobile commerce, open collaboration and the like, cloud computing is rapidly developed. Different from the prior parallel distributed computing, the generation of cloud computing can promote the revolutionary change of the whole internet mode and the enterprise management mode in concept.

A distributed cloud storage system (hereinafter, referred to as a storage system) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces to cooperatively work by using functions such as cluster application, grid technology, and a distributed storage file system, and provides a data storage function and a service access function to the outside.

At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data identification (ID, ID entry), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided in advance into stripes according to a group of capacity measures of objects stored in a logical volume (the measures often have a large margin with respect to the capacity of the actual objects to be stored) and Redundant Array of Independent Disks (RAID), and one logical volume can be understood as one stripe, thereby allocating physical storage space to the logical volume.

Database (Database), which can be regarded as an electronic file cabinet in short, a place for storing electronic files, a user can add, query, update, delete, etc. to data in files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.

A Database Management System (DBMS) is a computer software System designed for managing a Database, and generally has basic functions such as storage, interception, security assurance, and backup. The database management system may be categorized according to the database model it supports, such as relational, XML (Extensible Markup Language); or classified according to the type of computer supported, e.g., server cluster, mobile phone; or classified according to the Query Language used, such as SQL (Structured Query Language), XQuery; or by performance impulse emphasis, e.g., maximum size, maximum operating speed; or other classification schemes. Regardless of the manner of classification used, some DBMSs are capable of supporting multiple query languages across categories, for example, simultaneously.

Big data (Big data) refers to a data set which cannot be captured, managed and processed by a conventional software tool within a certain time range, and is a massive, high-growth-rate and diversified information asset which can have stronger decision-making power, insight discovery power and flow optimization capability only by a new processing mode. With the advent of the cloud era, big data has attracted more and more attention, and the big data needs special technology to effectively process a large amount of data within a tolerance elapsed time. The method is suitable for the technology of big data, and comprises a large-scale parallel processing database, data mining, a distributed file system, a distributed database, a cloud computing platform, the Internet and an extensible storage system.

The scheme provided by the embodiment of the application relates to the technologies of machine learning, image recognition and the like of artificial intelligence, and is specifically explained by the following embodiments:

the road identification method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. Specifically, the terminal 102 may perform image acquisition on a road, and transmit the acquired image to the server 104, the server 104 may use the image transmitted by the terminal 102 as a target image to be subjected to road identification, of course, the terminal 102 may also perform video acquisition on the road, and transmit the acquired video to the server 104, and the server 104 may extract a video image from the video transmitted by the terminal 102, and use the extracted video image as the target image to be subjected to road identification. The server 104 may perform road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image, determine a pixel conversion value corresponding to each target pixel point in the target image based on the target road detection value, arrange the pixel conversion values corresponding to the target pixel points according to a pixel arrangement sequence to obtain a conversion image corresponding to the target image, determine conversion value change information of the target pixel points in the conversion image relative to reference pixel points in the conversion image, determine edge pixels corresponding to a road in the target image based on the conversion value change information, and obtain the target road in the target image based on edge pixel identification. The server 104 may determine user movement prompting information based on the position of the target road and the position of the terminal 102, send the user movement prompting information to the terminal 102, and the terminal 102 may prompt the user corresponding to the terminal to move according to the user movement prompting information.

The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the terminal 102 may also be a blind guiding device, where at least one of an image capturing device and a video capturing device may be installed on the blind guiding device, the blind guiding device is used to guide the blind to walk, and the blind guiding device may include at least one of blind guiding glasses or a blind guiding stick. The server 104 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.

It is to be understood that the above application scenario is only an example, and does not constitute a limitation to the road identification method provided in the embodiment of the present application, and the method provided in the embodiment of the present application may also be applied in other application scenarios, for example, the road identification method provided in the present application may be executed by the terminal 102, the terminal 102 may determine the user movement prompt information based on the position of the target road and the position of the terminal 102, and perform movement prompt on the user corresponding to the terminal 102 based on the user movement prompt information. The terminal 102 may also upload the target road in the obtained target image to the server 104, and the server 104 may store the target road in the target image, or may forward the target road in the target image to other terminal devices.

In some embodiments, as shown in fig. 2, a road identification method is provided, where the method may be executed by a terminal or a server, or may be executed by both the terminal and the server, and in this embodiment, the method is described as being applied to the server 104 in fig. 1, and includes the following steps:

s202, acquiring a target image to be subjected to road identification.

The road identification refers to identifying one or more types of roads from the image, and the plurality of types refers to at least two types. The type of road may include at least one of a motorway, a non-motorway, or a sidewalk, wherein the sidewalk may include at least one of a crosswalk or a blind road.

The target image refers to a road image to be subjected to road recognition, and the road image refers to an image including a road. The target image may be a road image pre-stored in the server, or may be a road image acquired by the server in real time from the terminal, for example, the road image may be acquired by blind guiding glasses in real time.

Specifically, the terminal can collect road images in real time, send the collected road images to the server, and the server can take the received road images as target images to be subjected to road identification, perform road detection on the target images, and determine positions of roads in the target images.

In some embodiments, the server may store, in advance, at least two road images corresponding to one or more user types, where the multiple types refer to at least two types, and the user types may be divided according to a moving manner of the user on a road, for example, the user type may include at least one of a user driving a motor vehicle, a user driving a non-motor vehicle, or a user walking, where the user walking may include a user walking with visual impairment or a user walking without visual impairment, and the user with visual impairment may also be referred to as a blind person. The road image corresponding to the user type refers to a road image acquired by a terminal carried by the user of the user type, and can be acquired by blind guiding glasses worn by the blind or a mobile phone carried by the blind, for example.

In some embodiments, the server may obtain a road image corresponding to the target user type from the stored road images, and use the road image corresponding to the target user type as a target image to be subjected to road recognition. The target user type may be preset or determined as needed, for example, a walking user with visual impairment.

And S204, performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image.

The road detection means detecting a road in an image, and the target pixel points are pixel points in the target image. The road detection value is used for representing the probability that the pixel is the road pixel, the target road detection value is used for representing the probability that the target pixel is the road pixel, the higher the target road detection value is, the higher the probability that the target pixel is the road pixel is, the smaller the target road detection value is, the lower the probability that the target pixel is the road pixel is, and the target road detection value can be 0.7 or 0.9 and the like, for example. The road pixel points refer to pixel points forming a road in the image, namely pixel points at the position of the road in the image.

Specifically, the server may perform road detection on the target image using a road detection model, which is a model for detecting a road in the image, and may detect a position of the road in the image. The road detection model may be an artificial intelligence based neural network model, for example, may be a deep learning model, for example, may be at least one of a semantic segmentation model or an instance segmentation model. The semantic segmentation model may include At least one of UNet (U-type Network), FCN (full volume Network), yolcat (you Only Look At coefficiencts), or deep lab, and which model is specifically adopted may be determined based on hardware operation restrictions of the device that deploys the trained road detection model. The road detection model may be, for example, a road segmentation model, which may determine the location of the road from the image. The road detection model may determine the location of a motorway when the user is a blind person walking, or a sidewalk when the user is a blind person walking.

In some embodiments, the road detection model may include a feature extraction network, and the server may obtain the trained road detection model, input the target image into the feature extraction network of the trained road detection model for feature extraction, use the extracted features as image extraction features, and determine, based on the image extraction features, a target road detection value corresponding to each target pixel point in the target image.

In some embodiments, the server may obtain, from the target image, an image region corresponding to a feature value in the image extraction feature, to obtain an image region corresponding to each feature value, and determine, based on the feature value, a target road detection value corresponding to each target pixel point in the image region, for example, the feature value may be used as the target road detection value corresponding to each target pixel point in the corresponding image region, or the feature value may be scaled, the scaled feature value is used as the scaled feature value, and the scaled feature value is used as the target road detection value corresponding to each target pixel point in the corresponding image region.

S206, determining pixel conversion values corresponding to all target pixel points in the target image based on the target road detection values, and arranging the pixel conversion values corresponding to the target pixel points according to the pixel point arrangement sequence to obtain a conversion image corresponding to the target image.

The pixel conversion value is determined according to the target road detection value, the pixel conversion value can be any one of a road pixel value or a shielding pixel value, the road pixel value is used for indicating that the pixel point is a road pixel point, the shielding pixel value is used for indicating that the pixel point is a non-road pixel point, and the non-road pixel point is a pixel point except the road pixel point in the image. And converting the pixel values corresponding to the pixel points in the image into pixel conversion values. The pixel arrangement sequence refers to the position of the target pixel in the target image.

Specifically, the server may obtain, from each target pixel point of the target image, a target pixel point satisfying a screening condition of the road detection value based on the target road detection value, as a road pixel point, use the road pixel value as a pixel conversion value corresponding to the road pixel point, use the target pixel point not satisfying the screening condition of the road detection value as a non-road pixel point, and use the masking pixel value as a pixel conversion value corresponding to the non-road pixel point in the target image. The road detection value screening condition may include at least one of the target road detection value being greater than the detection value threshold value or the road detection value being ranked before the detection value ranking threshold value. The masked pixel value is different from the road pixel value, and the masked pixel value is used to indicate that the pixel point is a non-road pixel point, for example, the road pixel value may be 1, and the masked pixel value may be 0.

The detection value threshold may be preset, and may be a fixed value such as 0.8 or 0.9, or the detection value threshold may be calculated for each target road detection value, for example, the server may perform a mean calculation for each target road detection value to obtain a road detection value mean value, obtain a preset detection value coefficient, perform a multiplication operation on the detection value coefficient and the detection value mean value, and use the result of the multiplication operation as the detection value threshold. The detection value coefficient may be a value greater than 1, for example, 1.2.

The road detection value sequence is a sequence in which the target road detection values are arranged in a road detection value sequence, the road detection value sequence is a sequence in which the target road detection values are arranged in descending order, and the higher the target road detection value is, the earlier the target road detection value is in the road detection value sequence.

The detection value ranking threshold may be set in advance, for example, may be a fixed value such as 9 or 10, or may be calculated from the number of target road detection values, and for example, the server may acquire a preset ranking coefficient, multiply the ranking coefficient by the number of target road detection values, and use the result of the multiplication as the detection value ranking threshold. The ranking factor may be a value less than 1, and may be a fixed value such as 0.5 or 0.3.

In some embodiments, the server may compare the target road detection value with a detection value threshold, and when the target road detection value is greater than the detection value threshold, take the target pixel as a road pixel, and when the target road detection value is less than the detection value threshold, take the target pixel as a non-road pixel.

In some embodiments, the server may compare the target road detection value to a detection value threshold, when the target road detection value is greater than the detection value threshold, the target pixel is used as a primary selected pixel, the server can obtain the road pixels based on the primary selected pixel, for example, the server can use each primary selected pixel as a road pixel, or further screening each initially selected pixel point to obtain the road pixel points, for example, the server may arrange the pixel points in the order from large to small based on the target road detection values, the primary selection pixel points are arranged to obtain a primary selection pixel point sequence, the larger the target road detection value is, the more forward the primary selection pixel points are sorted in the primary selection pixel point sequence, the server can obtain all the primary selection pixel points of which the pixel points are sorted before a pixel point sorting threshold from the primary selection pixel point sequence, and all the primary selection pixel points of which the pixel points are sorted before the pixel point sorting threshold are used as road pixel points. The pixel point sorting refers to the sorting of the primary selected pixel points in the primary selected pixel point sequence. In the embodiment, the primary selection pixel points are obtained by screening according to the target road detection value, and then all the primary selection pixel points are sequenced, so that the sequencing speed can be increased, and the calculation efficiency is improved.

In some embodiments, the server may arrange the target pixel points according to positions of the target pixel points in the target image to obtain a converted image corresponding to the target image, where the positions of the same target pixel points in the target image are consistent with the positions of the same target pixel points in the converted image.

In some embodiments, the arranging the pixel conversion values corresponding to the target pixel point according to the pixel point arrangement order to obtain the conversion image corresponding to the target image includes: and updating the pixel value corresponding to the target pixel point in the target image into the pixel conversion value corresponding to the target pixel point, and taking the updated target image as a conversion image.

S208, determining the transformation value change information of the target pixel point in the transformation image relative to the reference pixel point in the transformation image, and determining the edge pixel corresponding to the road in the target image based on the transformation value change information.

The reference pixel point is a pixel point having a position association relationship with the target pixel point, the position association relationship may include at least one of adjacent pixels or a pixel distance smaller than a first pixel distance threshold, the pixel distance refers to a distance between two pixel points, the first pixel distance threshold may be preset or may be set as needed, and the reference pixel point may be, for example, a pixel point adjacent to the target pixel point.

The conversion value change information is used for reflecting the difference between the pixel conversion value of the target pixel point and the pixel conversion value of the reference pixel point, the larger the conversion value change information is, the larger the difference between the pixel conversion value of the target pixel point and the pixel conversion value of the reference pixel point is, and the smaller the conversion value change information is, the smaller the difference between the pixel conversion value of the target pixel point and the pixel conversion value of the reference pixel point is. Edge pixels refer to pixel points located at the edge of the road.

Specifically, the server may acquire a pixel conversion value corresponding to the target pixel point as a target pixel conversion value, acquire a pixel conversion value of a reference pixel point corresponding to the target pixel point as a reference pixel conversion value, calculate a difference between the target pixel conversion value and the reference pixel conversion value as a conversion value difference, obtain conversion value change information based on the conversion value difference using the calculated difference as a conversion value difference.

In some embodiments, the reference pixel of the target pixel may include an adjacent pixel of the target pixel, where the adjacent pixel refers to a pixel adjacent to the target pixel in a pixel arrangement direction, and the pixel arrangement direction includes at least one of a horizontal pixel arrangement direction or a vertical pixel arrangement direction. The horizontal pixel arrangement direction is, for example, a horizontal direction, and the vertical pixel arrangement direction is, for example, a vertical direction. The adjacent pixel points can comprise a transverse adjacent pixel point and a longitudinal adjacent pixel point, wherein the transverse adjacent pixel point refers to a pixel point adjacent to the target pixel point at the transverse adjacent pixel point, and the longitudinal adjacent pixel point refers to a pixel point adjacent to the target pixel point at the longitudinal adjacent pixel point. The server may obtain a lateral conversion value difference based on a difference between a pixel conversion value of a target pixel point and a pixel conversion value of a laterally adjacent pixel point, obtain a longitudinal conversion value difference based on a difference between the pixel conversion value of the target pixel point and a pixel conversion value of a longitudinally adjacent pixel point, and determine conversion value change information based on the lateral conversion value difference and the longitudinal conversion value difference.

In some embodiments, the server may perform a statistical operation based on the lateral conversion value difference and the longitudinal conversion value difference to obtain a conversion value change degree, perform a ratio operation based on the lateral conversion value difference and the longitudinal conversion value difference to obtain a conversion value change direction angle, and use the conversion value change degree and the conversion value change direction angle as conversion value change information. The conversion value change degree is the change magnitude of the pixel conversion value and is used for reflecting the change speed of the pixel conversion value, and the conversion value change direction angle is the change direction of the pixel conversion value.

And S210, identifying and obtaining a target road in the target image based on the edge pixels.

Specifically, the target road refers to a road in the target image. The server may fit each edge pixel, and use a straight line obtained by the fitting as a road edge line of the target road, where the road edge line is a straight line where an edge of the road is located.

In some embodiments, the server may obtain a plurality of edge pixels from each edge pixel to form an edge pixel set, where the plurality refers to at least two, for example, the server may obtain all edge pixels to form the edge pixel set, or obtain a specific number of edge pixels from each edge pixel to form the edge pixel set, where the specific number may be preset or calculated according to a total number of edge pixels, where the total number of edge pixels refers to a number of all edge pixels, and may obtain a number coefficient, where the number coefficient is multiplied by the total number of edge pixels, a result of the operation is used as the feature number, and the number coefficient may be a preset positive number smaller than 1, and may be 0.6, for example. The server can determine edge lines corresponding to each edge pixel in the edge pixel set, the edge lines are straight lines passing through the edge pixels, the edge lines can correspond to edge line parameters, the edge line parameters can uniquely determine a straight line, the edge lines corresponding to different edge line parameters are different, and different edge lines can be obtained by adjusting the edge line parameters. The server can adjust parameters of edge lines corresponding to the edge pixels in the edge pixel set respectively, and obtain road edge lines corresponding to the target road based on the adjusted edge lines.

In some embodiments, the edge line parameters include an edge line distance and an edge line included angle, where the edge line distance refers to a distance between the edge line and a reference position, and the reference position may be a preset position or a position set according to needs, for example, may be a coordinate origin, that is, an intersection of coordinate axes. The edge line included angle is an included angle between the edge line and a reference direction line, and the reference direction line may be preset or may be set according to needs, and may be any one of a horizontal axis (X axis) or a vertical axis (Y axis), for example. The server can adjust the edge line parameters corresponding to each edge pixel in the edge pixel point set until the parameter adjustment stopping condition is met, and determine the road edge line corresponding to the target road based on the edge line corresponding to the edge line parameter meeting the parameter adjustment stopping condition. Wherein the parameter adjustment stop condition may include at least one of a difference between respective edge line distances being less than a distance difference threshold or a difference between respective edge line angles being less than an angle difference threshold. The distance difference threshold and the angle difference threshold may be preset.

In some embodiments, the server may take any one of the edge lines corresponding to the edge line parameter of the parameter adjustment stop condition as the road edge line corresponding to the target road, for example, the edge pixel set includes 10 edge pixels, the adjusted edge lines corresponding to the 10 edge pixels are obtained by adjusting the edge line parameter of the edge line corresponding to the 10 edge pixels, and when the parameter adjustment stop condition is satisfied, the adjusted edge line corresponding to any one of the 10 edge pixels may be taken as the road edge line of the target road.

In some embodiments, the server may obtain each edge line parameter satisfying the parameter adjustment stop condition, perform statistical operation on each edge line parameter to obtain a parameter statistical value, and use a straight line determined by the parameter statistical value as a road edge line. The statistical operation may be, for example, a mean operation. The parameter statistics include at least one of an angle statistic or a distance statistic.

In some embodiments, the server may determine a road edge line of the target road based on the edge pixels, may determine a road center line and a direction of the road based on the road edge line, and may calculate a relative positional relationship between the target road and the terminal, thereby obtaining the relative positional relationship between the target road and the user.

In the method for identifying the road, a target image to be identified is obtained, the target image is subjected to road detection to obtain a target road detection value corresponding to each target pixel point in the target image, pixel conversion values corresponding to each target pixel point in the target image are determined based on the target road detection value, the pixel conversion values corresponding to the target pixel points are arranged according to the pixel point arrangement sequence to obtain a conversion image corresponding to the target image, conversion value change information of the target pixel points in the conversion image relative to reference pixel points in the conversion image is determined, edge pixels corresponding to the road in the target image are determined based on the conversion value change information, the target road in the target image is obtained based on the edge pixel identification, and the image can be converted based on the target road detection value, so that the conversion image better reflects the road information in the image, therefore, the corresponding edge pixels in the target image can be accurately detected based on the conversion value change information of the target pixel points relative to the reference pixel points in the conversion image, and the accuracy of road identification is improved.

The road identification method can realize real-time detection of the blind road and the pedestrian crossing based on the segmentation network, and real-time segmentation and detection of the blind road and the pedestrian crossing in the image in the video stream are realized by utilizing the image processing and deep neural network technology, so that the positions of the blind road area and the pedestrian crossing area in the image can be accurately obtained, and the relative positions of the blind road and the pedestrian crossing and the blind person can be determined.

The road identification method can be deployed in a mobile phone or wearable equipment, for example, can be deployed in blind guiding glasses, when the road identification method is deployed in the blind guiding glasses, the blind guiding glasses can identify the position of the road in the image by using the collected road image through the road identification method provided by the application, for example, the edge line and the central line of the road in the image are identified, the positions and the directions of the blind road and the pedestrian crossing can be pointed out, and the blind can be guided to travel, so that the travel mode of the blind is improved, the travel safety of the blind is improved, and the travel danger of the blind is reduced.

In some embodiments, the performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image includes: acquiring a target user type corresponding to a target image; acquiring a trained road detection model corresponding to the target user type, wherein the trained road detection model is obtained by training a target training image corresponding to the target user type; and inputting the target image into a trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image.

The target user type refers to a type of a user to which a terminal acquiring the target image belongs, that is, the target image is a road image acquired by a terminal carried by a user of the target user type, for example, when the target image is a road image acquired by a terminal carried by a walking user with visual disorder, the target user is the walking user with visual disorder, and the target user refers to a user of the target user type. The target training image is a road image corresponding to the target user type and is used for training a road detection model corresponding to the target user type. The target training images may be images taken under different environmental factors, for example, images taken under different lighting or weather conditions.

The road image corresponding to the user type may be a road image collected by a terminal carried by the user of the user type, for example, a road image collected by blind-guide glasses carried by a user with visual impairment. The road image corresponding to the user type can also be a road image acquired by a virtual user corresponding to the user type. The virtual user refers to a device with a movement mode consistent with a walking mode of a real user on a road, the virtual user can move on the road according to the walking mode of the real user on the road, and the virtual user can be at least one of a robot imitating the walking of a blind person or an aircraft with a movement mode consistent with the walking mode of the blind person. For example, the server may obtain road behavior information of the user, and control the virtual user to move based on the road behavior information, where the road behavior information refers to movement information of the target user on a road, and may include a speed of the movement.

Specifically, the server may store trained road detection models corresponding to various user types, where the trained road detection model corresponding to each user type may be a model trained by using a road image corresponding to the user type, for example, the trained road detection model corresponding to the user with the visual impairment may be a model trained by using a road image corresponding to the user with the visual impairment.

In some embodiments, the server may input the target image into a trained road detection model for road detection, obtain a target road detection value corresponding to each target pixel point in the target image, and determine a pixel transformation value corresponding to the target pixel point based on the target road detection value. The road detection model may be, for example, the segmentation algorithm model in (b) of fig. 3, the image in the real-time video stream including the blind road in (a) of fig. 3 is input into the segmentation algorithm model, a road detection value corresponding to each pixel point of the image in the video stream may be obtained, a pixel conversion value corresponding to each pixel point may be obtained according to the road detection value, a converted image is obtained based on the pixel conversion value, fig. 3 (c) is the converted image, it can be seen from the image that the pixel conversion value of the pixel point in the blind road region is a pixel value corresponding to a white region, and the pixel conversion value of the pixel point outside the blind road region is a pixel value corresponding to a black region.

In this embodiment, a target user type corresponding to a target image is obtained, a trained road detection model corresponding to the target user type is obtained, the target image is input into the trained road detection model for road detection, and a target road detection value corresponding to each target pixel point in the target image is obtained.

In some embodiments, the step of obtaining a trained road detection model comprises: determining a target user type corresponding to a road detection model to be trained; acquiring candidate training images corresponding to a plurality of target user types from the candidate training image set as target training images; acquiring a road position corresponding to a road in a target training image, and determining a pixel label value corresponding to each training pixel point in the target training image based on the road position; and training the road detection model to be trained based on the target training image and the pixel label value to obtain the trained road detection model.

The candidate training image set may include a plurality of candidate training images, where the plurality refers to at least two candidate training images, and the candidate training images belong to road images. The candidate training images may include candidate training images corresponding to a plurality of user types. The trained road detection model may be obtained through one or more times of training, that is, through one or more times of parameter adjustment, where the parameter adjustment refers to adjusting model parameters of the road detection model. The model parameters refer to variable parameters inside the model, and may also be referred to as neural network weights (weights) for the neural network model.

The road detection model to be trained refers to a road detection model to be trained, and may be an untrained road detection model or a road detection model trained in one or more rounds. The road detection model may be a deep neural network model, the deep neural network model may have a multi-layer neural network structure, the neural network structure may include a plurality of stacked convolutional layers and may further include a pooling layer, and the neural network structure may further be connected across layers. Feature extraction refers to extracting image information to obtain features of an image. For example, feature extraction may be performed using a convolution kernel to obtain a feature map (feature map) output by each neural network layer, where the feature map is a feature of an image obtained by processing, for example, performing convolution processing on an input image using model parameters. The road detection model may be a model based on a semantic segmentation algorithm.

The road position refers to a position of a road in the road image. The training pixel points refer to pixel points in the target training image. The pixel label value is used for determining the type of the training pixel point, the type of the training pixel point may be any one of a road pixel point and a non-road pixel point, and the pixel label value may be any one of a road label value and a non-road label value, where the road label value is used for indicating that the training pixel point is a road pixel point, the non-road label value is used for indicating that the training pixel point is a non-road pixel point, the road label value is different from the non-road label value, the road label value and the non-road label value may be preset, or may be determined as needed, the road label value may be 1, and the non-road label value may be 0, for example. As shown in fig. 4, (a1) in fig. 4 is an image including only a pedestrian crossing, (b1) in fig. 4 is an image including only a blind sidewalk, (c1) in fig. 4 is an image including both a pedestrian crossing and a blind sidewalk, (a2) in fig. 4 shows pixel label values corresponding to the respective pixel points in (a1), (b2) in fig. 4 shows pixel label values corresponding to the respective pixel points in (b1), and (c2) in fig. 4 shows pixel label values corresponding to the respective pixel points in (c 1).

Specifically, when it is determined that the road detection model to be trained is a model for performing road detection on a road image of a target user type, the server may acquire a candidate training image corresponding to the target user type from the candidate training image set, and use the candidate training image corresponding to the target user type as the target training image. The server can determine each training pixel point at the road position from the target training image, use the road label value as the pixel label value corresponding to the training pixel point at the road position, determine each training pixel point outside the road position, and use the non-road label value as the pixel label value corresponding to the training pixel point outside the road position.

In some embodiments, the server may input the target training image into a road detection model to be trained for road detection, so as to obtain a road detection result, where the road detection result may include a training road detection value corresponding to each training pixel point, and the training road detection value is a road detection value obtained by performing road detection on the target training image by the road detection model. The server can determine a model loss value based on the training road detection value and the pixel label value corresponding to each training pixel point, and adjust the model parameters of the road detection model based on the model loss value to obtain the trained road detection model. When the pixel label value of the training pixel point is a non-road label value, the model loss value and the training road detection value form a positive correlation relationship.

Wherein, the positive correlation refers to: under the condition that other conditions are not changed, the changing directions of the two variables are the same, and when one variable changes from large to small, the other variable also changes from large to small. It is understood that a positive correlation herein means that the direction of change is consistent, but does not require that when one variable changes at all, another variable must also change. For example, it may be set that the variable b is 100 when the variable a is 10 to 20, and the variable b is 120 when the variable a is 20 to 30. Thus, the change directions of a and b are both such that when a is larger, b is also larger. But b may be unchanged in the range of 10 to 20 a. The negative correlation relationship refers to: under the condition that other conditions are not changed, the changing directions of the two variables are opposite, and when one variable is changed from large to small, the other variable is changed from small to large. It is understood that the negative correlation herein means that the direction of change is reversed, but it is not required that when one variable changes at all, the other variable must also change.

In this embodiment, a target user type corresponding to a road detection model to be trained is determined, candidate training images corresponding to a plurality of target user types are obtained from a candidate training image set and serve as target training images, road positions corresponding to roads in the target training images are obtained, pixel label values corresponding to training pixel points in the target training images are determined based on the road positions, the road detection model to be trained is trained based on the target training images and the pixel label values, the trained road detection model is obtained, the road detection model is trained by using the road images corresponding to the target user types, the detection capability of the road detection model on the road images of the target user types is improved, and the road detection accuracy is improved.

In some embodiments, the step of obtaining the target training image comprises: acquiring road behavior information of a target user corresponding to the type of the target user; generating mobile control information corresponding to the virtual user based on the road behavior information; and controlling the virtual user to move along the road by using the movement control information, and acquiring an image acquired by the virtual user in the moving process as a target training image.

The target user refers to a user of a target user type, and the road behavior information refers to movement information of the target user on a road, which may include a movement speed. The movement control information is information for controlling the movement of the virtual user, and may control the virtual user to move along the road in such a manner that the target user travels on the road, for example, to move along the road at the speed at which the target user travels on the road. The height between the image acquisition equipment installed on the virtual user and the ground can be consistent with the height between the image acquisition equipment carried by the target user and the ground, the angle of the image acquired by the image acquisition equipment installed on the virtual user can be consistent with the angle of the image acquired by the image acquisition equipment carried by the target user, for example, the angle of the acquired image is the dead ahead, and the dead ahead refers to the dead ahead of the virtual user or the dead ahead of the target user. Of course, the angle at which the image is acquired may be other angles, and is not limited herein.

Specifically, when the virtual user moves along the road, the server may control the virtual user to perform image acquisition on the road to obtain a road image, the virtual user may send the acquired road image to the server, and the server may use the road image sent by the virtual user as the target training image.

In this embodiment, road behavior information of a target user corresponding to the type of the target user is obtained, movement control information corresponding to a virtual user is generated based on the road behavior information, the virtual user is controlled to move along a road by using the movement control information, an image acquired by the virtual user in the moving process is obtained and used as a target training image, so that the mode of acquiring the target training image is consistent with the mode of acquiring the image by equipment of a real user, and the accuracy of model training is improved when the target training image is used for model training.

In some embodiments, determining pixel transform values corresponding to respective target pixel points in the target image based on the target road detection values comprises: based on the target road detection value, acquiring target pixel points meeting the road detection value screening condition from all target pixel points of the target image, and taking the target pixel points as road pixel points; and taking the road pixel value corresponding to the road as the pixel conversion value corresponding to the road pixel point, and taking the shielding pixel value as the pixel conversion value corresponding to the non-road pixel point in the target image.

Wherein the road detection value screening condition may include at least one of the target road detection value being greater than the detection value threshold value or the road detection value being ranked before the detection value ranking threshold value. The non-road pixel points refer to target pixel points which do not meet the screening condition of the road detection values.

In this embodiment, based on the target road detection value, the target pixel points satisfying the road detection value screening condition are obtained from each target pixel point of the target image, and are used as the road pixel points, the road pixel values corresponding to the road are used as the pixel conversion values corresponding to the road pixel points, and the shielding pixel values are used as the pixel conversion values corresponding to the non-road pixel points in the target image, so that the road pixel points and the non-road pixel points are quickly and accurately marked out through the road detection value.

In some embodiments, determining transform value change information for a target pixel point in the transformed image relative to a reference pixel point in the transformed image comprises: obtaining a current pixel point from each pixel point of the transformed image; acquiring adjacent pixel points of the current pixel points from the transformed image, and determining reference pixel points corresponding to the current pixel points based on the adjacent pixel points; and obtaining the conversion value difference between the current pixel point and the reference pixel point corresponding to the current pixel point, and obtaining the conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference.

The current pixel point may be any pixel point in the transformed image. The adjacent pixel point of the current pixel point refers to a pixel point adjacent to the current pixel point in each pixel arrangement direction. The pixel arrangement direction may include at least one of a lateral arrangement direction or a longitudinal arrangement direction. There may be a plurality of adjacent pixels of the current pixel.

The adjacent pixel points can comprise at least one of a transverse adjacent pixel point or a longitudinal adjacent pixel point, the transverse adjacent pixel point is a pixel point adjacent to the current pixel point in the transverse arrangement direction, the longitudinal adjacent pixel point is a pixel point adjacent to the current pixel point in the longitudinal arrangement direction, the transverse adjacent pixel point can comprise at least one of a forward transverse adjacent pixel point or a backward transverse adjacent pixel point, the longitudinal adjacent pixel point can comprise at least one of a forward longitudinal adjacent pixel point or a backward longitudinal adjacent pixel point, the forward transverse adjacent pixel point is a transverse adjacent pixel point adjacent to the current pixel point and arranged in front of the current pixel point, and the backward transverse adjacent pixel point is a transverse adjacent pixel point adjacent to the current pixel point and arranged behind the current pixel point. The forward longitudinal adjacent pixel point refers to a longitudinal adjacent pixel point which is adjacent to the current pixel point and arranged in front of the current pixel point, and the backward longitudinal adjacent pixel point refers to a longitudinal adjacent pixel point which is adjacent to the current pixel point and arranged behind the current pixel point.

The reference pixel point may include at least one adjacent pixel point among adjacent pixel points of the current pixel point. For example, the reference pixel points may include at least one of backward horizontally adjacent pixel points or backward vertically adjacent pixel points.

Specifically, the server may determine the conversion value change information based on a difference between the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point. For example, the server may calculate differences between the current pixel conversion value and each of the reference pixel conversion values by using the pixel conversion value of the current pixel point as the current pixel conversion value and the pixel conversion value of the reference pixel point as the reference pixel conversion value, respectively, to obtain each conversion value difference, and obtain conversion value change information based on each conversion value difference.

In some embodiments, the server may perform a statistical operation on each of the transformed value differences to obtain a transformed value change degree, may perform a ratio operation on each of the transformed value differences to obtain a transformed value change degree, and may use the transformed value change degree and the transformed value change degree as transformed value change information.

In this embodiment, the current pixel point is obtained from each pixel point of the transformed image, the adjacent pixel point of the current pixel point is obtained from the transformed image, the adjacent pixel point is used as the reference pixel point corresponding to the current pixel point, the conversion value difference between the current pixel point and the reference pixel point corresponding to the current pixel point is obtained, the conversion value change information of the current pixel point relative to the reference pixel point is obtained based on the conversion value difference, and the accuracy of the conversion value change information is improved.

In some embodiments, the reference pixel point of the current pixel point includes an adjacent pixel point in a plurality of pixel arrangement directions, and the conversion value change information includes a conversion value change degree and a conversion value change direction angle; obtaining a conversion value difference between a current pixel point and a reference pixel point corresponding to the current pixel point, wherein obtaining conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference comprises: obtaining a conversion value difference corresponding to the current pixel point in each pixel arrangement direction based on the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point of the current pixel point in each pixel arrangement direction; and determining the conversion value change degree and the conversion value change direction angle of the current pixel point relative to the reference pixel point by combining the conversion value difference corresponding to each pixel arrangement direction.

The reference pixel point of the current pixel point includes an adjacent pixel point in each pixel arrangement direction, for example, the reference pixel point of the current pixel point may include at least one of a backward horizontally adjacent pixel point or a backward vertically adjacent pixel point. The backward horizontally adjacent pixel points refer to pixel points arranged behind the current pixel points in the horizontal arrangement direction, and the backward vertically adjacent pixel points refer to pixel points arranged behind the current pixel points in the vertical arrangement direction.

Specifically, the server may calculate a difference between the pixel transformation value of the current pixel point and the pixel transformation value of the backward horizontally adjacent pixel point, and obtain a horizontal difference value based on the difference, for example, a horizontal difference value corresponding to the current pixel point may be obtained based on a result obtained by subtracting the pixel transformation value of the current pixel point from the pixel transformation value of the backward horizontally adjacent pixel point, and for example, a result obtained by subtracting the pixel transformation value of the current pixel point from the pixel transformation value of the backward horizontally adjacent pixel point may be used as the horizontal difference value corresponding to the current pixel point. The server may calculate a difference between the pixel transformation value of the current pixel point and the pixel transformation value of the backward longitudinally adjacent pixel point, and obtain a longitudinal difference value corresponding to the current pixel point based on the difference, for example, obtain a longitudinal difference value based on a result obtained by subtracting the pixel transformation value of the current pixel point from the pixel transformation value of the backward longitudinally adjacent pixel point, and for example, obtain a result obtained by subtracting the pixel transformation value of the current pixel point from the pixel transformation value of the backward longitudinally adjacent pixel point as the longitudinal difference value corresponding to the current pixel point. The server may use the lateral difference value and the longitudinal difference value as the transformation value difference.

In some embodiments, the server may use the pixel transformation value corresponding to the current pixel point as the current pixel transformation value, use the pixel transformation value of the backward horizontally adjacent pixel point corresponding to the current pixel point as the backward horizontally transformation value, use the pixel transformation value of the backward vertically adjacent pixel point corresponding to the current pixel point as the first backward vertically transformation value, and use the pixel transformation value of the backward vertically adjacent pixel point corresponding to the backward horizontally adjacent pixel point of the current pixel point as the second backward vertically transformation value. The server may calculate a difference between the current pixel transformation value and the backward horizontal transformation value, and use the difference as a first difference value, for example, a result of subtracting the current pixel transformation value from the backward horizontal transformation value may be used as the first difference value, a difference between the first backward vertical transformation value and the second backward vertical transformation value may be calculated, and the difference may be used as a second difference value, for example, a result of subtracting the first backward vertical transformation value from the second backward vertical transformation value may be used as the second difference value, and perform a statistical operation, for example, a mean operation, on the first difference value and the second difference value, and use a result of the operation as a horizontal difference value corresponding to the current pixel point.

In some embodiments, the server may calculate a difference between the current pixel transformation value and the first backward longitudinal transformation value as a third difference value, for example, a result of subtracting the current pixel transformation value from the first backward longitudinal transformation value may be used as a third difference value, a difference between the backward lateral transformation value and the second backward longitudinal transformation value may be calculated as a fourth difference value, for example, a result of subtracting the backward lateral transformation value from the second backward longitudinal transformation value may be used as a fourth difference value, and perform a statistical operation, for example, a mean operation, on the third difference value and the fourth difference value, and use a result of the operation as a longitudinal difference value corresponding to the current pixel point.

In some embodiments, the server may determine a transformation value change degree and a transformation value change direction angle based on the lateral difference value and the longitudinal difference value, for example, the server may perform a statistical operation based on the lateral difference value and the longitudinal difference value to obtain the transformation value change degree, and perform a ratio operation based on the lateral difference value and the longitudinal difference value to obtain the transformation value change direction angle.

In this embodiment, based on the pixel conversion value of the current pixel and the pixel conversion value of the reference pixel in which the current pixel is located in each pixel arrangement direction, the conversion value difference corresponding to the current pixel in each pixel arrangement direction is obtained, and the conversion value change degree and the conversion value change direction angle of the current pixel relative to the reference pixel are determined by combining the conversion value difference corresponding to each pixel arrangement direction, so that the accuracy of the conversion value change information is improved.

In some embodiments, determining the transformation value change degree and the transformation value change direction angle of the current pixel point relative to the reference pixel point by combining the transformation value difference corresponding to each pixel arrangement direction includes: carrying out statistical operation on the conversion value difference corresponding to the current pixel point in each pixel arrangement direction to obtain the conversion value change degree; and taking the conversion value difference corresponding to each pixel arrangement direction of the current pixel point as the side length of the direction side corresponding to the pixel arrangement direction, determining the angle of the connecting side of the connecting direction side based on the side length, and taking the angle as the conversion value change direction angle.

The direction side is a line segment whose length is the difference of the transformed values, the direction side may include at least one of a transverse direction side and a longitudinal direction side, the transverse direction side is a line segment whose length is the difference of the transverse direction in the transverse pixel arrangement direction, and the longitudinal direction side is a line segment whose length is the difference of the longitudinal direction in the longitudinal pixel arrangement direction. The transverse direction side, the longitudinal direction side and the connecting side can form a right triangle, and the connecting side is the hypotenuse of the right triangle. The angle of the connecting edge refers to an included angle between the connecting edge and the arrangement direction of the transverse pixels.

Specifically, the transform value difference may include a lateral difference value and a longitudinal difference value. The server may perform a square operation on the horizontal difference value to obtain a horizontal square value, perform a square operation on the vertical difference value to obtain a vertical square value, perform a statistical operation on the horizontal square value and the vertical square value to obtain a conversion value change degree, for example, may perform a sum operation on the horizontal square value and the vertical square value, and use a result of the operation as the conversion value change degree, or may perform a sum operation on the horizontal square value and the vertical square value, perform an evolution operation on a result of the sum operation, and use a result of the evolution operation as the conversion value change degree.

In some embodiments, the server may calculate a ratio of the horizontal disparity value to the vertical disparity value to obtain a disparity ratio, and determine the transformation value change direction angle based on the disparity ratio, for example, the server may perform an arc tangent operation on the disparity ratio, and use the result of the operation as the transformation value change direction angle.

In this embodiment, a statistical operation is performed on the conversion value difference corresponding to each pixel arrangement direction of the current pixel point to obtain a conversion value change degree, the conversion value difference corresponding to each pixel arrangement direction of the current pixel point is used as a side length of a direction side corresponding to the pixel arrangement direction, an angle of a connection side connecting the direction sides is determined based on the side length, and the angle is used as a conversion value change direction angle, so that the accuracy of the conversion value change direction angle is improved.

In some embodiments, the transformation value change information includes a transformation value change degree and a transformation value change direction angle, and determining the edge pixel corresponding to the road in the target image based on the transformation value change information includes: acquiring pixel points meeting the change degree screening condition from all pixel points of the transformed image as candidate pixel points; obtaining the contrast conversion value change degree of the candidate pixel points on the conversion value change direction angle; determining the change degree difference of the change degree of the candidate pixel points relative to the change degree of the contrast change value; and when the variation difference is larger than the variation difference threshold, taking the candidate pixel point as an edge pixel corresponding to the road in the target image.

The change degree screening condition may include that the change degree of the conversion value is greater than the change degree of the conversion value of each associated pixel point, and the associated pixel point may be each pixel point whose distance from the target pixel point in the converted image is less than the second pixel distance threshold. The candidate pixel points refer to all pixel points which meet the change degree screening condition in the transformed image. The contrast conversion value change degree refers to a conversion value change degree corresponding to the contrast pixel point, and the contrast pixel point may include at least one of the pixel points whose distance from the candidate pixel point in the conversion value change direction angle is smaller than the third pixel distance threshold. The second pixel distance threshold and the third pixel distance threshold may be preset, or may be set as needed.

The change degree difference is a difference between the change degree of the conversion value of the candidate pixel and the change degree of the contrast conversion value, and for example, a result obtained by subtracting the change degree of the contrast conversion value from the change degree of the conversion value of the candidate pixel may be used as the change degree difference. The variation difference threshold may be preset, for example, may be 0, and may also be set as needed.

Specifically, the server can obtain the current target pixel point from each target pixel point of the transformed image, the current target pixel point can be any one of all target pixel points, the server can obtain all target pixel points of which the distance between the current target pixel point and the target pixel point is less than the pixel distance threshold value from all target pixel points (not including the current target pixel point) of the converted image, the target pixel points are used as associated pixel points corresponding to the current target pixel point, the conversion value change degrees of all the associated pixel points are obtained, the conversion value change degrees of the current target pixel point are respectively compared with the conversion value change degrees of all the associated pixel points, when it is determined that the degree of change of the transformation value of the current target pixel point is greater than the degree of change of the transformation value of each associated pixel point, and determining that the current target pixel point meets the variation degree screening condition, and taking the current target pixel point as a candidate pixel point.

In some embodiments, the server may compare the change degree difference with a change degree difference threshold by using a result obtained by subtracting the contrast change degree from the change degree of the candidate pixel point as a change degree difference, and when the change degree difference is greater than the change degree difference threshold, use the candidate pixel point as an edge pixel corresponding to the road in the target image.

In some embodiments, when the variance of the variance is less than the variance threshold, the server may update the pixel transform value of the candidate pixel point in the transformed image from the road pixel value to a non-edge pixel value indicating that the pixel point is not an edge pixel. Wherein the non-edge pixel value can be set according to the requirement, the non-edge pixel value is different from the road pixel value, for example, the non-edge pixel value can be the same as the mask pixel value, for example, the non-edge pixel value can be 0.

In this embodiment, a pixel point satisfying the condition of filtering the degree of change is obtained from each target pixel point of the transformed image, and is used as a candidate pixel point, the degree of change of the contrast transformation value of the candidate pixel point in the angle of the direction of change of the transformation value is obtained, the difference of the degree of change of the transformation value of the candidate pixel point relative to the degree of change of the contrast transformation value is determined, and when the difference of the degree of change is greater than the threshold of the difference of the degree of change, the candidate pixel point is used as an edge pixel corresponding to a road in the target image, so that the accuracy of the edge pixel is improved.

In some embodiments, the edge pixels are multiple, and identifying the target road in the target image based on the edge pixels includes: determining a current edge line corresponding to the edge pixel, determining the distance between the current edge line and the current edge line of the reference position, and determining the current edge line included angle between the current edge line and a preset reference direction line; taking the included angle between the current edge line distance and the current edge line as the current edge line parameter to be adjusted, adjusting the current edge line parameter of each current edge line until a parameter adjustment stopping condition is met, taking the edge direction line corresponding to the current edge line parameter meeting the parameter adjustment condition as the road edge line corresponding to the road in the target image, wherein the parameter adjustment stopping condition comprises at least one of the condition that the difference between the edge line distances is smaller than a distance difference threshold value or the difference between the included angles of the edge lines is smaller than an included angle difference threshold value; a target road in the target image is determined based on the road edge line.

Here, the edge line refers to a straight line passing through the edge pixel. The current edge line is a straight line passing through the edge pixel at the current moment, the current edge line distance is a distance between the current edge line and the reference position, and the current edge line included angle is an included angle between the current edge line and the reference direction line. The current edge image parameters comprise a current edge line distance and a current edge line included angle.

The reference position may be a preset position or a position set as needed, and may be, for example, a coordinate origin, that is, an intersection of coordinate axes. The reference direction line may be preset or may be set as needed, and may be any one of a horizontal axis (X axis) or a vertical axis (Y axis), for example.

Specifically, the server may calculate the difference between each current edge line distance and each other current edge line distance to obtain each edge line distance difference, for example, when the edge pixel is 3, there are 3 current edge lines, which are current edge line a, current edge line b and current edge line c, the current edge line distance of current edge line a is d1, the current edge line distance of current edge line b is d2, and the current edge line distance of current edge line c is d3, the difference between current edge line distance d1 and current edge line distance d2 may be calculated to obtain a first edge line distance difference, the difference between current edge line distance d1 and current edge line distance d3 may be calculated to obtain a second edge line distance difference, the difference between current edge line distance d2 and current edge line distance d3 may be calculated, and obtaining a third edge line distance difference, wherein the server can perform statistical operation, such as summation operation or mean value operation, on each edge line distance difference, the operation result is used as a distance difference statistical value, the distance difference statistical value is compared with a distance difference threshold value, and when the distance difference statistical value is smaller than the distance difference threshold value, the parameter adjustment stop condition is determined to be met.

In some embodiments, the server may calculate a difference between each current edge line included angle and another current edge line included angle to obtain a difference between each current edge line included angle, for example, if the current edge line included angle of the current edge line a is θ 1, the current edge line included angle of the current edge line b is θ 2, and the current edge line included angle of the current edge line c is θ 3, then calculate a difference between the current edge line included angle θ 1 and the current edge line included angle θ 2 to obtain a first difference between edge line included angles, calculate a difference between the current edge line included angle θ 1 and the current edge line included angle θ 3 to obtain a second difference between edge line included angles, calculate a difference between the current edge line included angle θ 2 and the current edge line included angle θ 3 to obtain a third difference between edge line included angles, and perform a statistical operation on the difference between each current edge line included angle, for example, a sum operation or a mean operation, the operation result is used as an included angle difference statistical value, the included angle difference statistical value is compared with an included angle difference threshold value, and when the included angle difference statistical value is smaller than the included angle difference threshold value, it is determined that the parameter adjustment stop condition is satisfied.

In some embodiments, the parameter adjustment stop condition is determined to be satisfied when the angle difference statistic is less than an angle difference threshold and the distance difference statistic is less than a distance difference threshold.

In some embodiments, the parameter adjustment stop condition may further include that the number of target parameters is greater than a parameter number threshold, and the number of target parameters is a maximum number of the same current edge line parameters in each current edge line parameter. Specifically, the server may use the current edge line distance and the current edge line angle as the current edge line parameters to be adjusted, adjust the current edge image parameters of each current edge line, that is, adjust each current edge line distance and each current edge line angle, classify each current edge line parameter, classify the same current edge line parameter into one class, obtain each current edge line parameter set, where the current edge line parameter in the current edge line parameter set is the same edge line parameter, count the number of each current edge line parameter set, obtain the parameter number corresponding to each current edge line parameter set, obtain the maximum parameter number from each parameter number, as the target parameter number, compare the target parameter number with the edge parameter number threshold, when it is determined that the target parameter number is greater than the parameter number threshold, when the condition that the parameter adjustment stop condition is satisfied is determined, the server may use the edge line corresponding to the current edge line parameter in the current edge line parameter set corresponding to the number of the target parameters as the road edge line corresponding to the road in the target image. The parameter number threshold may be preset, or may be calculated according to the number of edge pixels, for example, a number coefficient may be obtained, the number of edge pixels and the number coefficient are multiplied, a result of the operation is used as the parameter number threshold, and the number coefficient may be a preset positive number smaller than 1, and may be, for example, 0.8. The same current edge line parameter may include at least one of the same distance between the current edge lines or the same included angle between the current edge lines.

In some embodiments, the server may also use a tracking algorithm to fine tune the road marginality. The road edge lines can comprise left road edge lines and right road edge lines, the direction of a center line of the blind road can be calculated according to the left road edge lines and the right road edge lines, the server can send the positions of the road edge lines and the direction of the center line to the terminal through a protocol, the terminal can display the road edge lines and the center line in real time, and the terminal can also perform voice broadcast according to the direction of the center line to guide the blind person to move.

In this embodiment, a current edge line corresponding to an edge pixel is determined, a current edge line distance between the current edge line and a reference position is determined, a current edge line included angle between the current edge line and a preset reference direction line is determined, the current edge line distance and the current edge line included angle are used as current edge line parameters to be adjusted, the current edge line parameters of each current edge line are adjusted until a parameter adjustment stop condition is met, an edge line corresponding to the current edge line parameter meeting the parameter adjustment condition is used as a road edge line corresponding to a road in a target image, and since the parameter adjustment stop condition includes that a difference between each edge line distance is smaller than a distance difference threshold or a difference between each edge line included angle is smaller than at least one of angle difference thresholds, edge lines passing through as many edge pixels as possible at the same time can be obtained, and the target road in the target image is determined based on the road edge line, so that the road detection accuracy is improved.

In some embodiments, acquiring a target image to be subject to road recognition comprises: acquiring a target image uploaded by a terminal, and taking the target image as a target image to be subjected to road identification; the road detection of the target image to obtain the target road detection value corresponding to each target pixel point in the target image comprises the following steps: acquiring a target user type corresponding to a terminal, and acquiring a trained road detection model corresponding to the target user type, wherein the trained road detection model is obtained by training by using a target training image corresponding to the target user type; inputting the target image into a trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image; the method further comprises the following steps: determining user movement prompt information based on the position corresponding to the target road and the position of the terminal; and sending the user movement prompt information to the terminal so that the terminal carries out movement prompt according to the user movement prompt information.

The target user type corresponding to the terminal refers to the type of the user carrying the terminal. The position corresponding to the target road may be a position of the target road relative to the terminal, or a position of the target road in the image, where the position of the terminal is the position of the terminal, and the position of the terminal may be obtained in real time by a positioning system, and may be represented by latitude and longitude, or may be represented by other manners, which is not limited herein. The user movement prompt information is used for assisting the user to move, and may include at least one of a distance of the target road relative to the user or a direction of the target road relative to the user, for example, the user movement prompt information may be "go straight 50 meters ahead into the crosswalk" or "go crosswalk 50 meters ahead".

Specifically, the server may receive a target image sent by the terminal, and the server may determine a type of a target user according to the terminal, for example, when the terminal is a pair of blind-guiding glasses, it may be determined that the terminal corresponds to a blind person, and certainly, a user type corresponding to each user may also be stored in the server, and when the terminal sends the target image to the server, the terminal may also send a user identifier to the server, and the server may determine the user type based on the user identifier, and the user identifier is used to uniquely identify the user. The server may store a trained road detection model corresponding to each user type, and may acquire a model corresponding to a target user type from each trained road detection model, and perform road detection on a target image by using the acquired trained road detection model to obtain a target road detection value corresponding to each target pixel point in the target image. The server can determine pixel conversion values corresponding to the target pixel points based on the target road detection values, determine edge pixels of the target road in the target image based on the pixel conversion values, obtain road edge lines of the target road based on the edge pixels, and determine road center lines and road directions based on the road edge lines. As shown in fig. 5, the image in the video stream in (a) in fig. 5 is input to the trained model in (b) to obtain a road detection value corresponding to a pixel point, the converted image shown in (c) is obtained based on the road detection value, the road edge line and the road center line in (d) are obtained based on the converted image, and the direction of the arrow in (d) indicates the direction of the road.

In some embodiments, the server may obtain the location of the terminal in real time, for example, the terminal may be located by a location system and send location information to the server, and the server may determine a relative location between the target road and the terminal according to the location of the target road in the target image, generate user movement prompt information based on the relative location between the target road and the terminal, and send the user movement prompt information to the terminal. The terminal can display the user movement prompt information, play the user movement prompt information through voice, or convert the user movement prompt information into vibration information, and vibrate according to the vibration information, so that the user movement is assisted.

In this embodiment, a target image uploaded by a terminal is acquired, the target image is used as a target image to be subjected to road recognition, a target user type corresponding to the terminal is acquired, a trained road detection model corresponding to the target user type is acquired, the target image is input into the trained road detection model for road detection, a target road detection value corresponding to each target pixel point in the target image is acquired, user movement prompt information is determined based on a position corresponding to the target road and a position of the terminal, and the user movement prompt information is sent to the terminal, so that the terminal performs movement prompt according to the user movement prompt information, and therefore user movement can be assisted, and safety of movement on a road is improved.

In some embodiments, as shown in fig. 6, there is provided a road identification method, comprising the steps of:

and S602, determining a road detection model to be trained corresponding to the vision disorder user.

Wherein the road detection model may be a model based on a semantic segmentation algorithm. The model capable of separating blind roads and pedestrian crossings from background images can be obtained through training the road detection model.

S604, candidate training images collected by equipment carried by a plurality of vision disorder users are obtained from the candidate training image set and serve as target training images.

The target training image may include any one of a blind road or a pedestrian crossing. The target training images may be images taken under different environmental factors, for example, images taken under different lighting or weather conditions. The target training image may be an image acquired by simulating the visual angle of the blind, for example, a mobile phone or glasses with a camera may be hung on the chest like the blind, a video may be acquired in a real scene, and a video frame may be acquired from the video as the target training image, for example, an image may be acquired from the video in a manner of 1 frame per second. Therefore, the target training image is utilized for training, the influence of the model on light, weather, shooting angle and blind road species can be reduced, and the blind road and pedestrian crossing positions relative to the user can be detected more accurately and efficiently. The safety of the blind in going out is improved.

In some embodiments, the target training images may include positive samples that refer to images that include at least one of a blind road or a crosswalk, and negative samples that refer to images that do not include a blind road and a crosswalk.

S606, obtaining the road positions of blind roads and pedestrian crossings in the target training image, and determining pixel label values corresponding to training pixel points in the target training image based on the road positions.

The target training image can be outdoor data collected through the walking angle of the model blind person. The pixel label value of the pixel point at the road position is a road label value, and the pixel label value of the pixel point at the position other than the road position is a non-road label value. The road position refers to the position of a blind road or a pedestrian crossing in the image. The road position may be automatically obtained or manually marked by the user.

And S608, training the road detection model to be trained based on the target training image and the pixel label value to obtain the trained road detection model.

The trained road detection model may be a semantic segmentation model capable of detecting blind roads and pedestrian crossings from the image. After the trained road detection model is obtained, the trained road detection model can be deployed in a mobile phone or wearable equipment such as glasses.

S610, acquiring a road image sent by a terminal of a target user with visual impairment, and taking the road image as a target image to be subjected to road recognition.

S612, acquiring a trained road detection model corresponding to the vision disorder user, inputting the target image into the trained road detection model for road detection, and obtaining a target road detection value corresponding to each target pixel point in the target image.

The road segmentation model can determine the position of a blind road and the position of a pedestrian crossing from the image.

And S614, based on the target road detection value, acquiring target pixel points meeting the road detection value screening condition from all the target pixel points of the target image, taking the target pixel points as road pixel points, taking road pixel values corresponding to roads as pixel conversion values corresponding to the road pixel points, and taking the shielding pixel values as pixel conversion values corresponding to non-road pixel points in the target image.

And S616, arranging the pixel conversion values corresponding to the target pixel points according to the pixel point arrangement sequence to obtain a conversion image corresponding to the target image.

And S618, obtaining a current pixel point from each pixel point of the transformed image, obtaining an adjacent pixel point of the current pixel point from the transformed image, and determining a reference pixel point corresponding to the current pixel point based on the adjacent pixel point.

S620, obtaining the conversion value difference corresponding to the current pixel point in each pixel arrangement direction based on the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point of the current pixel point in each pixel arrangement direction.

And S622, carrying out statistical operation on the conversion value difference corresponding to the current pixel point in each pixel arrangement direction to obtain the conversion value change degree.

And S624, taking the conversion value difference corresponding to each pixel arrangement direction of the current pixel point as the side length of the direction side corresponding to the pixel arrangement direction, determining the angle of the connecting side of the connecting direction side based on the side length, and taking the angle as the conversion value change direction angle.

And S626, acquiring pixel points meeting the change degree screening condition from all pixel points of the converted image to serve as candidate pixel points, acquiring the contrast conversion value change degree of the candidate pixel points in the change direction angle of the conversion value, determining the change degree difference of the conversion value change degree of the candidate pixel points relative to the contrast conversion value change degree, and taking the candidate pixel points as edge pixels corresponding to the road in the target image when the change degree difference is greater than a change degree difference threshold.

S628, determining a current edge line corresponding to the edge pixel, determining a distance between the current edge line and the current edge line of the reference position, and determining an included angle between the current edge line and a preset reference direction line.

S630, taking the included angle between the distance of the current edge line and the current edge line as the current edge line parameter to be adjusted, and adjusting the current edge line parameter of each current edge line.

S632 determines whether the parameter adjustment stop condition is satisfied, if not, returns to step S630, and if so, executes step S634.

Wherein the parameter adjustment stopping condition comprises at least one of the difference between the distances of the edge lines being smaller than a distance difference threshold or the difference between the included angles of the edge lines being smaller than an included angle difference threshold.

And S634, taking the edge line corresponding to the current edge line parameter meeting the parameter adjusting condition as the road edge line corresponding to the road in the target image.

And S636, determining the target road in the target image based on the road edge line.

Wherein the server may determine the road centerline and the road direction based on the road edge lines. As shown in fig. 7A, the process of obtaining the road center line, the road edge line, and the road direction of the pedestrian crossing is shown, and as shown in fig. 7B, the process of obtaining the road center line, the road edge line, and the road direction of the blind road is shown.

In this embodiment, the server may calculate the directions of the blind road and the pedestrian crossing by using an edge detection algorithm, a statistical method, a tracking algorithm, and the like, so as to guide the blind to move forward.

It should be understood that although the various steps in the flow charts of fig. 2-7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-7 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.

In some embodiments, as shown in fig. 8, there is provided a road identification apparatus, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, and specifically includes: a target image obtaining module 802, a road detection value obtaining module 804, a transformed image obtaining module 806, an edge pixel determining module 808, and a target road obtaining module 810, wherein:

a target image obtaining module 802, configured to obtain a target image to be used for road identification.

A road detection value obtaining module 804, configured to perform road detection on the target image, so as to obtain a target road detection value corresponding to each target pixel point in the target image.

A transformed image obtaining module 806, configured to determine, based on the target road detection value, pixel transformation values corresponding to target pixel points in the target image, and arrange the pixel transformation values corresponding to the target pixel points according to a pixel point arrangement order to obtain a transformed image corresponding to the target image.

And the edge pixel determining module 808 is configured to determine conversion value change information of a target pixel point in the converted image relative to a reference pixel point in the converted image, and determine an edge pixel corresponding to a road in the target image based on the conversion value change information.

And a target road obtaining module 810, configured to obtain a target road in the target image based on edge pixel identification.

In some embodiments, the road detection value obtaining module 804 includes:

and the target user type acquisition unit is used for acquiring the target user type corresponding to the target image.

And the first road detection model acquisition unit is used for acquiring a trained road detection model corresponding to the target user type, and the trained road detection model is obtained by utilizing a target training image corresponding to the target user type for training.

And the first road detection value obtaining unit is used for inputting the target image into the trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image.

In some embodiments, the road detection model derivation module that derives a trained road detection model comprises:

and the target user type determining unit is used for determining the target user type corresponding to the road detection model to be trained.

And the first target training image obtaining unit is used for obtaining candidate training images corresponding to a plurality of target user types from the candidate training image set to be used as target training images.

And the pixel label value obtaining unit is used for obtaining the road position corresponding to the road in the target training image and determining the pixel label value corresponding to each training pixel point in the target training image based on the road position.

And the road detection model obtaining unit is used for training the road detection model to be trained based on the target training image and the pixel label value to obtain the trained road detection model.

In some embodiments, the target training image derivation module that derives the target training image includes:

and the road behavior information acquisition unit is used for acquiring the road behavior information of the target user corresponding to the target user type.

And the movement control information generating unit is used for generating movement control information corresponding to the virtual user based on the road behavior information.

And the second target training image obtaining unit is used for controlling the virtual user to move along the road by using the movement control information, and obtaining an image acquired by the virtual user in the moving process as a target training image.

In some embodiments, the transformed image derivation module 806 includes:

and the road pixel point obtaining unit is used for obtaining target pixel points meeting the screening condition of the road detection values from all the target pixel points of the target image based on the target road detection values, and the target pixel points are used as the road pixel points.

And the pixel conversion value obtaining unit is used for taking the road pixel value corresponding to the road as the pixel conversion value corresponding to the road pixel point and taking the shielding pixel value as the pixel conversion value corresponding to the non-road pixel point in the target image.

In some embodiments, the edge pixel determination module 808 includes:

and the current pixel point obtaining unit is used for obtaining the current pixel point from each pixel point of the transformed image.

And the reference pixel point obtaining unit is used for obtaining an adjacent pixel point of the current pixel point from the converted image and determining the reference pixel point corresponding to the current pixel point based on the adjacent pixel point.

And the conversion value change information obtaining unit is used for obtaining the conversion value difference between the current pixel point and the reference pixel point corresponding to the current pixel point and obtaining the conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference.

In some embodiments, the reference pixel point of the current pixel point includes an adjacent pixel point in a plurality of pixel arrangement directions, and the conversion value change information includes a conversion value change degree and a conversion value change direction angle; the conversion value change information obtaining unit is also used for obtaining the conversion value difference corresponding to the current pixel point in each pixel arrangement direction based on the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point of the current pixel point in each pixel arrangement direction; and determining the conversion value change degree and the conversion value change direction angle of the current pixel point relative to the reference pixel point by combining the conversion value difference corresponding to each pixel arrangement direction.

In some embodiments, the transformation value change information obtaining unit is further configured to perform statistical operation on a transformation value difference corresponding to each pixel arrangement direction of the current pixel point to obtain a transformation value change degree; and taking the conversion value difference corresponding to each pixel arrangement direction of the current pixel point as the side length of the direction side corresponding to the pixel arrangement direction, determining the angle of the connecting side of the connecting direction side based on the side length, and taking the angle as the conversion value change direction angle.

In some embodiments, the transform value change information includes a transform value change degree and a transform value change direction angle, and the edge pixel determination module 808 includes:

and the candidate pixel point obtaining unit is used for obtaining pixel points meeting the change degree screening condition from all the pixel points of the transformed image and taking the pixel points as candidate pixel points.

And the contrast transformation value change degree obtaining unit is used for obtaining the contrast transformation value change degree of the candidate pixel point on the change direction angle of the transformation value.

And the change degree difference obtaining unit is used for determining the change degree difference of the change degree of the conversion value of the candidate pixel point relative to the change degree of the contrast conversion value.

And the edge pixel obtaining unit is used for taking the candidate pixel points as edge pixels corresponding to the road in the target image when the change degree difference is greater than the change degree difference threshold value.

In some embodiments, the edge pixels are multiple, and the target road obtaining module 810 includes:

and the current edge line determining unit is used for determining a current edge line corresponding to the edge pixel, determining the distance between the current edge line and the current edge line of the reference position, and determining the current edge line included angle between the current edge direction line and a preset reference direction line.

And the road edge line obtaining unit is used for taking the current edge line distance and the current edge line included angle as current edge line parameters to be adjusted, adjusting the current edge line parameters of each current edge line until a parameter adjustment stopping condition is met, taking an edge direction line corresponding to the current edge line parameters meeting the parameter adjustment condition as a road edge line corresponding to the road in the target image, wherein the parameter adjustment stopping condition comprises at least one of the condition that the difference between the edge line distances is smaller than a distance difference threshold value or the difference between the edge line included angles is smaller than an included angle difference threshold value.

And a target road determination unit for determining a target road in the target image based on the road edge line.

In some embodiments, the target image obtaining module 802 is further configured to obtain a target image uploaded by the terminal, and use the target image as a target image to be subjected to road identification; the road detection value obtaining module 804 includes:

and the second road detection model acquisition unit is used for acquiring the target user type corresponding to the terminal and acquiring a trained road detection model corresponding to the target user type, wherein the trained road detection model is obtained by utilizing a target training image corresponding to the target user type for training.

And the second road detection value obtaining unit is used for inputting the target image into the trained road detection model for road detection to obtain the target road detection value corresponding to each target pixel point in the target image.

The device still includes:

and the user movement prompt information determining module is used for determining the user movement prompt information based on the position corresponding to the target road and the position of the terminal.

And the user movement prompt information sending module is used for sending the user movement prompt information to the terminal so that the terminal carries out movement prompt according to the user movement prompt information.

For the specific definition of the road identification device, reference may be made to the above definition of the road identification method, which is not described herein again. The various modules in the road identification device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a road identification method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 10. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data related to the road identification method. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a road identification method.

Those skilled in the art will appreciate that the configurations shown in fig. 9 and 10 are merely block diagrams of portions of configurations related to aspects of the present application, and do not constitute limitations on the computing devices to which aspects of the present application may be applied, as particular computing devices may include more or less components than shown, or combine certain components, or have a different arrangement of components.

In some embodiments, there is further provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above method embodiments when executing the computer program.

In some embodiments, a computer-readable storage medium is provided, in which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of road identification, the method comprising:

acquiring a target image to be subjected to road identification;

performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image;

determining pixel conversion values corresponding to target pixel points in the target image based on the target road detection values, and arranging the pixel conversion values corresponding to the target pixel points according to a pixel point arrangement sequence to obtain a conversion image corresponding to the target image;

determining transformation value change information of the target pixel points in the transformation image relative to reference pixel points in the transformation image, and determining edge pixels corresponding to roads in the target image based on the transformation value change information;

and identifying and obtaining a target road in the target image based on the edge pixels.

2. The method according to claim 1, wherein the performing road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image comprises:

acquiring a target user type corresponding to the target image;

acquiring a trained road detection model corresponding to the target user type, wherein the trained road detection model is obtained by utilizing a target training image corresponding to the target user type for training;

and inputting the target image into the trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image.

3. The method of claim 2, wherein the step of deriving the trained road detection model comprises:

determining a target user type corresponding to a road detection model to be trained;

acquiring a plurality of candidate training images corresponding to the target user types from a candidate training image set to serve as target training images;

acquiring a road position corresponding to a road in the target training image, and determining a pixel label value corresponding to each training pixel point in the target training image based on the road position;

and training the road detection model to be trained based on the target training image and the pixel label value to obtain a trained road detection model.

4. The method of claim 2, wherein the step of obtaining the target training image comprises:

acquiring road behavior information of a target user corresponding to the type of the target user;

generating mobile control information corresponding to the virtual user based on the road behavior information;

and controlling the virtual user to move along the road by using the movement control information, and acquiring an image acquired by the virtual user in the moving process as the target training image.

5. The method of claim 1, wherein the determining pixel transform values corresponding to respective target pixel points in the target image based on the target road detection values comprises:

based on the target road detection value, acquiring target pixel points meeting the road detection value screening condition from all target pixel points of the target image, and taking the target pixel points as road pixel points;

and taking the road pixel value corresponding to the road as the pixel conversion value corresponding to the road pixel point, and taking the shielding pixel value as the pixel conversion value corresponding to the non-road pixel point in the target image.

6. The method of claim 1, wherein the determining transformation value change information for the target pixel in the transformed image relative to a reference pixel in the transformed image comprises:

obtaining a current pixel point from each pixel point of the transformed image;

acquiring adjacent pixel points of the current pixel points from the converted image, and determining reference pixel points corresponding to the current pixel points based on the adjacent pixel points;

and obtaining the conversion value difference between the current pixel point and a reference pixel point corresponding to the current pixel point, and obtaining the conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference.

7. The method according to claim 6, wherein the reference pixel of the current pixel includes an adjacent pixel in a plurality of pixel arrangement directions, and the transformation value change information includes a transformation value change degree and a transformation value change direction angle;

the obtaining of the conversion value difference between the current pixel point and the reference pixel point corresponding to the current pixel point and the obtaining of the conversion value change information of the current pixel point relative to the reference pixel point based on the conversion value difference includes:

obtaining a conversion value difference corresponding to the current pixel point in each pixel arrangement direction based on the pixel conversion value of the current pixel point and the pixel conversion value of the reference pixel point of the current pixel point in each pixel arrangement direction;

and determining the conversion value change degree and the conversion value change direction angle of the current pixel point relative to the reference pixel point by combining the conversion value difference corresponding to each pixel arrangement direction.

8. The method of claim 7, wherein determining the transformation value change degree and the transformation value change direction angle of the current pixel point relative to the reference pixel point by combining the transformation value difference corresponding to each of the pixel arrangement directions comprises:

carrying out statistical operation on the conversion value difference corresponding to the current pixel point in each pixel arrangement direction to obtain the conversion value change degree;

and taking the conversion value difference corresponding to the current pixel point in each pixel arrangement direction as the side length of a direction side in the corresponding pixel arrangement direction, determining the angle of a connecting side connecting the direction sides based on the side length, and taking the angle as the conversion value change direction angle.

9. The method of claim 1, wherein the transformation value change information includes a transformation value change degree and a transformation value change direction angle, and wherein the determining the edge pixel corresponding to the road in the target image based on the transformation value change information includes:

obtaining pixel points meeting the change degree screening condition from all pixel points of the transformed image as candidate pixel points;

obtaining the contrast conversion value change degree of the candidate pixel point on the conversion value change direction angle;

determining the change degree difference of the change degree of the candidate pixel point relative to the change degree of the contrast change value;

and when the variation difference is larger than a variation difference threshold value, taking the candidate pixel point as an edge pixel corresponding to the road in the target image.

10. The method of claim 1, wherein the edge pixels are plural, and the identifying the target road in the target image based on the edge pixels comprises:

determining a current edge line corresponding to the edge pixel, determining a current edge line distance between the current edge line and a reference position, and determining a current edge line included angle between the current edge line and a preset reference direction line;

taking the included angle between the current edge line distance and the current edge line as the current edge line parameter to be adjusted, adjusting the current edge line parameter of each current edge line until a parameter adjustment stopping condition is met, and taking the edge line corresponding to the current edge line parameter meeting the parameter adjustment condition as the road edge line corresponding to the road in the target image, wherein the parameter adjustment stopping condition comprises at least one of the condition that the difference between the edge line distances is smaller than a distance difference threshold value or the difference between the included angles of the edge lines is smaller than an included angle difference threshold value;

determining a target road in the target image based on the road edge line.

11. The method of claim 1, wherein the obtaining the target image to be subject to road recognition comprises:

acquiring a target image uploaded by a terminal, and taking the target image as a target image to be subjected to road identification;

the road detection of the target image to obtain a target road detection value corresponding to each target pixel point in the target image comprises:

acquiring a target user type corresponding to the terminal, and acquiring a trained road detection model corresponding to the target user type, wherein the trained road detection model is obtained by utilizing a target training image corresponding to the target user type for training;

inputting the target image into the trained road detection model for road detection to obtain a target road detection value corresponding to each target pixel point in the target image;

the method further comprises the following steps:

determining user movement prompt information based on the position corresponding to the target road and the position of the terminal;

and sending the user movement prompt information to the terminal so that the terminal carries out movement prompt according to the user movement prompt information.

12. A road recognition device, characterized in that the device comprises:

the target image acquisition module is used for acquiring a target image to be subjected to road identification;

a road detection value obtaining module, configured to perform road detection on the target image to obtain a target road detection value corresponding to each target pixel point in the target image;

a transformed image obtaining module, configured to determine pixel transformation values corresponding to target pixel points in the target image based on the target road detection value, and arrange the pixel transformation values corresponding to the target pixel points according to a pixel point arrangement sequence to obtain a transformed image corresponding to the target image;

the edge pixel determining module is used for determining conversion value change information of the target pixel point in the conversion image relative to a reference pixel point in the conversion image and determining an edge pixel corresponding to a road in the target image based on the conversion value change information;

and the target road obtaining module is used for identifying and obtaining the target road in the target image based on the edge pixels.

13. The apparatus of claim 12, wherein the road detection value obtaining module comprises:

the target user type obtaining unit is used for obtaining a target user type corresponding to the target image;

a first road detection model obtaining unit, configured to obtain a trained road detection model corresponding to the target user type, where the trained road detection model is obtained by training a target training image corresponding to the target user type;

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 11 when executing the computer program.

15. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 11.